2024-05-14T10:59:24.478 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T10:59:24.479 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --archive /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738 --verbose --name vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/orig.config.yaml 2024-05-14T10:59:24.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T10:59:24.613 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738 branch: wip-vshankar-testing-20240509.053109-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: vshankar@redhat.com first_in_suite: false job_id: '7705738' kernel: flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing-20240509.053109-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: e2b755a64f0d72176b57299d02e880cf0620d780 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: e2b755a64f0d72176b57299d02e880cf0620d780 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240509.053109-debug sha1: e2b755a64f0d72176b57299d02e880cf0620d780 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6285 sha1: e2b755a64f0d72176b57299d02e880cf0620d780 sleep_before_teardown: 0 subset: 548/1024 suite: fs suite_branch: wip-vshankar-testing-20240509.053109-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e2b755a64f0d72176b57299d02e880cf0620d780 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUFfQvC/Eeg5MY0zk5NfdO/V0tles3akRJGBV52ASHZhy74RNaRRuEH/JG/4tzI0p1mm2wuxczjBQZrvBgj6oI= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMQCUVx0Oaewy2yohiXHF0W2rtGXKf/aJNUvheYs+LseaKr4ShASChwzQCxpjGn9M/gpHNvNQa1uUJxEZ4KHn4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_07:04:04 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T10:59:24.613 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa; will attempt to use it 2024-05-14T10:59:24.614 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks 2024-05-14T10:59:24.614 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T10:59:24.615 INFO:teuthology.task.internal:Checking packages... 2024-05-14T10:59:24.639 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e2b755a64f0d72176b57299d02e880cf0620d780' 2024-05-14T10:59:24.639 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T10:59:24.639 INFO:teuthology.packaging:ref: None 2024-05-14T10:59:24.639 INFO:teuthology.packaging:tag: None 2024-05-14T10:59:24.639 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240509.053109-debug 2024-05-14T10:59:24.639 INFO:teuthology.packaging:sha1: e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T10:59:24.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing-20240509.053109-debug 2024-05-14T10:59:24.791 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3588-ge2b755a6-1jammy 2024-05-14T10:59:24.793 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T10:59:24.800 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T10:59:24.800 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T10:59:24.839 INFO:teuthology.task.internal:Saving configuration 2024-05-14T10:59:24.855 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T10:59:24.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T10:59:24.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 10:53:45.720940', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGCfzvQCWmb6sOssS5k3PlyDVZ3Qvo5lvn3VI3YJkguW'} 2024-05-14T10:59:24.910 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 10:53:45.719103', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC2g8whzxd3W9TOpA7AYKl5U7xSXG5onlnGRTycWuxrF'} 2024-05-14T10:59:24.910 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T10:59:24.918 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-14T10:59:24.918 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-14T10:59:24.918 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T10:59:25.007 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdf09a5d3a0>, signals=[15]) 2024-05-14T10:59:25.007 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T10:59:25.015 INFO:teuthology.task.internal:Opening connections... 2024-05-14T10:59:25.015 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T10:59:25.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T10:59:25.101 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T10:59:25.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T10:59:25.190 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T10:59:25.207 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-05-14T10:59:25.213 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-05-14T10:59:25.213 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-05-14T10:59:25.258 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-14T10:59:25.258 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2024-05-14T10:59:25.258 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="22.04" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=jammy 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-14T10:59:25.259 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=jammy 2024-05-14T10:59:25.260 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-05-14T10:59:25.286 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-05-14T10:59:25.291 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-05-14T10:59:25.292 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-05-14T10:59:25.339 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Ubuntu" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="22.04" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_CODENAME=jammy 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:ID=ubuntu 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE=debian 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-14T10:59:25.340 INFO:teuthology.orchestra.run.smithi149.stdout:UBUNTU_CODENAME=jammy 2024-05-14T10:59:25.341 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-05-14T10:59:25.360 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T10:59:25.375 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T10:59:25.409 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T10:59:25.410 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-05-14T10:59:25.413 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-05-14T10:59:25.417 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T10:59:25.442 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T10:59:25.442 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-05-14T10:59:25.460 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-05-14T10:59:25.470 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T10:59:25.517 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T10:59:25.534 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-14T10:59:25.534 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}}, timeout 300 2024-05-14T10:59:25.535 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T10:59:25.535 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T10:59:25.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:59:25.540 DEBUG:teuthology.task.kernel:sha1 for mon.b is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.540 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-14T10:59:25.540 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-14T10:59:25.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T10:59:25.541 DEBUG:teuthology.task.kernel:sha1 for mon.a is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.541 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-14T10:59:25.541 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-05-14T10:59:25.587 INFO:teuthology.orchestra.run.smithi116.stdout:5.15.0-56-generic 2024-05-14T10:59:25.587 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.587 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-14T10:59:25.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.591 INFO:teuthology.orchestra.run.smithi149.stdout:5.15.0-56-generic 2024-05-14T10:59:25.591 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.591 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-14T10:59:25.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T10:59:25.735 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-05-14T10:59:25.735 DEBUG:teuthology.orchestra.run.smithi116:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-14T10:59:25.737 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-05-14T10:59:25.737 DEBUG:teuthology.orchestra.run.smithi149:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-14T10:59:25.749 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-14T10:59:25.752 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: git config --global init.defaultBranch 2024-05-14T10:59:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:hint: 2024-05-14T10:59:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-14T10:59:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-14T10:59:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:hint: 2024-05-14T10:59:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:hint: git branch -m 2024-05-14T10:59:25.890 INFO:teuthology.orchestra.run.smithi149.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-14T10:59:25.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: git config --global init.defaultBranch 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: 2024-05-14T10:59:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:hint: git branch -m 2024-05-14T10:59:25.905 INFO:teuthology.orchestra.run.smithi116.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-14T10:59:25.907 DEBUG:teuthology.orchestra.run.smithi116:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-14T10:59:25.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-14T10:59:25.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-14T10:59:26.015 DEBUG:teuthology.orchestra.run.smithi149:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-14T10:59:26.032 DEBUG:teuthology.orchestra.run.smithi116:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-14T11:01:04.838 INFO:teuthology.orchestra.run.smithi149.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-14T11:01:04.838 INFO:teuthology.orchestra.run.smithi149.stderr: * [new branch] main -> origin/main 2024-05-14T11:01:04.838 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20240513 -> 20240513 2024-05-14T11:01:04.871 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190312 -> 20190312 2024-05-14T11:01:04.871 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190416 -> 20190416 2024-05-14T11:01:04.871 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190514 -> 20190514 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190618 -> 20190618 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190717 -> 20190717 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190815 -> 20190815 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20190923 -> 20190923 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20191022 -> 20191022 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20191215 -> 20191215 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200122 -> 20200122 2024-05-14T11:01:04.872 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200316 -> 20200316 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200421 -> 20200421 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200519 -> 20200519 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200619 -> 20200619 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200721 -> 20200721 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200817 -> 20200817 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20200918 -> 20200918 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20201022 -> 20201022 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20201118 -> 20201118 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20201218 -> 20201218 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210208 -> 20210208 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210315 -> 20210315 2024-05-14T11:01:04.874 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210511 -> 20210511 2024-05-14T11:01:04.875 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210716 -> 20210716 2024-05-14T11:01:04.875 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210818 -> 20210818 2024-05-14T11:01:04.875 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20210919 -> 20210919 2024-05-14T11:01:04.875 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20211027 -> 20211027 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20211216 -> 20211216 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220209 -> 20220209 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220310 -> 20220310 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220411 -> 20220411 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220509 -> 20220509 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220610 -> 20220610 2024-05-14T11:01:04.877 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220708 -> 20220708 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220815 -> 20220815 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20220913 -> 20220913 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20221012 -> 20221012 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20221109 -> 20221109 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20221214 -> 20221214 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230117 -> 20230117 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230210 -> 20230210 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230310 -> 20230310 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230404 -> 20230404 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230515 -> 20230515 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230625 -> 20230625 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230804 -> 20230804 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20230919 -> 20230919 2024-05-14T11:01:04.878 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20231030 -> 20231030 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20231111 -> 20231111 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20231211 -> 20231211 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20240115 -> 20240115 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20240220 -> 20240220 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20240312 -> 20240312 2024-05-14T11:01:04.879 INFO:teuthology.orchestra.run.smithi149.stderr: * [new tag] 20240410 -> 20240410 2024-05-14T11:01:04.887 INFO:teuthology.orchestra.run.smithi116.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-14T11:01:04.888 INFO:teuthology.orchestra.run.smithi116.stderr: * [new branch] main -> origin/main 2024-05-14T11:01:04.888 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20240513 -> 20240513 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190312 -> 20190312 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190416 -> 20190416 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190514 -> 20190514 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190618 -> 20190618 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190717 -> 20190717 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190815 -> 20190815 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190923 -> 20190923 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20191022 -> 20191022 2024-05-14T11:01:04.922 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20191215 -> 20191215 2024-05-14T11:01:04.924 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200122 -> 20200122 2024-05-14T11:01:04.924 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200316 -> 20200316 2024-05-14T11:01:04.924 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200421 -> 20200421 2024-05-14T11:01:04.924 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200519 -> 20200519 2024-05-14T11:01:04.924 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200619 -> 20200619 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200721 -> 20200721 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200817 -> 20200817 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200918 -> 20200918 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20201022 -> 20201022 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20201118 -> 20201118 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20201218 -> 20201218 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210208 -> 20210208 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210315 -> 20210315 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210511 -> 20210511 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210716 -> 20210716 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210818 -> 20210818 2024-05-14T11:01:04.925 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20210919 -> 20210919 2024-05-14T11:01:04.926 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20211027 -> 20211027 2024-05-14T11:01:04.928 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20211216 -> 20211216 2024-05-14T11:01:04.928 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220209 -> 20220209 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220310 -> 20220310 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220411 -> 20220411 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220509 -> 20220509 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220610 -> 20220610 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220708 -> 20220708 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220815 -> 20220815 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20220913 -> 20220913 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20221012 -> 20221012 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20221109 -> 20221109 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20221214 -> 20221214 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230117 -> 20230117 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230210 -> 20230210 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230310 -> 20230310 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230404 -> 20230404 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230515 -> 20230515 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230625 -> 20230625 2024-05-14T11:01:04.929 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230804 -> 20230804 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20230919 -> 20230919 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20231030 -> 20231030 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20231111 -> 20231111 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20231211 -> 20231211 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20240115 -> 20240115 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20240220 -> 20240220 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20240312 -> 20240312 2024-05-14T11:01:04.930 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20240410 -> 20240410 2024-05-14T11:01:09.177 INFO:teuthology.orchestra.run.smithi149.stderr:Updating files: 45% (1564/3419) Updating files: 46% (1573/3419) Updating files: 47% (1607/3419) Updating files: 48% (1642/3419) Updating files: 49% (1676/3419) Updating files: 50% (1710/3419) Updating files: 51% (1744/3419) Updating files: 52% (1778/3419) Updating files: 53% (1813/3419) Updating files: 54% (1847/3419) Updating files: 55% (1881/3419) Updating files: 56% (1915/3419) Updating files: 57% (1949/3419) Updating files: 58% (1984/3419) Updating files: 59% (2018/3419) Updating files: 59% (2045/3419) Updating files: 60% (2052/3419) Updating files: 61% (2086/3419) Updating files: 62% (2120/3419) Updating files: 63% (2154/3419) Updating files: 64% (2189/3419) Updating files: 65% (2223/3419) Updating files: 66% (2257/3419) Updating files: 67% (2291/3419) Updating files: 68% (2325/3419) Updating files: 69% (2360/3419) Updating files: 70% (2394/3419) Updating files: 70% (2400/3419) Updating files: 71% (2428/3419) Updating files: 72% (2462/3419) Updating files: 73% (2496/3419) Updating files: 74% (2531/3419) Updating files: 75% (2565/3419) Updating files: 76% (2599/3419) Updating files: 77% (2633/3419) Updating files: 78% (2667/3419) Updating files: 79% (2702/3419) Updating files: 80% (2736/3419) Updating files: 81% (2770/3419) Updating files: 82% (2804/3419) Updating files: 83% (2838/3419) Updating files: 83% (2846/3419) Updating files: 84% (2872/3419) Updating files: 85% (2907/3419) Updating files: 86% (2941/3419) Updating files: 87% (2975/3419) Updating files: 88% (3009/3419) Updating files: 89% (3043/3419) Updating files: 90% (3078/3419) Updating files: 91% (3112/3419) Updating files: 92% (3146/3419) Updating files: 93% (3180/3419) Updating files: 94% (3214/3419) Updating files: 95% (3249/3419) Updating files: 96% (3283/3419) Updating files: 97% (3317/3419) Updating files: 98% (3351/3419) Updating files: 99% (3385/3419) Updating files: 100% (3419/3419) Updating files: 100% (3419/3419), done. 2024-05-14T11:01:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:Updating files: 45% (1570/3419) Updating files: 46% (1573/3419) Updating files: 47% (1607/3419) Updating files: 48% (1642/3419) Updating files: 49% (1676/3419) Updating files: 50% (1710/3419) Updating files: 51% (1744/3419) Updating files: 52% (1778/3419) Updating files: 53% (1813/3419) Updating files: 54% (1847/3419) Updating files: 55% (1881/3419) Updating files: 56% (1915/3419) Updating files: 57% (1949/3419) Updating files: 58% (1984/3419) Updating files: 59% (2018/3419) Updating files: 59% (2051/3419) Updating files: 60% (2052/3419) Updating files: 61% (2086/3419) Updating files: 62% (2120/3419) Updating files: 63% (2154/3419) Updating files: 64% (2189/3419) Updating files: 65% (2223/3419) Updating files: 66% (2257/3419) Updating files: 67% (2291/3419) Updating files: 68% (2325/3419) Updating files: 69% (2360/3419) Updating files: 70% (2394/3419) Updating files: 71% (2428/3419) Updating files: 72% (2462/3419) Updating files: 72% (2478/3419) Updating files: 73% (2496/3419) Updating files: 74% (2531/3419) Updating files: 75% (2565/3419) Updating files: 76% (2599/3419) Updating files: 77% (2633/3419) Updating files: 78% (2667/3419) Updating files: 79% (2702/3419) Updating files: 80% (2736/3419) Updating files: 81% (2770/3419) Updating files: 82% (2804/3419) Updating files: 83% (2838/3419) Updating files: 83% (2853/3419) Updating files: 84% (2872/3419) Updating files: 85% (2907/3419) Updating files: 86% (2941/3419) Updating files: 87% (2975/3419) Updating files: 88% (3009/3419) Updating files: 89% (3043/3419) Updating files: 90% (3078/3419) Updating files: 91% (3112/3419) Updating files: 92% (3146/3419) Updating files: 93% (3180/3419) Updating files: 94% (3214/3419) Updating files: 95% (3249/3419) Updating files: 96% (3283/3419) Updating files: 97% (3317/3419) Updating files: 98% (3351/3419) Updating files: 99% (3385/3419) Updating files: 100% (3419/3419) Updating files: 100% (3419/3419), done. 2024-05-14T11:01:09.182 INFO:teuthology.orchestra.run.smithi149.stdout:HEAD is now at b9d2bf23 Merge branch 'robot/pr-0-1715362889' into 'main' 2024-05-14T11:01:09.185 INFO:teuthology.orchestra.run.smithi116.stdout:HEAD is now at b9d2bf23 Merge branch 'robot/pr-0-1715362889' into 'main' 2024-05-14T11:01:09.203 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.b... 2024-05-14T11:01:09.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T11:01:09.209 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.a... 2024-05-14T11:01:09.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-14T11:01:09.346 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:09.347 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-gfb9a6a44107e_6.9.0-rc3-gfb9a6a44107e-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/ --input-file=- 2024-05-14T11:01:09.350 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi149.front.sepia.ceph.com to complete... 2024-05-14T11:01:09.356 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:09.356 DEBUG:teuthology.orchestra.run.smithi116:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-gfb9a6a44107e_6.9.0-rc3-gfb9a6a44107e-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/ --input-file=- 2024-05-14T11:01:09.359 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi116.front.sepia.ceph.com to complete... 2024-05-14T11:01:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14 11:01:13 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/linux-image-6.9.0-rc3-gfb9a6a44107e_6.9.0-rc3-gfb9a6a44107e-1_amd64.deb [44809390/44809390] -> "/tmp/linux-image.deb" [1] 2024-05-14T11:01:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:FINISHED --2024-05-14 11:01:13-- 2024-05-14T11:01:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:Total wall clock time: 3.9s 2024-05-14T11:01:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:Downloaded: 1 files, 43M in 3.6s (11.9 MB/s) 2024-05-14T11:01:13.315 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.a... 2024-05-14T11:01:13.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo dpkg -i /tmp/linux-image.deb 2024-05-14T11:01:13.563 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-gfb9a6a44107e. 2024-05-14T11:01:15.530 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-14T11:01:15.534 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-14T11:01:15.730 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-image-6.9.0-rc3-gfb9a6a44107e (6.9.0-rc3-gfb9a6a44107e-1) ... 2024-05-14T11:01:19.186 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-image-6.9.0-rc3-gfb9a6a44107e (6.9.0-rc3-gfb9a6a44107e-1) ... 2024-05-14T11:01:19.294 INFO:teuthology.orchestra.run.smithi116.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:19.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14 11:01:18 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-gfb9a6a44107e/linux-image-6.9.0-rc3-gfb9a6a44107e_6.9.0-rc3-gfb9a6a44107e-1_amd64.deb [44809390/44809390] -> "/tmp/linux-image.deb" [1] 2024-05-14T11:01:19.330 INFO:teuthology.orchestra.run.smithi149.stderr:FINISHED --2024-05-14 11:01:18-- 2024-05-14T11:01:19.330 INFO:teuthology.orchestra.run.smithi149.stderr:Total wall clock time: 10s 2024-05-14T11:01:19.330 INFO:teuthology.orchestra.run.smithi149.stderr:Downloaded: 1 files, 43M in 9.5s (4.47 MB/s) 2024-05-14T11:01:19.334 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on mon.b... 2024-05-14T11:01:19.335 DEBUG:teuthology.orchestra.run.smithi149:> sudo dpkg -i /tmp/linux-image.deb 2024-05-14T11:01:19.586 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-gfb9a6a44107e. 2024-05-14T11:01:21.559 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-14T11:01:21.564 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-14T11:01:21.701 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking linux-image-6.9.0-rc3-gfb9a6a44107e (6.9.0-rc3-gfb9a6a44107e-1) ... 2024-05-14T11:01:25.454 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up linux-image-6.9.0-rc3-gfb9a6a44107e (6.9.0-rc3-gfb9a6a44107e-1) ... 2024-05-14T11:01:25.584 INFO:teuthology.orchestra.run.smithi149.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:40.186 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub' 2024-05-14T11:01:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-14T11:01:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-05-14T11:01:40.508 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:40.814 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-14T11:01:40.825 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-14T11:01:40.886 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-14T11:01:40.891 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-14T11:01:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-14T11:01:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-14T11:01:41.045 INFO:teuthology.orchestra.run.smithi116.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-14T11:01:41.090 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-05-14T11:01:41.259 DEBUG:teuthology.orchestra.run.smithi116:> dpkg-deb -c /tmp/linux-image.deb 2024-05-14T11:01:41.266 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./ 2024-05-14T11:01:41.266 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./boot/ 2024-05-14T11:01:41.266 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7002012 2024-05-10 02:40 ./boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.284 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 172073 2024-05-10 02:40 ./boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.284 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13120512 2024-05-10 02:40 ./boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/ 2024-05-14T11:01:41.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/ 2024-05-14T11:01:41.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/postinst.d/ 2024-05-14T11:01:41.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/postrm.d/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/preinst.d/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/prerm.d/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 507569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-14T11:01:41.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 109905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 155249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 87009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-14T11:01:41.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/ 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/ 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/ 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpuid.ko 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/msr.ko 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/ 2024-05-14T11:01:41.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 444425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-14T11:01:41.310 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1035881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-14T11:01:41.310 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2849609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/af_alg.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/algif_hash.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/algif_skcipher.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/asymmetric_keys/ 2024-05-14T11:01:41.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/ 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-14T11:01:41.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cryptd.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_simd.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-14T11:01:41.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lrw.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 169065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-14T11:01:41.332 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xts.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_extlog.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/apei/ 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/apei/einj.ko 2024-05-14T11:01:41.333 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-14T11:01:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-14T11:01:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_legacy.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_radisys.ko 2024-05-14T11:01:41.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rz1000.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-14T11:01:41.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/ 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 163969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/ 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/ 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 140865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-14T11:01:41.338 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/ 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1241657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 203289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/ 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 143433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/ 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 313217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/ 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 117233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/ 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-14T11:01:41.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/applicom.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/ 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 186921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 179561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-14T11:01:41.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/ 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/nvram.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/ 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-14T11:01:41.341 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/ 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/ 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-palmas.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-si5351.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-twl6040.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-wm831x.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/ 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-14T11:01:41.342 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 89257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 333657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/ 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/ 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-14T11:01:41.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 751713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-14T11:01:41.395 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-14T11:01:41.395 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 146769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-14T11:01:41.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/ 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-gpio.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max14577.ko 2024-05-14T11:01:41.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max77693.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max8997.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-palmas.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/ 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/efi/ 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/ 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-14T11:01:41.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-arizona.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-da9052.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-da9055.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-dln2.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-generic.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-kempld.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max7300.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max730x.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max732x.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-14T11:01:41.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-sch.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-vx855.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/ 2024-05-14T11:01:41.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-a4tech.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-apple.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-belkin.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cherry.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-chicony.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cp2112.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cypress.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-14T11:01:41.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ezkey.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-generic.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kensington.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-14T11:01:41.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lg-g15.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 121369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-magicmouse.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-microsoft.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-monterey.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ntrig.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-14T11:01:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-14T11:01:41.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-14T11:01:41.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 184745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 319921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/ 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 153633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-14T11:01:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 261945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/ 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 382033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-14T11:01:41.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 89025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-14T11:01:41.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 113201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc2103.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-14T11:01:41.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hih6130.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-14T11:01:41.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 156465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/jc42.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-14T11:01:41.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 131505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-14T11:01:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 141353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-14T11:01:41.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-14T11:01:41.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-14T11:01:41.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 116777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 134441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-14T11:01:41.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/ 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/ 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/ 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-14T11:01:41.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-14T11:01:41.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-14T11:01:41.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/ 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 329697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1119913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-14T11:01:41.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 408433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 347217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 590929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 514641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 911665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 375545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-14T11:01:41.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 197121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/usnic/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 212553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 334401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 132585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 157937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 156673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-14T11:01:41.422 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/ 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/evbug.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/ff-memless.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/ 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/ 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-14T11:01:41.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lm8323.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lm8333.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/qt1070.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/qt2160.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-14T11:01:41.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/stowaway.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/matrix-keymap.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/ 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 160017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 114273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 428185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/psmouse.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-14T11:01:41.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/ 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 140969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ct82c710.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/parkbd.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/pcips2.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ps2mult.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serport.ko 2024-05-14T11:01:41.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/ 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-88pm860x.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-adp5520.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-bd2802.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-da903x.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-da9052.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-gpio.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3533.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm355x.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3642.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-14T11:01:41.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp8788.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-max8997.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-mc13783.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca9532.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca955x.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca963x.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-tca6507.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-wm8350.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-14T11:01:41.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/ 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/ 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 790753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 116825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bufio.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 199041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 134945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-14T11:01:41.429 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-era.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-mirror.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-region-hash.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 139817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-snapshot.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-14T11:01:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 234457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-zero.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/ 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 264329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 125009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 157681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 444665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/ 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/ 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 214841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-14T11:01:41.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 150761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/ 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm800.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm805.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm80x.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/arizona-i2c.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 148633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/arizona.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/bcm590xx.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/dln2.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/janz-cmodio.ko 2024-05-14T11:01:41.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/kempld-core.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lm3533-core.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lp3943.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/max8907.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/menf21bmc.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/retu-mfd.ko 2024-05-14T11:01:41.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 99425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/si476x-core.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/tps65010.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/tps6507x.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wm8994.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ad525x_dpot.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/ 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-14T11:01:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/ 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/core.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/ 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ds1682.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/dummy-irq.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/echo/ 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/echo/echo.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/ 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-14T11:01:41.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/genwqe/ 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 191649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hmc6352.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ibmasm/ 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 92345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ics932s401.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/ 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/ 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 154169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-14T11:01:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 389033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/phantom.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ti-st/ 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/ 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 234905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/ 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 620961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-14T11:01:41.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/ 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/ 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 139817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 587065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/ 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 158649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-14T11:01:41.438 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 109521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/ 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/ 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/ 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/ 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-14T11:01:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 128409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-14T11:01:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 105633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 228985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1733353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 167185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 161417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 440625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 432201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 257201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-14T11:01:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 446177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-14T11:01:41.442 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-14T11:01:41.442 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1311433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 206689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 269313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-14T11:01:41.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 169089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/ 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/ 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 356377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-14T11:01:41.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fujitsu/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 366217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 755793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 324577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1196297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-14T11:01:41.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 362593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 671801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 121537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 966705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 193585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/ 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-14T11:01:41.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 150777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/ 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 905033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 344145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1412417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 117353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-14T11:01:41.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 131425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 181081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 171417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/ 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-14T11:01:41.483 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 300817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 105689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 885129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 100689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 264161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/ 2024-05-14T11:01:41.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/ 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1350385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/ 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/ 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/ 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 100785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-14T11:01:41.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/ 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 644537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/ 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 128497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 207993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/ 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/ 2024-05-14T11:01:41.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/ 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 98689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/ 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/ 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/ 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/defxx.ko 2024-05-14T11:01:41.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/skfp/ 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 275801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/ 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 290441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/ 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/ 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-14T11:01:41.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/ 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-14T11:01:41.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 121105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-14T11:01:41.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/plip/ 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/plip/plip.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sb1000.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/ 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-14T11:01:41.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/ 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 137321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/ 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 176497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/ 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 383593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-14T11:01:41.492 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 175537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 335857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 168145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/ 2024-05-14T11:01:41.493 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pci-stub.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/ 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 179961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 133753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/ 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/broadcom/ 2024-05-14T11:01:41.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/samsung/ 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/ 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/ 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/ 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/ 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-14T11:01:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/ 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 87481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 164769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/ 2024-05-14T11:01:41.496 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/ 2024-05-14T11:01:41.497 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/da9030_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/da9052-battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/gpio-charger.ko 2024-05-14T11:01:41.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max14577_charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max17042_battery.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max8903_charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max8925_power.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/sbs-battery.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/test_power.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm831x_power.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm8350_power.ko 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/ 2024-05-14T11:01:41.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/ 2024-05-14T11:01:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/ 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9052.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9055.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9063.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-14T11:01:41.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-14T11:01:41.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8907.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8925.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8997.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8998.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-palmas.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-14T11:01:41.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-14T11:01:41.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 164569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/ 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 340721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/ 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 468081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 380673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic94xx/ 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 243705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/ 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 135617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-14T11:01:41.505 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 304993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1527113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 297801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 144401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 390713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-14T11:01:41.506 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 117481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 173017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/ 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 267577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/ 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 95329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 194017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fdomain.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/ 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 366529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-14T11:01:41.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 251505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/imm.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 327841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 129785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/ 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 397785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/ 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 386849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 178177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-14T11:01:41.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/ 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 274009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/ 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2736001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/ 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 389177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 104769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/ 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 836945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/ 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 158729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-14T11:01:41.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/ 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/ 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 443745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 152033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ppa.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/ 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2324809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/ 2024-05-14T11:01:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 697337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qlogicfas408.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 322865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 219857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 357153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 121185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 138121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 205025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-14T11:01:41.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/xen-scsifront.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 174913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/ 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/ 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 920169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/ 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-14T11:01:41.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1170033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-14T11:01:41.550 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 149721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/ 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/ 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/ 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-14T11:01:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-14T11:01:41.552 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 99313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/moxa.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/mxser.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 134401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/ 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/ 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/altera_uart.ko 2024-05-14T11:01:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/ 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/rp2.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/ 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 693081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/ 2024-05-14T11:01:41.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_mf624.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_netx.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/ 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/c67x00/ 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ 2024-05-14T11:01:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 189209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/ 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/common/ 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/common/ulpi.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc2/ 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/ 2024-05-14T11:01:41.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 574593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/ 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/ 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 131345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 146617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-14T11:01:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/ 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-14T11:01:41.558 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 204049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/ 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 182561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-14T11:01:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 87465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 95441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 195969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/ 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-14T11:01:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 104377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811_cs.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/ 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/cytherm.ko 2024-05-14T11:01:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ehset.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/lvstest.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/ 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-14T11:01:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbtest.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/mon/ 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/mon/usbmon.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/musb/ 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 396209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/ 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ 2024-05-14T11:01:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/f81232.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 188609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 152305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-14T11:01:41.564 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 122329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/metro-usb.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 89969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mxuport.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-14T11:01:41.565 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 193857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-14T11:01:41.566 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 151137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usbserial.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-14T11:01:41.567 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 201193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/ 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 122321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/ 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/ 2024-05-14T11:01:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 206921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 134265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/ 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 138721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/ 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/ 2024-05-14T11:01:41.569 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/bd6107.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-14T11:01:41.570 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/ 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/ 2024-05-14T11:01:41.571 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 129433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/acquirewdt.ko 2024-05-14T11:01:41.572 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/advantechwdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/dw_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-14T11:01:41.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/retu_wdt.ko 2024-05-14T11:01:41.574 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-14T11:01:41.575 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/ 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/ 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 213913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-scsiback.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/ 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ 2024-05-14T11:01:41.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 192617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/adfs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/adfs/adfs.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 197849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/afs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1261113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/afs/kafs.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/autofs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 127377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/autofs/autofs4.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/bfs/ 2024-05-14T11:01:41.577 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/bfs/bfs.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 4489593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 376857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1586065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 130073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/configfs/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 139081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/configfs/configfs.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/ 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-14T11:01:41.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 813225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/efs/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/efs/efs.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1906817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/freevxfs/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/freevxfs/freevxfs.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/ 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1486521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-14T11:01:41.579 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 183305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 282097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hpfs/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 249009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hpfs/hpfs.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 533073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 361369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/ 2024-05-14T11:01:41.580 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 591585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/ 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/ 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/ 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/ 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 147689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1320249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv2.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 151641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2518505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/ 2024-05-14T11:01:41.581 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/ 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2024049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/ 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 734025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/ 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-14T11:01:41.582 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ascii.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-14T11:01:41.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 95625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 144209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 167137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-14T11:01:41.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-14T11:01:41.585 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 302073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 548169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-14T11:01:41.586 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 3245617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-14T11:01:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-14T11:01:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-14T11:01:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-14T11:01:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/omfs/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/omfs/omfs.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 462873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx4/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx4/qnx4.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx6/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx6/qnx6.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/ 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_tree.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_v1.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_v2.ko 2024-05-14T11:01:41.627 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 684841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 3319041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 164185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/ 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-14T11:01:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 310017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 198233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 5192705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/locking/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/locking/locktorture.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/time/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/time/test_udelay.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-14T11:01:41.629 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc7.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/ 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/interval_tree_test.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/ 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/ 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-14T11:01:41.630 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/memory-notifier-error-inject.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/notifier-error-inject.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/percpu_test.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/pm-notifier-error-inject.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/ 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 152025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/rbtree_test.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/ 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test-kstrtox.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 4912441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_bpf.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_firmware.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_module.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_user_copy.ko 2024-05-14T11:01:41.631 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/xz/ 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/xz/xz_dec_test.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/ 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/ 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-14T11:01:41.632 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/ 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/ 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/ 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 236793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-14T11:01:41.633 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/ 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 879697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/ 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1030873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-14T11:01:41.634 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-14T11:01:41.635 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/ 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1243505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/ 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 153801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/ 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 244889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/ 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 507705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-14T11:01:41.636 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/hsr/ 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 128873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/hsr/hsr.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 357425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-14T11:01:41.637 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-14T11:01:41.638 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-14T11:01:41.639 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-14T11:01:41.640 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_gre.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 118881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-14T11:01:41.641 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/mip6.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-14T11:01:41.642 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-14T11:01:41.643 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/ 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/ 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 153217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/lapb/ 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/lapb/lapb.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/ 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-14T11:01:41.644 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 332889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc2.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/ 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 388873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/ 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 147745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-14T11:01:41.645 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 137833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 133681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 139073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 138265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 136177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 139897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 534113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-14T11:01:41.646 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 496817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-14T11:01:41.647 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 134377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 140801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 117201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-14T11:01:41.648 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 118369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 807985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-14T11:01:41.649 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_osf.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-14T11:01:41.650 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_synproxy.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tproxy.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/x_tables.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-14T11:01:41.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-14T11:01:41.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-14T11:01:41.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-14T11:01:41.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-14T11:01:41.655 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpudp.ko 2024-05-14T11:01:41.656 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/ 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/ 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/ 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 488825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/ 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/ 2024-05-14T11:01:41.657 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/phonet.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/pn_pep.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/ 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 318617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 202305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rxrpc/ 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1027897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rxrpc/rxrpc.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/ 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-14T11:01:41.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_cgroup.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-14T11:01:41.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq_codel.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-14T11:01:41.660 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-14T11:01:41.661 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/ 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1187313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/ 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/ 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 432409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1940889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/ 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 890729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/ 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1122337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-14T11:01:41.662 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/ 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/ 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 105609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/x25/ 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 165841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/x25/x25.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/ 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_algo.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 131865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_user.ko 2024-05-14T11:01:41.663 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 644769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 628542 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7706 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9941 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70990 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 146959 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 232835 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 210 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72602 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 879 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 258450 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 313019 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/ 2024-05-14T11:01:41.664 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/ 2024-05-14T11:01:41.665 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/doc/ 2024-05-14T11:01:41.665 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:41.665 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 173 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/changelog.Debian.gz 2024-05-14T11:01:41.665 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 691 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/copyright 2024-05-14T11:01:41.669 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.670 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.670 DEBUG:teuthology.orchestra.run.smithi116:> egrep '(submenu|menuentry.*6.9.0-rc3-gfb9a6a44107e).*{' /boot/grub/grub.cfg 2024-05-14T11:01:41.677 INFO:teuthology.orchestra.run.smithi116.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:41.677 INFO:teuthology.orchestra.run.smithi116.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-gfb9a6a44107e-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:41.677 INFO:teuthology.orchestra.run.smithi116.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-gfb9a6a44107e-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:41.677 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-14T11:01:41.677 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:41.678 DEBUG:teuthology.orchestra.run.smithi116:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-14T11:01:41.722 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi116.front.sepia.ceph.com to complete... 2024-05-14T11:01:42.153 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub' 2024-05-14T11:01:42.154 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-14T11:01:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-05-14T11:01:42.354 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:42.360 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:42.597 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-14T11:01:42.604 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-14T11:01:42.646 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-14T11:01:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-14T11:01:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-14T11:01:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-14T11:01:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-14T11:01:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-05-14T11:01:48.012 INFO:teuthology.orchestra.run.smithi149.stderr:Sourcing file `/etc/default/grub' 2024-05-14T11:01:48.024 INFO:teuthology.orchestra.run.smithi149.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-14T11:01:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-05-14T11:01:48.317 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:48.334 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:48.666 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-14T11:01:48.678 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-14T11:01:48.745 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-14T11:01:48.750 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-14T11:01:48.886 INFO:teuthology.orchestra.run.smithi149.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-14T11:01:48.886 INFO:teuthology.orchestra.run.smithi149.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-14T11:01:48.887 INFO:teuthology.orchestra.run.smithi149.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-14T11:01:48.919 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-05-14T11:01:49.095 DEBUG:teuthology.orchestra.run.smithi149:> dpkg-deb -c /tmp/linux-image.deb 2024-05-14T11:01:49.145 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./ 2024-05-14T11:01:49.145 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./boot/ 2024-05-14T11:01:49.146 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7002012 2024-05-10 02:40 ./boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.163 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 172073 2024-05-10 02:40 ./boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.163 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13120512 2024-05-10 02:40 ./boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/postinst.d/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/postrm.d/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/preinst.d/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./etc/kernel/prerm.d/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/ 2024-05-14T11:01:49.183 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ 2024-05-14T11:01:49.184 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 507569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 109905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 155249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-14T11:01:49.187 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 87009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/ 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/ 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/ 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-14T11:01:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpuid.ko 2024-05-14T11:01:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/msr.ko 2024-05-14T11:01:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/ 2024-05-14T11:01:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 444425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-14T11:01:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1035881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-14T11:01:49.189 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 2849609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-14T11:01:49.211 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/af_alg.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/algif_hash.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/algif_skcipher.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/asymmetric_keys/ 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/ 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-14T11:01:49.212 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-14T11:01:49.213 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cryptd.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_simd.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-14T11:01:49.214 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lrw.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 169065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-14T11:01:49.215 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xts.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_extlog.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/apei/ 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/apei/einj.ko 2024-05-14T11:01:49.216 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 119033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 96825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 107737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-14T11:01:49.217 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-14T11:01:49.218 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_legacy.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-14T11:01:49.219 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_radisys.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rz1000.ko 2024-05-14T11:01:49.220 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 102121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-14T11:01:49.221 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/ 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 163969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/ 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/ 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 140865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/ 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1241657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 203289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-14T11:01:49.222 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 115265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 143433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 126617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 119873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 313217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 117233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/applicom.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/ 2024-05-14T11:01:49.223 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 186921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 179561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/ 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 103409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-14T11:01:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/nvram.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/ 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/ 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/ 2024-05-14T11:01:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-palmas.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-si5351.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-twl6040.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/clk/clk-wm831x.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/ 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 89257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 333657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/ 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/ 2024-05-14T11:01:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-14T11:01:49.227 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 751713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-14T11:01:49.281 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 146769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-14T11:01:49.282 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 96145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 68697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/ 2024-05-14T11:01:49.283 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-gpio.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max14577.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max77693.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-max8997.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-palmas.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/ 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/efi/ 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/ 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-14T11:01:49.284 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-arizona.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-da9052.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-da9055.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-dln2.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 81073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-generic.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-kempld.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max7300.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max730x.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-max732x.ko 2024-05-14T11:01:49.285 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-sch.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-vx855.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-14T11:01:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/ 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-a4tech.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-apple.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-belkin.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cherry.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-chicony.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 74393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cp2112.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-cypress.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-14T11:01:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ezkey.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-generic.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kensington.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-14T11:01:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lg-g15.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 68081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 121369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-magicmouse.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-microsoft.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-monterey.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ntrig.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-14T11:01:49.289 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-14T11:01:49.290 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-14T11:01:49.291 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 81113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 184745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 319921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/ 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 153633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-14T11:01:49.292 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 261945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/ 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 75065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 110681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 382033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 77153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-14T11:01:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 115057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 89025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 81473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 68865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-14T11:01:49.294 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 113201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc2103.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-14T11:01:49.295 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hih6130.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-14T11:01:49.296 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 156465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/jc42.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 86497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-14T11:01:49.297 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 131505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-14T11:01:49.298 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 141353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 72737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-14T11:01:49.299 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ 2024-05-14T11:01:49.300 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-14T11:01:49.300 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-14T11:01:49.300 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 101233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-14T11:01:49.302 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-14T11:01:49.303 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 71169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 96577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 75129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 83425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 116777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 134441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/ 2024-05-14T11:01:49.304 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/ 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/ 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-14T11:01:49.305 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 76969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-14T11:01:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-14T11:01:49.307 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/ 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 329697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1119913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 83905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-14T11:01:49.308 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 408433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 120921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 347217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 84945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 590929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/ 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 514641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/ 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 911665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ 2024-05-14T11:01:49.309 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 375545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 197121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/usnic/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 212553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 334401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 120673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 132585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 157937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ 2024-05-14T11:01:49.310 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 156673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/ 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/evbug.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/ff-memless.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/ 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/ 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-14T11:01:49.311 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lm8323.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/lm8333.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/qt1070.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/qt2160.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/stowaway.ko 2024-05-14T11:01:49.312 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/matrix-keymap.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/ 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 160017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 114273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 428185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/psmouse.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-14T11:01:49.313 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/ 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 140969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ct82c710.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/parkbd.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/pcips2.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/ps2mult.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serport.ko 2024-05-14T11:01:49.314 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/ 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-88pm860x.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-adp5520.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 77265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-bd2802.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-da903x.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-da9052.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-gpio.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3533.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm355x.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3642.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp8788.ko 2024-05-14T11:01:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-max8997.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-mc13783.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca9532.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca955x.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-pca963x.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-tca6507.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-wm8350.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-14T11:01:49.316 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/ 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/ 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 790753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 116825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bufio.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 58281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 199041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 134945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-era.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-14T11:01:49.317 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-mirror.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-region-hash.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 139817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-snapshot.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 234457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-14T11:01:49.318 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-zero.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/ 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 264329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 125009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 157681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 444665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/ 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/ 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 214841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 77985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 150761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-14T11:01:49.319 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 63873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/ 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm800.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm805.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/88pm80x.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/arizona-i2c.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 148633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/arizona.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/bcm590xx.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/dln2.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/janz-cmodio.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/kempld-core.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lm3533-core.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lp3943.ko 2024-05-14T11:01:49.320 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/max8907.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/menf21bmc.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/pcf50633.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/retu-mfd.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 99425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/si476x-core.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 58057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/tps65010.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/tps6507x.ko 2024-05-14T11:01:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wm8994.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ad525x_dpot.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/ 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 58633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-14T11:01:49.322 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/ 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/c2port/core.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/ 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ds1682.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/dummy-irq.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/echo/ 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/echo/echo.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/ 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-14T11:01:49.323 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/genwqe/ 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 191649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hmc6352.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ibmasm/ 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 92345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ics932s401.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/ 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-14T11:01:49.324 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/ 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 154169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 389033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/phantom.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ti-st/ 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/ 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 234905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/ 2024-05-14T11:01:49.325 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 620961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/ 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/ 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 139817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 587065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/ 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-14T11:01:49.326 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 158649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 73865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 109521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/ 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/ 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 84329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/ 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-14T11:01:49.327 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 115881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 124497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 128409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 110345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 101857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-14T11:01:49.328 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 88953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/ 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 105633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 228985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1733353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-14T11:01:49.329 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 167185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-14T11:01:49.370 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-14T11:01:49.370 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 161417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-14T11:01:49.370 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 440625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 432201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 257201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 446177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1311433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 206689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-14T11:01:49.371 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/ 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 269313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/ 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 77457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 169089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 76209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/ 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 75945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-14T11:01:49.372 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/ 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 356377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 63185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fujitsu/ 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 115569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 366217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-14T11:01:49.373 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 755793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 324577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1196297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 362593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 671801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 121537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 966705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-14T11:01:49.374 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 193585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/ 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 126881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 150777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/ 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 905033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 344145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1412417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-14T11:01:49.375 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 117353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 131425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 181081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/ 2024-05-14T11:01:49.376 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 171417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 58249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 300817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 105689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 885129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/ 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 81641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-14T11:01:49.377 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 100689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 264161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 142449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1350385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 80065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/ 2024-05-14T11:01:49.378 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 74289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 79457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 100785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 83441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/ 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 644537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/ 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 128497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 207993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-14T11:01:49.379 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 82633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 110729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 98689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 102513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/ 2024-05-14T11:01:49.380 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/defxx.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/skfp/ 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 275801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/ 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 290441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/ 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/ 2024-05-14T11:01:49.381 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/ 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-14T11:01:49.382 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 103377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 96025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 121105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-14T11:01:49.383 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/plip/ 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/plip/plip.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sb1000.ko 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/ 2024-05-14T11:01:49.384 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 80161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/ 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 137321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/ 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 176497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/ 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 383593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-14T11:01:49.385 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 175537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 335857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 126553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 168145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 103889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/ 2024-05-14T11:01:49.386 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/ 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pci-stub.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/ 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 179961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 133753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/ 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/broadcom/ 2024-05-14T11:01:49.387 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/samsung/ 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/ 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/ 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/ 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/ 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-14T11:01:49.388 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/ 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 87481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 73865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 164769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-14T11:01:49.389 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/ 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-14T11:01:49.390 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/ 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 68833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 73073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/da9030_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/da9052-battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/gpio-charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-14T11:01:49.391 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max14577_charger.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max17042_battery.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max8903_charger.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/max8925_power.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/sbs-battery.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/test_power.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm831x_power.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/power/supply/wm8350_power.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/ 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 73297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/ 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/ 2024-05-14T11:01:49.392 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/ 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/ 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/ 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/ 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-14T11:01:49.393 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9052.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9055.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-da9063.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 82225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-14T11:01:49.394 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8907.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8925.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8997.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max8998.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-palmas.ko 2024-05-14T11:01:49.395 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-14T11:01:49.396 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 164569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/ 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 340721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 120337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/ 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 468081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 380673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-14T11:01:49.397 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic94xx/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 243705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 135617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 304993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1527113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 297801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/ 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 144401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-14T11:01:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/ 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 390713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/ 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 88881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 117481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 173017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/ 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 267577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-14T11:01:49.399 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/ 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 95329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 194017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fdomain.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/ 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 366529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 251505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/imm.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 327841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 129785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-14T11:01:49.400 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/ 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 397785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 63121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/ 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 386849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 178177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/ 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 274009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/ 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 2736001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/ 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 88529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-14T11:01:49.401 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 389177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 104769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/ 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 836945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/ 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 158729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 96537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/ 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 101625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/ 2024-05-14T11:01:49.402 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 443745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 152033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ppa.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/ 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 2324809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 103601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/ 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 697337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qlogicfas408.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 322865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 219857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 357153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-14T11:01:49.403 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 121185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-14T11:01:49.440 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 93481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-14T11:01:49.440 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-14T11:01:49.440 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-14T11:01:49.440 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 138121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-14T11:01:49.440 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 57401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 205025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/xen-scsifront.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 174913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/ 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/ 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 920169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/ 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-14T11:01:49.441 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1170033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 149721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/ 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/ 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/ 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-14T11:01:49.442 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-14T11:01:49.443 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 99313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/moxa.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/mxser.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 134401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/ 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/ 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/altera_uart.ko 2024-05-14T11:01:49.444 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/ 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 120577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/rp2.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/ 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 693081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/ 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-14T11:01:49.445 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_mf624.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_netx.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/ 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/c67x00/ 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 189209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-14T11:01:49.446 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/ 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 101545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 63073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 107417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/common/ 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/common/ulpi.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc2/ 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/ 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 574593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-14T11:01:49.447 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/ 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/ 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 72249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 131345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 146617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-14T11:01:49.448 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 101249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 71257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 84937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/ 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-14T11:01:49.449 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 204049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/ 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 182561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-14T11:01:49.450 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 94841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 80521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 87465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 95441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 195969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/ 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 104377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 102273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811_cs.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-14T11:01:49.451 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/ 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/cytherm.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ehset.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-14T11:01:49.452 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/lvstest.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/ 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 84265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbtest.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-14T11:01:49.453 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/mon/ 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 91529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/mon/usbmon.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/musb/ 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 396209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/ 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-14T11:01:49.454 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 79681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 58329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/f81232.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 188609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 152305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 142833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 69217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-14T11:01:49.455 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 122329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/metro-usb.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 79681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 89969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mxuport.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 193857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-14T11:01:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70497 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 151137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usbserial.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-14T11:01:49.457 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 64049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 201193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-14T11:01:49.458 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/ 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 76041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 122321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/ 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/ 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 206921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 134265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 93673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/ 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 138721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 71281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-14T11:01:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 107905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/ 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/ 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 51745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/bd6107.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-14T11:01:49.460 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/ 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-14T11:01:49.461 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/ 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 129433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-14T11:01:49.462 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/acquirewdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/advantechwdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/dw_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-14T11:01:49.463 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-14T11:01:49.464 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/retu_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-14T11:01:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/ 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/ 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 213913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-scsiback.ko 2024-05-14T11:01:49.466 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 192617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/adfs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 78153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/adfs/adfs.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 197849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/afs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1261113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/afs/kafs.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/autofs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 127377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/autofs/autofs4.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/ 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 80521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-14T11:01:49.467 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/bfs/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/bfs/bfs.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 4489593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 376857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1586065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 130073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/configfs/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 139081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/configfs/configfs.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/ 2024-05-14T11:01:49.468 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 813225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/efs/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/efs/efs.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1906817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/freevxfs/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/freevxfs/freevxfs.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/ 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1486521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-14T11:01:49.469 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 183305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 282097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hpfs/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 249009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hpfs/hpfs.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 93033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 533073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 361369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/ 2024-05-14T11:01:49.470 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 112609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/ 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 591585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/ 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/ 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 102561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/ 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 74353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/ 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 147689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1320249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 82105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv2.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 151641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 2518505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-14T11:01:49.471 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/ 2024-05-14T11:01:49.472 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-14T11:01:49.472 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-14T11:01:49.472 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/ 2024-05-14T11:01:49.472 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 2024049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-14T11:01:49.516 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/ 2024-05-14T11:01:49.516 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 734025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/ 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ascii.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-14T11:01:49.517 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-14T11:01:49.518 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 95625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 144209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 167137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 119513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-14T11:01:49.519 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 302073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 548169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-14T11:01:49.520 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 3245617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 22113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/omfs/ 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/omfs/omfs.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/ 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 462873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 67977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx4/ 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx4/qnx4.ko 2024-05-14T11:01:49.521 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx6/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 56097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/qnx6/qnx6.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_tree.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_v1.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/quota/quota_v2.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 684841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 3319041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/ 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-14T11:01:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 164185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 115977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 310017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 198233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 5192705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/locking/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 97625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/locking/locktorture.ko 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/time/ 2024-05-14T11:01:49.523 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/time/test_udelay.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 72665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc7.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/ 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/interval_tree_test.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/ 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-14T11:01:49.524 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/ 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7281 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/memory-notifier-error-inject.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/notifier-error-inject.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/percpu_test.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/pm-notifier-error-inject.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/ 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 152025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/rbtree_test.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/ 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test-kstrtox.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 4912441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_bpf.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 61169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_firmware.ko 2024-05-14T11:01:49.525 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7121 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_module.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/test_user_copy.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/xz/ 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/xz/xz_dec_test.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/ 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/ 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-14T11:01:49.526 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/ 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13537 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/ 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 106793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/ 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 236793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-14T11:01:49.527 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/ 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 879697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/ 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 84761 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1030873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-14T11:01:49.528 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9569 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70529 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-14T11:01:49.529 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/ 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1243505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/ 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 153801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/ 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 244889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 71033 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 74993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/ 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 507705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-14T11:01:49.530 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/hsr/ 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 128873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/hsr/hsr.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 357425 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 59393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-14T11:01:49.531 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 75489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 77721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50169 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-14T11:01:49.532 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-14T11:01:49.533 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11665 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11737 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-14T11:01:49.534 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 54441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 47433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 103177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_gre.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 118881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 71089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/mip6.ko 2024-05-14T11:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-14T11:01:49.536 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 29817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 94089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 30297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-14T11:01:49.537 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/ 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 106369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/ 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 153217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 66241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/lapb/ 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 63345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/lapb/lapb.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/ 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-14T11:01:49.538 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 332889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc2.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/ 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 388873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/ 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 147745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 124041 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 124345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 126681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 126473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-14T11:01:49.539 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 137833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 72353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 133681 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 139073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 138265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 136177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 139897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 534113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20953 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-14T11:01:49.540 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24577 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12649 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15977 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12809 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 55217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 496817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-14T11:01:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 134377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24513 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 140801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 79873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 117201 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 46105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 118369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-14T11:01:49.542 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14185 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13345 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 807985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53049 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 65145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 50657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11081 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-14T11:01:49.543 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17921 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16473 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13945 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_osf.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15057 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 24993 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_synproxy.ko 2024-05-14T11:01:49.544 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17641 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tproxy.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 108393 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/x_tables.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 21865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15745 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45217 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15937 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13017 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-14T11:01:49.545 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10857 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15361 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23089 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16689 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18985 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20697 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15289 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16633 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-14T11:01:49.546 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11961 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14209 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10257 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12657 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 52817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-14T11:01:49.547 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19193 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9137 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11521 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 16161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11561 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13553 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 11881 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-14T11:01:49.548 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9273 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36489 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 19777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10753 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10849 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18449 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpudp.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12441 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/ 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 17233 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/ 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 14073 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-14T11:01:49.549 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/ 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 488825 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12009 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 10873 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/ 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15929 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/ 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 142025 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/phonet.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 68817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/phonet/pn_pep.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/ 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 318617 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 202305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 85105 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rxrpc/ 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1027897 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rxrpc/rxrpc.ko 2024-05-14T11:01:49.550 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/ 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39001 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 28721 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 27593 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 53465 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38705 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23969 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43417 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 45673 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32129 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 41457 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23225 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_cgroup.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 37305 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39601 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 49321 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-14T11:01:49.551 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 79145 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8713 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 15385 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 48785 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9241 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13369 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 8377 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 25097 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43481 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 43153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 39177 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq_codel.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 42889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 62817 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-14T11:01:49.552 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26585 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 81833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33545 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 36625 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12297 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33065 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 44793 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 32833 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 13505 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 31801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 60249 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 40889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 33433 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 38329 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 35777 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-14T11:01:49.553 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 34161 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/ 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1187313 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23913 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/ 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/ 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 432409 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70113 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-14T11:01:49.554 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1940889 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/ 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 890729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/ 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 1122337 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12153 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/ 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 18801 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/ 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 105609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-14T11:01:49.558 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 90265 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 123769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 12905 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 20401 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/x25/ 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 165841 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/x25/x25.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/ 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 26729 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_algo.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23353 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 131865 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_user.ko 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 644769 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 628542 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 7706 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 23609 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 9941 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-14T11:01:49.559 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 70990 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 146959 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 232835 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 210 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 72602 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 879 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 258450 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 313019 2024-05-10 02:40 ./lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/ 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/ 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/doc/ 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:drwxr-xr-x root/root 0 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/ 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 173 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/changelog.Debian.gz 2024-05-14T11:01:49.560 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r-- root/root 691 2024-05-10 02:40 ./usr/share/doc/linux-image-6.9.0-rc3-gfb9a6a44107e/copyright 2024-05-14T11:01:49.565 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.565 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.565 DEBUG:teuthology.orchestra.run.smithi149:> egrep '(submenu|menuentry.*6.9.0-rc3-gfb9a6a44107e).*{' /boot/grub/grub.cfg 2024-05-14T11:01:49.608 INFO:teuthology.orchestra.run.smithi149.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:49.608 INFO:teuthology.orchestra.run.smithi149.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-gfb9a6a44107e-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:49.608 INFO:teuthology.orchestra.run.smithi149.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-gfb9a6a44107e-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-14T11:01:49.609 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-14T11:01:49.609 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:49.609 DEBUG:teuthology.orchestra.run.smithi149:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-14T11:01:49.652 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi149.front.sepia.ceph.com to complete... 2024-05-14T11:01:50.108 INFO:teuthology.orchestra.run.smithi149.stderr:Sourcing file `/etc/default/grub' 2024-05-14T11:01:50.109 INFO:teuthology.orchestra.run.smithi149.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-14T11:01:50.128 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-05-14T11:01:50.307 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:50.313 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:01:50.597 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-14T11:01:50.602 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-14T11:01:50.645 INFO:teuthology.orchestra.run.smithi149.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-14T11:01:50.650 INFO:teuthology.orchestra.run.smithi149.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-14T11:01:50.784 INFO:teuthology.orchestra.run.smithi149.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-14T11:01:50.784 INFO:teuthology.orchestra.run.smithi149.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-14T11:01:50.785 INFO:teuthology.orchestra.run.smithi149.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-14T11:01:50.794 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-05-14T11:02:13.954 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-14T11:02:13.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:02:13.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:02:21.889 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-14T11:02:21.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:02:21.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:02:48.610 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-05-14T11:02:56.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-14T11:02:57.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:02:57.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:03:05.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:03:05.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:03:15.999 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-05-14T11:03:28.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:03:28.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:03:28.809 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-05-14T11:03:29.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:03:29.643 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-14T11:03:29.643 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-05-14T11:03:29.693 INFO:teuthology.orchestra.run.smithi116.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:03:29.693 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:03:29.693 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T11:03:29.694 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-14T11:03:30.694 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-14T11:03:30.694 DEBUG:teuthology.orchestra.run.smithi116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-14T11:03:30.778 INFO:teuthology.orchestra.run.smithi116.stdout:ttyS1 2024-05-14T11:03:30.833 DEBUG:teuthology.parallel:result is None 2024-05-14T11:04:05.554 DEBUG:teuthology.orchestra.remote:timed out 2024-05-14T11:04:17.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:04:17.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:04:18.030 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-14T11:04:18.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:04:18.709 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-14T11:04:18.709 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-14T11:04:18.760 INFO:teuthology.orchestra.run.smithi149.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:04:18.760 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-14T11:04:18.760 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T11:04:18.760 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-14T11:04:19.761 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-14T11:04:19.761 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-14T11:04:19.835 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-05-14T11:04:19.887 DEBUG:teuthology.parallel:result is None 2024-05-14T11:04:19.888 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T11:04:19.895 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T11:04:19.895 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T11:04:19.899 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T11:04:19.907 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T11:04:19.932 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T11:04:19.977 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T11:04:19.977 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T11:04:19.980 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T11:04:20.021 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T11:04:20.028 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T11:04:20.028 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T11:04:20.036 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T11:04:20.062 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T11:04:20.076 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T11:04:20.097 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T11:04:20.111 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T11:04:20.113 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T11:04:20.120 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T11:04:20.121 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T11:04:20.124 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T11:04:20.177 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T11:04:20.186 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T11:04:20.186 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T11:04:20.189 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T11:04:20.228 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T11:04:20.246 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T11:04:20.298 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:04:20.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T11:04:20.378 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T11:04:20.387 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T11:04:20.436 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:04:20.436 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T11:04:20.505 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-05-14T11:04:20.508 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-05-14T11:04:20.658 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T11:04:20.666 INFO:teuthology.task.internal:Starting timer... 2024-05-14T11:04:20.666 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T11:04:20.711 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T11:04:20.751 DEBUG:teuthology.task.selinux:Excluding smithi116: OS 'ubuntu' does not support SELinux 2024-05-14T11:04:20.751 DEBUG:teuthology.task.selinux:Excluding smithi149: OS 'ubuntu' does not support SELinux 2024-05-14T11:04:20.752 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T11:04:20.752 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-14T11:04:20.752 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T11:04:20.752 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T11:04:20.806 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T11:04:20.829 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T11:04:20.831 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi116.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T11:11:54.861 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-05-14T11:11:54.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:11:54.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:11:54.948 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-05-14T11:11:55.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-05-14T11:11:55.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:11:55.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T11:11:55.149 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-14T11:11:55.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-14T11:11:55.293 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T11:11:55.304 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T11:11:55.304 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T11:11:55.304 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T11:11:55.307 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T11:11:55.308 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T11:11:55.342 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Command line: ntpd -gq 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: ---------------------------------------------------- 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: ntp-4 is maintained by Network Time Foundation, 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: corporation. Support and training for ntp-4 are 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: available at https://www.nwtime.org/support 2024-05-14T11:11:55.343 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: ---------------------------------------------------- 2024-05-14T11:11:55.344 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: proto: precision = 0.042 usec (-24) 2024-05-14T11:11:55.344 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: basedate set to 2022-02-04 2024-05-14T11:11:55.344 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: gps base set to 2022-02-06 (week 2196) 2024-05-14T11:11:55.344 INFO:teuthology.orchestra.run.smithi116.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-14T11:11:55.344 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-14T11:11:55.345 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: restrict ::: KOD does nothing without LIMITED. 2024-05-14T11:11:55.345 INFO:teuthology.orchestra.run.smithi116.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen and drop on 0 v6wildcard [::]:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen normally on 2 lo 127.0.0.1:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen normally on 3 enp3s0f1 172.21.15.116:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen normally on 4 lo [::1]:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:3f67%5]:123 2024-05-14T11:11:55.346 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:11:55 ntpd[17256]: Listening on routing socket on fd #22 for interface updates 2024-05-14T11:11:55.377 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-14T11:11:55.377 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Command line: ntpd -gq 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: ---------------------------------------------------- 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: ntp-4 is maintained by Network Time Foundation, 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: corporation. Support and training for ntp-4 are 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: available at https://www.nwtime.org/support 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: ---------------------------------------------------- 2024-05-14T11:11:55.378 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: proto: precision = 0.080 usec (-23) 2024-05-14T11:11:55.379 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: basedate set to 2022-02-04 2024-05-14T11:11:55.379 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: gps base set to 2022-02-06 (week 2196) 2024-05-14T11:11:55.379 INFO:teuthology.orchestra.run.smithi149.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-14T11:11:55.379 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-14T11:11:55.380 INFO:teuthology.orchestra.run.smithi149.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: restrict ::: KOD does nothing without LIMITED. 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen and drop on 0 v6wildcard [::]:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen normally on 2 lo 127.0.0.1:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen normally on 3 enp3s0f1 172.21.15.149:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen normally on 4 lo [::1]:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7d57%5]:123 2024-05-14T11:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:11:55 ntpd[17262]: Listening on routing socket on fd #22 for interface updates 2024-05-14T11:11:56.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:56 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:56.377 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:56.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:56.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:56.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:56 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:57.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:57.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:57.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:57.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:57.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:57.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:57 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:57.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:57.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:57.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:57.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:57.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:57.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:57 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:58 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:58.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:58.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:58 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:11:59 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:11:59.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:59.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:11:59.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:11:59 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:00 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:00.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:00.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:00 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:01.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:01.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:01.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:01.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:01.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:01.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:01 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:01.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:01.378 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:01.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:01.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:01.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:01.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:01 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:02.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.343 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stdout:14 May 11:12:02 ntpd[17256]: ntpd: time slew +0.000137 s 2024-05-14T11:12:02.344 INFO:teuthology.orchestra.run.smithi116.stdout:ntpd: time slew +0.000137s 2024-05-14T11:12:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-14T11:12:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:14 May 11:12:02 ntpd[17256]: can't open /var/log/ntpstats/loopstats.20240514: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-14T11:12:02.379 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: can't open /var/log/ntpstats/rawstats.20240514: Permission denied 2024-05-14T11:12:02.380 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-14T11:12:02.380 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: can't open /var/log/ntpstats/peerstats.20240514: Permission denied 2024-05-14T11:12:02.380 INFO:teuthology.orchestra.run.smithi149.stdout:14 May 11:12:02 ntpd[17262]: ntpd: time slew +0.000243 s 2024-05-14T11:12:02.380 INFO:teuthology.orchestra.run.smithi149.stdout:ntpd: time slew +0.000243s 2024-05-14T11:12:02.381 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-14T11:12:02.381 INFO:teuthology.orchestra.run.smithi149.stderr:14 May 11:12:02 ntpd[17262]: can't open /var/log/ntpstats/loopstats.20240514: Permission denied 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout: remote refid st t when poll reach delay offset jitter 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================== 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.462 INFO:teuthology.orchestra.run.smithi116.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: remote refid st t when poll reach delay offset jitter 2024-05-14T11:12:02.474 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================== 2024-05-14T11:12:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-14T11:12:02.476 INFO:teuthology.run_tasks:Running task install... 2024-05-14T11:12:02.484 DEBUG:teuthology.task.install:project ceph 2024-05-14T11:12:02.484 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}} 2024-05-14T11:12:02.484 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'} 2024-05-14T11:12:02.484 INFO:teuthology.task.install:Using flavor: default 2024-05-14T11:12:02.491 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T11:12:02.491 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T11:12:02.491 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-key list | grep Ceph 2024-05-14T11:12:02.492 DEBUG:teuthology.orchestra.run.smithi149:> sudo apt-key list | grep Ceph 2024-05-14T11:12:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-14T11:12:02.621 INFO:teuthology.orchestra.run.smithi149.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-14T11:12:02.658 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-14T11:12:02.658 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph.com (release key) 2024-05-14T11:12:02.659 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-14T11:12:02.659 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-14T11:12:02.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T11:12:02.672 INFO:teuthology.orchestra.run.smithi149.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-14T11:12:02.672 INFO:teuthology.orchestra.run.smithi149.stdout:uid [ unknown] Ceph.com (release key) 2024-05-14T11:12:02.673 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-14T11:12:02.673 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-14T11:12:02.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T11:12:02.807 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default/ 2024-05-14T11:12:02.808 INFO:teuthology.task.install.deb:Package version is 19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:12:02.906 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default/ 2024-05-14T11:12:02.906 INFO:teuthology.task.install.deb:Package version is 19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:12:03.132 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:12:03.132 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-14T11:12:03.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get update 2024-05-14T11:12:03.329 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:12:03.329 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-14T11:12:03.349 DEBUG:teuthology.orchestra.run.smithi149:> sudo apt-get update 2024-05-14T11:12:03.453 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-14T11:12:03.554 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy InRelease 2024-05-14T11:12:03.584 INFO:teuthology.orchestra.run.smithi149.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy InRelease 2024-05-14T11:12:03.591 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-14T11:12:03.605 INFO:teuthology.orchestra.run.smithi149.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-14T11:12:03.627 INFO:teuthology.orchestra.run.smithi149.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-14T11:12:03.637 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-14T11:12:03.650 INFO:teuthology.orchestra.run.smithi149.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-14T11:12:03.661 INFO:teuthology.orchestra.run.smithi149.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-14T11:12:03.700 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-05-14T11:12:03.732 INFO:teuthology.orchestra.run.smithi149.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-14T11:12:03.774 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-05-14T11:12:03.876 INFO:teuthology.orchestra.run.smithi149.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-05-14T11:12:03.907 INFO:teuthology.orchestra.run.smithi149.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-05-14T11:12:04.117 INFO:teuthology.orchestra.run.smithi149.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-14T11:12:04.261 INFO:teuthology.orchestra.run.smithi149.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [311 kB] 2024-05-14T11:12:04.340 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-14T11:12:04.368 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-14T11:12:04.390 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-14T11:12:04.613 INFO:teuthology.orchestra.run.smithi149.stdout:Fetched 569 kB in 1s (548 kB/s) 2024-05-14T11:12:04.808 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 140 kB in 1s (97.2 kB/s) 2024-05-14T11:12:05.618 INFO:teuthology.orchestra.run.smithi149.stdout:Reading package lists... 2024-05-14T11:12:05.633 DEBUG:teuthology.orchestra.run.smithi149:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3588-ge2b755a6-1jammy cephadm=19.0.0-3588-ge2b755a6-1jammy ceph-mds=19.0.0-3588-ge2b755a6-1jammy ceph-mgr=19.0.0-3588-ge2b755a6-1jammy ceph-common=19.0.0-3588-ge2b755a6-1jammy ceph-fuse=19.0.0-3588-ge2b755a6-1jammy ceph-test=19.0.0-3588-ge2b755a6-1jammy ceph-volume=19.0.0-3588-ge2b755a6-1jammy radosgw=19.0.0-3588-ge2b755a6-1jammy python3-rados=19.0.0-3588-ge2b755a6-1jammy python3-rgw=19.0.0-3588-ge2b755a6-1jammy python3-cephfs=19.0.0-3588-ge2b755a6-1jammy python3-rbd=19.0.0-3588-ge2b755a6-1jammy libcephfs2=19.0.0-3588-ge2b755a6-1jammy libcephfs-dev=19.0.0-3588-ge2b755a6-1jammy librados2=19.0.0-3588-ge2b755a6-1jammy librbd1=19.0.0-3588-ge2b755a6-1jammy rbd-fuse=19.0.0-3588-ge2b755a6-1jammy python3-cephfs=19.0.0-3588-ge2b755a6-1jammy cephfs-shell=19.0.0-3588-ge2b755a6-1jammy cephfs-top=19.0.0-3588-ge2b755a6-1jammy cephfs-mirror=19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:12:05.707 INFO:teuthology.orchestra.run.smithi149.stdout:Reading package lists... 2024-05-14T11:12:05.796 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-05-14T11:12:05.811 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3588-ge2b755a6-1jammy cephadm=19.0.0-3588-ge2b755a6-1jammy ceph-mds=19.0.0-3588-ge2b755a6-1jammy ceph-mgr=19.0.0-3588-ge2b755a6-1jammy ceph-common=19.0.0-3588-ge2b755a6-1jammy ceph-fuse=19.0.0-3588-ge2b755a6-1jammy ceph-test=19.0.0-3588-ge2b755a6-1jammy ceph-volume=19.0.0-3588-ge2b755a6-1jammy radosgw=19.0.0-3588-ge2b755a6-1jammy python3-rados=19.0.0-3588-ge2b755a6-1jammy python3-rgw=19.0.0-3588-ge2b755a6-1jammy python3-cephfs=19.0.0-3588-ge2b755a6-1jammy python3-rbd=19.0.0-3588-ge2b755a6-1jammy libcephfs2=19.0.0-3588-ge2b755a6-1jammy libcephfs-dev=19.0.0-3588-ge2b755a6-1jammy librados2=19.0.0-3588-ge2b755a6-1jammy librbd1=19.0.0-3588-ge2b755a6-1jammy rbd-fuse=19.0.0-3588-ge2b755a6-1jammy python3-cephfs=19.0.0-3588-ge2b755a6-1jammy cephfs-shell=19.0.0-3588-ge2b755a6-1jammy cephfs-top=19.0.0-3588-ge2b755a6-1jammy cephfs-mirror=19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:12:05.884 INFO:teuthology.orchestra.run.smithi149.stdout:Building dependency tree... 2024-05-14T11:12:05.885 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-05-14T11:12:05.885 INFO:teuthology.orchestra.run.smithi149.stdout:Reading state information... 2024-05-14T11:12:06.061 INFO:teuthology.orchestra.run.smithi149.stdout:The following packages were automatically installed and are no longer required: 2024-05-14T11:12:06.061 INFO:teuthology.orchestra.run.smithi149.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-14T11:12:06.061 INFO:teuthology.orchestra.run.smithi149.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-14T11:12:06.061 INFO:teuthology.orchestra.run.smithi149.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-14T11:12:06.062 INFO:teuthology.orchestra.run.smithi149.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-14T11:12:06.062 INFO:teuthology.orchestra.run.smithi149.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-14T11:12:06.062 INFO:teuthology.orchestra.run.smithi149.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-14T11:12:06.062 INFO:teuthology.orchestra.run.smithi149.stdout:The following additional packages will be installed: 2024-05-14T11:12:06.063 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-14T11:12:06.064 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-14T11:12:06.064 INFO:teuthology.orchestra.run.smithi149.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-14T11:12:06.064 INFO:teuthology.orchestra.run.smithi149.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-14T11:12:06.064 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-14T11:12:06.065 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout:Suggested packages: 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-14T11:12:06.066 INFO:teuthology.orchestra.run.smithi149.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout:Recommended packages: 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi149.stdout: btrfs-tools 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-05-14T11:12:06.067 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-05-14T11:12:06.215 INFO:teuthology.orchestra.run.smithi149.stdout:The following NEW packages will be installed: 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-14T11:12:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-14T11:12:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-14T11:12:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-14T11:12:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-14T11:12:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-14T11:12:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-14T11:12:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-14T11:12:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: socat unzip xmlstarlet zip 2024-05-14T11:12:06.251 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-05-14T11:12:06.251 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-14T11:12:06.251 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-14T11:12:06.251 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-14T11:12:06.252 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-14T11:12:06.252 INFO:teuthology.orchestra.run.smithi116.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-14T11:12:06.252 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-14T11:12:06.252 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2024-05-14T11:12:06.253 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-14T11:12:06.253 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-14T11:12:06.254 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-14T11:12:06.255 INFO:teuthology.orchestra.run.smithi116.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout:Recommended packages: 2024-05-14T11:12:06.256 INFO:teuthology.orchestra.run.smithi116.stdout: btrfs-tools 2024-05-14T11:12:06.380 INFO:teuthology.orchestra.run.smithi149.stdout:0 upgraded, 124 newly installed, 0 to remove and 310 not upgraded. 2024-05-14T11:12:06.380 INFO:teuthology.orchestra.run.smithi149.stdout:Need to get 175 MB of archives. 2024-05-14T11:12:06.380 INFO:teuthology.orchestra.run.smithi149.stdout:After this operation, 888 MB of additional disk space will be used. 2024-05-14T11:12:06.381 INFO:teuthology.orchestra.run.smithi149.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-14T11:12:06.402 INFO:teuthology.orchestra.run.smithi149.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-14T11:12:06.405 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2024-05-14T11:12:06.405 INFO:teuthology.orchestra.run.smithi116.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-14T11:12:06.405 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-14T11:12:06.405 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-14T11:12:06.405 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-14T11:12:06.407 INFO:teuthology.orchestra.run.smithi116.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-14T11:12:06.407 INFO:teuthology.orchestra.run.smithi116.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-14T11:12:06.407 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-14T11:12:06.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-14T11:12:06.409 INFO:teuthology.orchestra.run.smithi116.stdout: socat unzip xmlstarlet zip 2024-05-14T11:12:06.413 INFO:teuthology.orchestra.run.smithi149.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-14T11:12:06.465 INFO:teuthology.orchestra.run.smithi149.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-14T11:12:06.492 INFO:teuthology.orchestra.run.smithi149.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-14T11:12:06.495 INFO:teuthology.orchestra.run.smithi149.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-14T11:12:06.510 INFO:teuthology.orchestra.run.smithi149.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-14T11:12:06.563 INFO:teuthology.orchestra.run.smithi149.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-14T11:12:06.565 INFO:teuthology.orchestra.run.smithi149.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-14T11:12:06.575 INFO:teuthology.orchestra.run.smithi149.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-14T11:12:06.578 INFO:teuthology.orchestra.run.smithi149.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-14T11:12:06.579 INFO:teuthology.orchestra.run.smithi149.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-14T11:12:06.586 INFO:teuthology.orchestra.run.smithi149.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-14T11:12:06.589 INFO:teuthology.orchestra.run.smithi149.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-14T11:12:06.591 INFO:teuthology.orchestra.run.smithi149.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-14T11:12:06.593 INFO:teuthology.orchestra.run.smithi149.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-14T11:12:06.593 INFO:teuthology.orchestra.run.smithi149.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-14T11:12:06.603 INFO:teuthology.orchestra.run.smithi116.stdout:0 upgraded, 124 newly installed, 0 to remove and 310 not upgraded. 2024-05-14T11:12:06.603 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 175 MB of archives. 2024-05-14T11:12:06.603 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 888 MB of additional disk space will be used. 2024-05-14T11:12:06.603 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-14T11:12:06.614 INFO:teuthology.orchestra.run.smithi149.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-14T11:12:06.616 INFO:teuthology.orchestra.run.smithi149.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-14T11:12:06.618 INFO:teuthology.orchestra.run.smithi149.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-14T11:12:06.618 INFO:teuthology.orchestra.run.smithi149.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-14T11:12:06.619 INFO:teuthology.orchestra.run.smithi149.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-14T11:12:06.620 INFO:teuthology.orchestra.run.smithi149.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-14T11:12:06.620 INFO:teuthology.orchestra.run.smithi149.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-14T11:12:06.621 INFO:teuthology.orchestra.run.smithi149.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-14T11:12:06.621 INFO:teuthology.orchestra.run.smithi149.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-14T11:12:06.635 INFO:teuthology.orchestra.run.smithi149.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-14T11:12:06.643 INFO:teuthology.orchestra.run.smithi149.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-14T11:12:06.643 INFO:teuthology.orchestra.run.smithi149.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-14T11:12:06.645 INFO:teuthology.orchestra.run.smithi149.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-14T11:12:06.646 INFO:teuthology.orchestra.run.smithi149.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-14T11:12:06.646 INFO:teuthology.orchestra.run.smithi149.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-14T11:12:06.646 INFO:teuthology.orchestra.run.smithi149.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-14T11:12:06.648 INFO:teuthology.orchestra.run.smithi149.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-14T11:12:06.648 INFO:teuthology.orchestra.run.smithi149.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-14T11:12:06.656 INFO:teuthology.orchestra.run.smithi149.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-14T11:12:06.662 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3588-ge2b755a6-1jammy [3,525 kB] 2024-05-14T11:12:06.669 INFO:teuthology.orchestra.run.smithi149.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-14T11:12:06.670 INFO:teuthology.orchestra.run.smithi149.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-14T11:12:06.670 INFO:teuthology.orchestra.run.smithi149.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-14T11:12:06.676 INFO:teuthology.orchestra.run.smithi149.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-14T11:12:06.676 INFO:teuthology.orchestra.run.smithi149.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-14T11:12:06.677 INFO:teuthology.orchestra.run.smithi149.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-14T11:12:06.677 INFO:teuthology.orchestra.run.smithi149.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-14T11:12:06.678 INFO:teuthology.orchestra.run.smithi149.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-14T11:12:06.680 INFO:teuthology.orchestra.run.smithi149.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-14T11:12:06.685 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-14T11:12:06.701 INFO:teuthology.orchestra.run.smithi149.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-14T11:12:06.702 INFO:teuthology.orchestra.run.smithi149.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-14T11:12:06.703 INFO:teuthology.orchestra.run.smithi149.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-14T11:12:06.705 INFO:teuthology.orchestra.run.smithi149.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-14T11:12:06.706 INFO:teuthology.orchestra.run.smithi149.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-14T11:12:06.706 INFO:teuthology.orchestra.run.smithi149.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-14T11:12:06.708 INFO:teuthology.orchestra.run.smithi149.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-14T11:12:06.731 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-14T11:12:06.771 INFO:teuthology.orchestra.run.smithi149.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-14T11:12:06.772 INFO:teuthology.orchestra.run.smithi149.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-14T11:12:06.773 INFO:teuthology.orchestra.run.smithi149.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-14T11:12:06.775 INFO:teuthology.orchestra.run.smithi149.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-14T11:12:06.776 INFO:teuthology.orchestra.run.smithi149.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-14T11:12:06.835 INFO:teuthology.orchestra.run.smithi149.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-14T11:12:06.837 INFO:teuthology.orchestra.run.smithi149.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-14T11:12:06.838 INFO:teuthology.orchestra.run.smithi149.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-14T11:12:06.854 INFO:teuthology.orchestra.run.smithi149.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-14T11:12:06.854 INFO:teuthology.orchestra.run.smithi149.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-14T11:12:06.855 INFO:teuthology.orchestra.run.smithi149.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-14T11:12:06.855 INFO:teuthology.orchestra.run.smithi149.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-14T11:12:06.856 INFO:teuthology.orchestra.run.smithi149.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-14T11:12:06.857 INFO:teuthology.orchestra.run.smithi149.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-14T11:12:06.858 INFO:teuthology.orchestra.run.smithi149.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-14T11:12:06.878 INFO:teuthology.orchestra.run.smithi149.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-14T11:12:06.882 INFO:teuthology.orchestra.run.smithi149.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-14T11:12:06.884 INFO:teuthology.orchestra.run.smithi149.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-14T11:12:06.885 INFO:teuthology.orchestra.run.smithi149.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-14T11:12:06.886 INFO:teuthology.orchestra.run.smithi149.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-14T11:12:06.890 INFO:teuthology.orchestra.run.smithi149.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-14T11:12:06.893 INFO:teuthology.orchestra.run.smithi149.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-14T11:12:06.893 INFO:teuthology.orchestra.run.smithi149.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-14T11:12:06.894 INFO:teuthology.orchestra.run.smithi149.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-14T11:12:06.903 INFO:teuthology.orchestra.run.smithi149.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-14T11:12:06.916 INFO:teuthology.orchestra.run.smithi149.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-14T11:12:06.917 INFO:teuthology.orchestra.run.smithi149.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-14T11:12:06.920 INFO:teuthology.orchestra.run.smithi149.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-14T11:12:06.920 INFO:teuthology.orchestra.run.smithi149.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-14T11:12:06.921 INFO:teuthology.orchestra.run.smithi149.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-14T11:12:06.921 INFO:teuthology.orchestra.run.smithi149.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-14T11:12:06.923 INFO:teuthology.orchestra.run.smithi149.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-14T11:12:06.924 INFO:teuthology.orchestra.run.smithi149.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-14T11:12:06.932 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-14T11:12:06.933 INFO:teuthology.orchestra.run.smithi149.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-14T11:12:06.953 INFO:teuthology.orchestra.run.smithi149.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-14T11:12:06.954 INFO:teuthology.orchestra.run.smithi149.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-14T11:12:06.956 INFO:teuthology.orchestra.run.smithi149.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-14T11:12:06.956 INFO:teuthology.orchestra.run.smithi149.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-14T11:12:07.026 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-14T11:12:07.039 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-14T11:12:07.091 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-14T11:12:07.265 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-14T11:12:07.274 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-14T11:12:07.301 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-14T11:12:07.310 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-14T11:12:07.450 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3588-ge2b755a6-1jammy [3,288 kB] 2024-05-14T11:12:07.474 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-14T11:12:07.755 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3588-ge2b755a6-1jammy [727 kB] 2024-05-14T11:12:07.826 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3588-ge2b755a6-1jammy [357 kB] 2024-05-14T11:12:07.859 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3588-ge2b755a6-1jammy [32.8 kB] 2024-05-14T11:12:07.861 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3588-ge2b755a6-1jammy [180 kB] 2024-05-14T11:12:07.869 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3588-ge2b755a6-1jammy [67.4 kB] 2024-05-14T11:12:07.870 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3588-ge2b755a6-1jammy [334 kB] 2024-05-14T11:12:07.905 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3588-ge2b755a6-1jammy [7,187 kB] 2024-05-14T11:12:08.054 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-14T11:12:08.070 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-14T11:12:08.085 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-14T11:12:08.094 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-14T11:12:08.095 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-14T11:12:08.111 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-14T11:12:08.148 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-14T11:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-14T11:12:08.165 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-14T11:12:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-14T11:12:08.336 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3588-ge2b755a6-1jammy [112 kB] 2024-05-14T11:12:08.336 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-14T11:12:08.337 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3588-ge2b755a6-1jammy [471 kB] 2024-05-14T11:12:08.351 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3588-ge2b755a6-1jammy [25.6 MB] 2024-05-14T11:12:08.418 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-14T11:12:08.425 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-14T11:12:08.499 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-14T11:12:08.528 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-14T11:12:08.539 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-14T11:12:08.580 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-14T11:12:08.758 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-14T11:12:08.772 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-14T11:12:08.779 INFO:teuthology.orchestra.run.smithi116.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-14T11:12:08.784 INFO:teuthology.orchestra.run.smithi116.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-14T11:12:08.969 INFO:teuthology.orchestra.run.smithi116.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-14T11:12:09.052 INFO:teuthology.orchestra.run.smithi116.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-14T11:12:09.064 INFO:teuthology.orchestra.run.smithi149.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3588-ge2b755a6-1jammy [3,525 kB] 2024-05-14T11:12:09.069 INFO:teuthology.orchestra.run.smithi116.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-14T11:12:09.255 INFO:teuthology.orchestra.run.smithi116.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-14T11:12:09.302 INFO:teuthology.orchestra.run.smithi116.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-14T11:12:09.316 INFO:teuthology.orchestra.run.smithi116.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-14T11:12:09.476 INFO:teuthology.orchestra.run.smithi116.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-14T11:12:09.477 INFO:teuthology.orchestra.run.smithi116.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-14T11:12:09.481 INFO:teuthology.orchestra.run.smithi116.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-14T11:12:09.490 INFO:teuthology.orchestra.run.smithi116.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-14T11:12:09.500 INFO:teuthology.orchestra.run.smithi116.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-14T11:12:09.685 INFO:teuthology.orchestra.run.smithi116.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-14T11:12:09.943 INFO:teuthology.orchestra.run.smithi116.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3588-ge2b755a6-1jammy [4,918 kB] 2024-05-14T11:12:09.947 INFO:teuthology.orchestra.run.smithi116.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-14T11:12:10.082 INFO:teuthology.orchestra.run.smithi116.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-14T11:12:10.125 INFO:teuthology.orchestra.run.smithi116.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-14T11:12:10.214 INFO:teuthology.orchestra.run.smithi116.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-14T11:12:10.223 INFO:teuthology.orchestra.run.smithi116.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-14T11:12:10.236 INFO:teuthology.orchestra.run.smithi116.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-14T11:12:10.278 INFO:teuthology.orchestra.run.smithi116.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-14T11:12:10.301 INFO:teuthology.orchestra.run.smithi116.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3588-ge2b755a6-1jammy [244 kB] 2024-05-14T11:12:10.313 INFO:teuthology.orchestra.run.smithi116.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3588-ge2b755a6-1jammy [125 kB] 2024-05-14T11:12:10.321 INFO:teuthology.orchestra.run.smithi116.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3588-ge2b755a6-1jammy [1,542 kB] 2024-05-14T11:12:10.432 INFO:teuthology.orchestra.run.smithi116.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3588-ge2b755a6-1jammy [6,149 kB] 2024-05-14T11:12:10.541 INFO:teuthology.orchestra.run.smithi116.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-14T11:12:10.542 INFO:teuthology.orchestra.run.smithi116.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-14T11:12:10.543 INFO:teuthology.orchestra.run.smithi116.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-14T11:12:10.708 INFO:teuthology.orchestra.run.smithi116.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-14T11:12:10.870 INFO:teuthology.orchestra.run.smithi116.stdout:Get:74 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3588-ge2b755a6-1jammy [22.5 MB] 2024-05-14T11:12:11.054 INFO:teuthology.orchestra.run.smithi116.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-14T11:12:11.457 INFO:teuthology.orchestra.run.smithi116.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-14T11:12:11.466 INFO:teuthology.orchestra.run.smithi116.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-14T11:12:11.468 INFO:teuthology.orchestra.run.smithi116.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-14T11:12:11.536 INFO:teuthology.orchestra.run.smithi116.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-14T11:12:11.537 INFO:teuthology.orchestra.run.smithi116.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-14T11:12:11.537 INFO:teuthology.orchestra.run.smithi116.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-14T11:12:11.539 INFO:teuthology.orchestra.run.smithi116.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-14T11:12:11.539 INFO:teuthology.orchestra.run.smithi116.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-14T11:12:11.540 INFO:teuthology.orchestra.run.smithi116.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-14T11:12:11.630 INFO:teuthology.orchestra.run.smithi149.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3588-ge2b755a6-1jammy [3,288 kB] 2024-05-14T11:12:11.706 INFO:teuthology.orchestra.run.smithi116.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-14T11:12:11.922 INFO:teuthology.orchestra.run.smithi116.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-14T11:12:12.251 INFO:teuthology.orchestra.run.smithi116.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-14T11:12:12.297 INFO:teuthology.orchestra.run.smithi116.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-14T11:12:12.324 INFO:teuthology.orchestra.run.smithi116.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-14T11:12:12.333 INFO:teuthology.orchestra.run.smithi116.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-14T11:12:12.378 INFO:teuthology.orchestra.run.smithi116.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-14T11:12:12.405 INFO:teuthology.orchestra.run.smithi116.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-14T11:12:12.406 INFO:teuthology.orchestra.run.smithi116.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-14T11:12:12.414 INFO:teuthology.orchestra.run.smithi116.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-14T11:12:12.416 INFO:teuthology.orchestra.run.smithi116.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-14T11:12:12.537 INFO:teuthology.orchestra.run.smithi149.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3588-ge2b755a6-1jammy [727 kB] 2024-05-14T11:12:12.608 INFO:teuthology.orchestra.run.smithi116.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3588-ge2b755a6-1jammy [14.0 kB] 2024-05-14T11:12:12.609 INFO:teuthology.orchestra.run.smithi116.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3588-ge2b755a6-1jammy [857 kB] 2024-05-14T11:12:12.612 INFO:teuthology.orchestra.run.smithi116.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-14T11:12:12.684 INFO:teuthology.orchestra.run.smithi116.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3588-ge2b755a6-1jammy [2,370 kB] 2024-05-14T11:12:12.710 INFO:teuthology.orchestra.run.smithi149.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3588-ge2b755a6-1jammy [357 kB] 2024-05-14T11:12:12.797 INFO:teuthology.orchestra.run.smithi149.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3588-ge2b755a6-1jammy [32.8 kB] 2024-05-14T11:12:12.798 INFO:teuthology.orchestra.run.smithi149.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3588-ge2b755a6-1jammy [180 kB] 2024-05-14T11:12:12.806 INFO:teuthology.orchestra.run.smithi149.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3588-ge2b755a6-1jammy [67.4 kB] 2024-05-14T11:12:12.808 INFO:teuthology.orchestra.run.smithi149.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3588-ge2b755a6-1jammy [334 kB] 2024-05-14T11:12:12.873 INFO:teuthology.orchestra.run.smithi116.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-14T11:12:12.893 INFO:teuthology.orchestra.run.smithi149.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3588-ge2b755a6-1jammy [7,187 kB] 2024-05-14T11:12:12.910 INFO:teuthology.orchestra.run.smithi116.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3588-ge2b755a6-1jammy [792 kB] 2024-05-14T11:12:12.975 INFO:teuthology.orchestra.run.smithi116.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3588-ge2b755a6-1jammy [148 kB] 2024-05-14T11:12:12.999 INFO:teuthology.orchestra.run.smithi116.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3588-ge2b755a6-1jammy [2,279 kB] 2024-05-14T11:12:13.143 INFO:teuthology.orchestra.run.smithi116.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-14T11:12:13.145 INFO:teuthology.orchestra.run.smithi116.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-14T11:12:13.161 INFO:teuthology.orchestra.run.smithi116.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-14T11:12:13.167 INFO:teuthology.orchestra.run.smithi116.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-14T11:12:13.198 INFO:teuthology.orchestra.run.smithi116.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3588-ge2b755a6-1jammy [8,625 kB] 2024-05-14T11:12:13.206 INFO:teuthology.orchestra.run.smithi116.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-14T11:12:13.220 INFO:teuthology.orchestra.run.smithi116.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-14T11:12:13.310 INFO:teuthology.orchestra.run.smithi116.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-14T11:12:13.313 INFO:teuthology.orchestra.run.smithi116.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-14T11:12:13.314 INFO:teuthology.orchestra.run.smithi116.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-14T11:12:13.498 INFO:teuthology.orchestra.run.smithi116.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-14T11:12:13.767 INFO:teuthology.orchestra.run.smithi116.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-14T11:12:14.047 INFO:teuthology.orchestra.run.smithi116.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3588-ge2b755a6-1jammy [14.2 kB] 2024-05-14T11:12:14.048 INFO:teuthology.orchestra.run.smithi116.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3588-ge2b755a6-1jammy [45.0 MB] 2024-05-14T11:12:14.150 INFO:teuthology.orchestra.run.smithi149.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3588-ge2b755a6-1jammy [112 kB] 2024-05-14T11:12:14.152 INFO:teuthology.orchestra.run.smithi149.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3588-ge2b755a6-1jammy [471 kB] 2024-05-14T11:12:14.225 INFO:teuthology.orchestra.run.smithi149.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3588-ge2b755a6-1jammy [25.6 MB] 2024-05-14T11:12:18.599 INFO:teuthology.orchestra.run.smithi116.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3588-ge2b755a6-1jammy [124 kB] 2024-05-14T11:12:18.600 INFO:teuthology.orchestra.run.smithi116.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3588-ge2b755a6-1jammy [236 kB] 2024-05-14T11:12:18.612 INFO:teuthology.orchestra.run.smithi116.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3588-ge2b755a6-1jammy [33.9 kB] 2024-05-14T11:12:18.614 INFO:teuthology.orchestra.run.smithi116.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3588-ge2b755a6-1jammy [23.4 kB] 2024-05-14T11:12:18.614 INFO:teuthology.orchestra.run.smithi116.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3588-ge2b755a6-1jammy [38.9 kB] 2024-05-14T11:12:18.617 INFO:teuthology.orchestra.run.smithi116.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3588-ge2b755a6-1jammy [14.1 MB] 2024-05-14T11:12:19.684 INFO:teuthology.orchestra.run.smithi149.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3588-ge2b755a6-1jammy [4,918 kB] 2024-05-14T11:12:20.025 INFO:teuthology.orchestra.run.smithi116.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3588-ge2b755a6-1jammy [92.5 kB] 2024-05-14T11:12:20.577 INFO:teuthology.orchestra.run.smithi149.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3588-ge2b755a6-1jammy [244 kB] 2024-05-14T11:12:20.603 INFO:teuthology.orchestra.run.smithi149.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3588-ge2b755a6-1jammy [125 kB] 2024-05-14T11:12:20.639 INFO:teuthology.orchestra.run.smithi149.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3588-ge2b755a6-1jammy [1,542 kB] 2024-05-14T11:12:20.661 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 175 MB in 14s (12.9 MB/s) 2024-05-14T11:12:20.808 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-14T11:12:20.874 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-14T11:12:20.877 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-14T11:12:20.897 INFO:teuthology.orchestra.run.smithi149.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3588-ge2b755a6-1jammy [6,149 kB] 2024-05-14T11:12:20.915 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-14T11:12:21.217 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-14T11:12:21.231 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-14T11:12:21.276 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-14T11:12:21.603 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cmd2. 2024-05-14T11:12:21.617 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-14T11:12:21.653 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-14T11:12:21.890 INFO:teuthology.orchestra.run.smithi149.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3588-ge2b755a6-1jammy [22.5 MB] 2024-05-14T11:12:21.998 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-14T11:12:22.012 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-14T11:12:22.099 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-14T11:12:22.401 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-14T11:12:22.415 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-14T11:12:22.452 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-14T11:12:22.972 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-14T11:12:22.985 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-14T11:12:23.030 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-14T11:12:23.483 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-14T11:12:23.497 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:23.550 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:23.911 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-14T11:12:23.925 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:23.961 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:24.274 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-14T11:12:24.287 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:24.331 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:24.667 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-14T11:12:24.682 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-14T11:12:24.725 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-14T11:12:25.028 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librados2. 2024-05-14T11:12:25.042 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../010-librados2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:25.078 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librados2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:25.523 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librbd1. 2024-05-14T11:12:25.538 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../011-librbd1_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:25.581 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librbd1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:25.584 INFO:teuthology.orchestra.run.smithi149.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3588-ge2b755a6-1jammy [14.0 kB] 2024-05-14T11:12:25.584 INFO:teuthology.orchestra.run.smithi149.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3588-ge2b755a6-1jammy [857 kB] 2024-05-14T11:12:25.768 INFO:teuthology.orchestra.run.smithi149.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3588-ge2b755a6-1jammy [2,370 kB] 2024-05-14T11:12:25.992 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs2. 2024-05-14T11:12:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:26.075 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:26.314 INFO:teuthology.orchestra.run.smithi149.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3588-ge2b755a6-1jammy [792 kB] 2024-05-14T11:12:26.336 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rados. 2024-05-14T11:12:26.350 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:26.386 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rados (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:26.494 INFO:teuthology.orchestra.run.smithi149.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3588-ge2b755a6-1jammy [148 kB] 2024-05-14T11:12:26.526 INFO:teuthology.orchestra.run.smithi149.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3588-ge2b755a6-1jammy [2,279 kB] 2024-05-14T11:12:26.672 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-14T11:12:26.686 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:26.730 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-argparse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:26.983 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cephfs. 2024-05-14T11:12:26.997 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:27.041 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cephfs (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:27.221 INFO:teuthology.orchestra.run.smithi149.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3588-ge2b755a6-1jammy [8,625 kB] 2024-05-14T11:12:27.385 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-14T11:12:27.400 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:27.443 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:27.747 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-prettytable. 2024-05-14T11:12:27.761 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-14T11:12:27.814 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-14T11:12:28.249 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rbd. 2024-05-14T11:12:28.264 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:28.308 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rbd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:28.643 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-14T11:12:28.658 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-14T11:12:28.702 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-14T11:12:28.979 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-14T11:12:29.048 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-14T11:12:29.048 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-14T11:12:29.324 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-14T11:12:29.339 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-14T11:12:29.374 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-14T11:12:29.702 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua5.1. 2024-05-14T11:12:29.717 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-14T11:12:29.760 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-14T11:12:30.080 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-any. 2024-05-14T11:12:30.095 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-14T11:12:30.146 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-14T11:12:30.399 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package zip. 2024-05-14T11:12:30.414 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-14T11:12:30.457 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking zip (3.0-12build2) ... 2024-05-14T11:12:30.769 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package unzip. 2024-05-14T11:12:30.783 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-14T11:12:30.818 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-14T11:12:31.222 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package luarocks. 2024-05-14T11:12:31.237 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-14T11:12:31.280 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-14T11:12:31.525 INFO:teuthology.orchestra.run.smithi149.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3588-ge2b755a6-1jammy [14.2 kB] 2024-05-14T11:12:31.556 INFO:teuthology.orchestra.run.smithi149.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3588-ge2b755a6-1jammy [45.0 MB] 2024-05-14T11:12:31.647 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librgw2. 2024-05-14T11:12:31.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../027-librgw2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:31.696 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librgw2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:32.241 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rgw. 2024-05-14T11:12:32.256 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:32.299 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:32.639 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-14T11:12:32.650 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-14T11:12:32.693 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-14T11:12:33.021 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libradosstriper1. 2024-05-14T11:12:33.036 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:33.071 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libradosstriper1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:33.466 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-common. 2024-05-14T11:12:33.481 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:33.524 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:34.696 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-base. 2024-05-14T11:12:34.711 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:34.763 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-base (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:35.334 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-14T11:12:35.348 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-14T11:12:35.392 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-14T11:12:35.653 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cheroot. 2024-05-14T11:12:35.667 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-14T11:12:35.702 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-14T11:12:36.000 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-14T11:12:36.015 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-14T11:12:36.058 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-14T11:12:36.361 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-14T11:12:36.376 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-14T11:12:36.449 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-14T11:12:36.730 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-14T11:12:36.745 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-14T11:12:36.780 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-14T11:12:37.058 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tempora. 2024-05-14T11:12:37.073 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-14T11:12:37.116 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-14T11:12:37.419 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-portend. 2024-05-14T11:12:37.434 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-14T11:12:37.477 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-14T11:12:37.805 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-14T11:12:37.820 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-14T11:12:37.863 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-14T11:12:38.299 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-14T11:12:38.315 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-14T11:12:38.983 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-14T11:12:39.311 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-natsort. 2024-05-14T11:12:39.326 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-14T11:12:39.369 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-14T11:12:39.639 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-logutils. 2024-05-14T11:12:39.654 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-14T11:12:39.689 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-14T11:12:39.984 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-14T11:12:39.999 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-14T11:12:40.075 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-14T11:12:40.378 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-mako. 2024-05-14T11:12:40.393 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-14T11:12:40.436 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-14T11:12:40.661 INFO:teuthology.orchestra.run.smithi149.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3588-ge2b755a6-1jammy [124 kB] 2024-05-14T11:12:40.662 INFO:teuthology.orchestra.run.smithi149.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3588-ge2b755a6-1jammy [236 kB] 2024-05-14T11:12:40.668 INFO:teuthology.orchestra.run.smithi149.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3588-ge2b755a6-1jammy [33.9 kB] 2024-05-14T11:12:40.677 INFO:teuthology.orchestra.run.smithi149.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3588-ge2b755a6-1jammy [23.4 kB] 2024-05-14T11:12:40.679 INFO:teuthology.orchestra.run.smithi149.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3588-ge2b755a6-1jammy [38.9 kB] 2024-05-14T11:12:40.687 INFO:teuthology.orchestra.run.smithi149.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3588-ge2b755a6-1jammy [14.1 MB] 2024-05-14T11:12:40.764 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-14T11:12:40.779 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-14T11:12:40.822 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-14T11:12:41.134 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-14T11:12:41.149 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-14T11:12:41.217 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-14T11:12:41.495 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webob. 2024-05-14T11:12:41.510 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-14T11:12:41.553 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-14T11:12:41.889 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-waitress. 2024-05-14T11:12:41.904 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-14T11:12:41.950 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-14T11:12:41.952 INFO:teuthology.orchestra.run.smithi149.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3588-ge2b755a6-1jammy [92.5 kB] 2024-05-14T11:12:42.242 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tempita. 2024-05-14T11:12:42.257 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-14T11:12:42.292 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-14T11:12:42.553 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-paste. 2024-05-14T11:12:42.568 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-14T11:12:42.592 INFO:teuthology.orchestra.run.smithi149.stdout:Fetched 175 MB in 36s (4,895 kB/s) 2024-05-14T11:12:42.603 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-14T11:12:42.752 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-14T11:12:42.818 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-14T11:12:42.822 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-14T11:12:42.858 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-14T11:12:42.923 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-14T11:12:42.937 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-14T11:12:42.972 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-14T11:12:43.161 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-14T11:12:43.175 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-14T11:12:43.219 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-14T11:12:43.259 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-14T11:12:43.273 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-14T11:12:43.317 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-14T11:12:43.547 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-cmd2. 2024-05-14T11:12:43.561 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-14T11:12:43.606 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-14T11:12:43.645 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webtest. 2024-05-14T11:12:43.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-14T11:12:43.703 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-14T11:12:43.959 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-14T11:12:43.973 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-14T11:12:43.981 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pecan. 2024-05-14T11:12:43.996 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-14T11:12:44.031 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-14T11:12:44.059 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-14T11:12:44.369 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-certifi. 2024-05-14T11:12:44.384 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-14T11:12:44.395 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-14T11:12:44.410 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-14T11:12:44.427 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-14T11:12:44.454 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-14T11:12:44.772 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-idna. 2024-05-14T11:12:44.782 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-14T11:12:44.787 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-14T11:12:44.796 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-14T11:12:44.830 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-14T11:12:44.840 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-14T11:12:45.150 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-urllib3. 2024-05-14T11:12:45.165 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-14T11:12:45.208 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-14T11:12:45.227 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-14T11:12:45.241 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:45.284 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:45.536 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-requests. 2024-05-14T11:12:45.551 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-14T11:12:45.594 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-14T11:12:45.671 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-14T11:12:45.686 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:45.730 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:45.906 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-14T11:12:45.921 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-14T11:12:45.955 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-14T11:12:46.066 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-14T11:12:46.080 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-14T11:12:46.124 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:12:46.250 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-14T11:12:46.265 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:46.308 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:46.444 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-14T11:12:46.458 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-14T11:12:46.494 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-14T11:12:46.671 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-14T11:12:46.686 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:46.729 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:46.805 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package librados2. 2024-05-14T11:12:46.820 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../010-librados2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:46.863 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking librados2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:47.065 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-14T11:12:47.081 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-14T11:12:47.123 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-14T11:12:47.394 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-openssl. 2024-05-14T11:12:47.409 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-14T11:12:47.443 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-14T11:12:47.500 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package librbd1. 2024-05-14T11:12:47.514 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../011-librbd1_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:47.566 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking librbd1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:47.763 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr. 2024-05-14T11:12:47.778 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:47.821 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:48.060 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libcephfs2. 2024-05-14T11:12:48.074 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:48.127 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libcephfs2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:48.232 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mon. 2024-05-14T11:12:48.248 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:48.291 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mon (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:48.438 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-rados. 2024-05-14T11:12:48.452 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:48.496 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-rados (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:48.866 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-14T11:12:48.878 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-osd. 2024-05-14T11:12:48.880 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:48.892 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:48.927 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-osd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:48.932 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-ceph-argparse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:49.235 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-cephfs. 2024-05-14T11:12:49.249 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:49.301 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-cephfs (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:49.679 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-14T11:12:49.694 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:49.699 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph. 2024-05-14T11:12:49.711 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../068-ceph_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:49.746 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:49.758 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:50.074 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-fuse. 2024-05-14T11:12:50.089 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:50.133 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:50.166 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-prettytable. 2024-05-14T11:12:50.180 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-14T11:12:50.232 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-14T11:12:50.509 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-rbd. 2024-05-14T11:12:50.519 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mds. 2024-05-14T11:12:50.524 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:50.534 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:50.568 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-rbd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:50.577 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mds (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:50.914 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephadm. 2024-05-14T11:12:50.928 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../071-cephadm_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:50.971 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-14T11:12:50.971 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:50.986 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-14T11:12:51.054 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-14T11:12:51.283 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-14T11:12:51.297 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-14T11:12:51.341 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-14T11:12:51.382 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-14T11:12:51.397 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-14T11:12:51.448 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-14T11:12:51.694 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-14T11:12:51.709 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-14T11:12:51.743 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-14T11:12:51.793 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-14T11:12:51.808 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-14T11:12:51.859 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-14T11:12:52.221 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package lua5.1. 2024-05-14T11:12:52.235 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-14T11:12:52.279 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-14T11:12:52.582 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package lua-any. 2024-05-14T11:12:52.598 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-14T11:12:52.640 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-14T11:12:52.731 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-babel. 2024-05-14T11:12:52.745 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-14T11:12:52.789 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-14T11:12:52.885 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package zip. 2024-05-14T11:12:52.900 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-14T11:12:52.934 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking zip (3.0-12build2) ... 2024-05-14T11:12:53.143 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jinja2. 2024-05-14T11:12:53.156 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-14T11:12:53.201 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-14T11:12:53.212 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package unzip. 2024-05-14T11:12:53.227 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-14T11:12:53.262 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-14T11:12:53.683 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package luarocks. 2024-05-14T11:12:53.698 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-14T11:12:53.704 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-14T11:12:53.718 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:53.741 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-14T11:12:53.762 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:54.141 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package librgw2. 2024-05-14T11:12:54.141 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-14T11:12:54.155 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../027-librgw2_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:54.155 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-14T11:12:54.198 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking librgw2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:54.199 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-14T11:12:54.502 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-routes. 2024-05-14T11:12:54.517 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-14T11:12:54.560 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-14T11:12:54.751 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-rgw. 2024-05-14T11:12:54.766 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:54.801 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-rgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:54.896 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-14T11:12:54.910 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:54.955 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:55.112 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-14T11:12:55.127 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-14T11:12:55.170 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-14T11:12:55.465 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libradosstriper1. 2024-05-14T11:12:55.480 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:55.523 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libradosstriper1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:55.960 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-common. 2024-05-14T11:12:55.975 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:56.018 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:56.118 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-14T11:12:56.130 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-14T11:12:56.168 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-14T11:12:56.680 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-joblib. 2024-05-14T11:12:56.692 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-14T11:12:56.729 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-14T11:12:57.066 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-14T11:12:57.078 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-14T11:12:57.131 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-base. 2024-05-14T11:12:57.147 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:12:57.193 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-14T11:12:57.198 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-base (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:57.496 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn. 2024-05-14T11:12:57.509 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-14T11:12:57.580 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-14T11:12:57.911 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-14T11:12:57.925 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-14T11:12:57.969 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-14T11:12:58.196 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-14T11:12:58.210 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:12:58.254 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:12:58.297 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-cheroot. 2024-05-14T11:12:58.311 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-14T11:12:58.354 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-14T11:12:58.675 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-14T11:12:58.688 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-14T11:12:58.733 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-14T11:12:59.019 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-14T11:12:59.027 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-14T11:12:59.060 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-14T11:12:59.216 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cachetools. 2024-05-14T11:12:59.229 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-14T11:12:59.274 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-14T11:12:59.396 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-14T11:12:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-14T11:12:59.446 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-14T11:12:59.719 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-14T11:12:59.732 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-14T11:12:59.749 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-tempora. 2024-05-14T11:12:59.760 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-14T11:12:59.777 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-14T11:12:59.799 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-14T11:13:00.088 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-14T11:13:00.093 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-portend. 2024-05-14T11:13:00.101 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-14T11:13:00.103 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-14T11:13:00.130 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-14T11:13:00.143 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-14T11:13:00.400 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rsa. 2024-05-14T11:13:00.412 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-14T11:13:00.441 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-14T11:13:00.446 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-14T11:13:00.455 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-14T11:13:00.519 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-14T11:13:00.762 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-google-auth. 2024-05-14T11:13:00.775 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-14T11:13:00.820 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-14T11:13:00.824 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-14T11:13:00.834 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-14T11:13:00.874 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-14T11:13:01.148 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-14T11:13:01.162 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-14T11:13:01.177 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-natsort. 2024-05-14T11:13:01.187 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-14T11:13:01.206 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-14T11:13:01.227 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-14T11:13:01.526 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-websocket. 2024-05-14T11:13:01.530 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-logutils. 2024-05-14T11:13:01.540 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-14T11:13:01.540 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-14T11:13:01.581 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-14T11:13:01.584 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-14T11:13:01.887 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-14T11:13:01.901 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-14T11:13:01.908 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-14T11:13:01.918 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-14T11:13:01.938 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-14T11:13:01.958 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-14T11:13:02.261 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-mako. 2024-05-14T11:13:02.270 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-14T11:13:02.311 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-14T11:13:02.609 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-14T11:13:02.622 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:02.647 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-14T11:13:02.651 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:02.657 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-14T11:13:02.706 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-14T11:13:02.953 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-14T11:13:02.966 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-14T11:13:03.000 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-14T11:13:03.003 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-14T11:13:03.009 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-14T11:13:03.050 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-14T11:13:03.319 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-webob. 2024-05-14T11:13:03.328 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-14T11:13:03.339 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-14T11:13:03.352 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-14T11:13:03.369 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-14T11:13:03.389 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-14T11:13:03.667 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package jq. 2024-05-14T11:13:03.680 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-14T11:13:03.705 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-waitress. 2024-05-14T11:13:03.715 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-14T11:13:03.734 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-14T11:13:03.757 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-14T11:13:03.945 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package socat. 2024-05-14T11:13:03.958 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-14T11:13:04.003 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-14T11:13:04.100 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-tempita. 2024-05-14T11:13:04.112 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-14T11:13:04.150 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-14T11:13:04.289 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package xmlstarlet. 2024-05-14T11:13:04.303 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-14T11:13:04.347 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-14T11:13:04.452 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-paste. 2024-05-14T11:13:04.463 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-14T11:13:04.502 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-14T11:13:04.735 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-test. 2024-05-14T11:13:04.748 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:04.785 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-test (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:04.805 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-14T11:13:04.815 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-14T11:13:04.856 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-14T11:13:05.167 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-14T11:13:05.177 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-14T11:13:05.216 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-14T11:13:05.536 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-webtest. 2024-05-14T11:13:05.546 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-14T11:13:05.586 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-14T11:13:05.864 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pecan. 2024-05-14T11:13:05.874 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-14T11:13:05.906 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-14T11:13:06.235 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-certifi. 2024-05-14T11:13:06.245 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-14T11:13:06.285 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-14T11:13:06.638 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-idna. 2024-05-14T11:13:06.647 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-14T11:13:06.688 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-14T11:13:06.689 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-volume. 2024-05-14T11:13:06.700 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:06.731 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-volume (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:07.007 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-urllib3. 2024-05-14T11:13:07.017 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-14T11:13:07.057 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-14T11:13:07.077 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-14T11:13:07.088 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:07.118 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-mirror (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:07.377 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-requests. 2024-05-14T11:13:07.387 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-14T11:13:07.421 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-colorama. 2024-05-14T11:13:07.426 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-14T11:13:07.430 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-14T11:13:07.471 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-14T11:13:07.732 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-shell. 2024-05-14T11:13:07.741 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:07.763 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-14T11:13:07.773 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-14T11:13:07.781 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-shell (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:07.813 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-14T11:13:08.025 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-top. 2024-05-14T11:13:08.035 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:08.076 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-top (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:08.108 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-14T11:13:08.118 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:08.157 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:08.328 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-14T11:13:08.337 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:08.369 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs-dev (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:08.529 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-14T11:13:08.539 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:08.579 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:08.647 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-14T11:13:08.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-14T11:13:08.697 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-14T11:13:08.923 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-14T11:13:08.933 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-14T11:13:08.973 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-14T11:13:09.218 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-14T11:13:09.231 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-14T11:13:09.268 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-14T11:13:09.284 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-openssl. 2024-05-14T11:13:09.294 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-14T11:13:09.334 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-14T11:13:09.638 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package nvme-cli. 2024-05-14T11:13:09.649 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-14T11:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr. 2024-05-14T11:13:09.681 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:09.688 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-14T11:13:09.720 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:10.085 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package pkg-config. 2024-05-14T11:13:10.096 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-14T11:13:10.107 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mon. 2024-05-14T11:13:10.117 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:10.134 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-14T11:13:10.148 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mon (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:10.397 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-14T11:13:10.407 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-14T11:13:10.471 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-14T11:13:10.710 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-osd. 2024-05-14T11:13:10.720 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:10.760 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-osd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:10.891 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-14T11:13:10.902 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-14T11:13:10.940 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-14T11:13:11.252 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastescript. 2024-05-14T11:13:11.266 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-14T11:13:11.309 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-14T11:13:11.546 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph. 2024-05-14T11:13:11.556 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../068-ceph_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:11.596 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:11.655 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pluggy. 2024-05-14T11:13:11.667 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-14T11:13:11.704 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-14T11:13:11.899 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-fuse. 2024-05-14T11:13:11.909 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:11.949 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:11.966 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-psutil. 2024-05-14T11:13:11.978 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-14T11:13:12.007 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-14T11:13:12.310 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mds. 2024-05-14T11:13:12.320 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:12.335 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-py. 2024-05-14T11:13:12.347 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-14T11:13:12.359 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mds (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:12.385 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-14T11:13:12.713 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pygments. 2024-05-14T11:13:12.726 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-14T11:13:12.737 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package cephadm. 2024-05-14T11:13:12.747 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../071-cephadm_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:12.763 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-14T11:13:12.779 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:13.259 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-14T11:13:13.270 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-14T11:13:13.274 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-14T11:13:13.284 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-14T11:13:13.308 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-14T11:13:13.323 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-14T11:13:13.593 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-14T11:13:13.603 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-14T11:13:13.605 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-toml. 2024-05-14T11:13:13.617 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-14T11:13:13.642 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:13.654 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-14T11:13:13.957 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pytest. 2024-05-14T11:13:13.969 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-14T11:13:14.024 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-14T11:13:14.361 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplejson. 2024-05-14T11:13:14.373 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-14T11:13:14.410 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-14T11:13:14.606 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-babel. 2024-05-14T11:13:14.616 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-14T11:13:14.656 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:14.680 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-14T11:13:14.692 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-14T11:13:14.730 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-14T11:13:15.010 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-jinja2. 2024-05-14T11:13:15.020 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-14T11:13:15.060 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-14T11:13:15.372 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-14T11:13:15.382 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:15.422 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:15.450 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package radosgw. 2024-05-14T11:13:15.461 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../122-radosgw_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:15.500 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking radosgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:15.716 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-14T11:13:15.726 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-14T11:13:15.766 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-14T11:13:16.036 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-routes. 2024-05-14T11:13:16.046 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-14T11:13:16.086 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-14T11:13:16.211 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package rbd-fuse. 2024-05-14T11:13:16.223 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:16.295 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking rbd-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:16.447 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-14T11:13:16.457 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:16.488 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:16.595 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-14T11:13:16.867 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-14T11:13:16.992 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-14T11:13:17.251 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-14T11:13:17.652 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-14T11:13:17.736 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-14T11:13:17.745 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-14T11:13:17.785 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-14T11:13:18.272 INFO:teuthology.orchestra.run.smithi116.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-14T11:13:18.280 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-joblib. 2024-05-14T11:13:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-14T11:13:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-14T11:13:18.339 INFO:teuthology.orchestra.run.smithi116.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-14T11:13:18.386 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:18.726 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-14T11:13:18.735 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-14T11:13:18.852 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-14T11:13:19.147 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-sklearn. 2024-05-14T11:13:19.159 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-14T11:13:19.196 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-14T11:13:19.197 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user cephadm....done 2024-05-14T11:13:19.351 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-14T11:13:19.644 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-14T11:13:19.895 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-14T11:13:19.897 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-14T11:13:19.908 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:19.947 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:20.046 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-14T11:13:20.306 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-14T11:13:20.574 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-14T11:13:20.699 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-14T11:13:20.850 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-cachetools. 2024-05-14T11:13:20.862 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-14T11:13:20.908 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-14T11:13:21.009 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-14T11:13:21.236 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-14T11:13:21.247 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-14T11:13:21.286 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-14T11:13:21.420 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-14T11:13:21.614 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-14T11:13:21.626 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-14T11:13:21.664 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-14T11:13:21.679 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-14T11:13:21.939 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-14T11:13:21.975 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-rsa. 2024-05-14T11:13:21.986 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-14T11:13:22.033 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-14T11:13:22.246 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-14T11:13:22.362 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-google-auth. 2024-05-14T11:13:22.374 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-14T11:13:22.403 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-14T11:13:22.483 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-14T11:13:22.723 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-14T11:13:22.735 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-14T11:13:22.750 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-14T11:13:22.773 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-14T11:13:23.019 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-14T11:13:23.068 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-websocket. 2024-05-14T11:13:23.080 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-14T11:13:23.118 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-14T11:13:23.287 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-argparse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:23.388 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-14T11:13:23.400 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-14T11:13:23.462 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-14T11:13:23.554 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-14T11:13:23.672 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-14T11:13:23.798 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-14T11:13:24.065 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-14T11:13:24.135 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-14T11:13:24.146 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:24.176 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:24.479 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-14T11:13:24.490 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-14T11:13:24.516 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-14T11:13:24.529 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-14T11:13:24.801 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-14T11:13:24.890 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-14T11:13:24.902 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-14T11:13:24.940 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-14T11:13:25.185 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-14T11:13:25.223 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-14T11:13:25.288 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-14T11:13:25.318 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package jq. 2024-05-14T11:13:25.330 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-14T11:13:25.368 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-14T11:13:25.430 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-14T11:13:25.570 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package socat. 2024-05-14T11:13:25.582 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-14T11:13:25.620 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-14T11:13:25.807 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:25.915 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package xmlstarlet. 2024-05-14T11:13:25.926 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-14T11:13:25.932 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-14T11:13:25.964 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-14T11:13:26.200 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-14T11:13:26.294 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-test. 2024-05-14T11:13:26.305 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:26.335 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-test (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:26.476 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-14T11:13:26.794 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up zip (3.0-12build2) ... 2024-05-14T11:13:26.954 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-14T11:13:27.639 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-14T11:13:27.907 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-14T11:13:28.074 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-14T11:13:28.200 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-14T11:13:28.314 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ceph-volume. 2024-05-14T11:13:28.326 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:28.364 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ceph-volume (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:28.518 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-14T11:13:28.920 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-14T11:13:29.035 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-14T11:13:29.047 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:29.085 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking cephfs-mirror (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:29.204 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-14T11:13:29.346 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-colorama. 2024-05-14T11:13:29.358 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-14T11:13:29.421 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-14T11:13:29.580 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-14T11:13:29.673 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package cephfs-shell. 2024-05-14T11:13:29.684 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:29.723 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking cephfs-shell (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:29.882 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-14T11:13:29.993 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package cephfs-top. 2024-05-14T11:13:30.004 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3588-ge2b755a6-1jammy_all.deb ... 2024-05-14T11:13:30.042 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking cephfs-top (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:30.208 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-14T11:13:30.295 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-14T11:13:30.307 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:30.345 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libcephfs-dev (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:30.559 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-14T11:13:30.681 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-14T11:13:30.693 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-14T11:13:30.722 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-14T11:13:30.819 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-14T11:13:30.940 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:31.036 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-14T11:13:31.048 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-14T11:13:31.087 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-14T11:13:31.233 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-idna (3.3-1) ... 2024-05-14T11:13:31.448 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package nvme-cli. 2024-05-14T11:13:31.461 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-14T11:13:31.497 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-14T11:13:31.583 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-14T11:13:31.961 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package pkg-config. 2024-05-14T11:13:31.973 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-14T11:13:32.010 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-14T11:13:32.246 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-14T11:13:32.259 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-14T11:13:32.296 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-14T11:13:32.699 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-14T11:13:32.711 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-14T11:13:32.740 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-14T11:13:32.803 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-14T11:13:33.022 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:13:33.068 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pastescript. 2024-05-14T11:13:33.080 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-14T11:13:33.118 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-14T11:13:33.156 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-14T11:13:33.438 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pluggy. 2024-05-14T11:13:33.440 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-14T11:13:33.450 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-14T11:13:33.487 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-14T11:13:33.541 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-14T11:13:33.659 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-14T11:13:33.799 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-psutil. 2024-05-14T11:13:33.810 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-14T11:13:33.849 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-14T11:13:33.968 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-14T11:13:34.202 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-py. 2024-05-14T11:13:34.214 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-14T11:13:34.245 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-14T11:13:34.251 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-14T11:13:34.580 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pygments. 2024-05-14T11:13:34.592 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-14T11:13:34.604 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:13:34.629 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-14T11:13:34.731 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-14T11:13:35.048 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-14T11:13:35.059 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-14T11:13:35.070 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-14T11:13:35.108 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-14T11:13:35.300 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-14T11:13:35.420 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-toml. 2024-05-14T11:13:35.435 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-14T11:13:35.477 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-14T11:13:35.567 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-14T11:13:35.764 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-pytest. 2024-05-14T11:13:35.777 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-14T11:13:35.822 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-14T11:13:35.835 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-14T11:13:36.123 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-14T11:13:36.125 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package python3-simplejson. 2024-05-14T11:13:36.138 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-14T11:13:36.175 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-14T11:13:36.396 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-14T11:13:36.436 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-14T11:13:36.450 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-14T11:13:36.478 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-14T11:13:36.513 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-14T11:13:36.798 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-14T11:13:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-14T11:13:37.148 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package radosgw. 2024-05-14T11:13:37.160 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../122-radosgw_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:37.198 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking radosgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:37.208 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-14T11:13:37.543 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-14T11:13:37.810 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-14T11:13:37.909 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package rbd-fuse. 2024-05-14T11:13:37.923 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3588-ge2b755a6-1jammy_amd64.deb ... 2024-05-14T11:13:37.945 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-14T11:13:37.968 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking rbd-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:38.071 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:38.159 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-14T11:13:38.276 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-14T11:13:38.398 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-14T11:13:38.556 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-14T11:13:38.557 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-14T11:13:38.657 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-14T11:13:38.888 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-14T11:13:38.950 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-14T11:13:39.218 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-14T11:13:39.270 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-14T11:13:39.352 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-14T11:13:39.628 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:13:39.746 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-14T11:13:39.830 INFO:teuthology.orchestra.run.smithi149.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-14T11:13:39.871 INFO:teuthology.orchestra.run.smithi149.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-14T11:13:39.908 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:40.072 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-14T11:13:40.390 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-14T11:13:40.444 INFO:teuthology.orchestra.run.smithi149.stdout:Adding system user cephadm....done 2024-05-14T11:13:40.598 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-14T11:13:40.666 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-14T11:13:40.882 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-14T11:13:40.886 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-14T11:13:41.133 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-14T11:13:41.154 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-14T11:13:41.285 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-14T11:13:41.430 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-14T11:13:41.544 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-14T11:13:41.698 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-14T11:13:41.812 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-14T11:13:41.938 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-14T11:13:42.083 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-14T11:13:42.374 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-14T11:13:42.376 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-14T11:13:42.729 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-14T11:13:42.783 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-14T11:13:42.855 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librados2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:42.981 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:43.026 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-14T11:13:43.124 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-14T11:13:43.285 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-14T11:13:43.426 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:43.552 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libradosstriper1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:43.570 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-14T11:13:43.687 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-14T11:13:43.888 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-14T11:13:43.988 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librbd1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:44.114 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-modules-core (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:44.148 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-14T11:13:44.265 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:44.416 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-14T11:13:44.476 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-14T11:13:44.476 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-14T11:13:44.675 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-ceph-argparse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:44.934 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-14T11:13:45.060 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-14T11:13:45.091 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs-dev (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:45.186 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-14T11:13:45.209 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rados (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:45.335 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librgw2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:45.454 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-14T11:13:45.494 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-top (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:45.580 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-14T11:13:45.620 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-14T11:13:45.847 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-14T11:13:46.306 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-14T11:13:46.361 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-14T11:13:46.426 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-14T11:13:46.552 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-14T11:13:46.899 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rbd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:46.928 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:47.054 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-14T11:13:47.100 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up rbd-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:47.218 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:47.330 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-14T11:13:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cephfs (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:47.478 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-shell (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:47.604 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:47.606 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-14T11:13:47.907 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up zip (3.0-12build2) ... 2024-05-14T11:13:48.033 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-14T11:13:48.157 INFO:teuthology.orchestra.run.smithi116.stdout:Adding group ceph....done 2024-05-14T11:13:48.508 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user ceph....done 2024-05-14T11:13:48.669 INFO:teuthology.orchestra.run.smithi116.stdout:Setting system user ceph properties....done 2024-05-14T11:13:48.688 INFO:teuthology.orchestra.run.smithi116.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-14T11:13:48.727 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-14T11:13:48.833 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-14T11:13:48.995 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-14T11:13:49.120 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-14T11:13:49.219 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-14T11:13:49.246 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-14T11:13:49.572 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-14T11:13:49.957 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-14T11:13:50.208 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-14T11:13:50.210 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-mirror (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:50.413 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-14T11:13:50.414 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-14T11:13:50.602 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-14T11:13:50.886 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-14T11:13:51.086 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-test (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:51.196 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up radosgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:51.204 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-14T11:13:51.564 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-14T11:13:51.713 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-14T11:13:51.714 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-14T11:13:51.806 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-14T11:13:51.966 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:52.275 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-idna (3.3-1) ... 2024-05-14T11:13:52.314 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-base (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:52.618 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-14T11:13:52.706 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-14T11:13:53.357 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mds (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:53.589 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-14T11:13:53.589 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-14T11:13:53.849 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-14T11:13:54.084 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:13:54.201 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:54.210 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-14T11:13:54.414 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-14T11:13:54.414 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-14T11:13:54.486 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-14T11:13:54.612 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-14T11:13:54.738 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-14T11:13:55.031 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-14T11:13:55.077 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-osd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:55.290 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-14T11:13:55.414 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-14T11:13:55.414 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-14T11:13:55.541 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:13:55.668 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-14T11:13:55.977 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-14T11:13:56.029 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:56.155 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:56.237 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-14T11:13:56.314 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mon (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:56.521 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-14T11:13:56.524 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-14T11:13:56.524 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-14T11:13:56.797 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-14T11:13:57.048 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-14T11:13:57.157 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:57.274 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:57.307 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-14T11:13:57.384 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-dashboard (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:57.433 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-14T11:13:57.526 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-volume (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:13:57.720 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-14T11:13:57.871 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-14T11:13:57.896 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-14T11:13:58.005 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-14T11:13:58.140 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-14T11:13:58.164 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-14T11:13:58.465 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-14T11:13:58.716 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-14T11:13:58.834 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-14T11:13:58.960 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-14T11:13:59.051 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-14T11:13:59.266 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-14T11:13:59.295 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-14T11:13:59.696 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-14T11:13:59.996 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-14T11:14:00.001 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-14T11:14:00.076 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-05-14T11:14:00.139 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-14T11:14:00.252 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-05-14T11:14:00.253 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-14T11:14:00.425 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-14T11:14:00.426 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-14T11:14:00.426 INFO:teuthology.orchestra.run.smithi116.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-14T11:14:00.426 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-14T11:14:00.427 INFO:teuthology.orchestra.run.smithi116.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-14T11:14:00.428 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2024-05-14T11:14:00.428 INFO:teuthology.orchestra.run.smithi116.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-14T11:14:00.428 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-14T11:14:00.428 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-14T11:14:00.432 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-14T11:14:00.482 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2024-05-14T11:14:00.483 INFO:teuthology.orchestra.run.smithi116.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-14T11:14:00.483 INFO:teuthology.orchestra.run.smithi116.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-14T11:14:00.484 INFO:teuthology.orchestra.run.smithi116.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-14T11:14:00.484 INFO:teuthology.orchestra.run.smithi116.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-14T11:14:00.484 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-14T11:14:00.484 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-14T11:14:00.484 INFO:teuthology.orchestra.run.smithi116.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-14T11:14:00.485 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages will be upgraded: 2024-05-14T11:14:00.486 INFO:teuthology.orchestra.run.smithi116.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-14T11:14:00.533 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-14T11:14:00.646 INFO:teuthology.orchestra.run.smithi116.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-14T11:14:00.646 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 55.7 MB of archives. 2024-05-14T11:14:00.646 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-14T11:14:00.646 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-14T11:14:00.784 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-14T11:14:00.820 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-14T11:14:00.827 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-14T11:14:00.868 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-14T11:14:00.869 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-14T11:14:00.869 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-14T11:14:00.882 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-14T11:14:00.882 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-14T11:14:00.884 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-14T11:14:00.885 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-14T11:14:00.886 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-14T11:14:00.887 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-14T11:14:00.898 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-14T11:14:00.898 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-14T11:14:00.899 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-14T11:14:00.901 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-14T11:14:00.919 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-14T11:14:01.211 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-14T11:14:01.359 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-14T11:14:01.361 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-14T11:14:01.361 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-14T11:14:01.364 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-14T11:14:01.364 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-14T11:14:01.368 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-14T11:14:01.370 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-14T11:14:01.377 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-14T11:14:01.378 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-14T11:14:01.416 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-14T11:14:01.420 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-14T11:14:01.437 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-14T11:14:01.512 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-14T11:14:01.516 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-14T11:14:01.516 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-14T11:14:01.517 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-14T11:14:01.517 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-14T11:14:01.521 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-14T11:14:01.529 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-14T11:14:01.531 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-14T11:14:01.805 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-14T11:14:01.839 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-14T11:14:01.839 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-14T11:14:01.839 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-14T11:14:01.840 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-14T11:14:01.840 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-14T11:14:01.840 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-14T11:14:01.855 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-14T11:14:01.855 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-14T11:14:01.992 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-14T11:14:02.231 INFO:teuthology.orchestra.run.smithi116.stdout:Preconfiguring packages ... 2024-05-14T11:14:02.259 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-14T11:14:02.373 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 55.7 MB in 1s (40.9 MB/s) 2024-05-14T11:14:02.519 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-14T11:14:02.661 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-05-14T11:14:02.664 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:02.787 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-14T11:14:02.805 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:03.172 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-14T11:14:03.438 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:03.448 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-14T11:14:03.801 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-14T11:14:03.935 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up librados2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:04.053 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:04.179 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-14T11:14:04.514 INFO:teuthology.orchestra.run.smithi116.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-14T11:14:04.515 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libcephfs2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:04.641 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libradosstriper1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:04.767 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-14T11:14:04.768 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package flex. 2024-05-14T11:14:04.838 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-05-14T11:14:04.842 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-14T11:14:05.060 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up librbd1 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:05.187 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr-modules-core (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:05.319 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-14T11:14:05.321 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:05.481 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-14T11:14:05.561 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-14T11:14:05.561 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-14T11:14:05.887 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:06.435 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-05-14T11:14:06.439 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:06.481 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libcephfs-dev (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:06.547 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:06.607 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-rados (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:06.708 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up librgw2 (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:06.834 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up cephfs-top (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:06.942 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:06.960 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-14T11:14:07.079 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:07.385 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:07.762 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package bison. 2024-05-14T11:14:07.837 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-05-14T11:14:07.841 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-14T11:14:07.889 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-14T11:14:08.239 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-rbd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:08.275 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-14T11:14:08.290 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-14T11:14:08.334 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-14T11:14:08.348 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up rbd-fuse (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:08.457 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-rgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:08.558 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up python3-cephfs (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:08.670 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package dump. 2024-05-14T11:14:08.684 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up cephfs-shell (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:08.686 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-14T11:14:08.736 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking dump (0.4b47-3) ... 2024-05-14T11:14:08.852 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-common (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:09.007 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package indent. 2024-05-14T11:14:09.023 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:09.065 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-14T11:14:09.318 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-14T11:14:09.334 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-14T11:14:09.377 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-14T11:14:09.421 INFO:teuthology.orchestra.run.smithi149.stdout:Adding group ceph....done 2024-05-14T11:14:09.688 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-14T11:14:09.704 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-14T11:14:09.755 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-14T11:14:09.806 INFO:teuthology.orchestra.run.smithi149.stdout:Adding system user ceph....done 2024-05-14T11:14:10.000 INFO:teuthology.orchestra.run.smithi149.stdout:Setting system user ceph properties....done 2024-05-14T11:14:10.016 INFO:teuthology.orchestra.run.smithi149.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-14T11:14:10.016 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-14T11:14:10.032 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-14T11:14:10.074 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-14T11:14:10.156 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-14T11:14:10.411 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-14T11:14:10.426 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:10.461 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:10.544 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-14T11:14:10.663 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-14T11:14:10.679 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:10.713 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:10.991 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libjson-perl. 2024-05-14T11:14:11.007 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-14T11:14:11.049 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-14T11:14:11.203 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up cephfs-mirror (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:11.352 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-14T11:14:11.368 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-14T11:14:11.410 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-14T11:14:11.411 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-14T11:14:11.411 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-14T11:14:12.021 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-test (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:12.147 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up radosgw (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:12.665 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-14T11:14:12.665 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-14T11:14:12.706 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-14T11:14:12.721 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-14T11:14:12.763 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:13.016 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-14T11:14:13.032 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-14T11:14:13.074 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:13.315 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-base (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:13.394 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-14T11:14:13.409 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-14T11:14:13.451 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:13.639 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-14T11:14:13.780 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-14T11:14:13.795 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-14T11:14:13.829 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-14T11:14:14.133 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-14T11:14:14.148 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-14T11:14:14.191 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-14T11:14:14.292 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mds (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:14.516 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-14T11:14:14.517 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-14T11:14:14.535 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-14T11:14:14.551 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:14.594 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:14.938 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-14T11:14:14.954 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:14.996 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:15.257 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-14T11:14:15.273 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:15.315 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:15.343 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:15.545 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-14T11:14:15.545 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-14T11:14:15.668 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-14T11:14:15.684 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-14T11:14:15.726 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:16.166 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-osd (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:16.213 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-14T11:14:16.227 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-14T11:14:16.263 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-14T11:14:16.477 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-14T11:14:16.478 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-14T11:14:16.591 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package modemmanager. 2024-05-14T11:14:16.607 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-14T11:14:16.649 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:17.059 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:17.210 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:17.264 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager. 2024-05-14T11:14:17.280 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-14T11:14:17.330 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-14T11:14:17.353 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mon (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:17.560 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-14T11:14:17.560 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-14T11:14:17.954 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ppp. 2024-05-14T11:14:17.971 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-14T11:14:18.187 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr-cephadm (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:18.315 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-14T11:14:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:18.473 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-mgr-dashboard (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:18.615 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ceph-volume (19.0.0-3588-ge2b755a6-1jammy) ... 2024-05-14T11:14:18.626 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package pptp-linux. 2024-05-14T11:14:18.640 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-14T11:14:18.684 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-14T11:14:18.920 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-14T11:14:18.934 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-14T11:14:18.968 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-14T11:14:18.970 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-14T11:14:19.077 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-14T11:14:19.229 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-14T11:14:19.265 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-14T11:14:19.279 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-14T11:14:19.323 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client-common (238) ... 2024-05-14T11:14:19.635 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-14T11:14:19.650 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:19.693 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:20.264 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-common. 2024-05-14T11:14:20.279 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-14T11:14:20.322 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-14T11:14:20.348 INFO:teuthology.orchestra.run.smithi116.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-14T11:14:20.380 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-common (238) ... 2024-05-14T11:14:20.854 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-14. 2024-05-14T11:14:20.868 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:20.915 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:21.043 INFO:teuthology.orchestra.run.smithi149.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-14T11:14:21.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-14T11:14:21.122 INFO:teuthology.orchestra.run.smithi149.stdout:Reading package lists... 2024-05-14T11:14:21.302 INFO:teuthology.orchestra.run.smithi149.stdout:Building dependency tree... 2024-05-14T11:14:21.303 INFO:teuthology.orchestra.run.smithi149.stdout:Reading state information... 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout:The following packages were automatically installed and are no longer required: 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-14T11:14:21.481 INFO:teuthology.orchestra.run.smithi149.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-14T11:14:21.482 INFO:teuthology.orchestra.run.smithi149.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-14T11:14:21.482 INFO:teuthology.orchestra.run.smithi149.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-14T11:14:21.482 INFO:teuthology.orchestra.run.smithi149.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-14T11:14:21.482 INFO:teuthology.orchestra.run.smithi149.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-14T11:14:21.483 INFO:teuthology.orchestra.run.smithi149.stdout:The following additional packages will be installed: 2024-05-14T11:14:21.483 INFO:teuthology.orchestra.run.smithi149.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-14T11:14:21.483 INFO:teuthology.orchestra.run.smithi149.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-14T11:14:21.484 INFO:teuthology.orchestra.run.smithi149.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-14T11:14:21.484 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-14T11:14:21.484 INFO:teuthology.orchestra.run.smithi149.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-14T11:14:21.485 INFO:teuthology.orchestra.run.smithi149.stdout:Suggested packages: 2024-05-14T11:14:21.485 INFO:teuthology.orchestra.run.smithi149.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-14T11:14:21.485 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-14T11:14:21.485 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-14T11:14:21.539 INFO:teuthology.orchestra.run.smithi149.stdout:The following NEW packages will be installed: 2024-05-14T11:14:21.540 INFO:teuthology.orchestra.run.smithi149.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-14T11:14:21.540 INFO:teuthology.orchestra.run.smithi149.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-14T11:14:21.540 INFO:teuthology.orchestra.run.smithi149.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-14T11:14:21.541 INFO:teuthology.orchestra.run.smithi149.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-14T11:14:21.541 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-14T11:14:21.541 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-14T11:14:21.541 INFO:teuthology.orchestra.run.smithi149.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-14T11:14:21.542 INFO:teuthology.orchestra.run.smithi149.stdout:The following packages will be upgraded: 2024-05-14T11:14:21.543 INFO:teuthology.orchestra.run.smithi149.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-14T11:14:21.665 INFO:teuthology.orchestra.run.smithi149.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-14T11:14:21.665 INFO:teuthology.orchestra.run.smithi149.stdout:Need to get 55.7 MB of archives. 2024-05-14T11:14:21.665 INFO:teuthology.orchestra.run.smithi149.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-14T11:14:21.665 INFO:teuthology.orchestra.run.smithi149.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-14T11:14:22.226 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql. 2024-05-14T11:14:22.239 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-14T11:14:22.278 INFO:teuthology.orchestra.run.smithi149.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-14T11:14:22.282 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql (14+238) ... 2024-05-14T11:14:22.295 INFO:teuthology.orchestra.run.smithi149.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-14T11:14:22.398 INFO:teuthology.orchestra.run.smithi149.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-14T11:14:22.398 INFO:teuthology.orchestra.run.smithi149.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-14T11:14:22.399 INFO:teuthology.orchestra.run.smithi149.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-14T11:14:22.416 INFO:teuthology.orchestra.run.smithi149.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-14T11:14:22.417 INFO:teuthology.orchestra.run.smithi149.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-14T11:14:22.420 INFO:teuthology.orchestra.run.smithi149.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-14T11:14:22.421 INFO:teuthology.orchestra.run.smithi149.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-14T11:14:22.422 INFO:teuthology.orchestra.run.smithi149.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-14T11:14:22.423 INFO:teuthology.orchestra.run.smithi149.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-14T11:14:22.432 INFO:teuthology.orchestra.run.smithi149.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-14T11:14:22.452 INFO:teuthology.orchestra.run.smithi149.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-14T11:14:22.453 INFO:teuthology.orchestra.run.smithi149.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-14T11:14:22.454 INFO:teuthology.orchestra.run.smithi149.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-14T11:14:22.501 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client. 2024-05-14T11:14:22.515 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-14T11:14:22.551 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client (14+238) ... 2024-05-14T11:14:22.744 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-14T11:14:22.759 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-14T11:14:22.803 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-14T11:14:22.865 INFO:teuthology.orchestra.run.smithi149.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-14T11:14:22.867 INFO:teuthology.orchestra.run.smithi149.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-14T11:14:22.867 INFO:teuthology.orchestra.run.smithi149.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-14T11:14:22.870 INFO:teuthology.orchestra.run.smithi149.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-14T11:14:22.870 INFO:teuthology.orchestra.run.smithi149.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-14T11:14:22.875 INFO:teuthology.orchestra.run.smithi149.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-14T11:14:22.877 INFO:teuthology.orchestra.run.smithi149.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-14T11:14:22.885 INFO:teuthology.orchestra.run.smithi149.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-14T11:14:22.886 INFO:teuthology.orchestra.run.smithi149.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-14T11:14:22.926 INFO:teuthology.orchestra.run.smithi149.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-14T11:14:22.932 INFO:teuthology.orchestra.run.smithi149.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-14T11:14:22.943 INFO:teuthology.orchestra.run.smithi149.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-14T11:14:23.010 INFO:teuthology.orchestra.run.smithi149.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-14T11:14:23.013 INFO:teuthology.orchestra.run.smithi149.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-14T11:14:23.014 INFO:teuthology.orchestra.run.smithi149.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-14T11:14:23.014 INFO:teuthology.orchestra.run.smithi149.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-14T11:14:23.015 INFO:teuthology.orchestra.run.smithi149.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-14T11:14:23.027 INFO:teuthology.orchestra.run.smithi149.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-14T11:14:23.030 INFO:teuthology.orchestra.run.smithi149.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-14T11:14:23.155 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package tcl8.6. 2024-05-14T11:14:23.170 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-14T11:14:23.208 INFO:teuthology.orchestra.run.smithi149.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-14T11:14:23.208 INFO:teuthology.orchestra.run.smithi149.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-14T11:14:23.208 INFO:teuthology.orchestra.run.smithi149.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-14T11:14:23.209 INFO:teuthology.orchestra.run.smithi149.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-14T11:14:23.209 INFO:teuthology.orchestra.run.smithi149.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-14T11:14:23.209 INFO:teuthology.orchestra.run.smithi149.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-14T11:14:23.213 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-14T11:14:23.223 INFO:teuthology.orchestra.run.smithi149.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-14T11:14:23.224 INFO:teuthology.orchestra.run.smithi149.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-14T11:14:23.508 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package tcl. 2024-05-14T11:14:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-14T11:14:23.570 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-14T11:14:23.603 INFO:teuthology.orchestra.run.smithi149.stdout:Preconfiguring packages ... 2024-05-14T11:14:23.753 INFO:teuthology.orchestra.run.smithi149.stdout:Fetched 55.7 MB in 2s (33.3 MB/s) 2024-05-14T11:14:23.877 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package wpasupplicant. 2024-05-14T11:14:23.892 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-14T11:14:23.927 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-14T11:14:24.039 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-05-14T11:14:24.043 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:24.177 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:24.273 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-14T11:14:24.287 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-14T11:14:24.331 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-14T11:14:24.709 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-14T11:14:24.724 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-14T11:14:24.775 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-14T11:14:24.810 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:25.061 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client-common (238) ... 2024-05-14T11:14:25.298 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-14T11:14:25.424 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-14T11:14:25.550 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:25.676 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-14T11:14:25.794 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-14T11:14:25.897 INFO:teuthology.orchestra.run.smithi149.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-14T11:14:25.919 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:26.157 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package flex. 2024-05-14T11:14:26.228 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-05-14T11:14:26.232 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:26.272 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-14T11:14:26.792 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-14T11:14:26.871 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-14T11:14:26.967 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-14T11:14:26.978 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-14T11:14:27.320 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-14T11:14:27.320 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-14T11:14:27.368 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:27.933 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-05-14T11:14:27.936 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:27.970 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:28.077 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:28.095 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-14T11:14:28.222 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:28.347 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-14T11:14:28.473 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-14T11:14:28.482 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-14T11:14:28.608 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:28.608 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-14T11:14:28.759 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-14T11:14:28.876 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-14T11:14:28.923 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:29.309 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package bison. 2024-05-14T11:14:29.381 INFO:teuthology.orchestra.run.smithi149.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-05-14T11:14:29.385 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-14T11:14:29.436 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-14T11:14:29.589 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-14T11:14:29.715 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up dump (0.4b47-3) ... 2024-05-14T11:14:29.810 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-14T11:14:29.854 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-14T11:14:29.869 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-14T11:14:29.900 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:29.904 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-14T11:14:30.018 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-14T11:14:30.127 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-14T11:14:30.190 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package dump. 2024-05-14T11:14:30.205 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-14T11:14:30.253 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-14T11:14:30.273 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking dump (0.4b47-3) ... 2024-05-14T11:14:30.379 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-14T11:14:30.469 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-14T11:14:30.539 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-14T11:14:30.543 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package indent. 2024-05-14T11:14:30.558 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:30.602 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-14T11:14:30.673 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-14T11:14:30.846 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-14T11:14:30.849 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-14T11:14:30.861 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-14T11:14:30.904 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-14T11:14:30.976 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:31.118 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-14T11:14:31.200 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-14T11:14:31.216 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-14T11:14:31.258 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-14T11:14:31.261 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client (14+238) ... 2024-05-14T11:14:31.387 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-14T11:14:31.544 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-14T11:14:31.560 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-14T11:14:31.594 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-14T11:14:31.931 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-14T11:14:31.947 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:31.989 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:32.225 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-14T11:14:32.242 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-14T11:14:32.283 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:32.341 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout:which means they are currently configured by ifupdown: 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout:- enp3s0f1 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout:remove their configuration from /etc/network/interfaces. 2024-05-14T11:14:32.342 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:32.474 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-14T11:14:32.528 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libjson-perl. 2024-05-14T11:14:32.544 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-14T11:14:32.586 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-14T11:14:32.879 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-14T11:14:32.906 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-14T11:14:32.922 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-14T11:14:32.964 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-14T11:14:33.277 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-14T11:14:33.982 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:34.125 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:34.251 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-14T11:14:34.325 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-14T11:14:34.342 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-14T11:14:34.351 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-common (238) ... 2024-05-14T11:14:34.383 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:34.644 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-14T11:14:34.660 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-14T11:14:34.703 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:34.972 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-14T11:14:34.988 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-14T11:14:35.030 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:35.320 INFO:teuthology.orchestra.run.smithi116.stdout:Adding user postgres to group ssl-cert 2024-05-14T11:14:35.392 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-14T11:14:35.408 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-14T11:14:35.450 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-14T11:14:35.580 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:35.580 INFO:teuthology.orchestra.run.smithi116.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-14T11:14:35.753 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-14T11:14:35.769 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-14T11:14:35.803 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-14T11:14:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-14T11:14:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:Removing obsolete dictionary files: 2024-05-14T11:14:36.131 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-14T11:14:36.146 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:36.189 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:36.533 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-14T11:14:36.567 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-14T11:14:36.582 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:36.625 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:36.894 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-14T11:14:36.909 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:36.953 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:37.180 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-14T11:14:37.195 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-14T11:14:37.230 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:37.324 INFO:teuthology.orchestra.run.smithi116.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-14T11:14:37.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:This user must also own the server process. 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:The database cluster will be initialized with locale "en_US". 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:The default text search configuration will be set to "english". 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:Data page checksums are disabled. 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:14:37.404 INFO:teuthology.orchestra.run.smithi116.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-14T11:14:37.405 INFO:teuthology.orchestra.run.smithi116.stdout:creating subdirectories ... ok 2024-05-14T11:14:37.405 INFO:teuthology.orchestra.run.smithi116.stdout:selecting dynamic shared memory implementation ... posix 2024-05-14T11:14:37.429 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default max_connections ... 100 2024-05-14T11:14:37.463 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default shared_buffers ... 128MB 2024-05-14T11:14:37.477 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default time zone ... Etc/UTC 2024-05-14T11:14:37.478 INFO:teuthology.orchestra.run.smithi116.stdout:creating configuration files ... ok 2024-05-14T11:14:37.642 INFO:teuthology.orchestra.run.smithi116.stdout:running bootstrap script ... ok 2024-05-14T11:14:37.717 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-14T11:14:37.732 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-14T11:14:37.767 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-14T11:14:38.103 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package modemmanager. 2024-05-14T11:14:38.119 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-14T11:14:38.153 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:38.245 INFO:teuthology.orchestra.run.smithi116.stdout:performing post-bootstrap initialization ... ok 2024-05-14T11:14:38.752 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package network-manager. 2024-05-14T11:14:38.761 INFO:teuthology.orchestra.run.smithi116.stdout:syncing data to disk ... ok 2024-05-14T11:14:38.768 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-14T11:14:38.842 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-14T11:14:39.256 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-14T11:14:39.576 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package ppp. 2024-05-14T11:14:39.591 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-14T11:14:39.921 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-14T11:14:40.256 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package pptp-linux. 2024-05-14T11:14:40.270 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-14T11:14:40.314 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-14T11:14:40.575 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-14T11:14:40.589 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-14T11:14:40.633 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-14T11:14:40.903 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-14T11:14:40.917 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-14T11:14:40.952 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-client-common (238) ... 2024-05-14T11:14:41.273 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-14T11:14:41.285 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:41.323 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:42.003 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-common. 2024-05-14T11:14:42.016 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-14T11:14:42.078 INFO:teuthology.orchestra.run.smithi149.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-14T11:14:42.109 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-common (238) ... 2024-05-14T11:14:42.177 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:42.294 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-14T11:14:42.420 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql (14+238) ... 2024-05-14T11:14:42.546 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:14:42.610 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-14. 2024-05-14T11:14:42.622 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-14T11:14:42.670 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:42.705 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:42.966 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-14T11:14:42.966 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-14T11:14:43.716 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-14T11:14:43.749 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql. 2024-05-14T11:14:43.761 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-14T11:14:43.799 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql (14+238) ... 2024-05-14T11:14:43.834 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-14T11:14:44.025 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-client. 2024-05-14T11:14:44.037 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-14T11:14:44.067 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-client (14+238) ... 2024-05-14T11:14:44.152 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-14T11:14:44.279 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-14T11:14:44.294 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-14T11:14:44.306 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-14T11:14:44.344 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-14T11:14:44.688 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package tcl8.6. 2024-05-14T11:14:44.703 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-14T11:14:44.746 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-14T11:14:45.066 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package tcl. 2024-05-14T11:14:45.080 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-14T11:14:45.137 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-14T11:14:45.436 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package wpasupplicant. 2024-05-14T11:14:45.450 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-14T11:14:45.493 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-14T11:14:45.648 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-14T11:14:45.748 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-14T11:14:46.240 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-14T11:14:46.254 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-14T11:14:46.290 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-14T11:14:46.297 DEBUG:teuthology.parallel:result is None 2024-05-14T11:14:46.298 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-14T11:14:46.660 INFO:teuthology.orchestra.run.smithi149.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-14T11:14:46.676 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-14T11:14:46.726 INFO:teuthology.orchestra.run.smithi149.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-14T11:14:47.010 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-client-common (238) ... 2024-05-14T11:14:47.216 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-14T11:14:47.350 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-14T11:14:47.460 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:47.585 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-14T11:14:47.712 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-14T11:14:47.838 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:48.780 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-14T11:14:48.877 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-14T11:14:49.254 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-14T11:14:49.254 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-14T11:14:49.980 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-14T11:14:50.106 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-14T11:14:50.240 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-14T11:14:50.358 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-14T11:14:50.467 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-14T11:14:50.585 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:50.719 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-14T11:14:50.845 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-14T11:14:51.931 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-14T11:14:52.177 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up dump (0.4b47-3) ... 2024-05-14T11:14:52.272 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-14T11:14:52.329 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:14:52.455 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-14T11:14:52.581 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-14T11:14:52.707 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-14T11:14:52.833 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-14T11:14:52.923 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-14T11:14:52.993 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-14T11:14:53.102 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-14T11:14:53.295 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-14T11:14:53.421 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:53.547 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-14T11:14:53.715 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-client (14+238) ... 2024-05-14T11:14:53.841 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout:which means they are currently configured by ifupdown: 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout:- enp3s0f1 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-14T11:14:54.417 INFO:teuthology.orchestra.run.smithi149.stdout:remove their configuration from /etc/network/interfaces. 2024-05-14T11:14:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:54.550 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-14T11:14:54.942 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-14T11:14:55.338 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-14T11:14:56.019 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-14T11:14:56.170 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-14T11:14:56.305 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-14T11:14:56.431 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-common (238) ... 2024-05-14T11:14:57.354 INFO:teuthology.orchestra.run.smithi149.stdout:Adding user postgres to group ssl-cert 2024-05-14T11:14:57.597 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:57.597 INFO:teuthology.orchestra.run.smithi149.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-14T11:14:57.851 INFO:teuthology.orchestra.run.smithi149.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-14T11:14:57.852 INFO:teuthology.orchestra.run.smithi149.stdout:Removing obsolete dictionary files: 2024-05-14T11:14:58.564 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-14T11:14:59.015 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-14T11:14:59.340 INFO:teuthology.orchestra.run.smithi149.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-14T11:14:59.360 INFO:teuthology.orchestra.run.smithi149.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:This user must also own the server process. 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:The database cluster will be initialized with locale "en_US". 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout:The default text search configuration will be set to "english". 2024-05-14T11:14:59.405 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:59.406 INFO:teuthology.orchestra.run.smithi149.stdout:Data page checksums are disabled. 2024-05-14T11:14:59.406 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:14:59.406 INFO:teuthology.orchestra.run.smithi149.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-14T11:14:59.407 INFO:teuthology.orchestra.run.smithi149.stdout:creating subdirectories ... ok 2024-05-14T11:14:59.407 INFO:teuthology.orchestra.run.smithi149.stdout:selecting dynamic shared memory implementation ... posix 2024-05-14T11:14:59.430 INFO:teuthology.orchestra.run.smithi149.stdout:selecting default max_connections ... 100 2024-05-14T11:14:59.463 INFO:teuthology.orchestra.run.smithi149.stdout:selecting default shared_buffers ... 128MB 2024-05-14T11:14:59.477 INFO:teuthology.orchestra.run.smithi149.stdout:selecting default time zone ... Etc/UTC 2024-05-14T11:14:59.478 INFO:teuthology.orchestra.run.smithi149.stdout:creating configuration files ... ok 2024-05-14T11:14:59.645 INFO:teuthology.orchestra.run.smithi149.stdout:running bootstrap script ... ok 2024-05-14T11:15:00.248 INFO:teuthology.orchestra.run.smithi149.stdout:performing post-bootstrap initialization ... ok 2024-05-14T11:15:00.866 INFO:teuthology.orchestra.run.smithi149.stdout:syncing data to disk ... ok 2024-05-14T11:15:01.395 INFO:teuthology.orchestra.run.smithi149.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-14T11:15:04.415 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:15:04.683 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-14T11:15:04.809 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up postgresql (14+238) ... 2024-05-14T11:15:04.935 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-14T11:15:05.036 INFO:teuthology.orchestra.run.smithi149.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-14T11:15:05.295 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-14T11:15:05.295 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-14T11:15:06.080 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-14T11:15:06.172 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-14T11:15:06.491 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-14T11:15:06.659 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-14T11:15:08.037 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-14T11:15:08.138 INFO:teuthology.orchestra.run.smithi149.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-14T11:15:08.682 INFO:teuthology.orchestra.run.smithi149.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-14T11:15:08.688 DEBUG:teuthology.parallel:result is None 2024-05-14T11:15:08.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T11:15:09.061 DEBUG:teuthology.orchestra.run.smithi116:> dpkg-query -W -f '${Version}' ceph 2024-05-14T11:15:09.085 INFO:teuthology.orchestra.run.smithi116.stdout:19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:15:09.086 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:15:09.086 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588-ge2b755a6-1jammy is installed. 2024-05-14T11:15:09.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T11:15:09.231 DEBUG:teuthology.orchestra.run.smithi149:> dpkg-query -W -f '${Version}' ceph 2024-05-14T11:15:09.253 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:15:09.253 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588-ge2b755a6-1jammy 2024-05-14T11:15:09.253 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588-ge2b755a6-1jammy is installed. 2024-05-14T11:15:09.255 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T11:15:09.256 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:09.256 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T11:15:09.278 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:09.278 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T11:15:09.316 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T11:15:09.316 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:09.316 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T11:15:09.339 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T11:15:09.399 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:09.399 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T11:15:09.422 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T11:15:09.479 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T11:15:09.479 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:09.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T11:15:09.500 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T11:15:09.563 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:09.563 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T11:15:09.583 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T11:15:09.643 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T11:15:09.644 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:09.644 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T11:15:09.665 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T11:15:09.727 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:09.727 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T11:15:09.746 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T11:15:09.811 INFO:teuthology.run_tasks:Running task ceph... 2024-05-14T11:15:09.897 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-14T11:15:09.898 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2024-05-14T11:15:09.901 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /var/log/ceph 2024-05-14T11:15:09.920 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-14T11:15:09.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-14T11:15:09.966 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-14T11:15:09.985 INFO:tasks.ceph:Creating extra log directories... 2024-05-14T11:15:09.985 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-14T11:15:10.035 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-14T11:15:10.060 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-14T11:15:10.061 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-14T11:15:10.061 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-14T11:15:10.061 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:10.061 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-14T11:15:10.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-14T11:15:10.196 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:10.196 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-14T11:15:10.205 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-14T11:15:10.303 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-14T11:15:10.303 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-14T11:15:10.303 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738', 'branch': 'wip-vshankar-testing-20240509.053109-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7705738', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240509.053109-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240509.053109-debug', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6285, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'sleep_before_teardown': 0, 'subset': '548/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240509.053109-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi116.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUFfQvC/Eeg5MY0zk5NfdO/V0tles3akRJGBV52ASHZhy74RNaRRuEH/JG/4tzI0p1mm2wuxczjBQZrvBgj6oI=', 'smithi149.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMQCUVx0Oaewy2yohiXHF0W2rtGXKf/aJNUvheYs+LseaKr4ShASChwzQCxpjGn9M/gpHNvNQa1uUJxEZ4KHn4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-14_07:04:04', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-14T11:15:10.304 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-14T11:15:10.307 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-14T11:15:10.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2024-05-14T11:15:10.358 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/run/ceph 2024-05-14T11:15:10.377 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:10.377 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T11:15:10.428 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T11:15:10.428 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 887 Links: 1 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-05-14 11:08:30.025765008 +0000 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-05-14 11:08:29.761767366 +0000 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-05-14 11:08:29.761767366 +0000 2024-05-14T11:15:10.484 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-05-14 11:08:29.761767366 +0000 2024-05-14T11:15:10.484 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T11:15:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-05-14T11:15:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-05-14T11:15:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000415801 s, 1.2 MB/s 2024-05-14T11:15:10.547 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T11:15:10.601 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-05-14T11:15:10.656 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T11:15:10.656 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:10.656 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 905 Links: 1 2024-05-14T11:15:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-05-14 11:08:30.561760221 +0000 2024-05-14T11:15:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-05-14 11:08:30.301762543 +0000 2024-05-14T11:15:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-05-14 11:08:30.301762543 +0000 2024-05-14T11:15:10.657 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-05-14 11:08:30.301762543 +0000 2024-05-14T11:15:10.657 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T11:15:10.717 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-05-14T11:15:10.717 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-05-14T11:15:10.717 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000385965 s, 1.3 MB/s 2024-05-14T11:15:10.719 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T11:15:10.774 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-05-14T11:15:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 923 Links: 1 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-05-14 11:08:31.121755219 +0000 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-05-14 11:08:30.861757541 +0000 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-05-14 11:08:30.861757541 +0000 2024-05-14T11:15:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-05-14 11:08:30.861757541 +0000 2024-05-14T11:15:10.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T11:15:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-05-14T11:15:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-05-14T11:15:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000382492 s, 1.3 MB/s 2024-05-14T11:15:10.888 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T11:15:10.941 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-05-14T11:15:10.991 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-05-14 11:08:31.681750217 +0000 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-05-14 11:08:31.413752611 +0000 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-05-14 11:08:31.413752611 +0000 2024-05-14T11:15:10.992 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-05-14 11:08:31.413752611 +0000 2024-05-14T11:15:10.992 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T11:15:11.054 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-05-14T11:15:11.054 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-05-14T11:15:11.054 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000398002 s, 1.3 MB/s 2024-05-14T11:15:11.056 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T11:15:11.110 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:11.110 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:11.110 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T11:15:11.119 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T11:15:11.119 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-05-14T11:15:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T11:15:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:11.169 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 894 Links: 1 2024-05-14T11:15:11.169 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:11.169 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-14 11:08:30.049540143 +0000 2024-05-14T11:15:11.170 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-14 11:08:29.761542708 +0000 2024-05-14T11:15:11.170 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-14 11:08:29.761542708 +0000 2024-05-14T11:15:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-14 11:08:29.761542708 +0000 2024-05-14T11:15:11.170 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T11:15:11.231 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-14T11:15:11.231 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-14T11:15:11.231 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000419734 s, 1.2 MB/s 2024-05-14T11:15:11.233 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T11:15:11.291 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 908 Links: 1 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-14 11:08:30.589535331 +0000 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-14 11:08:30.325537684 +0000 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-14 11:08:30.325537684 +0000 2024-05-14T11:15:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-14 11:08:30.325537684 +0000 2024-05-14T11:15:11.341 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T11:15:11.407 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-14T11:15:11.408 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-14T11:15:11.408 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.00037499 s, 1.4 MB/s 2024-05-14T11:15:11.410 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T11:15:11.467 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 925 Links: 1 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-14 11:08:31.149530342 +0000 2024-05-14T11:15:11.521 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-14 11:08:30.869532836 +0000 2024-05-14T11:15:11.522 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-14 11:08:30.869532836 +0000 2024-05-14T11:15:11.522 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-14 11:08:30.869532836 +0000 2024-05-14T11:15:11.522 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T11:15:11.583 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-14T11:15:11.583 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-14T11:15:11.583 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000393584 s, 1.3 MB/s 2024-05-14T11:15:11.585 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T11:15:11.644 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 945 Links: 1 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-14 11:08:31.705525388 +0000 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-14 11:08:31.445527705 +0000 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-14 11:08:31.445527705 +0000 2024-05-14T11:15:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-14 11:08:31.445527705 +0000 2024-05-14T11:15:11.694 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T11:15:11.755 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-14T11:15:11.756 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-14T11:15:11.756 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000667932 s, 767 kB/s 2024-05-14T11:15:11.758 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T11:15:11.811 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:11.811 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-14T11:15:11.811 INFO:tasks.ceph:Generating config... 2024-05-14T11:15:11.812 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-14T11:15:11.812 INFO:tasks.ceph:[client] debug client = 20 2024-05-14T11:15:11.812 INFO:tasks.ceph:[client] debug ms = 1 2024-05-14T11:15:11.812 INFO:tasks.ceph:[client] pid file = 2024-05-14T11:15:11.812 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-14T11:15:11.813 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-14T11:15:11.813 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-14T11:15:11.814 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-14T11:15:11.814 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-14T11:15:11.814 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-14T11:15:11.814 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-14T11:15:11.815 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-14T11:15:11.815 INFO:tasks.ceph:Setting up mon.a... 2024-05-14T11:15:11.815 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-14T11:15:11.873 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.keyring 2024-05-14T11:15:11.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-14T11:15:11.945 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-14T11:15:11.964 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.116'), ('mon.b', '172.21.15.149'), ('mon.c', '[v2:172.21.15.149:3301,v1:172.21.15.149:6790]')] 2024-05-14T11:15:11.965 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-14T11:15:11.965 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:11.965 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-14T11:15:12.015 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.116 --add b 172.21.15.149 --addv c '[v2:172.21.15.149:3301,v1:172.21.15.149:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:12.114 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:12.114 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: generated fsid 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:15:12.114 INFO:teuthology.orchestra.run.smithi116.stdout:setting min_mon_release = quincy 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:epoch 0 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:fsid 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:last_changed 2024-05-14T11:15:12.118123+0000 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:created 2024-05-14T11:15:12.118123+0000 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:min_mon_release 17 (quincy) 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:election_strategy: 1 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:2: [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] mon.c 2024-05-14T11:15:12.115 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-14T11:15:12.118 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-14T11:15:12.126 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277... 2024-05-14T11:15:12.127 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:15:12.173 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:15:12.212 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-05-14T11:15:12.212 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-05-14T11:15:12.212 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.213 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-05-14T11:15:12.214 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.215 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:15:12.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-05-14T11:15:12.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:15:12.218 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.219 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-05-14T11:15:12.220 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:15:12.221 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:15:12.222 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-14T11:15:12.223 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-14T11:15:12.224 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-14T11:15:12.225 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-14T11:15:12.226 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-14T11:15:12.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 20 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 20 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 10 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-14T11:15:12.228 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:15:12.229 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:15:12.230 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: mon op complaint time = 120 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:15:12.231 INFO:teuthology.orchestra.run.smithi149.stdout: client mount timeout = 600 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:[mds] 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds = 20 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds balancer = 20 2024-05-14T11:15:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug frag = True 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug scatterstat = True 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: mds op complaint time = 180 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: mds verify scatter = True 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:15:12.233 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:15:12.235 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-14T11:15:12.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-14T11:15:12.344 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-14T11:15:12.344 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:12.344 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-14T11:15:12.353 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:12.353 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-14T11:15:12.406 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:15:12.406 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:12.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-14T11:15:12.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-14T11:15:12.477 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:12.477 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:12.526 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:15:12.526 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:12.527 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-14T11:15:12.527 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-14T11:15:12.561 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:12.561 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:12.613 INFO:tasks.ceph:Setting up mon nodes... 2024-05-14T11:15:12.613 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-14T11:15:12.613 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-14T11:15:12.677 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-14T11:15:12.686 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-14T11:15:12.756 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-14T11:15:12.774 INFO:tasks.ceph:Setting up mds nodes... 2024-05-14T11:15:12.774 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-14T11:15:12.839 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-14T11:15:12.844 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-14T11:15:12.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-14T11:15:12.970 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-14T11:15:12.975 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-14T11:15:12.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-14T11:15:13.063 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-14T11:15:13.068 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-14T11:15:13.088 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-14T11:15:13.191 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-14T11:15:13.196 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-14T11:15:13.214 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-14T11:15:13.215 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T11:15:13.267 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-14T11:15:13.284 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-14T11:15:13.334 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-14T11:15:13.351 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-14T11:15:13.402 INFO:teuthology.orchestra.run.smithi149.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-14T11:15:13.419 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-14T11:15:13.470 INFO:teuthology.orchestra.run.smithi149.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-14T11:15:13.488 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-14T11:15:13.488 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-14T11:15:13.488 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-14T11:15:13.507 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:13.507 INFO:tasks.ceph:role: osd.0 2024-05-14T11:15:13.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:15:13.507 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:13.577 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:13.578 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:13.975 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-05-14T11:15:13.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-05-14T11:15:13.978 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-14T11:15:14.075 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-14T11:15:14.090 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:14.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:14.091 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-14T11:15:14.154 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:14.154 INFO:tasks.ceph:role: osd.1 2024-05-14T11:15:14.154 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:15:14.154 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:14.227 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:14.626 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-05-14T11:15:14.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-05-14T11:15:14.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-14T11:15:14.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-14T11:15:14.750 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:14.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:14.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-14T11:15:14.814 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:14.814 INFO:tasks.ceph:role: osd.2 2024-05-14T11:15:14.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:15:14.814 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-14T11:15:14.881 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:15.289 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-05-14T11:15:15.292 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-05-14T11:15:15.292 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-14T11:15:15.410 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-14T11:15:15.426 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:15.428 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-14T11:15:15.491 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:15.491 INFO:tasks.ceph:role: osd.3 2024-05-14T11:15:15.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:15:15.492 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:15.564 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:15.565 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:15.933 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-05-14T11:15:15.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-05-14T11:15:15.935 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-14T11:15:16.009 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-14T11:15:16.068 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:16.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:16.069 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:16.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.199+0000 7fb1761887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-14T11:15:16.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.203+0000 7fb1761887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-14T11:15:16.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.203+0000 7fb1761887c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-14T11:15:16.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.203+0000 7fb1761887c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-14T11:15:16.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.203+0000 7fb1761887c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-14T11:15:16.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:16.203+0000 7fb1761887c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-14T11:15:19.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-14T11:15:19.280 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:19.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-14T11:15:19.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-14T11:15:19.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-14T11:15:19.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-14T11:15:19.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-14T11:15:19.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:19.387+0000 7fdeb86657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-14T11:15:22.425 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-14T11:15:22.446 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:22.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-14T11:15:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-14T11:15:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-14T11:15:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-14T11:15:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-14T11:15:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:22.555+0000 7fb7c4ce67c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-14T11:15:25.588 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-14T11:15:25.608 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:25.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.715+0000 7f5b0c6597c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-14T11:15:25.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.719+0000 7f5b0c6597c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-14T11:15:25.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.719+0000 7f5b0c6597c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-14T11:15:25.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.719+0000 7f5b0c6597c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-14T11:15:25.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.719+0000 7f5b0c6597c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-14T11:15:25.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:25.719+0000 7f5b0c6597c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-14T11:15:28.765 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-14T11:15:28.787 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-14T11:15:28.809 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:28.810 INFO:tasks.ceph:role: osd.4 2024-05-14T11:15:28.810 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:15:28.810 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:28.883 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:28.883 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:28.883 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:29.288 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-14T11:15:29.291 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-14T11:15:29.291 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-14T11:15:29.428 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-14T11:15:29.445 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:29.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:29.446 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-14T11:15:29.509 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:29.509 INFO:tasks.ceph:role: osd.5 2024-05-14T11:15:29.510 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:15:29.510 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-14T11:15:29.581 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:29.581 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:29.581 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:29.581 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:29.582 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:29.943 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-14T11:15:29.946 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-14T11:15:29.946 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-14T11:15:30.023 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-14T11:15:30.093 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:30.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:30.094 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-14T11:15:30.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:30.159 INFO:tasks.ceph:role: osd.6 2024-05-14T11:15:30.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:15:30.159 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-14T11:15:30.230 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:30.231 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:30.620 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-14T11:15:30.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-14T11:15:30.623 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-14T11:15:30.727 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-14T11:15:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:30.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:30.745 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-14T11:15:30.814 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-14T11:15:30.814 INFO:tasks.ceph:role: osd.7 2024-05-14T11:15:30.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:15:30.814 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-14T11:15:30.890 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-14T11:15:30.891 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-14T11:15:31.286 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-14T11:15:31.289 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-14T11:15:31.289 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-14T11:15:31.410 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-14T11:15:31.427 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: /sbin/restorecon: command not found 2024-05-14T11:15:31.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:15:31.428 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:31.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.569+0000 7fb9ebbfd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-14T11:15:31.576 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.573+0000 7fb9ebbfd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-14T11:15:31.576 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.573+0000 7fb9ebbfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-14T11:15:31.576 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.573+0000 7fb9ebbfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-14T11:15:31.576 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.573+0000 7fb9ebbfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-14T11:15:31.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:31.585+0000 7fb9ebbfd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-14T11:15:34.618 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-14T11:15:34.640 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:34.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.749+0000 7fa89fdb67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-14T11:15:34.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.749+0000 7fa89fdb67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-14T11:15:34.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.749+0000 7fa89fdb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-14T11:15:34.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.749+0000 7fa89fdb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-14T11:15:34.754 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.749+0000 7fa89fdb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-14T11:15:34.755 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:34.753+0000 7fa89fdb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-14T11:15:37.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-14T11:15:37.823 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:37.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.925+0000 7f5292e7e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-14T11:15:37.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.929+0000 7f5292e7e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-14T11:15:37.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.929+0000 7f5292e7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-14T11:15:37.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.929+0000 7f5292e7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-14T11:15:37.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.929+0000 7f5292e7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-14T11:15:37.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:37.929+0000 7f5292e7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-14T11:15:40.305 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:15:40.308 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:15:40.997 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-14T11:15:41.017 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:41.122 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-14T11:15:41.124 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-14T11:15:41.124 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-14T11:15:41.124 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-14T11:15:41.124 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-14T11:15:41.125 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:41.125+0000 7fe421eac7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-14T11:15:44.165 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-14T11:15:44.185 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-14T11:15:44.185 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.186 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-14T11:15:44.208 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-14T11:15:44.268 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.268 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-14T11:15:44.332 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.332 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-14T11:15:44.392 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.392 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-14T11:15:44.452 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-14T11:15:44.511 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.511 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-14T11:15:44.571 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.571 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-14T11:15:44.591 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.591 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-14T11:15:44.652 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-14T11:15:44.712 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.712 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-14T11:15:44.773 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.774 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-14T11:15:44.832 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.833 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-14T11:15:44.892 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:44.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-14T11:15:44.952 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.952 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-14T11:15:44.961 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:44.961 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-14T11:15:45.012 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:45.012 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-14T11:15:45.020 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:45.020 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-14T11:15:45.069 INFO:tasks.ceph:Adding keys to all mons... 2024-05-14T11:15:45.070 DEBUG:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-14T11:15:45.073 DEBUG:teuthology.orchestra.run.smithi149:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-14T11:15:45.113 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2024-05-14T11:15:45.113 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ== 2024-05-14T11:15:45.113 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:15:45.113 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg== 2024-05-14T11:15:45.113 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA== 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:15:45.114 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw== 2024-05-14T11:15:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:[client.1] 2024-05-14T11:15:45.116 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA== 2024-05-14T11:15:45.116 INFO:teuthology.orchestra.run.smithi116.stdout:[client.3] 2024-05-14T11:15:45.116 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g== 2024-05-14T11:15:45.125 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.y] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.0] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.1] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.2] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA== 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.3] 2024-05-14T11:15:45.126 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.x] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.4] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.5] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.6] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.7] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg== 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout:[client.0] 2024-05-14T11:15:45.127 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout:[client.2] 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw== 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA== 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout:[client.3] 2024-05-14T11:15:45.128 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g== 2024-05-14T11:15:45.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-14T11:15:45.163 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-14T11:15:45.233 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.236 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.304 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.307 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.382 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.453 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.516 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.520 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.586 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.589 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.654 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-14T11:15:45.658 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-14T11:15:45.722 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.728 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.798 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.802 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-14T11:15:45.869 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.873 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.938 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:45.942 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:46.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:46.010 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:46.075 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:46.078 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-14T11:15:46.142 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.145 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.212 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.215 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.284 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.352 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.355 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-14T11:15:46.422 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-14T11:15:46.422 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-14T11:15:46.441 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-14T11:15:46.565 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-14T11:15:46.584 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-14T11:15:46.603 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-14T11:15:46.723 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-14T11:15:46.743 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-14T11:15:46.808 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-14T11:15:46.930 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-14T11:15:46.949 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:46.952 DEBUG:teuthology.orchestra.run.smithi149:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-14T11:15:46.997 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-14T11:15:46.998 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-14T11:15:46.998 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-14T11:15:47.003 INFO:tasks.ceph.mon.a:Started 2024-05-14T11:15:47.003 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-14T11:15:47.003 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-14T11:15:47.040 INFO:tasks.ceph.mon.b:Started 2024-05-14T11:15:47.041 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-14T11:15:47.041 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-14T11:15:47.043 INFO:tasks.ceph.mon.c:Started 2024-05-14T11:15:47.044 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-14T11:15:47.044 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-14T11:15:47.044 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-14T11:15:47.047 INFO:tasks.ceph.mgr.y:Started 2024-05-14T11:15:47.047 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-14T11:15:47.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-14T11:15:47.050 INFO:tasks.ceph.mgr.x:Started 2024-05-14T11:15:47.050 INFO:tasks.ceph:Setting crush tunables to default 2024-05-14T11:15:47.050 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd crush tunables default 2024-05-14T11:15:47.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 Processor -- start 2024-05-14T11:15:47.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 -- start start 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8aa80ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa80cec10 con 0x7f8aa80ce800 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa80d1d40 con 0x7f8aa81621b0 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aaf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa80d1f90 con 0x7f8aa80b9020 2024-05-14T11:15:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56298/0 (socket says 172.21.15.116:56298) 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 learned_addr learned my addr 172.21.15.116:0/1664801948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 msgr2=0x7f8aa80d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 msgr2=0x7f8aa80d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8a9f400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8aa80ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8a9f400640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 msgr2=0x7f8aa80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8a9f400640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 msgr2=0x7f8aa80ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8a9f400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8aa80ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T11:15:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.202+0000 7f8a9f400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8aa80ce2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T11:15:47.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41066509 0 0) 0x7f8aa00014a0 con 0x7f8aa80ce800 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a8c003530 con 0x7f8aa80ce800 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887917744 0 0) 0x7f8a8c003530 con 0x7f8aa80ce800 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 msgr2=0x7f8aa80ce2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8aa80ce2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 msgr2=0x7f8aa80d2d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 --2- 172.21.15.116:0/1664801948 >> v2:172.21.15.116:3300/0 conn(0x7f8aa81621b0 0x7f8aa80d2d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa80d2270 con 0x7f8aa80ce800 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f8aa0001c50 con 0x7f8aa80ce800 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 >> v1:172.21.15.149:6789/0 conn(0x7f8aa80ce800 legacy=0x7f8aa80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8a8c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8a8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a8c004d50 0x7f8a8c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80d1d40 con 0x7f8a8c003a10 2024-05-14T11:15:47.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80d1f90 con 0x7f8a8c004d50 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8c00ffc0 con 0x7f8aa80b9020 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8a8c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aad000640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a8c004d50 0x7f8a8c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8a9f400640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8a8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 msgr2=0x7f8a8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:47.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa80b9020 0x7f8a8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a8c004d50 msgr2=0x7f8a8c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:47.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a8c004d50 0x7f8a8c0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aa7e00640 1 -- 172.21.15.116:0/1664801948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c003530 con 0x7f8a8c003a10 2024-05-14T11:15:47.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.362+0000 7f8aad000640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a8c004d50 0x7f8a8c0051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:47.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8a8c003e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009880 tx=0x7f8a9801c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f345f1119c976f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:47.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800f040 con 0x7f8a8c003a10 2024-05-14T11:15:47.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a980b0070 con 0x7f8a8c003a10 2024-05-14T11:15:47.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aa7400640 1 -- 172.21.15.116:0/1664801948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980ab860 con 0x7f8a8c003a10 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 -- 172.21.15.116:0/1664801948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 msgr2=0x7f8a8c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 --2- 172.21.15.116:0/1664801948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8a8c003e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009880 tx=0x7f8a9801c5f0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aad000640 1 -- 172.21.15.116:0/1664801948 reap_dead start 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 -- 172.21.15.116:0/1664801948 shutdown_connections 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 -- 172.21.15.116:0/1664801948 >> 172.21.15.116:0/1664801948 conn(0x7f8aa80d0770 msgr2=0x7f8aa80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 -- 172.21.15.116:0/1664801948 shutdown_connections 2024-05-14T11:15:47.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.366+0000 7f8aaf000640 1 -- 172.21.15.116:0/1664801948 wait complete. 2024-05-14T11:15:47.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.370+0000 7f8aaf000640 1 Processor -- start 2024-05-14T11:15:47.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.370+0000 7f8aaf000640 1 -- start start 2024-05-14T11:15:47.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.370+0000 7f8aaf000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa814cb40 0x7f8aa814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa8150110 0x7f8aa81508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80b3bd0 con 0x7f8a8c003a10 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80b3a50 con 0x7f8aa8150110 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa815ea80 con 0x7f8aa814cb40 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56332/0 (socket says 172.21.15.116:56332) 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 -- 172.21.15.116:0/3765410140 learned_addr learned my addr 172.21.15.116:0/3765410140 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8a9f400640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa814cb40 0x7f8aa814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aad000640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa8150110 0x7f8aa81508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 -- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa814cb40 msgr2=0x7f8aa814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa814cb40 0x7f8aa814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 -- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa8150110 msgr2=0x7f8aa81508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa8150110 0x7f8aa81508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 -- 172.21.15.116:0/3765410140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa8150520 con 0x7f8a8c003a10 2024-05-14T11:15:47.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa7e00640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a9801c420 tx=0x7f8a8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:47.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980042c0 con 0x7f8a8c003a10 2024-05-14T11:15:47.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a98004480 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98003860 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa814f5c0 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa815adc0 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a980bf430 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8a980ab500 con 0x7f8a8c003a10 2024-05-14T11:15:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.374+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa80cfb70 con 0x7f8a8c003a10 2024-05-14T11:15:47.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.386+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8aa80cfb70 con 0x7f8a8c003a10 2024-05-14T11:15:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8aa806a9c0 con 0x7f8a8c003a10 2024-05-14T11:15:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aa4c00640 1 -- 172.21.15.116:0/3765410140 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8a980c5060 con 0x7f8a8c003a10 2024-05-14T11:15:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:adjusted tunables profile to default 2024-05-14T11:15:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 msgr2=0x7f8aa80b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a9801c420 tx=0x7f8a8c005840 comp rx=0 tx=0).stop 2024-05-14T11:15:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 shutdown_connections 2024-05-14T11:15:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa8150110 0x7f8aa81508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8aa814cb40 0x7f8aa814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 --2- 172.21.15.116:0/3765410140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a8c003a10 0x7f8aa80b7d90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 >> 172.21.15.116:0/3765410140 conn(0x7f8aa80d0770 msgr2=0x7f8aa80686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 shutdown_connections 2024-05-14T11:15:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:47.450+0000 7f8aaf000640 1 -- 172.21.15.116:0/3765410140 wait complete. 2024-05-14T11:15:47.472 INFO:tasks.ceph:check_enable_crimson: False 2024-05-14T11:15:47.472 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-14T11:15:47.472 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:47.472 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-14T11:15:47.493 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:47.493 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-14T11:15:47.560 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:47.560 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-14T11:15:47.569 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:47.570+0000 7fcf900cd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T11:15:47.582 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:47.584+0000 7f5fb5ec8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T11:15:47.589 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:15:47.589 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-14T11:15:47.652 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:47.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-14T11:15:47.672 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:47.672 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-14T11:15:47.733 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:47.733 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-14T11:15:47.793 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:15:47.793 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-14T11:15:47.853 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new b1924236-7cff-4e7b-a5d4-b5cb84beb123 0 2024-05-14T11:15:48.040 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 Processor -- start 2024-05-14T11:15:48.040 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 -- start start 2024-05-14T11:15:48.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3cfc06d710 0x7f3cfc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.042 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc0705b0 0x7f3cfc0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cfc0d1490 con 0x7f3cfc06fd60 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3cfc15cfd0 con 0x7f3cfc06d710 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3d01000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc06e060 con 0x7f3cfc0705b0 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3cf9400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3cfc06d710 0x7f3cfc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3cf9400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3cfc06d710 0x7f3cfc06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47870/0 (socket says 172.21.15.149:47870) 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3cf9400640 1 -- 172.21.15.149:0/3641073351 learned_addr learned my addr 172.21.15.149:0/3641073351 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.040+0000 7f3cfa800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc0705b0 0x7f3cfc0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf9400640 1 -- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc0705b0 msgr2=0x7f3cfc0709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf8a00640 1 -- 172.21.15.149:0/3641073351 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935298364 0 0) 0x7f3cfc0d1490 con 0x7f3cfc06fd60 2024-05-14T11:15:48.043 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf9400640 1 --2- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc0705b0 0x7f3cfc0709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf9400640 1 -- 172.21.15.149:0/3641073351 >> v1:172.21.15.116:6789/0 conn(0x7f3cfc06fd60 legacy=0x7f3cfc15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.044 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf9400640 1 -- 172.21.15.149:0/3641073351 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc070170 con 0x7f3cfc06d710 2024-05-14T11:15:48.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf9400640 1 --2- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc06d710 0x7f3cfc06db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800bb10 tx=0x7f3ce801eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511a67c6f4be2e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.045 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf8a00640 1 -- 172.21.15.149:0/3641073351 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80b4070 con 0x7f3cfc06d710 2024-05-14T11:15:48.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf8a00640 1 -- 172.21.15.149:0/3641073351 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce80a7c70 con 0x7f3cfc06d710 2024-05-14T11:15:48.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3cf8a00640 1 -- 172.21.15.149:0/3641073351 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80af660 con 0x7f3cfc06d710 2024-05-14T11:15:48.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3d01000640 1 -- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc06d710 msgr2=0x7f3cfc06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.044+0000 7f3d01000640 1 --2- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc06d710 0x7f3cfc06db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800bb10 tx=0x7f3ce801eba0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 -- 172.21.15.149:0/3641073351 shutdown_connections 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 --2- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc0705b0 0x7f3cfc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 --2- 172.21.15.149:0/3641073351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc06d710 0x7f3cfc06db20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 -- 172.21.15.149:0/3641073351 >> 172.21.15.149:0/3641073351 conn(0x7f3cfc0d1e50 msgr2=0x7f3cfc072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 -- 172.21.15.149:0/3641073351 shutdown_connections 2024-05-14T11:15:48.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 -- 172.21.15.149:0/3641073351 wait complete. 2024-05-14T11:15:48.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 Processor -- start 2024-05-14T11:15:48.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.048+0000 7f3d01000640 1 -- start start 2024-05-14T11:15:48.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cfc06fd60 0x7f3cfc14adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 0x7f3cfc14b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc0b47b0 con 0x7f3cfc06fd60 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc15dab0 con 0x7f3cfc0705b0 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc0b4930 con 0x7f3cfc06d710 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cfa800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 0x7f3cfc14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:40852/0 (socket says 172.21.15.149:40852) 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 -- 172.21.15.149:0/1064471188 learned_addr learned my addr 172.21.15.149:0/1064471188 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9400640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cfc06fd60 0x7f3cfc14adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 -- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 msgr2=0x7f3cfc14b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 0x7f3cfc14b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 -- 172.21.15.149:0/1064471188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cfc06fd60 msgr2=0x7f3cfc14adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cfc06fd60 0x7f3cfc14adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 -- 172.21.15.149:0/1064471188 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc14bbc0 con 0x7f3cfc06d710 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cfa800640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 0x7f3cfc14b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf9e00640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c3c0 tx=0x7f3cec00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec017070 con 0x7f3cfc06d710 2024-05-14T11:15:48.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cec003600 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0039f0 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc148b00 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc1490c0 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3cec01a070 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f3cec012740 con 0x7f3cfc06d710 2024-05-14T11:15:48.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.052+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc06a9c0 con 0x7f3cfc06d710 2024-05-14T11:15:48.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.056+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3cec021020 con 0x7f3cfc06d710 2024-05-14T11:15:48.072 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:48.070+0000 7fcf900cd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T11:15:48.097 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:48.096+0000 7f5fb5ec8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T11:15:48.122 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.120+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b1924236-7cff-4e7b-a5d4-b5cb84beb123", "id": 0} v 0) -- 0x7f3cfc0cf220 con 0x7f3cfc06d710 2024-05-14T11:15:48.127 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3cf2000640 1 -- 172.21.15.149:0/1064471188 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1924236-7cff-4e7b-a5d4-b5cb84beb123", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f3cec003c60 con 0x7f3cfc06d710 2024-05-14T11:15:48.127 INFO:teuthology.orchestra.run.smithi149.stdout:0 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 msgr2=0x7f3cfc1f8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c3c0 tx=0x7f3cec00c880 comp rx=0 tx=0).stop 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 shutdown_connections 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cfc0705b0 0x7f3cfc14b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cfc06fd60 0x7f3cfc14adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 --2- 172.21.15.149:0/1064471188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3cfc06d710 0x7f3cfc1f8910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 >> 172.21.15.149:0/1064471188 conn(0x7f3cfc0d1e50 msgr2=0x7f3cfc0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 shutdown_connections 2024-05-14T11:15:48.128 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.128+0000 7f3d01000640 1 -- 172.21.15.149:0/1064471188 wait complete. 2024-05-14T11:15:48.147 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 9f040550-c15e-418b-aaaa-27267ad5a416 1 2024-05-14T11:15:48.293 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:48.294+0000 7fcf900cd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T11:15:48.294 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.292+0000 7fbf37e00640 1 Processor -- start 2024-05-14T11:15:48.294 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.292+0000 7fbf37e00640 1 -- start start 2024-05-14T11:15:48.296 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf300ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.296 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 0x7fbf300b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbf300d1e60 con 0x7fbf300b40c0 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fbf300b4a10 con 0x7fbf300d1510 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf37e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbf300b3060 con 0x7fbf300cdf80 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf34a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf300ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf34a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47894/0 (socket says 172.21.15.149:47894) 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf34a00640 1 -- 172.21.15.149:0/2058874027 learned_addr learned my addr 172.21.15.149:0/2058874027 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35e00640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 0x7fbf300b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 -- 172.21.15.149:0/2058874027 >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 msgr2=0x7fbf300d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 -- 172.21.15.149:0/2058874027 >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 msgr2=0x7fbf300b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 0x7fbf300b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35400640 1 -- 172.21.15.149:0/2058874027 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf300b3300 con 0x7fbf300cdf80 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf34a00640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.297 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.296+0000 7fbf35e00640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 0x7fbf300b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:48.299 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf35400640 1 --2- 172.21.15.149:0/2058874027 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf300ce390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf300b5060 tx=0x7fbf2401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53a97a31e8ba953 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf2fe00640 1 -- 172.21.15.149:0/2058874027 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240ab610 con 0x7fbf300cdf80 2024-05-14T11:15:48.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf2fe00640 1 -- 172.21.15.149:0/2058874027 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf240b0070 con 0x7fbf300cdf80 2024-05-14T11:15:48.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf2fe00640 1 -- 172.21.15.149:0/2058874027 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240ab9d0 con 0x7fbf300cdf80 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 -- 172.21.15.149:0/2058874027 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 msgr2=0x7fbf300ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 --2- 172.21.15.149:0/2058874027 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf300ce390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf300b5060 tx=0x7fbf2401c450 comp rx=0 tx=0).stop 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 -- 172.21.15.149:0/2058874027 shutdown_connections 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.116:3300/0 conn(0x7fbf300b40c0 0x7fbf300b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 --2- 172.21.15.149:0/2058874027 >> v2:172.21.15.149:3300/0 conn(0x7fbf300d1510 0x7fbf300d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 --2- 172.21.15.149:0/2058874027 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf300ce390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 -- 172.21.15.149:0/2058874027 >> 172.21.15.149:0/2058874027 conn(0x7fbf300d0770 msgr2=0x7fbf300d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 -- 172.21.15.149:0/2058874027 shutdown_connections 2024-05-14T11:15:48.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.300+0000 7fbf37e00640 1 -- 172.21.15.149:0/2058874027 wait complete. 2024-05-14T11:15:48.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 Processor -- start 2024-05-14T11:15:48.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- start start 2024-05-14T11:15:48.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300b40c0 0x7fbf3014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf30152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.305 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30160320 con 0x7fbf300b40c0 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf301601a0 con 0x7fbf300d1510 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3006fdc0 con 0x7fbf300cdf80 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf30152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47896/0 (socket says 172.21.15.149:47896) 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf35e00640 1 -- 172.21.15.149:0/565871776 learned_addr learned my addr 172.21.15.149:0/565871776 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf35400640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300b40c0 0x7fbf3014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 -- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 msgr2=0x7fbf30157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 -- 172.21.15.149:0/565871776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300b40c0 msgr2=0x7fbf3014b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300b40c0 0x7fbf3014b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 -- 172.21.15.149:0/565871776 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301fd480 con 0x7fbf300cdf80 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf35e00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.306 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf34a00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf30152480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf30163940 tx=0x7fbf1c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.307 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004030 con 0x7fbf300cdf80 2024-05-14T11:15:48.307 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0041d0 con 0x7fbf300cdf80 2024-05-14T11:15:48.307 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00c940 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf301fc900 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.304+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301fcec0 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.308+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf1c0059a0 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.308+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbf1c006020 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.308+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3006a9c0 con 0x7fbf300cdf80 2024-05-14T11:15:48.308 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:48.308+0000 7f5fb5ec8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T11:15:48.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.312+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbf1c017020 con 0x7fbf300cdf80 2024-05-14T11:15:48.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.380+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9f040550-c15e-418b-aaaa-27267ad5a416", "id": 1} v 0) -- 0x7fbf300b9590 con 0x7fbf300cdf80 2024-05-14T11:15:48.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf2d600640 1 -- 172.21.15.149:0/565871776 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f040550-c15e-418b-aaaa-27267ad5a416", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbf1c005e20 con 0x7fbf300cdf80 2024-05-14T11:15:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-05-14T11:15:48.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 msgr2=0x7fbf30152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf30152480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbf30163940 tx=0x7fbf1c00ed40 comp rx=0 tx=0).stop 2024-05-14T11:15:48.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 shutdown_connections 2024-05-14T11:15:48.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf300d1510 0x7fbf30157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf300cdf80 0x7fbf30152480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 --2- 172.21.15.149:0/565871776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf300b40c0 0x7fbf3014b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 >> 172.21.15.149:0/565871776 conn(0x7fbf300d0770 msgr2=0x7fbf3006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 shutdown_connections 2024-05-14T11:15:48.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.388+0000 7fbf37e00640 1 -- 172.21.15.149:0/565871776 wait complete. 2024-05-14T11:15:48.407 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new b023f361-28b5-43af-96fe-d518ea584db2 2 2024-05-14T11:15:48.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.552+0000 7f31ffe00640 1 Processor -- start 2024-05-14T11:15:48.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.552+0000 7f31ffe00640 1 -- start start 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31ffe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 0x7f31f80d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31ffe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 0x7f31f80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31ffe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f80d1e60 con 0x7f31f8160080 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31ffe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f31f80b2f40 con 0x7f31f80b3fe0 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31ffe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f31f80b3220 con 0x7f31f80d1510 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fde00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 0x7f31f80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 0x7f31f80d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fd400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f31f8160080 0x7f31f80cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.149:47424/0 (socket says 172.21.15.149:47424) 2024-05-14T11:15:48.556 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fd400640 1 -- 172.21.15.149:0/1245028568 learned_addr learned my addr 172.21.15.149:0/1245028568 (peer_addr_for_me v1:172.21.15.149:0/0) 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fca00640 1 -- 172.21.15.149:0/1245028568 >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 msgr2=0x7f31f80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fca00640 1 --2- 172.21.15.149:0/1245028568 >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 0x7f31f80b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fca00640 1 -- 172.21.15.149:0/1245028568 >> v1:172.21.15.149:6789/0 conn(0x7f31f8160080 legacy=0x7f31f80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fca00640 1 -- 172.21.15.149:0/1245028568 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f80b3500 con 0x7f31f80d1510 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31fde00640 1 --2- 172.21.15.149:0/1245028568 >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 0x7f31f80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.557 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.556+0000 7f31f7e00640 1 -- 172.21.15.149:0/1245028568 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099955708 0 0) 0x7f31ec001120 con 0x7f31f8160080 2024-05-14T11:15:48.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31fca00640 1 --2- 172.21.15.149:0/1245028568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 0x7f31f80d1920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097c0 tx=0x7f31e401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25610dd1784bb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31f7e00640 1 -- 172.21.15.149:0/1245028568 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40b0070 con 0x7f31f80d1510 2024-05-14T11:15:48.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31f7e00640 1 -- 172.21.15.149:0/1245028568 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4004230 con 0x7f31f80d1510 2024-05-14T11:15:48.559 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31f7e00640 1 -- 172.21.15.149:0/1245028568 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4003620 con 0x7f31f80d1510 2024-05-14T11:15:48.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 -- 172.21.15.149:0/1245028568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 msgr2=0x7f31f80d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 --2- 172.21.15.149:0/1245028568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 0x7f31f80d1920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097c0 tx=0x7f31e401c350 comp rx=0 tx=0).stop 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 -- 172.21.15.149:0/1245028568 shutdown_connections 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 --2- 172.21.15.149:0/1245028568 >> v2:172.21.15.149:3300/0 conn(0x7f31f80b3fe0 0x7f31f80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 --2- 172.21.15.149:0/1245028568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80d1510 0x7f31f80d1920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 -- 172.21.15.149:0/1245028568 >> 172.21.15.149:0/1245028568 conn(0x7f31f80d0770 msgr2=0x7f31f80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 -- 172.21.15.149:0/1245028568 shutdown_connections 2024-05-14T11:15:48.561 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.560+0000 7f31ffe00640 1 -- 172.21.15.149:0/1245028568 wait complete. 2024-05-14T11:15:48.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 Processor -- start 2024-05-14T11:15:48.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- start start 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 0x7f31f80c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31f8160080 0x7f31f80c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31f806fdd0 con 0x7f31f8160080 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f31f80ce690 con 0x7f31f80d1510 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f31f806ff50 con 0x7f31f80b3fe0 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 0x7f31f80c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:40886/0 (socket says 172.21.15.149:40886) 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 -- 172.21.15.149:0/4148038619 learned_addr learned my addr 172.21.15.149:0/4148038619 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fde00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31f8160080 0x7f31f80c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 -- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 msgr2=0x7f31f80c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 0x7f31f80c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 -- 172.21.15.149:0/4148038619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31f8160080 msgr2=0x7f31f80c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31f8160080 0x7f31f80c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 -- 172.21.15.149:0/4148038619 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f8152c30 con 0x7f31f80b3fe0 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fca00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 0x7f31f80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31fd400640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c430 tx=0x7f31ec00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0074e0 con 0x7f31f80b3fe0 2024-05-14T11:15:48.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec0047f0 con 0x7f31f80b3fe0 2024-05-14T11:15:48.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f8152f10 con 0x7f31f80b3fe0 2024-05-14T11:15:48.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007680 con 0x7f31f80b3fe0 2024-05-14T11:15:48.567 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f80c9410 con 0x7f31f80b3fe0 2024-05-14T11:15:48.568 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31ec011070 con 0x7f31f80b3fe0 2024-05-14T11:15:48.568 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.564+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f31ec0137e0 con 0x7f31f80b3fe0 2024-05-14T11:15:48.568 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.568+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f806a9c0 con 0x7f31f80b3fe0 2024-05-14T11:15:48.569 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.568+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f31f80d1e60 con 0x7f31f80b3fe0 2024-05-14T11:15:48.636 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.636+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b023f361-28b5-43af-96fe-d518ea584db2", "id": 2} v 0) -- 0x7f31f80b9590 con 0x7f31f80b3fe0 2024-05-14T11:15:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31f5600640 1 -- 172.21.15.149:0/4148038619 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b023f361-28b5-43af-96fe-d518ea584db2", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f31ec004ce0 con 0x7f31f80b3fe0 2024-05-14T11:15:48.642 INFO:teuthology.orchestra.run.smithi149.stdout:2 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 msgr2=0x7f31f8154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c430 tx=0x7f31ec00c830 comp rx=0 tx=0).stop 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 shutdown_connections 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31f80d1510 0x7f31f80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31f8160080 0x7f31f80c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 --2- 172.21.15.149:0/4148038619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f31f80b3fe0 0x7f31f8154fb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 >> 172.21.15.149:0/4148038619 conn(0x7f31f80d0770 msgr2=0x7f31f81568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 shutdown_connections 2024-05-14T11:15:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.644+0000 7f31ffe00640 1 -- 172.21.15.149:0/4148038619 wait complete. 2024-05-14T11:15:48.663 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 062cdb74-9d77-4afb-ba9b-ad53cb5be49a 3 2024-05-14T11:15:48.810 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.808+0000 7fab5fe00640 1 Processor -- start 2024-05-14T11:15:48.810 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.808+0000 7fab5fe00640 1 -- start start 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab58154ae0 0x7fab580ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fab580ce900 0x7fab580c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab580ced10 con 0x7fab580c8900 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fab580cdd40 con 0x7fab58154ae0 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fab580cd130 con 0x7fab580ce900 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fab580ce900 0x7fab580c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fab580ce900 0x7fab580c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47932/0 (socket says 172.21.15.149:47932) 2024-05-14T11:15:48.812 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 -- 172.21.15.149:0/3274394620 learned_addr learned my addr 172.21.15.149:0/3274394620 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 -- 172.21.15.149:0/3274394620 >> v2:172.21.15.116:3300/0 conn(0x7fab58154ae0 msgr2=0x7fab580ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 --2- 172.21.15.149:0/3274394620 >> v2:172.21.15.116:3300/0 conn(0x7fab58154ae0 0x7fab580ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 -- 172.21.15.149:0/3274394620 >> v1:172.21.15.116:6789/0 conn(0x7fab580c8900 legacy=0x7fab58161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:48.813 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.812+0000 7fab5ca00640 1 -- 172.21.15.149:0/3274394620 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab580cd410 con 0x7fab580ce900 2024-05-14T11:15:48.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5ca00640 1 --2- 172.21.15.149:0/3274394620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab580ce900 0x7fab580c83c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fab5815e130 tx=0x7fab4401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da254f05a63c59fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab57e00640 1 -- 172.21.15.149:0/3274394620 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440b0070 con 0x7fab580ce900 2024-05-14T11:15:48.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab57e00640 1 -- 172.21.15.149:0/3274394620 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab440ab600 con 0x7fab580ce900 2024-05-14T11:15:48.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab57e00640 1 -- 172.21.15.149:0/3274394620 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440ab9d0 con 0x7fab580ce900 2024-05-14T11:15:48.816 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 -- 172.21.15.149:0/3274394620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab580ce900 msgr2=0x7fab580c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.816 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 --2- 172.21.15.149:0/3274394620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab580ce900 0x7fab580c83c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fab5815e130 tx=0x7fab4401c4b0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 -- 172.21.15.149:0/3274394620 shutdown_connections 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 --2- 172.21.15.149:0/3274394620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab580ce900 0x7fab580c83c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 --2- 172.21.15.149:0/3274394620 >> v2:172.21.15.116:3300/0 conn(0x7fab58154ae0 0x7fab580ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 -- 172.21.15.149:0/3274394620 >> 172.21.15.149:0/3274394620 conn(0x7fab580712c0 msgr2=0x7fab580716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 -- 172.21.15.149:0/3274394620 shutdown_connections 2024-05-14T11:15:48.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.816+0000 7fab5fe00640 1 -- 172.21.15.149:0/3274394620 wait complete. 2024-05-14T11:15:48.819 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 Processor -- start 2024-05-14T11:15:48.819 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- start start 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab580ce900 0x7fab580ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab5815f150 con 0x7fab580ce900 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab5815efd0 con 0x7fab58154ae0 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fab580b5590 con 0x7fab580c8900 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47948/0 (socket says 172.21.15.149:47948) 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 -- 172.21.15.149:0/73508394 learned_addr learned my addr 172.21.15.149:0/73508394 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:40896/0 (socket says 172.21.15.149:40896) 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5ca00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab580ce900 0x7fab580ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 -- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 msgr2=0x7fab581540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 -- 172.21.15.149:0/73508394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab580ce900 msgr2=0x7fab580ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab580ce900 0x7fab580ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 -- 172.21.15.149:0/73508394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab580cae80 con 0x7fab58154ae0 2024-05-14T11:15:48.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5d400640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:48.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5de00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b990 tx=0x7fab5000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:48.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50019070 con 0x7fab58154ae0 2024-05-14T11:15:48.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5000cba0 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab580c98a0 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50014440 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab580c9e60 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.820+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab5001c070 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.824+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fab5001d840 con 0x7fab58154ae0 2024-05-14T11:15:48.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.824+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5806a9c0 con 0x7fab58154ae0 2024-05-14T11:15:48.826 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.824+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fab50023070 con 0x7fab58154ae0 2024-05-14T11:15:48.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.892+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "062cdb74-9d77-4afb-ba9b-ad53cb5be49a", "id": 3} v 0) -- 0x7fab580b2730 con 0x7fab58154ae0 2024-05-14T11:15:48.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab55600640 1 -- 172.21.15.149:0/73508394 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "062cdb74-9d77-4afb-ba9b-ad53cb5be49a", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fab5001d520 con 0x7fab58154ae0 2024-05-14T11:15:48.897 INFO:teuthology.orchestra.run.smithi149.stdout:3 2024-05-14T11:15:48.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 msgr2=0x7fab581f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:48.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab5fe00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b990 tx=0x7fab5000be50 comp rx=0 tx=0).stop 2024-05-14T11:15:48.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 shutdown_connections 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab5fe00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58154ae0 0x7fab581f9db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.896+0000 7fab5fe00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab580ce900 0x7fab580ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.900+0000 7fab5fe00640 1 --2- 172.21.15.149:0/73508394 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fab580c8900 0x7fab581540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.900+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 >> 172.21.15.149:0/73508394 conn(0x7fab580712c0 msgr2=0x7fab5815e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.900+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 shutdown_connections 2024-05-14T11:15:48.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:48.900+0000 7fab5fe00640 1 -- 172.21.15.149:0/73508394 wait complete. 2024-05-14T11:15:48.917 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new a42da358-4ee2-4bd3-9203-bcd1ad5ac93a 4 2024-05-14T11:15:48.930 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:48.930+0000 7fcf900cd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T11:15:48.949 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:48.948+0000 7f5fb5ec8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T11:15:48.992 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:48.990+0000 7fcf900cd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T11:15:49.016 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.016+0000 7f5fb5ec8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T11:15:49.052 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.050+0000 7fcf900cd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T11:15:49.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.064+0000 7f194b200640 1 Processor -- start 2024-05-14T11:15:49.066 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.064+0000 7f194b200640 1 -- start start 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194b200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f19440b6400 0x7f19440b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194b200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1944146d30 0x7f1944147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194b200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19440b6d50 con 0x7f19440b07e0 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194b200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f19440b1300 con 0x7f1944146d30 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194b200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f19440b1560 con 0x7f19440b6400 2024-05-14T11:15:49.067 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f19440b6400 0x7f19440b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f19440b6400 0x7f19440b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47964/0 (socket says 172.21.15.149:47964) 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 -- 172.21.15.149:0/63758420 learned_addr learned my addr 172.21.15.149:0/63758420 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f194a800640 1 --2- 172.21.15.149:0/63758420 >> v2:172.21.15.116:3300/0 conn(0x7f1944146d30 0x7f1944147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 -- 172.21.15.149:0/63758420 >> v2:172.21.15.116:3300/0 conn(0x7f1944146d30 msgr2=0x7f1944147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 --2- 172.21.15.149:0/63758420 >> v2:172.21.15.116:3300/0 conn(0x7f1944146d30 0x7f1944147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 -- 172.21.15.149:0/63758420 >> v1:172.21.15.116:6789/0 conn(0x7f19440b07e0 legacy=0x7f19440b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.068 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.068+0000 7f1949e00640 1 -- 172.21.15.149:0/63758420 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440b1840 con 0x7f19440b6400 2024-05-14T11:15:49.071 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f1949e00640 1 --2- 172.21.15.149:0/63758420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f19440b6810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1944074950 tx=0x7f193801c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115c23412aacc73f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.071 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f1948a00640 1 -- 172.21.15.149:0/63758420 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380b2070 con 0x7f19440b6400 2024-05-14T11:15:49.071 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f1948a00640 1 -- 172.21.15.149:0/63758420 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19380ad680 con 0x7f19440b6400 2024-05-14T11:15:49.071 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f1948a00640 1 -- 172.21.15.149:0/63758420 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380ada50 con 0x7f19440b6400 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- 172.21.15.149:0/63758420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 msgr2=0x7f19440b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 --2- 172.21.15.149:0/63758420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f19440b6810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1944074950 tx=0x7f193801c490 comp rx=0 tx=0).stop 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- 172.21.15.149:0/63758420 shutdown_connections 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 --2- 172.21.15.149:0/63758420 >> v2:172.21.15.116:3300/0 conn(0x7f1944146d30 0x7f1944147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 --2- 172.21.15.149:0/63758420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f19440b6810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- 172.21.15.149:0/63758420 >> 172.21.15.149:0/63758420 conn(0x7f1944072050 msgr2=0x7f1944073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- 172.21.15.149:0/63758420 shutdown_connections 2024-05-14T11:15:49.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- 172.21.15.149:0/63758420 wait complete. 2024-05-14T11:15:49.074 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 Processor -- start 2024-05-14T11:15:49.074 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.072+0000 7f194b200640 1 -- start start 2024-05-14T11:15:49.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1944146d30 0x7f19441611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f194416c790 con 0x7f1944146d30 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19440d2060 con 0x7f19440b6400 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f194416c910 con 0x7f19440b07e0 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:40902/0 (socket says 172.21.15.149:40902) 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949e00640 1 -- 172.21.15.149:0/1052824619 learned_addr learned my addr 172.21.15.149:0/1052824619 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47968/0 (socket says 172.21.15.149:47968) 2024-05-14T11:15:49.077 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194a800640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1944146d30 0x7f19441611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 -- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 msgr2=0x7f194415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 -- 172.21.15.149:0/1052824619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1944146d30 msgr2=0x7f19441611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1944146d30 0x7f19441611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 -- 172.21.15.149:0/1052824619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194415b080 con 0x7f19440b6400 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949e00640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:49.078 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f1949400640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19440b53c0 tx=0x7f193400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.079 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934017070 con 0x7f19440b6400 2024-05-14T11:15:49.079 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19340052d0 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19440ceab0 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934003800 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19440cf070 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f193401a070 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1934011590 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.076+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194406e800 con 0x7f19440b6400 2024-05-14T11:15:49.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.080+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f19440b6d50 con 0x7f19440b6400 2024-05-14T11:15:49.081 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.080+0000 7f5fb5ec8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T11:15:49.141 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.142+0000 7fcf900cd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T11:15:49.148 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.148+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a42da358-4ee2-4bd3-9203-bcd1ad5ac93a", "id": 4} v 0) -- 0x7f194406a9c0 con 0x7f19440b6400 2024-05-14T11:15:49.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.152+0000 7f193e000640 1 -- 172.21.15.149:0/1052824619 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a42da358-4ee2-4bd3-9203-bcd1ad5ac93a", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f19340119f0 con 0x7f19440b6400 2024-05-14T11:15:49.154 INFO:teuthology.orchestra.run.smithi149.stdout:4 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 msgr2=0x7f194406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19440b53c0 tx=0x7f193400c820 comp rx=0 tx=0).stop 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 shutdown_connections 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19440b6400 0x7f194406d930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1944146d30 0x7f19441611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 --2- 172.21.15.149:0/1052824619 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f19440b07e0 0x7f194415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 >> 172.21.15.149:0/1052824619 conn(0x7f1944072050 msgr2=0x7f19440b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 shutdown_connections 2024-05-14T11:15:49.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.156+0000 7f194b200640 1 -- 172.21.15.149:0/1052824619 wait complete. 2024-05-14T11:15:49.173 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.172+0000 7f5fb5ec8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T11:15:49.174 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new f6f87186-b4aa-4e02-a8ab-e048753ba149 5 2024-05-14T11:15:49.202 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.202+0000 7fcf900cd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T11:15:49.236 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.236+0000 7f5fb5ec8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T11:15:49.290 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.290+0000 7fcf900cd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T11:15:49.331 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.332+0000 7f5fb5ec8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T11:15:49.353 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.354+0000 7fcf900cd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T11:15:49.370 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.368+0000 7fa787e00640 1 Processor -- start 2024-05-14T11:15:49.370 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.368+0000 7fa787e00640 1 -- start start 2024-05-14T11:15:49.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa787e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 0x7fa7800ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa787e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa7800b40c0 0x7fa7800b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa787e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7800b4a10 con 0x7fa7800cdf80 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa787e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa7800b3060 con 0x7fa7800b40c0 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa787e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa7800b3300 con 0x7fa7800d1c20 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa784a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 0x7fa7800ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa784a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 0x7fa7800ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47974/0 (socket says 172.21.15.149:47974) 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa784a00640 1 -- 172.21.15.149:0/4125001340 learned_addr learned my addr 172.21.15.149:0/4125001340 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa785e00640 1 --2- 172.21.15.149:0/4125001340 >> v2:172.21.15.116:3300/0 conn(0x7fa7800b40c0 0x7fa7800b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa785e00640 1 -- 172.21.15.149:0/4125001340 >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 msgr2=0x7fa7800ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa785e00640 1 --2- 172.21.15.149:0/4125001340 >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 0x7fa7800ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa785e00640 1 -- 172.21.15.149:0/4125001340 >> v1:172.21.15.116:6789/0 conn(0x7fa7800cdf80 legacy=0x7fa7800d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.372+0000 7fa785e00640 1 -- 172.21.15.149:0/4125001340 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7800b35e0 con 0x7fa7800b40c0 2024-05-14T11:15:49.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa785e00640 1 --2- 172.21.15.149:0/4125001340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800b40c0 0x7fa7800b44d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa780163940 tx=0x7fa77801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed28e9637d4526aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa77fe00640 1 -- 172.21.15.149:0/4125001340 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780b2070 con 0x7fa7800b40c0 2024-05-14T11:15:49.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa77fe00640 1 -- 172.21.15.149:0/4125001340 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7780a59c0 con 0x7fa7800b40c0 2024-05-14T11:15:49.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa77fe00640 1 -- 172.21.15.149:0/4125001340 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780a5d90 con 0x7fa7800b40c0 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 -- 172.21.15.149:0/4125001340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800b40c0 msgr2=0x7fa7800b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 --2- 172.21.15.149:0/4125001340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800b40c0 0x7fa7800b44d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa780163940 tx=0x7fa77801c900 comp rx=0 tx=0).stop 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 -- 172.21.15.149:0/4125001340 shutdown_connections 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 --2- 172.21.15.149:0/4125001340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800b40c0 0x7fa7800b44d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 --2- 172.21.15.149:0/4125001340 >> v2:172.21.15.149:3300/0 conn(0x7fa7800d1c20 0x7fa7800ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 -- 172.21.15.149:0/4125001340 >> 172.21.15.149:0/4125001340 conn(0x7fa7800d0770 msgr2=0x7fa7800d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 -- 172.21.15.149:0/4125001340 shutdown_connections 2024-05-14T11:15:49.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.376+0000 7fa787e00640 1 -- 172.21.15.149:0/4125001340 wait complete. 2024-05-14T11:15:49.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 Processor -- start 2024-05-14T11:15:49.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 -- start start 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7800b40c0 0x7fa780145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 0x7fa780144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa78006ff40 con 0x7fa7800cdf80 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa78006fdc0 con 0x7fa780144cb0 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa78016cd60 con 0x7fa7800b40c0 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa785e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 0x7fa780144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa785e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47984/0 (socket says 172.21.15.149:47984) 2024-05-14T11:15:49.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa785e00640 1 -- 172.21.15.149:0/641727894 learned_addr learned my addr 172.21.15.149:0/641727894 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa785400640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7800b40c0 0x7fa780145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 -- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7800b40c0 msgr2=0x7fa780145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7800b40c0 0x7fa780145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 -- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 msgr2=0x7fa7800cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 -- 172.21.15.149:0/641727894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7800ccfb0 con 0x7fa7800cdf80 2024-05-14T11:15:49.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa785e00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa784a00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 0x7fa780144770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e8a0 tx=0x7fa76c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c017070 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c00c940 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7801450c0 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.380+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00cd10 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.384+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7800cc9f0 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.384+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa76c01a070 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.384+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fa76c013470 con 0x7fa7800cdf80 2024-05-14T11:15:49.384 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.384+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78006a9c0 con 0x7fa7800cdf80 2024-05-14T11:15:49.386 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.384+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa76c0059a0 con 0x7fa7800cdf80 2024-05-14T11:15:49.399 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.400+0000 7f5fb5ec8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T11:15:49.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.452+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f6f87186-b4aa-4e02-a8ab-e048753ba149", "id": 5} v 0) -- 0x7fa78006e620 con 0x7fa7800cdf80 2024-05-14T11:15:49.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa77d600640 1 -- 172.21.15.149:0/641727894 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6f87186-b4aa-4e02-a8ab-e048753ba149", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fa76c005b80 con 0x7fa7800cdf80 2024-05-14T11:15:49.456 INFO:teuthology.orchestra.run.smithi149.stdout:5 2024-05-14T11:15:49.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 msgr2=0x7fa780144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 0x7fa780144770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e8a0 tx=0x7fa76c00ed60 comp rx=0 tx=0).stop 2024-05-14T11:15:49.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 shutdown_connections 2024-05-14T11:15:49.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa780144cb0 0x7fa7800cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7800cdf80 0x7fa780144770 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.458 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 --2- 172.21.15.149:0/641727894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7800b40c0 0x7fa780145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.458 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 >> 172.21.15.149:0/641727894 conn(0x7fa7800d0770 msgr2=0x7fa78015bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.458 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 shutdown_connections 2024-05-14T11:15:49.458 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.456+0000 7fa787e00640 1 -- 172.21.15.149:0/641727894 wait complete. 2024-05-14T11:15:49.476 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 500d5e45-1937-476c-99de-15b138f68674 6 2024-05-14T11:15:49.522 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.522+0000 7fcf900cd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T11:15:49.569 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.568+0000 7f5fb5ec8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T11:15:49.592 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:49.590+0000 7fcf900cd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T11:15:49.644 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:49.644+0000 7f5fb5ec8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T11:15:49.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 Processor -- start 2024-05-14T11:15:49.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 -- start start 2024-05-14T11:15:49.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 0x7f2e740d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740b3fe0 0x7f2e740b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e740d1e60 con 0x7f2e74160080 2024-05-14T11:15:49.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2e740b2f40 con 0x7f2e740b3fe0 2024-05-14T11:15:49.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.664+0000 7f2e7c200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2e740b3220 con 0x7f2e740d1510 2024-05-14T11:15:49.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740b3fe0 0x7f2e740b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e78e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 0x7f2e740d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e740b3fe0 0x7f2e740b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.149:56896/0 (socket says 172.21.15.149:56896) 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 -- 172.21.15.149:0/2353439673 learned_addr learned my addr 172.21.15.149:0/2353439673 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 -- 172.21.15.149:0/2353439673 >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 msgr2=0x7f2e740d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 --2- 172.21.15.149:0/2353439673 >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 0x7f2e740d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 -- 172.21.15.149:0/2353439673 >> v1:172.21.15.116:6789/0 conn(0x7f2e74160080 legacy=0x7f2e740cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e7a200640 1 -- 172.21.15.149:0/2353439673 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e740b3500 con 0x7f2e740b3fe0 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e73e00640 1 -- 172.21.15.149:0/2353439673 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210231890 0 0) 0x7f2e68001120 con 0x7f2e74160080 2024-05-14T11:15:49.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.668+0000 7f2e78e00640 1 --2- 172.21.15.149:0/2353439673 >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 0x7f2e740d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:49.670 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7a200640 1 --2- 172.21.15.149:0/2353439673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e740b3fe0 0x7f2e740b43f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e740d25c0 tx=0x7f2e6c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10270804a48b758 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.671 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e73e00640 1 -- 172.21.15.149:0/2353439673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c0a5760 con 0x7f2e740b3fe0 2024-05-14T11:15:49.671 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e73e00640 1 -- 172.21.15.149:0/2353439673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6c0a5900 con 0x7f2e740b3fe0 2024-05-14T11:15:49.671 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e73e00640 1 -- 172.21.15.149:0/2353439673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c0a5cd0 con 0x7f2e740b3fe0 2024-05-14T11:15:49.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- 172.21.15.149:0/2353439673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e740b3fe0 msgr2=0x7f2e740b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2353439673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e740b3fe0 0x7f2e740b43f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e740d25c0 tx=0x7f2e6c01c8a0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- 172.21.15.149:0/2353439673 shutdown_connections 2024-05-14T11:15:49.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2353439673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e740b3fe0 0x7f2e740b43f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2353439673 >> v2:172.21.15.149:3300/0 conn(0x7f2e740d1510 0x7f2e740d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.673 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- 172.21.15.149:0/2353439673 >> 172.21.15.149:0/2353439673 conn(0x7f2e740d0770 msgr2=0x7f2e740d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.673 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- 172.21.15.149:0/2353439673 shutdown_connections 2024-05-14T11:15:49.673 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- 172.21.15.149:0/2353439673 wait complete. 2024-05-14T11:15:49.674 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 Processor -- start 2024-05-14T11:15:49.674 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.672+0000 7f2e7c200640 1 -- start start 2024-05-14T11:15:49.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2e740b3fe0 0x7f2e74150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e74160080 0x7f2e74155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7406fdd0 con 0x7f2e74160080 2024-05-14T11:15:49.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2e740ce690 con 0x7f2e740d1510 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7406ff50 con 0x7f2e740b3fe0 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:48008/0 (socket says 172.21.15.149:48008) 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 -- 172.21.15.149:0/2302274478 learned_addr learned my addr 172.21.15.149:0/2302274478 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7a200640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e74160080 0x7f2e74155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e79800640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2e740b3fe0 0x7f2e74150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 -- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2e740b3fe0 msgr2=0x7f2e74150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2e740b3fe0 0x7f2e74150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 -- 172.21.15.149:0/2302274478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e74160080 msgr2=0x7f2e74155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e74160080 0x7f2e74155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 -- 172.21.15.149:0/2302274478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e741510c0 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e78e00640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e930 tx=0x7f2e6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6000ca00 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e741513a0 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000cdb0 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e74151960 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e6001a070 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2e60009380 con 0x7f2e740d1510 2024-05-14T11:15:49.681 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.676+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7406a9c0 con 0x7f2e740d1510 2024-05-14T11:15:49.682 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.684+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2e60013470 con 0x7f2e740d1510 2024-05-14T11:15:49.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.748+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "500d5e45-1937-476c-99de-15b138f68674", "id": 6} v 0) -- 0x7f2e740b9590 con 0x7f2e740d1510 2024-05-14T11:15:49.756 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e71600640 1 -- 172.21.15.149:0/2302274478 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "500d5e45-1937-476c-99de-15b138f68674", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2e60013650 con 0x7f2e740d1510 2024-05-14T11:15:49.757 INFO:teuthology.orchestra.run.smithi149.stdout:6 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 msgr2=0x7f2e74155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e930 tx=0x7f2e6000edf0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 shutdown_connections 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e740d1510 0x7f2e74155870 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e74160080 0x7f2e74155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 --2- 172.21.15.149:0/2302274478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2e740b3fe0 0x7f2e74150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 >> 172.21.15.149:0/2302274478 conn(0x7f2e740d0770 msgr2=0x7f2e74156490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 shutdown_connections 2024-05-14T11:15:49.758 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.756+0000 7f2e7c200640 1 -- 172.21.15.149:0/2302274478 wait complete. 2024-05-14T11:15:49.777 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 582cbeba-60c9-4f83-bf81-eb2ca3e9bd35 7 2024-05-14T11:15:49.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.920+0000 7fa90e800640 1 Processor -- start 2024-05-14T11:15:49.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.920+0000 7fa90e800640 1 -- start start 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa90e800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 0x7fa9080b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa90e800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9080d3160 con 0x7fa9080d2d50 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa90e800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9080d1f90 con 0x7fa9081621b0 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa90e800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9080d1d40 con 0x7fa9080ce9d0 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 0x7fa9080b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fa9081621b0 0x7fa9080b9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.149:47440/0 (socket says 172.21.15.149:47440) 2024-05-14T11:15:49.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907400640 1 -- 172.21.15.149:0/451147718 learned_addr learned my addr 172.21.15.149:0/451147718 (peer_addr_for_me v1:172.21.15.149:0/0) 2024-05-14T11:15:49.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa906000640 1 -- 172.21.15.149:0/451147718 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828464051 0 0) 0x7fa9080d1f90 con 0x7fa9081621b0 2024-05-14T11:15:49.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907e00640 1 -- 172.21.15.149:0/451147718 >> v1:172.21.15.149:6789/0 conn(0x7fa9081621b0 legacy=0x7fa9080b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907e00640 1 -- 172.21.15.149:0/451147718 >> v1:172.21.15.116:6789/0 conn(0x7fa9080d2d50 legacy=0x7fa9080ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:49.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.924+0000 7fa907e00640 1 -- 172.21.15.149:0/451147718 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9080d2270 con 0x7fa9080ce9d0 2024-05-14T11:15:49.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa907e00640 1 --2- 172.21.15.149:0/451147718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 0x7fa9080b2f30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa900009dc0 tx=0x7fa90001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bc3b9f06ef02a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa906000640 1 -- 172.21.15.149:0/451147718 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000b2070 con 0x7fa9080ce9d0 2024-05-14T11:15:49.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa906000640 1 -- 172.21.15.149:0/451147718 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9000a5850 con 0x7fa9080ce9d0 2024-05-14T11:15:49.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa906000640 1 -- 172.21.15.149:0/451147718 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000a5c20 con 0x7fa9080ce9d0 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 -- 172.21.15.149:0/451147718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 msgr2=0x7fa9080b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 --2- 172.21.15.149:0/451147718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 0x7fa9080b2f30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa900009dc0 tx=0x7fa90001c990 comp rx=0 tx=0).stop 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 -- 172.21.15.149:0/451147718 shutdown_connections 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 --2- 172.21.15.149:0/451147718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080ce9d0 0x7fa9080b2f30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 -- 172.21.15.149:0/451147718 >> 172.21.15.149:0/451147718 conn(0x7fa9080d0770 msgr2=0x7fa9080d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 -- 172.21.15.149:0/451147718 shutdown_connections 2024-05-14T11:15:49.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.928+0000 7fa90e800640 1 -- 172.21.15.149:0/451147718 wait complete. 2024-05-14T11:15:49.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 Processor -- start 2024-05-14T11:15:49.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 -- start start 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 0x7fa9081608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 0x7fa908071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa90815e600 con 0x7fa908071630 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa90815e480 con 0x7fa9081621b0 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa90e800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa90815e780 con 0x7fa9080d2d50 2024-05-14T11:15:49.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa906a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 0x7fa9081608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:40950/0 (socket says 172.21.15.149:40950) 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 -- 172.21.15.149:0/2804041021 learned_addr learned my addr 172.21.15.149:0/2804041021 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907e00640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 0x7fa908071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 -- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 msgr2=0x7fa9081608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 0x7fa9081608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 -- 172.21.15.149:0/2804041021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 msgr2=0x7fa908071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 0x7fa908071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.932+0000 7fa907400640 1 -- 172.21.15.149:0/2804041021 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa908153a40 con 0x7fa9080d2d50 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa906a00640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 0x7fa9081608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa907e00640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 0x7fa908071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa907400640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c420 tx=0x7fa8f800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:49.936 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80074e0 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f80047f0 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa908160e30 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8007680 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa908154300 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8f8011070 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fa8f8004be0 con 0x7fa9080d2d50 2024-05-14T11:15:49.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90806a9c0 con 0x7fa9080d2d50 2024-05-14T11:15:49.938 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:49.936+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa8f801d3f0 con 0x7fa9080d2d50 2024-05-14T11:15:50.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.048+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "582cbeba-60c9-4f83-bf81-eb2ca3e9bd35", "id": 7} v 0) -- 0x7fa9081546b0 con 0x7fa9080d2d50 2024-05-14T11:15:50.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.052+0000 7fa8ff400640 1 -- 172.21.15.149:0/2804041021 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "582cbeba-60c9-4f83-bf81-eb2ca3e9bd35", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fa8f801d5d0 con 0x7fa9080d2d50 2024-05-14T11:15:50.054 INFO:teuthology.orchestra.run.smithi149.stdout:7 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 msgr2=0x7fa9081694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c420 tx=0x7fa8f800c8e0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 shutdown_connections 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9081621b0 0x7fa9081608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa908071630 0x7fa908071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 --2- 172.21.15.149:0/2804041021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9080d2d50 0x7fa9081694c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 >> 172.21.15.149:0/2804041021 conn(0x7fa9080d0770 msgr2=0x7fa908169d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:50.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 shutdown_connections 2024-05-14T11:15:50.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:15:50.056+0000 7fa90e800640 1 -- 172.21.15.149:0/2804041021 wait complete. 2024-05-14T11:15:50.073 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-14T11:15:50.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-14T11:15:50.077 INFO:tasks.ceph.osd.0:Started 2024-05-14T11:15:50.077 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-14T11:15:50.077 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-14T11:15:50.080 INFO:tasks.ceph.osd.1:Started 2024-05-14T11:15:50.080 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-14T11:15:50.080 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-14T11:15:50.083 INFO:tasks.ceph.osd.2:Started 2024-05-14T11:15:50.083 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-14T11:15:50.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-14T11:15:50.086 INFO:tasks.ceph.osd.3:Started 2024-05-14T11:15:50.086 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-14T11:15:50.087 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-14T11:15:50.089 INFO:tasks.ceph.osd.4:Started 2024-05-14T11:15:50.090 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-14T11:15:50.090 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-14T11:15:50.092 INFO:tasks.ceph.osd.5:Started 2024-05-14T11:15:50.093 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-14T11:15:50.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-14T11:15:50.095 INFO:tasks.ceph.osd.6:Started 2024-05-14T11:15:50.096 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-14T11:15:50.096 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-14T11:15:50.099 INFO:tasks.ceph.osd.7:Started 2024-05-14T11:15:50.099 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:15:50.259 INFO:tasks.ceph.mgr.y.smithi116.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-14T11:15:50.259 INFO:tasks.ceph.mgr.y.smithi116.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-14T11:15:50.259 INFO:tasks.ceph.mgr.y.smithi116.stderr: from numpy import show_config as show_numpy_config 2024-05-14T11:15:50.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 Processor -- start 2024-05-14T11:15:50.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 -- start start 2024-05-14T11:15:50.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1248160cd0 0x7f1248159db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:50.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f12480d1a00 0x7f12480b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480b65c0 con 0x7f124815a2f0 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f124815a700 con 0x7f1248160cd0 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124f400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f12480b68a0 con 0x7f12480d1a00 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1248160cd0 0x7f1248159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1248160cd0 0x7f1248159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59594/0 (socket says 172.21.15.116:59594) 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124ca00640 1 -- 172.21.15.116:0/3401213471 learned_addr learned my addr 172.21.15.116:0/3401213471 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.286+0000 7f124d400640 1 --2- 172.21.15.116:0/3401213471 >> v2:172.21.15.149:3300/0 conn(0x7f12480d1a00 0x7f12480b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f124ca00640 1 -- 172.21.15.116:0/3401213471 >> v2:172.21.15.149:3300/0 conn(0x7f12480d1a00 msgr2=0x7f12480b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:50.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f124ca00640 1 --2- 172.21.15.116:0/3401213471 >> v2:172.21.15.149:3300/0 conn(0x7f12480d1a00 0x7f12480b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f124ca00640 1 -- 172.21.15.116:0/3401213471 >> v1:172.21.15.149:6789/0 conn(0x7f124815a2f0 legacy=0x7f12480d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f124ca00640 1 -- 172.21.15.116:0/3401213471 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12480b6b80 con 0x7f1248160cd0 2024-05-14T11:15:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f124ca00640 1 --2- 172.21.15.116:0/3401213471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1248160cd0 0x7f1248159db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1248160780 tx=0x7f123c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d71ae400f4434a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f1247400640 1 -- 172.21.15.116:0/3401213471 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0b1070 con 0x7f1248160cd0 2024-05-14T11:15:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f1247400640 1 -- 172.21.15.116:0/3401213471 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c0ac600 con 0x7f1248160cd0 2024-05-14T11:15:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.290+0000 7f1247400640 1 -- 172.21.15.116:0/3401213471 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0ac9d0 con 0x7f1248160cd0 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- 172.21.15.116:0/3401213471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1248160cd0 msgr2=0x7f1248159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 --2- 172.21.15.116:0/3401213471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1248160cd0 0x7f1248159db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1248160780 tx=0x7f123c01c490 comp rx=0 tx=0).stop 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- 172.21.15.116:0/3401213471 shutdown_connections 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 --2- 172.21.15.116:0/3401213471 >> v2:172.21.15.149:3300/0 conn(0x7f12480d1a00 0x7f12480b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 --2- 172.21.15.116:0/3401213471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1248160cd0 0x7f1248159db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- 172.21.15.116:0/3401213471 >> 172.21.15.116:0/3401213471 conn(0x7f1248071370 msgr2=0x7f1248071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- 172.21.15.116:0/3401213471 shutdown_connections 2024-05-14T11:15:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- 172.21.15.116:0/3401213471 wait complete. 2024-05-14T11:15:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 Processor -- start 2024-05-14T11:15:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 -- start start 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.294+0000 7f124f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 0x7f1248141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1248141aa0 0x7f1248158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f12480b1fa0 con 0x7f124815a2f0 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12480b1e20 con 0x7f1248141aa0 2024-05-14T11:15:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f12480cf970 con 0x7f1248160cd0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59604/0 (socket says 172.21.15.116:59604) 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 -- 172.21.15.116:0/1163460335 learned_addr learned my addr 172.21.15.116:0/1163460335 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124d400640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1248141aa0 0x7f1248158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1247e00640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 0x7f1248141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 -- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 msgr2=0x7f1248141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 0x7f1248141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 -- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1248141aa0 msgr2=0x7f1248158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1248141aa0 0x7f1248158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 -- 172.21.15.116:0/1163460335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124806e290 con 0x7f124815a2f0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1247e00640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 0x7f1248141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124ca00640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f123c01a0c0 tx=0x7f123c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0b1040 con 0x7f124815a2f0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c0bd070 con 0x7f124815a2f0 2024-05-14T11:15:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124806e570 con 0x7f124815a2f0 2024-05-14T11:15:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0b7620 con 0x7f124815a2f0 2024-05-14T11:15:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124806eb30 con 0x7f124815a2f0 2024-05-14T11:15:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f123c0b7b50 con 0x7f124815a2f0 2024-05-14T11:15:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f123c0c1c30 con 0x7f124815a2f0 2024-05-14T11:15:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.298+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124806a9c0 con 0x7f124815a2f0 2024-05-14T11:15:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.302+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f123c0c6020 con 0x7f124815a2f0 2024-05-14T11:15:50.311 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:50.310+0000 7fcf900cd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T11:15:50.323 INFO:tasks.ceph.mgr.x.smithi149.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-14T11:15:50.323 INFO:tasks.ceph.mgr.x.smithi149.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-14T11:15:50.323 INFO:tasks.ceph.mgr.x.smithi149.stderr: from numpy import show_config as show_numpy_config 2024-05-14T11:15:50.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.358+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1248146750 con 0x7f124815a2f0 2024-05-14T11:15:50.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f1244c00640 1 -- 172.21.15.116:0/1163460335 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f123c016110 con 0x7f124815a2f0 2024-05-14T11:15:50.361 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:15:50.361 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":10,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:15:50.050628+0000","last_up_change":"0.000000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 msgr2=0x7f1248145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f123c01a0c0 tx=0x7f123c01acd0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 shutdown_connections 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1248141aa0 0x7f1248158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1248160cd0 0x7f1248141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 --2- 172.21.15.116:0/1163460335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124815a2f0 0x7f1248145e20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 >> 172.21.15.116:0/1163460335 conn(0x7f1248071370 msgr2=0x7f12480d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 shutdown_connections 2024-05-14T11:15:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:15:50.362+0000 7f124f400640 1 -- 172.21.15.116:0/1163460335 wait complete. 2024-05-14T11:15:50.369 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:50.368+0000 7f5fb5ec8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T11:15:50.381 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-14T11:15:50.381 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-14T11:15:50.463 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:50.462+0000 7fcf900cd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T11:15:50.508 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-14T11:15:50.508+0000 7f83bdcc17c0 -1 Falling back to public interface 2024-05-14T11:15:50.517 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:50.516+0000 7f5fb5ec8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T11:15:50.644 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:50.642+0000 7fcf900cd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T11:15:50.693 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:50.692+0000 7f5fb5ec8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T11:15:50.720 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:50.718+0000 7fcf900cd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T11:15:50.738 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-05-14T11:15:50.738+0000 7fc24722b7c0 -1 Falling back to public interface 2024-05-14T11:15:50.750 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-05-14T11:15:50.750+0000 7f56ec6957c0 -1 Falling back to public interface 2024-05-14T11:15:50.762 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-05-14T11:15:50.762+0000 7f401c52d7c0 -1 Falling back to public interface 2024-05-14T11:15:50.762 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-05-14T11:15:50.762+0000 7fa45cd757c0 -1 Falling back to public interface 2024-05-14T11:15:50.764 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-14T11:15:50.764+0000 7f57508f97c0 -1 Falling back to public interface 2024-05-14T11:15:50.764 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-14T11:15:50.764+0000 7fb117d657c0 -1 Falling back to public interface 2024-05-14T11:15:50.768 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:50.768+0000 7f5fb5ec8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T11:15:50.775 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-14T11:15:50.776+0000 7fe74948c7c0 -1 Falling back to public interface 2024-05-14T11:15:50.878 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:50.878+0000 7fcf900cd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T11:15:50.927 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:50.928+0000 7f5fb5ec8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T11:15:51.185 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-14T11:15:51.326 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 Processor -- start 2024-05-14T11:15:51.326 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 -- start start 2024-05-14T11:15:51.328 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7bb4160e00 0x7f7bb406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:51.328 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 0x7f7bb406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:51.328 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb406c700 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb406efc0 con 0x7f7bb4160e00 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bba200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb40cdac0 con 0x7f7bb406f710 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7babe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7bb4160e00 0x7f7bb406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bb3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7bb40cd6b0 0x7f7bb41606f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58952/0 (socket says 172.21.15.116:58952) 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bb3400640 1 -- 172.21.15.116:0/1996628917 learned_addr learned my addr 172.21.15.116:0/1996628917 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bb3e00640 1 --2- 172.21.15.116:0/1996628917 >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 0x7f7bb406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7babe00640 1 -- 172.21.15.116:0/1996628917 >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 msgr2=0x7f7bb406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7babe00640 1 --2- 172.21.15.116:0/1996628917 >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 0x7f7bb406fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7babe00640 1 -- 172.21.15.116:0/1996628917 >> v1:172.21.15.116:6789/0 conn(0x7f7bb40cd6b0 legacy=0x7f7bb41606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7babe00640 1 -- 172.21.15.116:0/1996628917 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb40ccaf0 con 0x7f7bb4160e00 2024-05-14T11:15:51.329 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.326+0000 7f7bb3e00640 1 --2- 172.21.15.116:0/1996628917 >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 0x7f7bb406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:51.331 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7babe00640 1 --2- 172.21.15.116:0/1996628917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb4160e00 0x7f7bb406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40b3ad0 tx=0x7f7b9c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8f3237461842c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:51.331 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bb2a00640 1 -- 172.21.15.116:0/1996628917 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b0070 con 0x7f7bb4160e00 2024-05-14T11:15:51.332 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bb2a00640 1 -- 172.21.15.116:0/1996628917 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb4070870 con 0x7f7bb4160e00 2024-05-14T11:15:51.332 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bb2a00640 1 -- 172.21.15.116:0/1996628917 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0ab820 con 0x7f7bb4160e00 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bba200640 1 -- 172.21.15.116:0/1996628917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb4160e00 msgr2=0x7f7bb406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bba200640 1 --2- 172.21.15.116:0/1996628917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb4160e00 0x7f7bb406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40b3ad0 tx=0x7f7b9c01c480 comp rx=0 tx=0).stop 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.330+0000 7f7bba200640 1 -- 172.21.15.116:0/1996628917 shutdown_connections 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 --2- 172.21.15.116:0/1996628917 >> v2:172.21.15.149:3300/0 conn(0x7f7bb406f710 0x7f7bb406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 --2- 172.21.15.116:0/1996628917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb4160e00 0x7f7bb406ea80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- 172.21.15.116:0/1996628917 >> 172.21.15.116:0/1996628917 conn(0x7f7bb40b4890 msgr2=0x7f7bb40b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- 172.21.15.116:0/1996628917 shutdown_connections 2024-05-14T11:15:51.333 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- 172.21.15.116:0/1996628917 wait complete. 2024-05-14T11:15:51.335 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 Processor -- start 2024-05-14T11:15:51.335 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- start start 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb406f710 0x7f7bb409a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 0x7f7bb4166130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41699c0 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4169840 con 0x7f7bb40729c0 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.334+0000 7f7bba200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb4169b40 con 0x7f7bb406f710 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:51.338 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59630/0 (socket says 172.21.15.116:59630) 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 -- 172.21.15.116:0/1336797006 learned_addr learned my addr 172.21.15.116:0/1336797006 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bb3400640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb406f710 0x7f7bb409a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bb3e00640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 0x7f7bb4166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 -- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb406f710 msgr2=0x7f7bb409a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb406f710 0x7f7bb409a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 -- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 msgr2=0x7f7bb4166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 0x7f7bb4166130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 -- 172.21.15.116:0/1336797006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb4073130 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bb3e00640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 0x7f7bb4166130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7babe00640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ae040 tx=0x7f7b9c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b0040 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0b9070 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0b4890 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.339 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb4073410 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.340 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb40739d0 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.340 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b9c0b4a30 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.340 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7b9c0becd0 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.340 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.338+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb406a290 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.341 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.342+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7b9c016070 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.398 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bb4072e50 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.399 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bab400640 1 -- 172.21.15.116:0/1336797006 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7b9c0be460 con 0x7f7bb40cd6b0 2024-05-14T11:15:51.399 INFO:teuthology.misc.health.smithi116.stdout: 2024-05-14T11:15:51.399 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":10,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:15:50.050628+0000","last_up_change":"0.000000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:15:51.400 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 msgr2=0x7f7bb4165bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:51.400 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ae040 tx=0x7f7b9c019220 comp rx=0 tx=0).stop 2024-05-14T11:15:51.400 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 shutdown_connections 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb40729c0 0x7f7bb4166130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bb40cd6b0 0x7f7bb4165bf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 --2- 172.21.15.116:0/1336797006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb406f710 0x7f7bb409a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 >> 172.21.15.116:0/1336797006 conn(0x7f7bb40b4890 msgr2=0x7f7bb40cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 shutdown_connections 2024-05-14T11:15:51.401 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:51.398+0000 7f7bba200640 1 -- 172.21.15.116:0/1336797006 wait complete. 2024-05-14T11:15:51.418 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-14T11:15:51.562 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:51.562+0000 7fcf900cd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T11:15:51.629 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:51.628+0000 7f5fb5ec8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T11:15:51.650 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:51.650+0000 7fcf900cd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T11:15:51.716 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:51.716+0000 7f5fb5ec8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T11:15:51.853 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:51.854+0000 7fcf900cd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T11:15:51.918 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:51.916+0000 7f5fb5ec8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T11:15:51.937 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:51.938+0000 7fcf900cd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T11:15:52.003 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:52.004+0000 7f5fb5ec8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T11:15:52.269 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:52.266+0000 7fcf900cd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T11:15:52.332 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:52.332+0000 7f5fb5ec8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T11:15:52.339 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:52.338+0000 7fcf900cd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T11:15:52.419 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-14T11:15:52.420+0000 7f5fb5ec8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T11:15:53.381 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:53.381+0000 7fcf35400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-14T11:15:53.381 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-05-14T11:15:53.381+0000 7fcf35400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-14T11:15:54.494 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-14T11:15:54.492+0000 7f83bdcc17c0 -1 osd.6 0 log_to_monitors true 2024-05-14T11:15:54.615 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-05-14T11:15:54.613+0000 7fc24722b7c0 -1 osd.1 0 log_to_monitors true 2024-05-14T11:15:54.649 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-14T11:15:54.648+0000 7fb117d657c0 -1 osd.7 0 log_to_monitors true 2024-05-14T11:15:54.702 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-05-14T11:15:54.701+0000 7f56ec6957c0 -1 osd.0 0 log_to_monitors true 2024-05-14T11:15:54.715 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-14T11:15:54.716+0000 7fe74948c7c0 -1 osd.5 0 log_to_monitors true 2024-05-14T11:15:54.844 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-05-14T11:15:54.841+0000 7f401c52d7c0 -1 osd.3 0 log_to_monitors true 2024-05-14T11:15:54.844 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-05-14T11:15:54.841+0000 7fa45cd757c0 -1 osd.2 0 log_to_monitors true 2024-05-14T11:15:54.953 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-14T11:15:54.952+0000 7f57508f97c0 -1 osd.4 0 log_to_monitors true 2024-05-14T11:15:58.225 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-14T11:15:58.464 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.461+0000 7f488ba00640 1 Processor -- start 2024-05-14T11:15:58.464 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.461+0000 7f488ba00640 1 -- start start 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c034050 0x7f487c0e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f487c0528d0 0x7f487c039ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f487c052ce0 0x7f487c03a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f487c0528d0 0x7f487c039ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f487c0380d0 con 0x7f487c0528d0 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f487c0515f0 con 0x7f487c052ce0 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f488ba00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f487c050e80 con 0x7f487c034050 2024-05-14T11:15:58.467 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f487c0528d0 0x7f487c039ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39608/0 (socket says 172.21.15.116:39608) 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 -- 172.21.15.116:0/2999623764 learned_addr learned my addr 172.21.15.116:0/2999623764 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4889a00640 1 --2- 172.21.15.116:0/2999623764 >> v2:172.21.15.149:3300/0 conn(0x7f487c052ce0 0x7f487c03a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 -- 172.21.15.116:0/2999623764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c034050 msgr2=0x7f487c0e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 --2- 172.21.15.116:0/2999623764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c034050 0x7f487c0e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 -- 172.21.15.116:0/2999623764 >> v2:172.21.15.149:3300/0 conn(0x7f487c052ce0 msgr2=0x7f487c03a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 --2- 172.21.15.116:0/2999623764 >> v2:172.21.15.149:3300/0 conn(0x7f487c052ce0 0x7f487c03a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.468 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.465+0000 7f4883e00640 1 -- 172.21.15.116:0/2999623764 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c037b50 con 0x7f487c0528d0 2024-05-14T11:15:58.472 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.469+0000 7f4883e00640 1 --2- 172.21.15.116:0/2999623764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c039ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c640 tx=0x7f4870020830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdc544c56d65287 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:58.472 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.469+0000 7f4883400640 1 -- 172.21.15.116:0/2999623764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487000d2d0 con 0x7f487c0528d0 2024-05-14T11:15:58.472 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.469+0000 7f4883400640 1 -- 172.21.15.116:0/2999623764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48700aa500 con 0x7f487c0528d0 2024-05-14T11:15:58.473 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.469+0000 7f4883400640 1 -- 172.21.15.116:0/2999623764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48700aa8d0 con 0x7f487c0528d0 2024-05-14T11:15:58.473 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 -- 172.21.15.116:0/2999623764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 msgr2=0x7f487c039ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 --2- 172.21.15.116:0/2999623764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c039ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c640 tx=0x7f4870020830 comp rx=0 tx=0).stop 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 -- 172.21.15.116:0/2999623764 shutdown_connections 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 --2- 172.21.15.116:0/2999623764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c039ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 --2- 172.21.15.116:0/2999623764 >> v2:172.21.15.149:3300/0 conn(0x7f487c052ce0 0x7f487c03a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 --2- 172.21.15.116:0/2999623764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c034050 0x7f487c0e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 -- 172.21.15.116:0/2999623764 >> 172.21.15.116:0/2999623764 conn(0x7f487c054d00 msgr2=0x7f487c055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:58.474 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 -- 172.21.15.116:0/2999623764 shutdown_connections 2024-05-14T11:15:58.479 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.473+0000 7f488ba00640 1 -- 172.21.15.116:0/2999623764 wait complete. 2024-05-14T11:15:58.479 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.477+0000 7f488ba00640 1 Processor -- start 2024-05-14T11:15:58.480 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.477+0000 7f488ba00640 1 -- start start 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 0x7f487c0e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 0x7f487c18d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f487c0e4bf0 con 0x7f487c0528d0 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f487c0e4a70 con 0x7f487c034050 2024-05-14T11:15:58.483 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f488ba00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f487c051160 con 0x7f487c052ce0 2024-05-14T11:15:58.484 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f4889000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 0x7f487c0e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.484 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f4883e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.484 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f4883e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39622/0 (socket says 172.21.15.116:39622) 2024-05-14T11:15:58.485 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f4883e00640 1 -- 172.21.15.116:0/798565546 learned_addr learned my addr 172.21.15.116:0/798565546 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:15:58.485 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.481+0000 7f4889a00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 0x7f487c18d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.485 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 -- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 msgr2=0x7f487c18d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:58.486 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 0x7f487c18d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.486 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 -- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 msgr2=0x7f487c18d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:58.486 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.486 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 -- 172.21.15.116:0/798565546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c0d6b00 con 0x7f487c034050 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4883e00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889000640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 0x7f487c0e50a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f487800b470 tx=0x7f487800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4889a00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 0x7f487c18d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878015020 con 0x7f487c034050 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4878011850 con 0x7f487c034050 2024-05-14T11:15:58.487 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878011c40 con 0x7f487c034050 2024-05-14T11:15:58.488 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c0d6de0 con 0x7f487c034050 2024-05-14T11:15:58.488 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.485+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c0d73a0 con 0x7f487c034050 2024-05-14T11:15:58.490 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.489+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c004870 con 0x7f487c034050 2024-05-14T11:15:58.492 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.489+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 4) ==== 93151+0+0 (secure 0 0 0) 0x7f4878018020 con 0x7f487c034050 2024-05-14T11:15:58.493 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.493+0000 7f4880c00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 0x7f4864071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:15:58.494 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.493+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f487808e140 con 0x7f487c034050 2024-05-14T11:15:58.494 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.493+0000 7f4883e00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 0x7f4864071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:15:58.495 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.493+0000 7f4883e00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 0x7f4864071900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4870020870 tx=0x7f4870009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:15:58.503 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.501+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f487801f020 con 0x7f487c034050 2024-05-14T11:15:58.697 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-14T11:15:58.696+0000 7fe744400640 -1 osd.5 0 waiting for initial osdmap 2024-05-14T11:15:58.733 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-14T11:15:58.732+0000 7fe73a400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.801 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.801+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f487c005580 con 0x7f487c034050 2024-05-14T11:15:58.805 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.805+0000 7f4880c00640 1 -- 172.21.15.116:0/798565546 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f487805bf80 con 0x7f487c034050 2024-05-14T11:15:58.806 INFO:teuthology.misc.health.smithi116.stdout: 2024-05-14T11:15:58.806 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":12,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:15:56.384727+0000","last_up_change":"0.000000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:15:58.809 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 msgr2=0x7f4864071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:58.809 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 0x7f4864071900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4870020870 tx=0x7f4870009040 comp rx=0 tx=0).stop 2024-05-14T11:15:58.810 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 msgr2=0x7f487c0e50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:15:58.810 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 0x7f487c0e50a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f487800b470 tx=0x7f487800b870 comp rx=0 tx=0).stop 2024-05-14T11:15:58.810 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 shutdown_connections 2024-05-14T11:15:58.810 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f486406f3f0 0x7f4864071900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f487c052ce0 0x7f487c18d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f487c0528d0 0x7f487c18d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 --2- 172.21.15.116:0/798565546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f487c034050 0x7f487c0e50a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 >> 172.21.15.116:0/798565546 conn(0x7f487c054d00 msgr2=0x7f487c035040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 shutdown_connections 2024-05-14T11:15:58.811 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:15:58.809+0000 7f488ba00640 1 -- 172.21.15.116:0/798565546 wait complete. 2024-05-14T11:15:58.841 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-14T11:15:58.844 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-14T11:15:58.844+0000 7f83b8c00640 -1 osd.6 0 waiting for initial osdmap 2024-05-14T11:15:58.861 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-14T11:15:58.860+0000 7f574c200640 -1 osd.4 0 waiting for initial osdmap 2024-05-14T11:15:58.866 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-14T11:15:58.864+0000 7fb113600640 -1 osd.7 0 waiting for initial osdmap 2024-05-14T11:15:58.875 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-14T11:15:58.876+0000 7f83aec00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.892 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-14T11:15:58.892+0000 7f5741800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.895 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-14T11:15:58.896+0000 7fb108c00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.909 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-05-14T11:15:58.909+0000 7f56e7600640 -1 osd.0 0 waiting for initial osdmap 2024-05-14T11:15:58.922 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-05-14T11:15:58.917+0000 7fa457c00640 -1 osd.2 0 waiting for initial osdmap 2024-05-14T11:15:58.943 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-05-14T11:15:58.941+0000 7f4017400640 -1 osd.3 0 waiting for initial osdmap 2024-05-14T11:15:58.950 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-05-14T11:15:58.949+0000 7f56dd600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.962 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-05-14T11:15:58.961+0000 7fa44dc00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.974 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-05-14T11:15:58.973+0000 7f400d400640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:15:58.998 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-05-14T11:15:58.997+0000 7fc242000640 -1 osd.1 0 waiting for initial osdmap 2024-05-14T11:15:59.034 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-05-14T11:15:59.033+0000 7fc237600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T11:16:05.647 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:05.832 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.793+0000 7f234f800640 1 Processor -- start 2024-05-14T11:16:05.832 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.793+0000 7f234f800640 1 -- start start 2024-05-14T11:16:05.832 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234f800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f23480ce5d0 0x7f23480b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.832 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f2348073dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234f800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23480cef10 con 0x7f2348074310 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234f800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f23480cfdc0 con 0x7f23480ce5d0 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f23480cf1b0 con 0x7f23480b2480 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234d800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2348074310 0x7f23480d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50832/0 (socket says 172.21.15.116:50832) 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234d800640 1 -- 172.21.15.116:0/2951824137 learned_addr learned my addr 172.21.15.116:0/2951824137 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f234ce00640 1 --2- 172.21.15.116:0/2951824137 >> v2:172.21.15.149:3300/0 conn(0x7f23480ce5d0 0x7f23480b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f2347e00640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f2348073dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.797+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092355260 0 0) 0x7f23480cef10 con 0x7f2348074310 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f232c003510 con 0x7f2348074310 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1449870412 0 0) 0x7f232c003510 con 0x7f2348074310 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f234ce00640 1 -- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 msgr2=0x7f2348073dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f234ce00640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f2348073dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f234ce00640 1 -- 172.21.15.116:0/2951824137 >> v1:172.21.15.116:6789/0 conn(0x7f2348074310 legacy=0x7f23480d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:05.833 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f234ce00640 1 -- 172.21.15.116:0/2951824137 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23480cf490 con 0x7f23480ce5d0 2024-05-14T11:16:05.834 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.829+0000 7f2347e00640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f2348073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:05.835 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234ce00640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480b1f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2330008260 tx=0x7f233001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e24cde6ec3dd0a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:05.835 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23300b0070 con 0x7f23480ce5d0 2024-05-14T11:16:05.835 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2330008a20 con 0x7f23480ce5d0 2024-05-14T11:16:05.835 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f2347400640 1 -- 172.21.15.116:0/2951824137 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340001a80 con 0x7f23480ce5d0 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 -- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 msgr2=0x7f23480b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480b1f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2330008260 tx=0x7f233001c300 comp rx=0 tx=0).stop 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 -- 172.21.15.116:0/2951824137 shutdown_connections 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f2348073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 --2- 172.21.15.116:0/2951824137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480b1f40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 -- 172.21.15.116:0/2951824137 >> 172.21.15.116:0/2951824137 conn(0x7f23480b6170 msgr2=0x7f23480b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 -- 172.21.15.116:0/2951824137 shutdown_connections 2024-05-14T11:16:05.837 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.833+0000 7f234f800640 1 -- 172.21.15.116:0/2951824137 wait complete. 2024-05-14T11:16:05.840 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.837+0000 7f234f800640 1 Processor -- start 2024-05-14T11:16:05.840 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.837+0000 7f234f800640 1 -- start start 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f234808a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b2890 con 0x7f2348074310 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2348074b30 con 0x7f23480ce5d0 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f23480d0250 con 0x7f23480b2480 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39662/0 (socket says 172.21.15.116:39662) 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 -- 172.21.15.116:0/4156692782 learned_addr learned my addr 172.21.15.116:0/4156692782 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234d800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f2347e00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f234808a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 -- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 msgr2=0x7f234808a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:05.844 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f234808a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:05.845 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 -- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 msgr2=0x7f23480957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:05.845 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:05.845 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 -- 172.21.15.116:0/4156692782 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348094530 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234ce00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23300ae040 tx=0x7f233000aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23300b0040 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2330004200 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23480c4480 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23300059a0 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.841+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348094d30 con 0x7f2348074310 2024-05-14T11:16:05.846 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234806a290 con 0x7f2348074310 2024-05-14T11:16:05.847 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23300bf020 con 0x7f2348074310 2024-05-14T11:16:05.847 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f2344c00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 0x7f2324071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:05.848 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2330128d30 con 0x7f2348074310 2024-05-14T11:16:05.848 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f2347e00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 0x7f2324071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:05.848 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.845+0000 7f2347e00640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 0x7f2324071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23480c82b0 tx=0x7f233800f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:05.853 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:05.853+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f233012e020 con 0x7f2348074310 2024-05-14T11:16:06.017 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23480b5800 con 0x7f2348074310 2024-05-14T11:16:06.018 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f2344c00640 1 -- 172.21.15.116:0/4156692782 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f2330015ef0 con 0x7f2348074310 2024-05-14T11:16:06.019 INFO:teuthology.misc.health.smithi116.stdout: 2024-05-14T11:16:06.019 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":16,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:02.734861+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 msgr2=0x7f2324071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 0x7f2324071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23480c82b0 tx=0x7f233800f040 comp rx=0 tx=0).stop 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 msgr2=0x7f23480c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23300ae040 tx=0x7f233000aac0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 shutdown_connections 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232406f400 0x7f2324071910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23480ce5d0 0x7f23480957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480b2480 0x7f234808a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.021 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 --2- 172.21.15.116:0/4156692782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348074310 0x7f23480c4e90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.022 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 >> 172.21.15.116:0/4156692782 conn(0x7f23480b6170 msgr2=0x7f2348070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:06.022 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.017+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 shutdown_connections 2024-05-14T11:16:06.022 INFO:teuthology.misc.health.smithi116.stderr:2024-05-14T11:16:06.021+0000 7f234f800640 1 -- 172.21.15.116:0/4156692782 wait complete. 2024-05-14T11:16:06.039 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-14T11:16:06.039 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-14T11:16:06.039 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:16:06.039 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:16:06.042 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:16:06.042 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:16:06.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:16:06.045 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:16:06.045 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:16:06.045 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:16:06.048 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:16:06.049 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:16:06.049 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:16:06.052 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:16:06.052 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-14T11:16:06.052 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:06.174 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:16:06.174 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:16:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 Processor -- start 2024-05-14T11:16:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 -- start start 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f66b80d2d50 0x7f66b80ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80cec10 con 0x7f66b81621b0 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80d3160 con 0x7f66b80ce800 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66bea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f66b80d1d40 con 0x7f66b80d2d50 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f66b81621b0 0x7f66b80b9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50848/0 (socket says 172.21.15.116:50848) 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7e00640 1 -- 172.21.15.116:0/2654836847 learned_addr learned my addr 172.21.15.116:0/2654836847 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66afe00640 1 --2- 172.21.15.116:0/2654836847 >> v2:172.21.15.116:3300/0 conn(0x7f66b80d2d50 0x7f66b80ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2228331826 0 0) 0x7f66a8001120 con 0x7f66b81621b0 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f669c003530 con 0x7f66b81621b0 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905269884 0 0) 0x7f66b80d3160 con 0x7f66b80ce800 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a8001120 con 0x7f66b80ce800 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66afe00640 1 -- 172.21.15.116:0/2654836847 >> v1:172.21.15.149:6789/0 conn(0x7f66b80ce800 legacy=0x7f66b80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66afe00640 1 -- 172.21.15.116:0/2654836847 >> v1:172.21.15.116:6789/0 conn(0x7f66b81621b0 legacy=0x7f66b80b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.205+0000 7f66afe00640 1 -- 172.21.15.116:0/2654836847 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80d1f90 con 0x7f66b80d2d50 2024-05-14T11:16:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66afe00640 1 --2- 172.21.15.116:0/2654836847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b80d2d50 0x7f66b80ce2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c8e1c2d999774b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00b0070 con 0x7f66b80d2d50 2024-05-14T11:16:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a00ab600 con 0x7f66b80d2d50 2024-05-14T11:16:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66b7400640 1 -- 172.21.15.116:0/2654836847 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00ab9d0 con 0x7f66b80d2d50 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 -- 172.21.15.116:0/2654836847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b80d2d50 msgr2=0x7f66b80ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 --2- 172.21.15.116:0/2654836847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b80d2d50 0x7f66b80ce2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 -- 172.21.15.116:0/2654836847 shutdown_connections 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 --2- 172.21.15.116:0/2654836847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b80d2d50 0x7f66b80ce2c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 -- 172.21.15.116:0/2654836847 >> 172.21.15.116:0/2654836847 conn(0x7f66b80d0770 msgr2=0x7f66b80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 -- 172.21.15.116:0/2654836847 shutdown_connections 2024-05-14T11:16:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.209+0000 7f66bea00640 1 -- 172.21.15.116:0/2654836847 wait complete. 2024-05-14T11:16:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 Processor -- start 2024-05-14T11:16:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 -- start start 2024-05-14T11:16:06.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b80ce800 0x7f66b80c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b80d2d50 0x7f66b815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b3bd0 con 0x7f66b81621b0 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b3a50 con 0x7f66b80ce800 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f66b815ea80 con 0x7f66b80d2d50 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39686/0 (socket says 172.21.15.116:39686) 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.213+0000 7f66bca00640 1 -- 172.21.15.116:0/1281248685 learned_addr learned my addr 172.21.15.116:0/1281248685 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66afe00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b80d2d50 0x7f66b815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b7e00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b80ce800 0x7f66b80c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 -- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b80d2d50 msgr2=0x7f66b815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b80d2d50 0x7f66b815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 -- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b80ce800 msgr2=0x7f66b80c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b80ce800 0x7f66b80c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 -- 172.21.15.116:0/1281248685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80c9fc0 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bca00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c760 tx=0x7f66b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003b80 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0003d20 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000aa30 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b8155490 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8155a50 con 0x7f66b81621b0 2024-05-14T11:16:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8159c90 con 0x7f66b81621b0 2024-05-14T11:16:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66b000abd0 con 0x7f66b81621b0 2024-05-14T11:16:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 0x7f6694071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.217+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f66b008edb0 con 0x7f66b81621b0 2024-05-14T11:16:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.221+0000 7f66b7e00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 0x7f6694071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.221+0000 7f66b7e00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 0x7f6694071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005890 tx=0x7f66a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.225+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f66b0093070 con 0x7f66b81621b0 2024-05-14T11:16:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.417+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b80d3410 con 0x7f66b81621b0 2024-05-14T11:16:06.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66b4c00640 1 -- 172.21.15.116:0/1281248685 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f66b0060e20 con 0x7f66b81621b0 2024-05-14T11:16:06.421 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:06.421 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T11:16:06.422 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 2 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 msgr2=0x7f6694071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 0x7f6694071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005890 tx=0x7f66a8009040 comp rx=0 tx=0).stop 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 msgr2=0x7f66b815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c760 tx=0x7f66b000cc20 comp rx=0 tx=0).stop 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 shutdown_connections 2024-05-14T11:16:06.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f669406f400 0x7f6694071910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66b81621b0 0x7f66b815be60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b80d2d50 0x7f66b815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 --2- 172.21.15.116:0/1281248685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b80ce800 0x7f66b80c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 >> 172.21.15.116:0/1281248685 conn(0x7f66b80d0770 msgr2=0x7f66b81606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 shutdown_connections 2024-05-14T11:16:06.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:06.421+0000 7f66bea00640 1 -- 172.21.15.116:0/1281248685 wait complete. 2024-05-14T11:16:06.668 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:16:06.668 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:16:08.444 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 Processor -- start 2024-05-14T11:16:08.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 -- start start 2024-05-14T11:16:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89c4160a00 0x7f89c4160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f89c40d0280 0x7f89c40d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c40c7510 con 0x7f89c40cc9d0 2024-05-14T11:16:08.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f89c40c85b0 con 0x7f89c4160a00 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89cc600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f89c415a7d0 con 0x7f89c40d0280 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89c9c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89c4160a00 0x7f89c4160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89c9c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89c4160a00 0x7f89c4160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40638/0 (socket says 172.21.15.116:40638) 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89c9c00640 1 -- 172.21.15.116:0/37098125 learned_addr learned my addr 172.21.15.116:0/37098125 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89ca600640 1 --2- 172.21.15.116:0/37098125 >> v2:172.21.15.149:3300/0 conn(0x7f89c40d0280 0x7f89c40d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89c9c00640 1 -- 172.21.15.116:0/37098125 >> v2:172.21.15.149:3300/0 conn(0x7f89c40d0280 msgr2=0x7f89c40d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.601+0000 7f89c9c00640 1 --2- 172.21.15.116:0/37098125 >> v2:172.21.15.149:3300/0 conn(0x7f89c40d0280 0x7f89c40d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c9c00640 1 -- 172.21.15.116:0/37098125 >> v1:172.21.15.149:6789/0 conn(0x7f89c40cc9d0 legacy=0x7f89c40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c9c00640 1 -- 172.21.15.116:0/37098125 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c40c77f0 con 0x7f89c4160a00 2024-05-14T11:16:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c9c00640 1 --2- 172.21.15.116:0/37098125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c4160e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89c4074f60 tx=0x7f89b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5373d1a4e3c364e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c3e00640 1 -- 172.21.15.116:0/37098125 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80aa650 con 0x7f89c4160a00 2024-05-14T11:16:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c3e00640 1 -- 172.21.15.116:0/37098125 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b80af070 con 0x7f89c4160a00 2024-05-14T11:16:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.605+0000 7f89c3e00640 1 -- 172.21.15.116:0/37098125 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80aaa10 con 0x7f89c4160a00 2024-05-14T11:16:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- 172.21.15.116:0/37098125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 msgr2=0x7f89c4160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 --2- 172.21.15.116:0/37098125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c4160e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89c4074f60 tx=0x7f89b801c4f0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- 172.21.15.116:0/37098125 shutdown_connections 2024-05-14T11:16:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 --2- 172.21.15.116:0/37098125 >> v2:172.21.15.149:3300/0 conn(0x7f89c40d0280 0x7f89c40d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 --2- 172.21.15.116:0/37098125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c4160e10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- 172.21.15.116:0/37098125 >> 172.21.15.116:0/37098125 conn(0x7f89c4071190 msgr2=0x7f89c40715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- 172.21.15.116:0/37098125 shutdown_connections 2024-05-14T11:16:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- 172.21.15.116:0/37098125 wait complete. 2024-05-14T11:16:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 Processor -- start 2024-05-14T11:16:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.609+0000 7f89cc600640 1 -- start start 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c40cc9d0 0x7f89c40b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89c40d0280 0x7f89c4144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89c416cde0 con 0x7f89c4160a00 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89c416cc60 con 0x7f89c40cc9d0 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f89c416cf60 con 0x7f89c40d0280 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40640/0 (socket says 172.21.15.116:40640) 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 -- 172.21.15.116:0/246922195 learned_addr learned my addr 172.21.15.116:0/246922195 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c9c00640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c40cc9d0 0x7f89c40b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c9200640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89c40d0280 0x7f89c4144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 -- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89c40d0280 msgr2=0x7f89c4144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89c40d0280 0x7f89c4144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 -- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c40cc9d0 msgr2=0x7f89c40b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c40cc9d0 0x7f89c40b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 -- 172.21.15.116:0/246922195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c40b7f20 con 0x7f89c4160a00 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89ca600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007d80 tx=0x7f89bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc014420 con 0x7f89c4160a00 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc019070 con 0x7f89c4160a00 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c4145400 con 0x7f89c4160a00 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc014800 con 0x7f89c4160a00 2024-05-14T11:16:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c40b7160 con 0x7f89c4160a00 2024-05-14T11:16:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c40753e0 con 0x7f89c4160a00 2024-05-14T11:16:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.613+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89bc0149a0 con 0x7f89c4160a00 2024-05-14T11:16:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.617+0000 7f89c1600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 0x7f89a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.617+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f89bc08f5f0 con 0x7f89c4160a00 2024-05-14T11:16:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.617+0000 7f89c9c00640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 0x7f89a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.617+0000 7f89c9c00640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 0x7f89a4071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89b80ae000 tx=0x7f89b80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.621+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f89bc05c1e0 con 0x7f89c4160a00 2024-05-14T11:16:08.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89c406a9c0 con 0x7f89c4160a00 2024-05-14T11:16:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89c1600640 1 -- 172.21.15.116:0/246922195 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f89bc0604e0 con 0x7f89c4160a00 2024-05-14T11:16:08.814 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:08.814 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T11:16:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 3 2024-05-14T11:16:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 msgr2=0x7f89a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 0x7f89a4071910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89b80ae000 tx=0x7f89b80a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 msgr2=0x7f89c40af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007d80 tx=0x7f89bc00e4b0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 shutdown_connections 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89a406f400 0x7f89a4071910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89c4160a00 0x7f89c40af540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89c40d0280 0x7f89c4144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 --2- 172.21.15.116:0/246922195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89c40cc9d0 0x7f89c40b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 >> 172.21.15.116:0/246922195 conn(0x7f89c4071190 msgr2=0x7f89c4159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 shutdown_connections 2024-05-14T11:16:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.813+0000 7f89cc600640 1 -- 172.21.15.116:0/246922195 wait complete. 2024-05-14T11:16:08.836 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:16:08.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 Processor -- start 2024-05-14T11:16:08.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 -- start start 2024-05-14T11:16:08.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f48b01621b0 0x7f48b00d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b00d1f90 con 0x7f48b00b9020 2024-05-14T11:16:08.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f48b00d1d40 con 0x7f48b01621b0 2024-05-14T11:16:08.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.981+0000 7f48b5c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48b00d2270 con 0x7f48b00ce9d0 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f48b01621b0 0x7f48b00d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48af400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f48b01621b0 0x7f48b00d2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:35524/0 (socket says 172.21.15.116:35524) 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48af400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:59506/0 (socket says 172.21.15.116:59506) 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 -- 172.21.15.116:0/3942272806 learned_addr learned my addr 172.21.15.116:0/3942272806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 -- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 msgr2=0x7f48b00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 -- 172.21.15.116:0/3942272806 >> v1:172.21.15.149:6789/0 conn(0x7f48b00b9020 legacy=0x7f48b00ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 -- 172.21.15.116:0/3942272806 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b00d2550 con 0x7f48b01621b0 2024-05-14T11:16:08.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48af400640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48aea00640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b00d2d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009870 tx=0x7f48a001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f190cbecec5daf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.985+0000 7f48ad600640 1 -- 172.21.15.116:0/3942272806 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a00b2070 con 0x7f48b01621b0 2024-05-14T11:16:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48ad600640 1 -- 172.21.15.116:0/3942272806 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a00a9070 con 0x7f48b01621b0 2024-05-14T11:16:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48ad600640 1 -- 172.21.15.116:0/3942272806 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a00ad880 con 0x7f48b01621b0 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 msgr2=0x7f48b00d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b00d2d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009870 tx=0x7f48a001c560 comp rx=0 tx=0).stop 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- 172.21.15.116:0/3942272806 shutdown_connections 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00ce9d0 0x7f48b00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3942272806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b00d2d50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- 172.21.15.116:0/3942272806 >> 172.21.15.116:0/3942272806 conn(0x7f48b00d0770 msgr2=0x7f48b00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- 172.21.15.116:0/3942272806 shutdown_connections 2024-05-14T11:16:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- 172.21.15.116:0/3942272806 wait complete. 2024-05-14T11:16:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 Processor -- start 2024-05-14T11:16:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.989+0000 7f48b5c00640 1 -- start start 2024-05-14T11:16:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00b9020 0x7f48b015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b01513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48b015e600 con 0x7f48b0151910 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48b015e480 con 0x7f48b01621b0 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48b015e780 con 0x7f48b00b9020 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40650/0 (socket says 172.21.15.116:40650) 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 -- 172.21.15.116:0/3106741354 learned_addr learned my addr 172.21.15.116:0/3106741354 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48aea00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00b9020 0x7f48b015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48ae000640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b01513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 -- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00b9020 msgr2=0x7f48b015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00b9020 0x7f48b015cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 -- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 msgr2=0x7f48b01513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b01513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 -- 172.21.15.116:0/3106741354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b015d0f0 con 0x7f48b0151910 2024-05-14T11:16:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48af400640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48a800efc0 tx=0x7f48a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8018030 con 0x7f48b0151910 2024-05-14T11:16:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a800cd30 con 0x7f48b0151910 2024-05-14T11:16:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b015fe70 con 0x7f48b0151910 2024-05-14T11:16:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003bc0 con 0x7f48b0151910 2024-05-14T11:16:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b0160430 con 0x7f48b0151910 2024-05-14T11:16:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.993+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b006a9c0 con 0x7f48b0151910 2024-05-14T11:16:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.997+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f48a801b070 con 0x7f48b0151910 2024-05-14T11:16:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.997+0000 7f48a6a00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.997+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f48a808f8b0 con 0x7f48b0151910 2024-05-14T11:16:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.997+0000 7f48aea00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:08.997+0000 7f48aea00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48a00af000 tx=0x7f48a00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:09.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.001+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f48a8022070 con 0x7f48b0151910 2024-05-14T11:16:09.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.161+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f48b00cf850 con 0x7f48b0151910 2024-05-14T11:16:09.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.161+0000 7f48a6a00640 1 -- 172.21.15.116:0/3106741354 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f48a8010040 con 0x7f48b0151910 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:16:09.164 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:16:09.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 msgr2=0x7f4890071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48a00af000 tx=0x7f48a00a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 msgr2=0x7f48b01fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48a800efc0 tx=0x7f48a800c490 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 shutdown_connections 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48b01621b0 0x7f48b01513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b0151910 0x7f48b01fc510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 --2- 172.21.15.116:0/3106741354 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b00b9020 0x7f48b015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 >> 172.21.15.116:0/3106741354 conn(0x7f48b00d0770 msgr2=0x7f48b0168f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 shutdown_connections 2024-05-14T11:16:09.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.165+0000 7f48b5c00640 1 -- 172.21.15.116:0/3106741354 wait complete. 2024-05-14T11:16:09.185 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:16:09.185 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:16:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.333+0000 7f63bb400640 1 Processor -- start 2024-05-14T11:16:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.333+0000 7f63bb400640 1 -- start start 2024-05-14T11:16:09.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63bb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f63b4165060 0x7f63b4167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63bb400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b415c800 con 0x7f63b406cf90 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63bb400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b415caa0 con 0x7f63b40b5f30 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63bb400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f63b415d0f0 con 0x7f63b4165060 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63b406cf90 0x7f63b406d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39882/0 (socket says 172.21.15.116:39882) 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3e00640 1 -- 172.21.15.116:0/1920751750 learned_addr learned my addr 172.21.15.116:0/1920751750 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b9400640 1 --2- 172.21.15.116:0/1920751750 >> v2:172.21.15.149:3300/0 conn(0x7f63b4165060 0x7f63b4167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863370422 0 0) 0x7f63b415c800 con 0x7f63b406cf90 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f639c003530 con 0x7f63b406cf90 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 803190804 0 0) 0x7f63b415caa0 con 0x7f63b40b5f30 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63b415c800 con 0x7f63b40b5f30 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b9400640 1 -- 172.21.15.116:0/1920751750 >> v1:172.21.15.149:6789/0 conn(0x7f63b40b5f30 legacy=0x7f63b40b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b9400640 1 -- 172.21.15.116:0/1920751750 >> v1:172.21.15.116:6789/0 conn(0x7f63b406cf90 legacy=0x7f63b406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.337+0000 7f63b9400640 1 -- 172.21.15.116:0/1920751750 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b415cd80 con 0x7f63b4165060 2024-05-14T11:16:09.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63b9400640 1 --2- 172.21.15.116:0/1920751750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b4165060 0x7f63b4167480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00aed0 tx=0x7f63ac01da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d99e7ab70282b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a66b0 con 0x7f63b4165060 2024-05-14T11:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac0a6850 con 0x7f63b4165060 2024-05-14T11:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63b3400640 1 -- 172.21.15.116:0/1920751750 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a6c20 con 0x7f63b4165060 2024-05-14T11:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 -- 172.21.15.116:0/1920751750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b4165060 msgr2=0x7f63b4167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 --2- 172.21.15.116:0/1920751750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b4165060 0x7f63b4167480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00aed0 tx=0x7f63ac01da70 comp rx=0 tx=0).stop 2024-05-14T11:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 -- 172.21.15.116:0/1920751750 shutdown_connections 2024-05-14T11:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 --2- 172.21.15.116:0/1920751750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b4165060 0x7f63b4167480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 -- 172.21.15.116:0/1920751750 >> 172.21.15.116:0/1920751750 conn(0x7f63b40cde80 msgr2=0x7f63b40ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 -- 172.21.15.116:0/1920751750 shutdown_connections 2024-05-14T11:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 -- 172.21.15.116:0/1920751750 wait complete. 2024-05-14T11:16:09.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.341+0000 7f63bb400640 1 Processor -- start 2024-05-14T11:16:09.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- start start 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b40b5f30 0x7f63b41fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63b4165060 0x7f63b414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4169b50 con 0x7f63b406cf90 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63b41699d0 con 0x7f63b40b5f30 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f63b4169cd0 con 0x7f63b4165060 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40652/0 (socket says 172.21.15.116:40652) 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 -- 172.21.15.116:0/1248426910 learned_addr learned my addr 172.21.15.116:0/1248426910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b9400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63b4165060 0x7f63b414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b3e00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b40b5f30 0x7f63b41fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 -- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63b4165060 msgr2=0x7f63b414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63b4165060 0x7f63b414ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 -- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b40b5f30 msgr2=0x7f63b41fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b40b5f30 0x7f63b41fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 -- 172.21.15.116:0/1248426910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b4156910 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b8a00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c400 tx=0x7f63a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8017070 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a8005310 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b4166250 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80054c0 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b4154420 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.345+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b4156ae0 con 0x7f63b406cf90 2024-05-14T11:16:09.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.349+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63a801a070 con 0x7f63b406cf90 2024-05-14T11:16:09.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.349+0000 7f63b0c00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 0x7f6394071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:09.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.349+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f63a808e220 con 0x7f63b406cf90 2024-05-14T11:16:09.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.349+0000 7f63b3e00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 0x7f6394071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:09.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.349+0000 7f63b3e00640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 0x7f6394071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b4150ad0 tx=0x7f63a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:09.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.353+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f63a805ae00 con 0x7f63b406cf90 2024-05-14T11:16:09.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:09.509+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f63b406aa30 con 0x7f63b406cf90 2024-05-14T11:16:10.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63b0c00640 1 -- 172.21.15.116:0/1248426910 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f63a808ce20 con 0x7f63b406cf90 2024-05-14T11:16:10.209 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:16:10.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 msgr2=0x7f6394071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 0x7f6394071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b4150ad0 tx=0x7f63a0009040 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 msgr2=0x7f63b414a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c400 tx=0x7f63a800c8c0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 shutdown_connections 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f639406f430 0x7f6394071940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63b40b5f30 0x7f63b41fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63b4165060 0x7f63b414ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 --2- 172.21.15.116:0/1248426910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63b406cf90 0x7f63b414a330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 >> 172.21.15.116:0/1248426910 conn(0x7f63b40cde80 msgr2=0x7f63b40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 shutdown_connections 2024-05-14T11:16:10.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.209+0000 7f63bb400640 1 -- 172.21.15.116:0/1248426910 wait complete. 2024-05-14T11:16:10.238 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-14T11:16:10.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 Processor -- start 2024-05-14T11:16:10.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 -- start start 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3b740d1e40 0x7f3b740cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b7406f0a0 con 0x7f3b740cd1c0 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b740cbb60 con 0x7f3b7406c700 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b79800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7406f420 con 0x7f3b740d1e40 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b6be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3b740d1e40 0x7f3b740cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b72a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3b740cd1c0 0x7f3b740d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39892/0 (socket says 172.21.15.116:39892) 2024-05-14T11:16:10.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b72a00640 1 -- 172.21.15.116:0/1877813942 learned_addr learned my addr 172.21.15.116:0/1877813942 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:10.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b6be00640 1 -- 172.21.15.116:0/1877813942 >> v1:172.21.15.149:6789/0 conn(0x7f3b7406c700 legacy=0x7f3b7406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b72000640 1 -- 172.21.15.116:0/1877813942 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268848818 0 0) 0x7f3b7406f0a0 con 0x7f3b740cd1c0 2024-05-14T11:16:10.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b6be00640 1 -- 172.21.15.116:0/1877813942 >> v1:172.21.15.116:6789/0 conn(0x7f3b740cd1c0 legacy=0x7f3b740d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.393+0000 7f3b6be00640 1 -- 172.21.15.116:0/1877813942 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7406f700 con 0x7f3b740d1e40 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b6be00640 1 --2- 172.21.15.116:0/1877813942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b740d1e40 0x7f3b740cd5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0097c0 tx=0x7f3b5c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9feeb2436067207 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b72000640 1 -- 172.21.15.116:0/1877813942 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0aa600 con 0x7f3b740d1e40 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b72000640 1 -- 172.21.15.116:0/1877813942 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c0af070 con 0x7f3b740d1e40 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b72000640 1 -- 172.21.15.116:0/1877813942 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0aa9c0 con 0x7f3b740d1e40 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 -- 172.21.15.116:0/1877813942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b740d1e40 msgr2=0x7f3b740cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 --2- 172.21.15.116:0/1877813942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b740d1e40 0x7f3b740cd5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0097c0 tx=0x7f3b5c01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 -- 172.21.15.116:0/1877813942 shutdown_connections 2024-05-14T11:16:10.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 --2- 172.21.15.116:0/1877813942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b740d1e40 0x7f3b740cd5d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 -- 172.21.15.116:0/1877813942 >> 172.21.15.116:0/1877813942 conn(0x7f3b740b4720 msgr2=0x7f3b740b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:10.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 -- 172.21.15.116:0/1877813942 shutdown_connections 2024-05-14T11:16:10.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.397+0000 7f3b79800640 1 -- 172.21.15.116:0/1877813942 wait complete. 2024-05-14T11:16:10.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 Processor -- start 2024-05-14T11:16:10.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 -- start start 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 0x7f3b741571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 0x7f3b740ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7416a9c0 con 0x7f3b7415d170 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7415cf10 con 0x7f3b7406c700 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b79800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7416ab40 con 0x7f3b740cd1c0 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40676/0 (socket says 172.21.15.116:40676) 2024-05-14T11:16:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 -- 172.21.15.116:0/2386245646 learned_addr learned my addr 172.21.15.116:0/2386245646 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:10.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b6be00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 0x7f3b740ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:10.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b72a00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 0x7f3b741571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 -- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 msgr2=0x7f3b740ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 0x7f3b740ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 -- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 msgr2=0x7f3b741571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 0x7f3b741571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b73400640 1 -- 172.21.15.116:0/2386245646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7415dac0 con 0x7f3b7415d170 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b6be00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 0x7f3b740ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.401+0000 7f3b72a00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 0x7f3b741571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b73400640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c760 tx=0x7f3b6c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003b80 con 0x7f3b7415d170 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6c003d20 con 0x7f3b7415d170 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b740c82d0 con 0x7f3b7415d170 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00aa10 con 0x7f3b7415d170 2024-05-14T11:16:10.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b74158780 con 0x7f3b7415d170 2024-05-14T11:16:10.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7406a9c0 con 0x7f3b7415d170 2024-05-14T11:16:10.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b6c00abb0 con 0x7f3b7415d170 2024-05-14T11:16:10.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 0x7f3b50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:10.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f3b6c08efd0 con 0x7f3b7415d170 2024-05-14T11:16:10.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.405+0000 7f3b72a00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 0x7f3b50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:10.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.409+0000 7f3b72a00640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 0x7f3b50071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000f70 tx=0x7f3b64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:10.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.413+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3b7415dac0 con 0x7f3b7415d170 2024-05-14T11:16:10.524 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:16:10.527 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:16:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:10.573+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3b74069de0 con 0x7f3b7415d170 2024-05-14T11:16:11.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b6aa00640 1 -- 172.21.15.116:0/2386245646 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f3b740cbb60 con 0x7f3b7415d170 2024-05-14T11:16:11.218 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 msgr2=0x7f3b50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 0x7f3b50071940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000f70 tx=0x7f3b64009040 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 msgr2=0x7f3b7415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00c760 tx=0x7f3b6c00cc20 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 shutdown_connections 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5006f430 0x7f3b50071940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b7415d170 0x7f3b7415d580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b740cd1c0 0x7f3b740ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 --2- 172.21.15.116:0/2386245646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b7406c700 0x7f3b741571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 >> 172.21.15.116:0/2386245646 conn(0x7f3b740b4720 msgr2=0x7f3b74074020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:11.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 shutdown_connections 2024-05-14T11:16:11.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.217+0000 7f3b79800640 1 -- 172.21.15.116:0/2386245646 wait complete. 2024-05-14T11:16:11.254 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:16:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.401+0000 7fadae000640 1 Processor -- start 2024-05-14T11:16:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.401+0000 7fadae000640 1 -- start start 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadae000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fada8074580 0x7fada815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadae000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80cdd00 0x7fada80ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadae000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada80ce650 con 0x7fada806c700 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadae000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fada80d1eb0 con 0x7fada80cdd00 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadae000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fada815e000 con 0x7fada8074580 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80cdd00 0x7fada80ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80cdd00 0x7fada80ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40682/0 (socket says 172.21.15.116:40682) 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 -- 172.21.15.116:0/2616092506 learned_addr learned my addr 172.21.15.116:0/2616092506 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fadacc00640 1 --2- 172.21.15.116:0/2616092506 >> v2:172.21.15.149:3300/0 conn(0x7fada8074580 0x7fada815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 -- 172.21.15.116:0/2616092506 >> v2:172.21.15.149:3300/0 conn(0x7fada8074580 msgr2=0x7fada815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 --2- 172.21.15.116:0/2616092506 >> v2:172.21.15.149:3300/0 conn(0x7fada8074580 0x7fada815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 -- 172.21.15.116:0/2616092506 >> v1:172.21.15.149:6789/0 conn(0x7fada806c700 legacy=0x7fada806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.405+0000 7fad9fe00640 1 -- 172.21.15.116:0/2616092506 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada80d2150 con 0x7fada80cdd00 2024-05-14T11:16:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fad9fe00640 1 --2- 172.21.15.116:0/2616092506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80cdd00 0x7fada80ce110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada8072550 tx=0x7fad9001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e115204edfa73b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fada7e00640 1 -- 172.21.15.116:0/2616092506 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad900b5070 con 0x7fada80cdd00 2024-05-14T11:16:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fada7e00640 1 -- 172.21.15.116:0/2616092506 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad900a8a10 con 0x7fada80cdd00 2024-05-14T11:16:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fada7e00640 1 -- 172.21.15.116:0/2616092506 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad900a8de0 con 0x7fada80cdd00 2024-05-14T11:16:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 -- 172.21.15.116:0/2616092506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80cdd00 msgr2=0x7fada80ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 --2- 172.21.15.116:0/2616092506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80cdd00 0x7fada80ce110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada8072550 tx=0x7fad9001ed10 comp rx=0 tx=0).stop 2024-05-14T11:16:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 -- 172.21.15.116:0/2616092506 shutdown_connections 2024-05-14T11:16:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 --2- 172.21.15.116:0/2616092506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80cdd00 0x7fada80ce110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 --2- 172.21.15.116:0/2616092506 >> v2:172.21.15.149:3300/0 conn(0x7fada8074580 0x7fada815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 -- 172.21.15.116:0/2616092506 >> 172.21.15.116:0/2616092506 conn(0x7fada80d12f0 msgr2=0x7fada80d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 -- 172.21.15.116:0/2616092506 shutdown_connections 2024-05-14T11:16:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.409+0000 7fadae000640 1 -- 172.21.15.116:0/2616092506 wait complete. 2024-05-14T11:16:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 Processor -- start 2024-05-14T11:16:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- start start 2024-05-14T11:16:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8074580 0x7fada8160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80cdd00 0x7fada8160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fada80cda50 con 0x7fada806c700 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fada80cd740 con 0x7fada8074580 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fada80c8080 con 0x7fada80cdd00 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40688/0 (socket says 172.21.15.116:40688) 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 -- 172.21.15.116:0/1810971510 learned_addr learned my addr 172.21.15.116:0/1810971510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadad600640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80cdd00 0x7fada8160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fad9fe00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8074580 0x7fada8160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 -- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80cdd00 msgr2=0x7fada8160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80cdd00 0x7fada8160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 -- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8074580 msgr2=0x7fada8160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8074580 0x7fada8160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 -- 172.21.15.116:0/1810971510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada80b0500 con 0x7fada806c700 2024-05-14T11:16:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadacc00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e870 tx=0x7fad9800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98017070 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9800c940 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada80b10d0 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9800ccf0 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.413+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada80cf060 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada806a9c0 con 0x7fada806c700 2024-05-14T11:16:11.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad9801a070 con 0x7fada806c700 2024-05-14T11:16:11.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fada5600640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:11.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fad9808f900 con 0x7fada806c700 2024-05-14T11:16:11.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fad9fe00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:11.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.417+0000 7fad9fe00640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad900b2000 tx=0x7fad900aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:11.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.421+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fad9805c1e0 con 0x7fada806c700 2024-05-14T11:16:11.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:11.609+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fada80690e0 con 0x7fada806c700 2024-05-14T11:16:12.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.217+0000 7fada5600640 1 -- 172.21.15.116:0/1810971510 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fad980604e0 con 0x7fada806c700 2024-05-14T11:16:12.220 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T11:16:12.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 msgr2=0x7fad84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:12.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad900b2000 tx=0x7fad900aa070 comp rx=0 tx=0).stop 2024-05-14T11:16:12.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 msgr2=0x7fada815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:12.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e870 tx=0x7fad9800ec70 comp rx=0 tx=0).stop 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 shutdown_connections 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8074580 0x7fada8160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80cdd00 0x7fada8160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 --2- 172.21.15.116:0/1810971510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806c700 0x7fada815c800 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 >> 172.21.15.116:0/1810971510 conn(0x7fada80d12f0 msgr2=0x7fada8072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 shutdown_connections 2024-05-14T11:16:12.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.221+0000 7fadae000640 1 -- 172.21.15.116:0/1810971510 wait complete. 2024-05-14T11:16:12.272 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-14T11:16:12.272 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-14T11:16:12.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 Processor -- start 2024-05-14T11:16:12.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 -- start start 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 0x7f56b006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56b006d710 0x7f56b006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00d04c0 con 0x7f56b00cda30 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f56b00d0270 con 0x7f56b006d710 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b8400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f56b00b0c30 con 0x7f56b00cd620 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56b006d710 0x7f56b006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56b006d710 0x7f56b006fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40696/0 (socket says 172.21.15.116:40696) 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 -- 172.21.15.116:0/1837113847 learned_addr learned my addr 172.21.15.116:0/1837113847 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b5a00640 1 --2- 172.21.15.116:0/1837113847 >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 0x7f56b006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 -- 172.21.15.116:0/1837113847 >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 msgr2=0x7f56b006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 --2- 172.21.15.116:0/1837113847 >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 0x7f56b006f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 -- 172.21.15.116:0/1837113847 >> v1:172.21.15.149:6789/0 conn(0x7f56b00cda30 legacy=0x7f56b0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b6400640 1 -- 172.21.15.116:0/1837113847 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b00d07a0 con 0x7f56b006d710 2024-05-14T11:16:12.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.429+0000 7f56b5a00640 1 --2- 172.21.15.116:0/1837113847 >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 0x7f56b006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:12.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b6400640 1 --2- 172.21.15.116:0/1837113847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b006d710 0x7f56b006fbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56b00b6890 tx=0x7f56a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc73b0a04cd65074 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:12.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56afe00640 1 -- 172.21.15.116:0/1837113847 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80a5590 con 0x7f56b006d710 2024-05-14T11:16:12.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56afe00640 1 -- 172.21.15.116:0/1837113847 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a80a5730 con 0x7f56b006d710 2024-05-14T11:16:12.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56afe00640 1 -- 172.21.15.116:0/1837113847 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80a5b00 con 0x7f56b006d710 2024-05-14T11:16:12.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 -- 172.21.15.116:0/1837113847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b006d710 msgr2=0x7f56b006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:12.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 --2- 172.21.15.116:0/1837113847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b006d710 0x7f56b006fbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56b00b6890 tx=0x7f56a801c9f0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 -- 172.21.15.116:0/1837113847 shutdown_connections 2024-05-14T11:16:12.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 --2- 172.21.15.116:0/1837113847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b006d710 0x7f56b006fbf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 --2- 172.21.15.116:0/1837113847 >> v2:172.21.15.149:3300/0 conn(0x7f56b00cd620 0x7f56b006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 -- 172.21.15.116:0/1837113847 >> 172.21.15.116:0/1837113847 conn(0x7f56b00cde80 msgr2=0x7f56b00ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:12.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 -- 172.21.15.116:0/1837113847 shutdown_connections 2024-05-14T11:16:12.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.433+0000 7f56b8400640 1 -- 172.21.15.116:0/1837113847 wait complete. 2024-05-14T11:16:12.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 Processor -- start 2024-05-14T11:16:12.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 -- start start 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b006d710 0x7f56b0154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 0x7f56b0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56b016ae40 con 0x7f56b0158ad0 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56b016acc0 con 0x7f56b006d710 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b8400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56b016afc0 con 0x7f56b00cd620 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40706/0 (socket says 172.21.15.116:40706) 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 -- 172.21.15.116:0/2061011862 learned_addr learned my addr 172.21.15.116:0/2061011862 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:12.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b5a00640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b006d710 0x7f56b0154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b5000640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 0x7f56b0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 -- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 msgr2=0x7f56b0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 0x7f56b0154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 -- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b006d710 msgr2=0x7f56b0154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b006d710 0x7f56b0154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b6400640 1 -- 172.21.15.116:0/2061011862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b015b470 con 0x7f56b0158ad0 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.437+0000 7f56b5000640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 0x7f56b0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:12.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56b6400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56a80af040 tx=0x7f56a8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8009510 con 0x7f56b0158ad0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a80b8470 con 0x7f56b0158ad0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80b8860 con 0x7f56b0158ad0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b015b750 con 0x7f56b0158ad0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b015bd10 con 0x7f56b0158ad0 2024-05-14T11:16:12.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b006aa30 con 0x7f56b0158ad0 2024-05-14T11:16:12.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.441+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56a80c0020 con 0x7f56b0158ad0 2024-05-14T11:16:12.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.445+0000 7f56ad600640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 0x7f5690071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:12.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.445+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f56a812a6d0 con 0x7f56b0158ad0 2024-05-14T11:16:12.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.445+0000 7f56b5a00640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 0x7f5690071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.445+0000 7f56b5a00640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 0x7f5690071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.445+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f56a80165e0 con 0x7f56b0158ad0 2024-05-14T11:16:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:12.613+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f56b00cfb40 con 0x7f56b0158ad0 2024-05-14T11:16:13.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56ad600640 1 -- 172.21.15.116:0/2061011862 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f56a80f76f0 con 0x7f56b0158ad0 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 msgr2=0x7f5690071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 0x7f5690071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4008040 comp rx=0 tx=0).stop 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 msgr2=0x7f56b015c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56a80af040 tx=0x7f56a8019790 comp rx=0 tx=0).stop 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 shutdown_connections 2024-05-14T11:16:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569006f400 0x7f5690071910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56b0158ad0 0x7f56b015c510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56b00cd620 0x7f56b0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 --2- 172.21.15.116:0/2061011862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b006d710 0x7f56b0154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 >> 172.21.15.116:0/2061011862 conn(0x7f56b00cde80 msgr2=0x7f56b0156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 shutdown_connections 2024-05-14T11:16:13.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.221+0000 7f56b8400640 1 -- 172.21.15.116:0/2061011862 wait complete. 2024-05-14T11:16:13.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:16:13.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.401+0000 7f4f04a00640 1 Processor -- start 2024-05-14T11:16:13.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.401+0000 7f4f04a00640 1 -- start start 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4f04a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 0x7f4f000d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4f04a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4f000af380 0x7f4f000af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4f04a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f000d1fc0 con 0x7f4f00156fa0 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4f04a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4f000706f0 con 0x7f4f000af380 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4f04a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4f001573b0 con 0x7f4f000d1670 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efe800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4f000af380 0x7f4f000af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efde00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4f00156fa0 0x7f4f0006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39894/0 (socket says 172.21.15.116:39894) 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efde00640 1 -- 172.21.15.116:0/91743814 learned_addr learned my addr 172.21.15.116:0/91743814 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:13.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efd400640 1 --2- 172.21.15.116:0/91743814 >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 0x7f4f000d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efe800640 1 -- 172.21.15.116:0/91743814 >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 msgr2=0x7f4f000d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efe800640 1 --2- 172.21.15.116:0/91743814 >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 0x7f4f000d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efe800640 1 -- 172.21.15.116:0/91743814 >> v1:172.21.15.116:6789/0 conn(0x7f4f00156fa0 legacy=0x7f4f0006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efe800640 1 -- 172.21.15.116:0/91743814 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f00160510 con 0x7f4f000af380 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efca00640 1 -- 172.21.15.116:0/91743814 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243717528 0 0) 0x7f4ef00014a0 con 0x7f4f00156fa0 2024-05-14T11:16:13.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.405+0000 7f4efd400640 1 --2- 172.21.15.116:0/91743814 >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 0x7f4f000d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:13.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4efe800640 1 --2- 172.21.15.116:0/91743814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000af790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f00070c70 tx=0x7f4ef801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cda2a4adab5f01c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:13.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4efca00640 1 -- 172.21.15.116:0/91743814 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80b0070 con 0x7f4f000af380 2024-05-14T11:16:13.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4efca00640 1 -- 172.21.15.116:0/91743814 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef80ab600 con 0x7f4f000af380 2024-05-14T11:16:13.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4efca00640 1 -- 172.21.15.116:0/91743814 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80ab9d0 con 0x7f4f000af380 2024-05-14T11:16:13.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 -- 172.21.15.116:0/91743814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 msgr2=0x7f4f000af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 --2- 172.21.15.116:0/91743814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000af790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f00070c70 tx=0x7f4ef801c480 comp rx=0 tx=0).stop 2024-05-14T11:16:13.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 -- 172.21.15.116:0/91743814 shutdown_connections 2024-05-14T11:16:13.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 --2- 172.21.15.116:0/91743814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000af790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 --2- 172.21.15.116:0/91743814 >> v2:172.21.15.149:3300/0 conn(0x7f4f000d1670 0x7f4f000d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 -- 172.21.15.116:0/91743814 >> 172.21.15.116:0/91743814 conn(0x7f4f000b4620 msgr2=0x7f4f000b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:13.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 -- 172.21.15.116:0/91743814 shutdown_connections 2024-05-14T11:16:13.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.409+0000 7f4f04a00640 1 -- 172.21.15.116:0/91743814 wait complete. 2024-05-14T11:16:13.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 Processor -- start 2024-05-14T11:16:13.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- start start 2024-05-14T11:16:13.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f000d1670 0x7f4f001503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f00156fa0 0x7f4f00155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b0660 con 0x7f4f000af380 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b04e0 con 0x7f4f00156fa0 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4f00169810 con 0x7f4f000d1670 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40720/0 (socket says 172.21.15.116:40720) 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 -- 172.21.15.116:0/3346631420 learned_addr learned my addr 172.21.15.116:0/3346631420 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efe800640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f00156fa0 0x7f4f00155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efd400640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f000d1670 0x7f4f001503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 -- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f000d1670 msgr2=0x7f4f001503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f000d1670 0x7f4f001503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 -- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f00156fa0 msgr2=0x7f4f00155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f00156fa0 0x7f4f00155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:13.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 -- 172.21.15.116:0/3346631420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f00073920 con 0x7f4f000af380 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4efde00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0002240 tx=0x7f4ef000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0018070 con 0x7f4f000af380 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef000ce50 con 0x7f4f000af380 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f00073c00 con 0x7f4f000af380 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000a840 con 0x7f4f000af380 2024-05-14T11:16:13.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.413+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f000741c0 con 0x7f4f000af380 2024-05-14T11:16:13.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0006a9c0 con 0x7f4f000af380 2024-05-14T11:16:13.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ef001b070 con 0x7f4f000af380 2024-05-14T11:16:13.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4ef6000640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 0x7f4ee0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:13.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f4ef008e9e0 con 0x7f4f000af380 2024-05-14T11:16:13.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4efd400640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 0x7f4ee0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:13.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.417+0000 7f4efd400640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 0x7f4ee0071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f000d0a50 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:13.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.421+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4f000d1fc0 con 0x7f4f000af380 2024-05-14T11:16:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:13.581+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4f000d1030 con 0x7f4f000af380 2024-05-14T11:16:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.261+0000 7f4ef6000640 1 -- 172.21.15.116:0/3346631420 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f4ef005b140 con 0x7f4f000af380 2024-05-14T11:16:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data_ec' created 2024-05-14T11:16:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 msgr2=0x7f4ee0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 0x7f4ee0071940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f000d0a50 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 msgr2=0x7f4f000cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0002240 tx=0x7f4ef000cc00 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 shutdown_connections 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4ee006f430 0x7f4ee0071940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f00156fa0 0x7f4f00155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f000d1670 0x7f4f001503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 --2- 172.21.15.116:0/3346631420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f000af380 0x7f4f000cfcf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 >> 172.21.15.116:0/3346631420 conn(0x7f4f000b4620 msgr2=0x7f4f000b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 shutdown_connections 2024-05-14T11:16:15.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.265+0000 7f4f04a00640 1 -- 172.21.15.116:0/3346631420 wait complete. 2024-05-14T11:16:15.303 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-14T11:16:15.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 Processor -- start 2024-05-14T11:16:15.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 -- start start 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa14006fdc0 0x7fa14006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1400d04c0 con 0x7fa1400cd620 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1400d07a0 con 0x7fa1400cda30 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa147400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa1400d0270 con 0x7fa14006fdc0 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa137e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa14006fdc0 0x7fa14006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa144a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa1400cd620 0x7fa14006f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39896/0 (socket says 172.21.15.116:39896) 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa137e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa14006fdc0 0x7fa14006c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40736/0 (socket says 172.21.15.116:40736) 2024-05-14T11:16:15.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa144a00640 1 -- 172.21.15.116:0/278132598 learned_addr learned my addr 172.21.15.116:0/278132598 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:15.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa137e00640 1 -- 172.21.15.116:0/278132598 >> v1:172.21.15.149:6789/0 conn(0x7fa1400cda30 legacy=0x7fa14006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa137e00640 1 -- 172.21.15.116:0/278132598 >> v1:172.21.15.116:6789/0 conn(0x7fa1400cd620 legacy=0x7fa14006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa137e00640 1 -- 172.21.15.116:0/278132598 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1400b08c0 con 0x7fa14006fdc0 2024-05-14T11:16:15.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.457+0000 7fa13fe00640 1 -- 172.21.15.116:0/278132598 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239318284 0 0) 0x7fa1400d04c0 con 0x7fa1400cd620 2024-05-14T11:16:15.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa137e00640 1 --2- 172.21.15.116:0/278132598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14006fdc0 0x7fa14006c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdb7d7c68aac3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:15.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa13fe00640 1 -- 172.21.15.116:0/278132598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280b0070 con 0x7fa14006fdc0 2024-05-14T11:16:15.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa13fe00640 1 -- 172.21.15.116:0/278132598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1280ab600 con 0x7fa14006fdc0 2024-05-14T11:16:15.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa13fe00640 1 -- 172.21.15.116:0/278132598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280ab9d0 con 0x7fa14006fdc0 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 -- 172.21.15.116:0/278132598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14006fdc0 msgr2=0x7fa14006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 --2- 172.21.15.116:0/278132598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14006fdc0 0x7fa14006c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 -- 172.21.15.116:0/278132598 shutdown_connections 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 --2- 172.21.15.116:0/278132598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14006fdc0 0x7fa14006c700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 -- 172.21.15.116:0/278132598 >> 172.21.15.116:0/278132598 conn(0x7fa1400cde80 msgr2=0x7fa1400ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 -- 172.21.15.116:0/278132598 shutdown_connections 2024-05-14T11:16:15.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.461+0000 7fa147400640 1 -- 172.21.15.116:0/278132598 wait complete. 2024-05-14T11:16:15.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 Processor -- start 2024-05-14T11:16:15.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 -- start start 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1400cd620 0x7fa140136c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa140144bd0 0x7fa14014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa14016c4e0 con 0x7fa14014c8c0 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa1400b0c30 con 0x7fa1400cd620 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa147400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa14016c660 con 0x7fa140144bd0 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa145400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa145400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40738/0 (socket says 172.21.15.116:40738) 2024-05-14T11:16:15.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa145400640 1 -- 172.21.15.116:0/1125890794 learned_addr learned my addr 172.21.15.116:0/1125890794 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa137e00640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa140144bd0 0x7fa14014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.465+0000 7fa144a00640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1400cd620 0x7fa140136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 -- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa140144bd0 msgr2=0x7fa14014c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa140144bd0 0x7fa14014c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 -- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1400cd620 msgr2=0x7fa140136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1400cd620 0x7fa140136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 -- 172.21.15.116:0/1125890794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14014ccd0 con 0x7fa14014c8c0 2024-05-14T11:16:15.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa145400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c710 tx=0x7fa13800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138003b80 con 0x7fa14014c8c0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa138003d20 con 0x7fa14014c8c0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13800aa90 con 0x7fa14014c8c0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14015c430 con 0x7fa14014c8c0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1400b46d0 con 0x7fa14014c8c0 2024-05-14T11:16:15.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14015d4b0 con 0x7fa14014c8c0 2024-05-14T11:16:15.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.469+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa13800ac30 con 0x7fa14014c8c0 2024-05-14T11:16:15.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.473+0000 7fa13d600640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 0x7fa11c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:15.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.473+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fa13808f320 con 0x7fa14014c8c0 2024-05-14T11:16:15.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.473+0000 7fa144a00640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 0x7fa11c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:15.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.473+0000 7fa144a00640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 0x7fa11c071940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa130000f70 tx=0x7fa130007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:15.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.477+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa14014ccd0 con 0x7fa14014c8c0 2024-05-14T11:16:15.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:15.637+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa14006aa30 con 0x7fa14014c8c0 2024-05-14T11:16:16.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.261+0000 7fa13d600640 1 -- 172.21.15.116:0/1125890794 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fa13800ce20 con 0x7fa14014c8c0 2024-05-14T11:16:16.265 INFO:teuthology.orchestra.run.smithi116.stderr:set pool 4 allow_ec_overwrites to true 2024-05-14T11:16:16.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 msgr2=0x7fa11c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 0x7fa11c071940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa130000f70 tx=0x7fa130007240 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 msgr2=0x7fa14015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c710 tx=0x7fa13800cbd0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 shutdown_connections 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa11c06f430 0x7fa11c071940 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa130000f70 tx=0x7fa130007240 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14014c8c0 0x7fa14015b380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa140144bd0 0x7fa14014c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 --2- 172.21.15.116:0/1125890794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1400cd620 0x7fa140136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.265+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 >> 172.21.15.116:0/1125890794 conn(0x7fa1400cde80 msgr2=0x7fa1400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:16.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.269+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 shutdown_connections 2024-05-14T11:16:16.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.269+0000 7fa147400640 1 -- 172.21.15.116:0/1125890794 wait complete. 2024-05-14T11:16:16.307 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-14T11:16:16.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 Processor -- start 2024-05-14T11:16:16.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 -- start start 2024-05-14T11:16:16.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67180b47d0 0x7f67180ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:16.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67180b3060 con 0x7f67180cdf80 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67180b3300 con 0x7f67180d1c20 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671fc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f67180b35e0 con 0x7f67180b47d0 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671dc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67180b47d0 0x7f67180ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671d200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f67180cdf80 0x7f67180d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39908/0 (socket says 172.21.15.116:39908) 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.461+0000 7f671d200640 1 -- 172.21.15.116:0/2843563478 learned_addr learned my addr 172.21.15.116:0/2843563478 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f671dc00640 1 -- 172.21.15.116:0/2843563478 >> v1:172.21.15.149:6789/0 conn(0x7f67180d1c20 legacy=0x7f67180b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f671dc00640 1 -- 172.21.15.116:0/2843563478 >> v1:172.21.15.116:6789/0 conn(0x7f67180cdf80 legacy=0x7f67180d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f671dc00640 1 -- 172.21.15.116:0/2843563478 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67180b38c0 con 0x7f67180b47d0 2024-05-14T11:16:16.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f671dc00640 1 --2- 172.21.15.116:0/2843563478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180ce390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6710009d90 tx=0x7f671001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e3d4d189f2d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:16.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f6717e00640 1 -- 172.21.15.116:0/2843563478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100b2070 con 0x7f67180b47d0 2024-05-14T11:16:16.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f6717e00640 1 -- 172.21.15.116:0/2843563478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67100a5a20 con 0x7f67180b47d0 2024-05-14T11:16:16.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.465+0000 7f6717e00640 1 -- 172.21.15.116:0/2843563478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100a5df0 con 0x7f67180b47d0 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- 172.21.15.116:0/2843563478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 msgr2=0x7f67180ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 --2- 172.21.15.116:0/2843563478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180ce390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6710009d90 tx=0x7f671001c940 comp rx=0 tx=0).stop 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- 172.21.15.116:0/2843563478 shutdown_connections 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 --2- 172.21.15.116:0/2843563478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180ce390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- 172.21.15.116:0/2843563478 >> 172.21.15.116:0/2843563478 conn(0x7f67180d0770 msgr2=0x7f67180d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- 172.21.15.116:0/2843563478 shutdown_connections 2024-05-14T11:16:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- 172.21.15.116:0/2843563478 wait complete. 2024-05-14T11:16:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 Processor -- start 2024-05-14T11:16:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- start start 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67180cdf80 0x7f67181437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6718143ce0 0x7f6718142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f671816dd70 con 0x7f67180b47d0 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f671806ffd0 con 0x7f67180cdf80 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.469+0000 7f671fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f671816def0 con 0x7f6718143ce0 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40754/0 (socket says 172.21.15.116:40754) 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 -- 172.21.15.116:0/823766582 learned_addr learned my addr 172.21.15.116:0/823766582 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:16.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671dc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6718143ce0 0x7f6718142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f670fe00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67180cdf80 0x7f67181437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 -- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6718143ce0 msgr2=0x7f6718142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6718143ce0 0x7f6718142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 -- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67180cdf80 msgr2=0x7f67181437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67180cdf80 0x7f67181437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 -- 172.21.15.116:0/823766582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67181440f0 con 0x7f67180b47d0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671d200640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f670800c4d0 tx=0x7f670800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f67180b47d0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6708005cf0 con 0x7f67180b47d0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708007270 con 0x7f67180b47d0 2024-05-14T11:16:16.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67181418b0 con 0x7f67180b47d0 2024-05-14T11:16:16.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6718141e70 con 0x7f67180b47d0 2024-05-14T11:16:16.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671806a9c0 con 0x7f67180b47d0 2024-05-14T11:16:16.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.473+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f67180b47d0 2024-05-14T11:16:16.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.477+0000 7f6715600640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 0x7f66f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:16.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.477+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f670808e7c0 con 0x7f67180b47d0 2024-05-14T11:16:16.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.477+0000 7f670fe00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 0x7f66f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:16.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.477+0000 7f670fe00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 0x7f66f4071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6718158680 tx=0x7f6700009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:16.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.481+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f67181440f0 con 0x7f67180b47d0 2024-05-14T11:16:16.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:16.673+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f67180b9590 con 0x7f67180b47d0 2024-05-14T11:16:17.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.297+0000 7f6715600640 1 -- 172.21.15.116:0/823766582 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f670805adc0 con 0x7f67180b47d0 2024-05-14T11:16:17.301 INFO:teuthology.orchestra.run.smithi116.stderr:added data pool 4 to fsmap 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 msgr2=0x7f66f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 0x7f66f4071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6718158680 tx=0x7f6700009040 comp rx=0 tx=0).stop 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 msgr2=0x7f67180c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f670800c4d0 tx=0x7f670800c990 comp rx=0 tx=0).stop 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 shutdown_connections 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f66f406f400 0x7f66f4071910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67180cdf80 0x7f67181437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6718143ce0 0x7f6718142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 --2- 172.21.15.116:0/823766582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67180b47d0 0x7f67180c9a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 >> 172.21.15.116:0/823766582 conn(0x7f67180d0770 msgr2=0x7f67180707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:17.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 shutdown_connections 2024-05-14T11:16:17.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.301+0000 7f671fc00640 1 -- 172.21.15.116:0/823766582 wait complete. 2024-05-14T11:16:17.326 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:17.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 Processor -- start 2024-05-14T11:16:17.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 -- start start 2024-05-14T11:16:17.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f738c071bd0 0x7f738c0d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f738c0d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f738c0ccaa0 con 0x7f738c072470 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f738c0b1f00 con 0x7f738c071bd0 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7394200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f738c15e070 con 0x7f738c153c90 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7391800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f738c072470 0x7f738c0714c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39910/0 (socket says 172.21.15.116:39910) 2024-05-14T11:16:17.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7391800640 1 -- 172.21.15.116:0/1560488155 learned_addr learned my addr 172.21.15.116:0/1560488155 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166744648 0 0) 0x7f738c0ccaa0 con 0x7f738c072470 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7374003530 con 0x7f738c072470 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7390e00640 1 --2- 172.21.15.116:0/1560488155 >> v2:172.21.15.149:3300/0 conn(0x7f738c071bd0 0x7f738c0d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f7392200640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f738c0d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 261875482 0 0) 0x7f7374003530 con 0x7f738c072470 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 msgr2=0x7f738c0d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f738c0d1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 >> v2:172.21.15.149:3300/0 conn(0x7f738c071bd0 msgr2=0x7f738c0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:17.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 --2- 172.21.15.116:0/1560488155 >> v2:172.21.15.149:3300/0 conn(0x7f738c071bd0 0x7f738c0d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c15d2b0 con 0x7f738c072470 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f7380001700 con 0x7f738c072470 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 >> v1:172.21.15.116:6789/0 conn(0x7f738c072470 legacy=0x7f738c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7374003a10 0x7f7374003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374004810 0x7f7374004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f7374005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0b1f00 con 0x7f7374004810 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f738c15e070 con 0x7f7374003a10 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.473+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f737400ba80 con 0x7f738c153c90 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7374003a10 0x7f7374003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7392200640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f7374005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7390e00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374004810 0x7f7374004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 -- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 msgr2=0x7f7374005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c153c90 0x7f7374005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 -- 172.21.15.116:0/1560488155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374004810 msgr2=0x7f7374004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374004810 0x7f7374004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 -- 172.21.15.116:0/1560488155 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7374003530 con 0x7f7374003a10 2024-05-14T11:16:17.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7390e00640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374004810 0x7f7374004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:17.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f7391800640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7374003a10 0x7f7374003e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f738000c2f0 tx=0x7f738001f3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba56b7ae3d123045 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.477+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380007d60 con 0x7f7374003a10 2024-05-14T11:16:17.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380005b00 con 0x7f7374003a10 2024-05-14T11:16:17.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f738be00640 1 -- 172.21.15.116:0/1560488155 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380007070 con 0x7f7374003a10 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7374003a10 msgr2=0x7f7374003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 --2- 172.21.15.116:0/1560488155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7374003a10 0x7f7374003e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f738000c2f0 tx=0x7f738001f3f0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7392200640 1 -- 172.21.15.116:0/1560488155 reap_dead start 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- 172.21.15.116:0/1560488155 shutdown_connections 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- 172.21.15.116:0/1560488155 >> 172.21.15.116:0/1560488155 conn(0x7f738c073ca0 msgr2=0x7f738c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- 172.21.15.116:0/1560488155 shutdown_connections 2024-05-14T11:16:17.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- 172.21.15.116:0/1560488155 wait complete. 2024-05-14T11:16:17.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 Processor -- start 2024-05-14T11:16:17.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.481+0000 7f7394200640 1 -- start start 2024-05-14T11:16:17.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c0cbcc0 0x7f738c1efb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f738c06e440 0x7f738c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f738c15d870 con 0x7f7374003a10 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f738c15d050 con 0x7f738c06e440 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f738c15db50 con 0x7f738c0cbcc0 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40776/0 (socket says 172.21.15.116:40776) 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 -- 172.21.15.116:0/1350976179 learned_addr learned my addr 172.21.15.116:0/1350976179 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7390e00640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c0cbcc0 0x7f738c1efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7392200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f738c06e440 0x7f738c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 -- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c0cbcc0 msgr2=0x7f738c1efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c0cbcc0 0x7f738c1efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 -- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f738c06e440 msgr2=0x7f738c06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f738c06e440 0x7f738c06e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 -- 172.21.15.116:0/1350976179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c0c9470 con 0x7f7374003a10 2024-05-14T11:16:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7391800640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f738001daf0 tx=0x7f7374005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800047f0 con 0x7f7374003a10 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380004990 con 0x7f7374003a10 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380004d60 con 0x7f7374003a10 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c06ed90 con 0x7f7374003a10 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c0c96e0 con 0x7f7374003a10 2024-05-14T11:16:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.485+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c06a9c0 con 0x7f7374003a10 2024-05-14T11:16:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.489+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7380007070 con 0x7f7374003a10 2024-05-14T11:16:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.489+0000 7f7389600640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 0x7f736c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.489+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f738012c520 con 0x7f7374003a10 2024-05-14T11:16:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.489+0000 7f7390e00640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 0x7f736c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.489+0000 7f7390e00640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 0x7f736c071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f738c152fd0 tx=0x7f7378009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.493+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f73800c2030 con 0x7f7374003a10 2024-05-14T11:16:17.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.681+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f738c06fa50 con 0x7f7374003a10 2024-05-14T11:16:17.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7389600640 1 -- 172.21.15.116:0/1350976179 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f738c06ed90 con 0x7f7374003a10 2024-05-14T11:16:17.685 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:17.685 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:16.676555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:17.686 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 7 2024-05-14T11:16:17.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 msgr2=0x7f736c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 0x7f736c071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f738c152fd0 tx=0x7f7378009190 comp rx=0 tx=0).stop 2024-05-14T11:16:17.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 msgr2=0x7f738c0cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f738001daf0 tx=0x7f7374005840 comp rx=0 tx=0).stop 2024-05-14T11:16:17.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 shutdown_connections 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f736c06f400 0x7f736c071910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f738c06e440 0x7f738c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f738c0cbcc0 0x7f738c1efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 --2- 172.21.15.116:0/1350976179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7374003a10 0x7f738c0cb070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 >> 172.21.15.116:0/1350976179 conn(0x7f738c073ca0 msgr2=0x7f738c0c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 shutdown_connections 2024-05-14T11:16:17.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.685+0000 7f7394200640 1 -- 172.21.15.116:0/1350976179 wait complete. 2024-05-14T11:16:17.707 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:17.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 Processor -- start 2024-05-14T11:16:17.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 -- start start 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f43d015fe10 0x7f43d00d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d006eb90 con 0x7f43d0164050 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d0160220 con 0x7f43d006e780 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43d5800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f43d0164510 con 0x7f43d015fe10 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43cf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f43d015fe10 0x7f43d00d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43c6000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f43d0164050 0x7f43d00cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60680/0 (socket says 172.21.15.116:60680) 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.853+0000 7f43c6000640 1 -- 172.21.15.116:0/3010918065 learned_addr learned my addr 172.21.15.116:0/3010918065 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43cf400640 1 -- 172.21.15.116:0/3010918065 >> v1:172.21.15.149:6789/0 conn(0x7f43d006e780 legacy=0x7f43d0163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43cf400640 1 -- 172.21.15.116:0/3010918065 >> v1:172.21.15.116:6789/0 conn(0x7f43d0164050 legacy=0x7f43d00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43cf400640 1 -- 172.21.15.116:0/3010918065 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00cd1f0 con 0x7f43d015fe10 2024-05-14T11:16:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43cf400640 1 --2- 172.21.15.116:0/3010918065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d015fe10 0x7f43d00d0010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009d90 tx=0x7f43c801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0a030ec4796507 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43ce000640 1 -- 172.21.15.116:0/3010918065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b2070 con 0x7f43d015fe10 2024-05-14T11:16:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43ce000640 1 -- 172.21.15.116:0/3010918065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80a5970 con 0x7f43d015fe10 2024-05-14T11:16:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43ce000640 1 -- 172.21.15.116:0/3010918065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80a5d40 con 0x7f43d015fe10 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 -- 172.21.15.116:0/3010918065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d015fe10 msgr2=0x7f43d00d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 --2- 172.21.15.116:0/3010918065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d015fe10 0x7f43d00d0010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009d90 tx=0x7f43c801c8b0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 -- 172.21.15.116:0/3010918065 shutdown_connections 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 --2- 172.21.15.116:0/3010918065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d015fe10 0x7f43d00d0010 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 -- 172.21.15.116:0/3010918065 >> 172.21.15.116:0/3010918065 conn(0x7f43d00ce570 msgr2=0x7f43d00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 -- 172.21.15.116:0/3010918065 shutdown_connections 2024-05-14T11:16:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.857+0000 7f43d5800640 1 -- 172.21.15.116:0/3010918065 wait complete. 2024-05-14T11:16:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 Processor -- start 2024-05-14T11:16:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 -- start start 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 0x7f43d015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43d015eb40 0x7f43d0149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43d016dcf0 con 0x7f43d006e780 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43d00c5340 con 0x7f43d015eb40 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43d5800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f43d016de70 con 0x7f43d015fe10 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43cea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43cea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41152/0 (socket says 172.21.15.116:41152) 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.861+0000 7f43cea00640 1 -- 172.21.15.116:0/109774592 learned_addr learned my addr 172.21.15.116:0/109774592 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:17.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cf400640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43d015eb40 0x7f43d0149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c6000640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 0x7f43d015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 -- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 msgr2=0x7f43d015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 0x7f43d015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 -- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43d015eb40 msgr2=0x7f43d0149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43d015eb40 0x7f43d0149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 -- 172.21.15.116:0/109774592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d015ef50 con 0x7f43d006e780 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c6000640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 0x7f43d015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43cea00640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c3a0 tx=0x7f43c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0017070 con 0x7f43d006e780 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c00052d0 con 0x7f43d006e780 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d0148030 con 0x7f43d006e780 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0003650 con 0x7f43d006e780 2024-05-14T11:16:17.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01569c0 con 0x7f43d006e780 2024-05-14T11:16:17.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d006a9c0 con 0x7f43d006e780 2024-05-14T11:16:17.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43c001a070 con 0x7f43d006e780 2024-05-14T11:16:17.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 0x7f43ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:17.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.865+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f43c008e9a0 con 0x7f43d006e780 2024-05-14T11:16:17.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.869+0000 7f43c6000640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 0x7f43ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:17.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.869+0000 7f43c6000640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 0x7f43ac071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43d00a2e10 tx=0x7f43b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:17.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:17.873+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f43d015ef50 con 0x7f43d006e780 2024-05-14T11:16:18.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.029+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43d00b7a70 con 0x7f43d006e780 2024-05-14T11:16:18.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43c7400640 1 -- 172.21.15.116:0/109774592 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f43c005ae60 con 0x7f43d006e780 2024-05-14T11:16:18.034 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:18.034 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 msgr2=0x7f43ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 0x7f43ac071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43d00a2e10 tx=0x7f43b8009040 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 msgr2=0x7f43d00a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c3a0 tx=0x7f43c000c860 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 shutdown_connections 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f43ac06f400 0x7f43ac071910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43d015eb40 0x7f43d0149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43d015fe10 0x7f43d015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 --2- 172.21.15.116:0/109774592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d006e780 0x7f43d00a1750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 >> 172.21.15.116:0/109774592 conn(0x7f43d00ce570 msgr2=0x7f43d006f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 shutdown_connections 2024-05-14T11:16:18.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.033+0000 7f43d5800640 1 -- 172.21.15.116:0/109774592 wait complete. 2024-05-14T11:16:18.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:18.200 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:16:18.197+0000 7f4603200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:16:18.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 Processor -- start 2024-05-14T11:16:18.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 -- start start 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce1015db30 0x7fce100aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce100d1670 0x7fce100d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1006ff20 con 0x7fce10161600 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fce1006d710 con 0x7fce1015db30 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce15200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fce100ae100 con 0x7fce100d1670 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce1015db30 0x7fce100aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce1015db30 0x7fce100aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41158/0 (socket says 172.21.15.116:41158) 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 -- 172.21.15.116:0/323487776 learned_addr learned my addr 172.21.15.116:0/323487776 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0d400640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce100d1670 0x7fce100d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 -- 172.21.15.116:0/323487776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce100d1670 msgr2=0x7fce100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce100d1670 0x7fce100d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 -- 172.21.15.116:0/323487776 >> v1:172.21.15.149:6789/0 conn(0x7fce10161600 legacy=0x7fce10161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.201+0000 7fce0de00640 1 -- 172.21.15.116:0/323487776 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100d1fc0 con 0x7fce1015db30 2024-05-14T11:16:18.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce0de00640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce1015db30 0x7fce100aeaa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce00009870 tx=0x7fce0001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174a3c42384bcc44 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce0ca00640 1 -- 172.21.15.116:0/323487776 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b0070 con 0x7fce1015db30 2024-05-14T11:16:18.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce0ca00640 1 -- 172.21.15.116:0/323487776 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000ab600 con 0x7fce1015db30 2024-05-14T11:16:18.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce0ca00640 1 -- 172.21.15.116:0/323487776 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000ab9d0 con 0x7fce1015db30 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 -- 172.21.15.116:0/323487776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce1015db30 msgr2=0x7fce100aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce1015db30 0x7fce100aeaa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce00009870 tx=0x7fce0001c4d0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 -- 172.21.15.116:0/323487776 shutdown_connections 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce100d1670 0x7fce100d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 --2- 172.21.15.116:0/323487776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce1015db30 0x7fce100aeaa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 -- 172.21.15.116:0/323487776 >> 172.21.15.116:0/323487776 conn(0x7fce100b4620 msgr2=0x7fce100b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 -- 172.21.15.116:0/323487776 shutdown_connections 2024-05-14T11:16:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.205+0000 7fce15200640 1 -- 172.21.15.116:0/323487776 wait complete. 2024-05-14T11:16:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 Processor -- start 2024-05-14T11:16:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 -- start start 2024-05-14T11:16:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce1015db30 0x7fce1014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 0x7fce101594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce1015df40 con 0x7fce100d1670 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce100aefe0 con 0x7fce10161600 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce15200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fce1016af80 con 0x7fce1015db30 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41162/0 (socket says 172.21.15.116:41162) 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 -- 172.21.15.116:0/1642004785 learned_addr learned my addr 172.21.15.116:0/1642004785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0d400640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce1015db30 0x7fce1014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0e800640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 0x7fce101594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 -- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce1015db30 msgr2=0x7fce1014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce1015db30 0x7fce1014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 -- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 msgr2=0x7fce101594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 0x7fce101594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 -- 172.21.15.116:0/1642004785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce10156ac0 con 0x7fce100d1670 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0e800640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 0x7fce101594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:18.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.209+0000 7fce0de00640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce000ae040 tx=0x7fce0001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000ba020 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000b6620 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b0040 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce10151720 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce10074940 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1006a9c0 con 0x7fce100d1670 2024-05-14T11:16:18.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce000bd020 con 0x7fce100d1670 2024-05-14T11:16:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 0x7fcdf0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fce00129700 con 0x7fce100d1670 2024-05-14T11:16:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.213+0000 7fce0d400640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 0x7fcdf0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.217+0000 7fce0d400640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 0x7fcdf0071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce10151e80 tx=0x7fcdfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.221+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fce000c4030 con 0x7fce100d1670 2024-05-14T11:16:18.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.377+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce10068e40 con 0x7fce100d1670 2024-05-14T11:16:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce06000640 1 -- 172.21.15.116:0/1642004785 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fce100d1fc0 con 0x7fce100d1670 2024-05-14T11:16:18.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:18.382 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 msgr2=0x7fcdf0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 0x7fcdf0071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce10151e80 tx=0x7fcdfc008040 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 msgr2=0x7fce101572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce000ae040 tx=0x7fce0001ad10 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 shutdown_connections 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcdf006f400 0x7fcdf0071910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce10161600 0x7fce101594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fce1015db30 0x7fce1014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 --2- 172.21.15.116:0/1642004785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce100d1670 0x7fce101572a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 >> 172.21.15.116:0/1642004785 conn(0x7fce100b4620 msgr2=0x7fce100b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 shutdown_connections 2024-05-14T11:16:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.381+0000 7fce15200640 1 -- 172.21.15.116:0/1642004785 wait complete. 2024-05-14T11:16:18.419 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.561+0000 7feb85e00640 1 Processor -- start 2024-05-14T11:16:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.561+0000 7feb85e00640 1 -- start start 2024-05-14T11:16:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb85e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb80168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb85e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb8006ff20 con 0x7feb8015db30 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb85e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb80070470 con 0x7feb800d1670 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb85e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb8006d710 con 0x7feb80166750 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feb8015db30 0x7feb800aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60684/0 (socket says 172.21.15.116:60684) 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 -- 172.21.15.116:0/540246969 learned_addr learned my addr 172.21.15.116:0/540246969 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7f400640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb80168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167605755 0 0) 0x7feb8006ff20 con 0x7feb8015db30 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb64003530 con 0x7feb8015db30 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925678759 0 0) 0x7feb80070470 con 0x7feb800d1670 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb8006ff20 con 0x7feb800d1670 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2468170987 0 0) 0x7feb64003530 con 0x7feb8015db30 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 msgr2=0x7feb80168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb80168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 >> v1:172.21.15.149:6789/0 conn(0x7feb800d1670 legacy=0x7feb800d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb80070750 con 0x7feb8015db30 2024-05-14T11:16:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7feb70001700 con 0x7feb8015db30 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 >> v1:172.21.15.116:6789/0 conn(0x7feb8015db30 legacy=0x7feb800aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb64003c20 0x7feb64004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb64004810 0x7feb64004ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb640055e0 con 0x7feb64003c20 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb6400bf10 con 0x7feb64004810 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb6400c090 con 0x7feb80166750 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb64003c20 0x7feb64004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7f400640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7e000640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb64004810 0x7feb64004ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 -- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 msgr2=0x7feb6400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb80166750 0x7feb6400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 -- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb64004810 msgr2=0x7feb64004ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb64004810 0x7feb64004ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.565+0000 7feb7ea00640 1 -- 172.21.15.116:0/540246969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64003530 con 0x7feb64003c20 2024-05-14T11:16:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb7ea00640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb64003c20 0x7feb64004070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c1c0 tx=0x7feb7001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdd4b94aba62ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70007cb0 con 0x7feb64003c20 2024-05-14T11:16:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb700a9020 con 0x7feb64003c20 2024-05-14T11:16:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb7d600640 1 -- 172.21.15.116:0/540246969 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70005b00 con 0x7feb64003c20 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 -- 172.21.15.116:0/540246969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb64003c20 msgr2=0x7feb64004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 --2- 172.21.15.116:0/540246969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb64003c20 0x7feb64004070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c1c0 tx=0x7feb7001cb40 comp rx=0 tx=0).stop 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb7f400640 1 -- 172.21.15.116:0/540246969 reap_dead start 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 -- 172.21.15.116:0/540246969 shutdown_connections 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 -- 172.21.15.116:0/540246969 >> 172.21.15.116:0/540246969 conn(0x7feb800b4620 msgr2=0x7feb800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 -- 172.21.15.116:0/540246969 shutdown_connections 2024-05-14T11:16:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.569+0000 7feb85e00640 1 -- 172.21.15.116:0/540246969 wait complete. 2024-05-14T11:16:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 Processor -- start 2024-05-14T11:16:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 -- start start 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb64003c20 0x7feb8006c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 0x7feb8006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb80169590 con 0x7feb801fc6c0 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb800ae870 con 0x7feb80143b00 2024-05-14T11:16:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb85e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb80169710 con 0x7feb64003c20 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41178/0 (socket says 172.21.15.116:41178) 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 -- 172.21.15.116:0/2739517476 learned_addr learned my addr 172.21.15.116:0/2739517476 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7ea00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb64003c20 0x7feb8006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7e000640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 0x7feb8006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 -- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb64003c20 msgr2=0x7feb8006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb64003c20 0x7feb8006c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 -- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 msgr2=0x7feb8006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 0x7feb8006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7f400640 1 -- 172.21.15.116:0/2739517476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801651d0 con 0x7feb801fc6c0 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.573+0000 7feb7e000640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 0x7feb8006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb7f400640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb801626c0 tx=0x7feb780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780039a0 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7801b070 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801654b0 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78003b80 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb800cbfd0 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8006a9c0 con 0x7feb801fc6c0 2024-05-14T11:16:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb78004030 con 0x7feb801fc6c0 2024-05-14T11:16:18.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 0x7feb5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feb7808ff60 con 0x7feb801fc6c0 2024-05-14T11:16:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.577+0000 7feb7ea00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 0x7feb5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.581+0000 7feb7ea00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 0x7feb5c071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb700aa000 tx=0x7feb700b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.585+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feb78025020 con 0x7feb801fc6c0 2024-05-14T11:16:18.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.773+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb80166290 con 0x7feb801fc6c0 2024-05-14T11:16:18.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.773+0000 7feb77400640 1 -- 172.21.15.116:0/2739517476 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7feb7805cb00 con 0x7feb801fc6c0 2024-05-14T11:16:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:18.777 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:16.676555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:18.777 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 7 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 msgr2=0x7feb5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 0x7feb5c071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb700aa000 tx=0x7feb700b1040 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 msgr2=0x7feb801fcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb801626c0 tx=0x7feb780115a0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 shutdown_connections 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5c06f400 0x7feb5c071910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80143b00 0x7feb8006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb801fc6c0 0x7feb801fcad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 --2- 172.21.15.116:0/2739517476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb64003c20 0x7feb8006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 >> 172.21.15.116:0/2739517476 conn(0x7feb800b4620 msgr2=0x7feb800686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 shutdown_connections 2024-05-14T11:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.777+0000 7feb85e00640 1 -- 172.21.15.116:0/2739517476 wait complete. 2024-05-14T11:16:18.798 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.941+0000 7f5d8d400640 1 Processor -- start 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.941+0000 7f5d8d400640 1 -- start start 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d8d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5d8806f730 0x7f5d8806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d8d400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d88070080 con 0x7f5d880b0770 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d8d400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8806cf10 con 0x7f5d88072a30 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d8d400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5d8806db20 con 0x7f5d8806f730 2024-05-14T11:16:18.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d86800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5d880b0770 0x7f5d8806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60700/0 (socket says 172.21.15.116:60700) 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d86800640 1 -- 172.21.15.116:0/2757104683 learned_addr learned my addr 172.21.15.116:0/2757104683 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d7de00640 1 --2- 172.21.15.116:0/2757104683 >> v2:172.21.15.149:3300/0 conn(0x7f5d8806f730 0x7f5d8806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737564706 0 0) 0x7f5d78001120 con 0x7f5d880b0770 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d6c003530 con 0x7f5d880b0770 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1856859943 0 0) 0x7f5d6c003530 con 0x7f5d880b0770 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 >> v2:172.21.15.149:3300/0 conn(0x7f5d8806f730 msgr2=0x7f5d8806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 --2- 172.21.15.116:0/2757104683 >> v2:172.21.15.149:3300/0 conn(0x7f5d8806f730 0x7f5d8806fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 >> v1:172.21.15.149:6789/0 conn(0x7f5d88072a30 legacy=0x7f5d88072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8806d1f0 con 0x7f5d880b0770 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f5d780018f0 con 0x7f5d880b0770 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 >> v1:172.21.15.116:6789/0 conn(0x7f5d880b0770 legacy=0x7f5d8806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c003a60 0x7f5d6c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d6c004810 0x7f5d6c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 0x7f5d6c00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8806db20 con 0x7f5d6c005360 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c00d3a0 con 0x7f5d6c004810 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6c00d5a0 con 0x7f5d6c003a60 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 0x7f5d6c00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d86800640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c003a60 0x7f5d6c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d7de00640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d6c004810 0x7f5d6c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 -- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c003a60 msgr2=0x7f5d6c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c003a60 0x7f5d6c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 -- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d6c004810 msgr2=0x7f5d6c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d6c004810 0x7f5d6c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d87200640 1 -- 172.21.15.116:0/2757104683 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c003530 con 0x7f5d6c005360 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d7de00640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d6c004810 0x7f5d6c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.945+0000 7f5d86800640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c003a60 0x7f5d6c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d87200640 1 -- 172.21.15.116:0/2757104683 reap_dead start 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d87200640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 0x7f5d6c00cae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806ee20 tx=0x7f5d8001e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58ec47be9d2f6db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800c0e0 con 0x7f5d6c005360 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80003b80 con 0x7f5d6c005360 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d85e00640 1 -- 172.21.15.116:0/2757104683 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700096b0 con 0x7f5d6c005360 2024-05-14T11:16:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 -- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 msgr2=0x7f5d6c00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 0x7f5d6c00cae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806ee20 tx=0x7f5d8001e960 comp rx=0 tx=0).stop 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 -- 172.21.15.116:0/2757104683 shutdown_connections 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 --2- 172.21.15.116:0/2757104683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c005360 0x7f5d6c00cae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 -- 172.21.15.116:0/2757104683 >> 172.21.15.116:0/2757104683 conn(0x7f5d880b4620 msgr2=0x7f5d880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 -- 172.21.15.116:0/2757104683 shutdown_connections 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.949+0000 7f5d8d400640 1 -- 172.21.15.116:0/2757104683 wait complete. 2024-05-14T11:16:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 Processor -- start 2024-05-14T11:16:18.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- start start 2024-05-14T11:16:18.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c005360 0x7f5d880ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 0x7f5d881f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8806dfb0 con 0x7f5d881f8d60 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8806d7b0 con 0x7f5d88073320 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8806fb40 con 0x7f5d6c005360 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41194/0 (socket says 172.21.15.116:41194) 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 -- 172.21.15.116:0/4272569303 learned_addr learned my addr 172.21.15.116:0/4272569303 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d7de00640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 0x7f5d881f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d86800640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c005360 0x7f5d880ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 -- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c005360 msgr2=0x7f5d880ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c005360 0x7f5d880ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 -- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 msgr2=0x7f5d881f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 0x7f5d881f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 -- 172.21.15.116:0/4272569303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d880cd090 con 0x7f5d881f8d60 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d7de00640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 0x7f5d881f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d87200640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d8001d060 tx=0x7f5d6c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80001ae0 con 0x7f5d881f8d60 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80003980 con 0x7f5d881f8d60 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d800054e0 con 0x7f5d881f8d60 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8815e2f0 con 0x7f5d881f8d60 2024-05-14T11:16:18.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.953+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881f9750 con 0x7f5d881f8d60 2024-05-14T11:16:18.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.957+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d881f99d0 con 0x7f5d881f8d60 2024-05-14T11:16:18.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.957+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d80005680 con 0x7f5d881f8d60 2024-05-14T11:16:18.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.957+0000 7f5d7f400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 0x7f5d64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:18.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.957+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5d80129170 con 0x7f5d881f8d60 2024-05-14T11:16:18.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.957+0000 7f5d86800640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 0x7f5d64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:18.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.961+0000 7f5d86800640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 0x7f5d64071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d78001020 tx=0x7f5d78007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:18.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:18.961+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5d800bf030 con 0x7f5d881f8d60 2024-05-14T11:16:19.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.121+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d8806a9c0 con 0x7f5d881f8d60 2024-05-14T11:16:19.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.121+0000 7f5d7f400640 1 -- 172.21.15.116:0/4272569303 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f5d800186c0 con 0x7f5d881f8d60 2024-05-14T11:16:19.125 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:19.125 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 msgr2=0x7f5d64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 0x7f5d64071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d78001020 tx=0x7f5d78007b10 comp rx=0 tx=0).stop 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 msgr2=0x7f5d881f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d8001d060 tx=0x7f5d6c005840 comp rx=0 tx=0).stop 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 shutdown_connections 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d6406f400 0x7f5d64071910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88073320 0x7f5d881f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d881f8d60 0x7f5d881f9170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 --2- 172.21.15.116:0/4272569303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d6c005360 0x7f5d880ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 >> 172.21.15.116:0/4272569303 conn(0x7f5d880b4620 msgr2=0x7f5d8815c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 shutdown_connections 2024-05-14T11:16:19.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.125+0000 7f5d8d400640 1 -- 172.21.15.116:0/4272569303 wait complete. 2024-05-14T11:16:19.548 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:16:11.615741+0000', 'modified': '2024-05-14T11:16:16.676555+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/2939305126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2939305126}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2939305126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4284, 'qdb_cluster': [4284]} 2024-05-14T11:16:19.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:16:19.549 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:16:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 Processor -- start 2024-05-14T11:16:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 -- start start 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30080af420 0x7f3008074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3008074b70 0x7f30080d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f30080af830 0x7f30080d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f300815d340 con 0x7f30080af420 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f300815d620 con 0x7f30080af830 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f3010400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f300815d900 con 0x7f3008074b70 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30080af420 0x7f3008074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30080af420 0x7f3008074630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41208/0 (socket says 172.21.15.116:41208) 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 -- 172.21.15.116:0/566855433 learned_addr learned my addr 172.21.15.116:0/566855433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300e400640 1 --2- 172.21.15.116:0/566855433 >> v2:172.21.15.149:3300/0 conn(0x7f30080af830 0x7f30080d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300d000640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3008074b70 0x7f30080d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 -- 172.21.15.116:0/566855433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3008074b70 msgr2=0x7f30080d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3008074b70 0x7f30080d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 -- 172.21.15.116:0/566855433 >> v2:172.21.15.149:3300/0 conn(0x7f30080af830 msgr2=0x7f30080d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 --2- 172.21.15.116:0/566855433 >> v2:172.21.15.149:3300/0 conn(0x7f30080af830 0x7f30080d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.697+0000 7f300da00640 1 -- 172.21.15.116:0/566855433 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300815dbe0 con 0x7f30080af420 2024-05-14T11:16:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f300da00640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f3008074630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30080b41b0 tx=0x7f2ffc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf63ed66b58bd8c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3007e00640 1 -- 172.21.15.116:0/566855433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b4070 con 0x7f30080af420 2024-05-14T11:16:19.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3007e00640 1 -- 172.21.15.116:0/566855433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0a7ca0 con 0x7f30080af420 2024-05-14T11:16:19.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3007e00640 1 -- 172.21.15.116:0/566855433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0af660 con 0x7f30080af420 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 -- 172.21.15.116:0/566855433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 msgr2=0x7f3008074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f3008074630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30080b41b0 tx=0x7f2ffc01ebf0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 -- 172.21.15.116:0/566855433 shutdown_connections 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 --2- 172.21.15.116:0/566855433 >> v2:172.21.15.149:3300/0 conn(0x7f30080af830 0x7f30080d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3008074b70 0x7f30080d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 --2- 172.21.15.116:0/566855433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f3008074630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 -- 172.21.15.116:0/566855433 >> 172.21.15.116:0/566855433 conn(0x7f30080b4620 msgr2=0x7f30080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 -- 172.21.15.116:0/566855433 shutdown_connections 2024-05-14T11:16:19.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.701+0000 7f3010400640 1 -- 172.21.15.116:0/566855433 wait complete. 2024-05-14T11:16:19.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 Processor -- start 2024-05-14T11:16:19.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 -- start start 2024-05-14T11:16:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3008074b70 0x7f300809a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30080af830 0x7f30080cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f300806f960 con 0x7f30080af420 2024-05-14T11:16:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f300806f7e0 con 0x7f3008074b70 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3010400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30081604e0 con 0x7f30080af830 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41218/0 (socket says 172.21.15.116:41218) 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 -- 172.21.15.116:0/573594785 learned_addr learned my addr 172.21.15.116:0/573594785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300e400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30080af830 0x7f30080cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300da00640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3008074b70 0x7f300809a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 -- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30080af830 msgr2=0x7f30080cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30080af830 0x7f30080cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 -- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3008074b70 msgr2=0x7f300809a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3008074b70 0x7f300809a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 -- 172.21.15.116:0/573594785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30080b10a0 con 0x7f30080af420 2024-05-14T11:16:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f300d000640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400e870 tx=0x7f2ff400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3005600640 1 -- 172.21.15.116:0/573594785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4017070 con 0x7f30080af420 2024-05-14T11:16:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3005600640 1 -- 172.21.15.116:0/573594785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff400c940 con 0x7f30080af420 2024-05-14T11:16:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.705+0000 7f3005600640 1 -- 172.21.15.116:0/573594785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400ccf0 con 0x7f30080af420 2024-05-14T11:16:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.709+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30080b1380 con 0x7f30080af420 2024-05-14T11:16:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.709+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30080b1940 con 0x7f30080af420 2024-05-14T11:16:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.709+0000 7f3005600640 1 -- 172.21.15.116:0/573594785 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ff401a070 con 0x7f30080af420 2024-05-14T11:16:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3005600640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 0x7f2fe8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3005600640 1 -- 172.21.15.116:0/573594785 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ff408f8e0 con 0x7f30080af420 2024-05-14T11:16:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 0x7f2fe8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 Processor -- start 2024-05-14T11:16:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 -- start start 2024-05-14T11:16:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 0x7f2fe8071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0b1000 tx=0x7f2ffc0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108d8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1107706b0 0x55c1108d8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c110775620 con 0x55c11039e7c0 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c110690740 con 0x55c1107706b0 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55c1106de090 con 0x55c110786f80 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41230/0 (socket says 172.21.15.116:41230) 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 -- 172.21.15.116:0/2450752373 learned_addr learned my addr 172.21.15.116:0/2450752373 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300d000640 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108d8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300e400640 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1107706b0 0x55c1108d8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 -- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 msgr2=0x55c1108d8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108d8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 -- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1107706b0 msgr2=0x55c1108d8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1107706b0 0x55c1108d8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 -- 172.21.15.116:0/2450752373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1108720a0 con 0x55c11039e7c0 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f300da00640 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01f490 tx=0x7f2ffc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1e6e76d153978 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f2ffa000640 1 -- 172.21.15.116:0/2450752373 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0b4040 con 0x55c11039e7c0 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f2ffa000640 1 -- 172.21.15.116:0/2450752373 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0a7e50 con 0x55c11039e7c0 2024-05-14T11:16:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f2ffa000640 1 -- 172.21.15.116:0/2450752373 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0bab20 con 0x55c11039e7c0 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 -- 172.21.15.116:0/2450752373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 msgr2=0x55c1108d8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.713+0000 7f3017b14000 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01f490 tx=0x7f2ffc004740 comp rx=0 tx=0).stop 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- 172.21.15.116:0/2450752373 shutdown_connections 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1107706b0 0x55c1108d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108d8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- 172.21.15.116:0/2450752373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c1108d8320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- 172.21.15.116:0/2450752373 >> 172.21.15.116:0/2450752373 conn(0x55c11085d340 msgr2=0x55c11088cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- 172.21.15.116:0/2450752373 shutdown_connections 2024-05-14T11:16:19.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- 172.21.15.116:0/2450752373 wait complete. 2024-05-14T11:16:19.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- start start 2024-05-14T11:16:19.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 Processor -- start 2024-05-14T11:16:19.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c1108de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c1106da050 con 0x55c1107706b0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c110696ec0 con 0x55c11039e7c0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55c11067dc90 con 0x55c110786f80 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f3017b14000 10 client.0 _ll_register_callbacks cb 0x55c110879e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41238/0 (socket says 172.21.15.116:41238) 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 -- 172.21.15.116:0/1953558592 learned_addr learned my addr 172.21.15.116:0/1953558592 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300e400640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300da00640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c1108de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 -- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 msgr2=0x55c1108bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 -- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 msgr2=0x55c1108de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c1108de980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 -- 172.21.15.116:0/1953558592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1107886f0 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f300d000640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ff405ba70 tx=0x7f2ff405c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4427eb1d20db19e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.717+0000 7f2feea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff408b060 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ff405aa40 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc0a7e50 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2ff407a050 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 10 client.4395 fetch_fsmap learned FSMap version 7 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c1107846a0 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0a7e50 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2ff405ac60 con 0x55c1107706b0 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 10 client.4395 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 20 client.4395 populate_metadata read hostname 'smithi116' 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c1103837d0 0x55c1108bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 4 client.4395 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:16:19.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 --> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c110676be0 con 0x55c1103837d0 2024-05-14T11:16:19.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f300da00640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c1103837d0 0x55c1108bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f300da00640 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c1103837d0 0x55c1108bd460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe7a86f2429dc3d server_cookie=23bfd3f57eea8d6f in_seq=0 out_seq=0 2024-05-14T11:16:19.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 10 client.4395 ms_handle_connect on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 1 -- 172.21.15.116:0/1953558592 <== mds.0 v2:172.21.15.116:6834/2939305126 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c110676be0 con 0x55c1103837d0 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 10 client.4395 handle_command_reply: tid=0 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 client.4395 shutdown 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 msgr2=0x55c1108deec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ff405ba70 tx=0x7f2ff405c340 comp rx=0 tx=0).stop 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 shutdown_connections 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c1103837d0 0x55c1108bd460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c110786f80 0x55c1108bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1107706b0 0x55c1108deec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f2feea00640 0 client.4395 ms_handle_reset on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 --2- 172.21.15.116:0/1953558592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c1108de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 >> 172.21.15.116:0/1953558592 conn(0x55c11085d340 msgr2=0x55c11088d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 shutdown_connections 2024-05-14T11:16:19.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 1 -- 172.21.15.116:0/1953558592 wait complete. 2024-05-14T11:16:19.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.721+0000 7f3017b14000 20 client.4395 trim_cache size 0 max 16384 2024-05-14T11:16:19.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 Processor -- start 2024-05-14T11:16:19.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 -- start start 2024-05-14T11:16:19.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c1108edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 0x55c1108ee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3008205ec0 con 0x55c11039e7c0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30080b2e80 con 0x55c110786f80 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3008152160 con 0x55c1107706b0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41248/0 (socket says 172.21.15.116:41248) 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 -- 172.21.15.116:0/2245834546 learned_addr learned my addr 172.21.15.116:0/2245834546 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300e400640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 0x55c1108ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300d000640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c1108edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 -- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 msgr2=0x55c1108edb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c1108edb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 -- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 msgr2=0x55c1108ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 0x55c1108ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 -- 172.21.15.116:0/2245834546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1106b7af0 con 0x55c11039e7c0 2024-05-14T11:16:19.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300e400640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 0x55c1108ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:19.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f300da00640 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0251c0 tx=0x7f2ffc00b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e2ccf08e721b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.741+0000 7f2ff9600640 1 -- 172.21.15.116:0/2245834546 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0dd070 con 0x55c11039e7c0 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f2ff9600640 1 -- 172.21.15.116:0/2245834546 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0bab30 con 0x55c11039e7c0 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f2ff9600640 1 -- 172.21.15.116:0/2245834546 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0d51e0 con 0x55c11039e7c0 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- 172.21.15.116:0/2245834546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 msgr2=0x55c11088f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0251c0 tx=0x7f2ffc00b450 comp rx=0 tx=0).stop 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- 172.21.15.116:0/2245834546 shutdown_connections 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c110786f80 0x55c1108ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c1108edb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- 172.21.15.116:0/2245834546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c11039e7c0 0x55c11088f930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- 172.21.15.116:0/2245834546 >> 172.21.15.116:0/2245834546 conn(0x55c11085d340 msgr2=0x55c1108bb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- 172.21.15.116:0/2245834546 shutdown_connections 2024-05-14T11:16:19.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- 172.21.15.116:0/2245834546 wait complete. 2024-05-14T11:16:19.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- start start 2024-05-14T11:16:19.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 Processor -- start 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c11094a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c11094a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c1106b43a0 con 0x55c110786f80 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c1106737d0 con 0x55c11039e7c0 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55c110678220 con 0x55c1107706b0 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f3017b14000 10 client.0 _ll_register_callbacks cb 0x7f2ff4009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f300e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f300da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c11094a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f300e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41262/0 (socket says 172.21.15.116:41262) 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f300e400640 1 -- 172.21.15.116:0/2611029876 learned_addr learned my addr 172.21.15.116:0/2611029876 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.745+0000 7f300d000640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c11094a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 -- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 msgr2=0x55c11094a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c11094a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 -- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 msgr2=0x55c11094a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c11094a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 -- 172.21.15.116:0/2611029876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1106b6e40 con 0x55c110786f80 2024-05-14T11:16:19.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300da00640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c11094a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300e400640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3000017e60 tx=0x7f3000015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbb04e86b941d82 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300001e070 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3000023e00 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc0bab30 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f3000018830 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 10 client.4401 fetch_fsmap learned FSMap version 7 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c11075e4e0 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0bab30 con 0x55c110786f80 2024-05-14T11:16:19.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f300002a050 con 0x55c110786f80 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 10 client.4401 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 20 client.4401 populate_metadata read hostname 'smithi116' 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c110371830 0x55c11088e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 4 client.4401 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 --> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] -- command(tid 0: {"prefix": "status"}) -- 0x55c1106ebca0 con 0x55c110371830 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300da00640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c110371830 0x55c11088e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 10 client.4401 ms_handle_connect on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f300da00640 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c110371830 0x55c11088e4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e448806f18165f server_cookie=a72923f2df4c3440 in_seq=0 out_seq=0 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 1 -- 172.21.15.116:0/2611029876 <== mds.0 v2:172.21.15.116:6834/2939305126 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c1106ebca0 con 0x55c110371830 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 10 client.4401 handle_command_reply: tid=0 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 client.4401 shutdown 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 msgr2=0x55c11088d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3000017e60 tx=0x7f3000015700 comp rx=0 tx=0).stop 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 shutdown_connections 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55c110371830 0x55c11088e4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c110786f80 0x55c11088d7e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f2feea00640 0 client.4401 ms_handle_reset on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c1107706b0 0x55c11094a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 --2- 172.21.15.116:0/2611029876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c11039e7c0 0x55c11094a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 >> 172.21.15.116:0/2611029876 conn(0x55c11085d340 msgr2=0x55c1108ddcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 shutdown_connections 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 1 -- 172.21.15.116:0/2611029876 wait complete. 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3017b14000 20 client.4401 trim_cache size 0 max 16384 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:16:19.755 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4284, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 7.5308314950000002, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 7, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 24, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 24, 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 13.080189036 2024-05-14T11:16:19.756 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 msgr2=0x7f2fe8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 0x7f2fe8071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0b1000 tx=0x7f2ffc0a9070 comp rx=0 tx=0).stop 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 msgr2=0x7f30080cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400e870 tx=0x7f2ff400ec70 comp rx=0 tx=0).stop 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 shutdown_connections 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2fe806f400 0x7f2fe8071910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30080af830 0x7f30080cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30080af420 0x7f30080cecf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 --2- 172.21.15.116:0/573594785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3008074b70 0x7f300809a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.749+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 >> 172.21.15.116:0/573594785 conn(0x7f30080b4620 msgr2=0x7f3008156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.753+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 shutdown_connections 2024-05-14T11:16:19.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.753+0000 7f3010400640 1 -- 172.21.15.116:0/573594785 wait complete. 2024-05-14T11:16:19.772 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T11:16:19.772 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:19.772 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:16:19.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 Processor -- start 2024-05-14T11:16:19.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 -- start start 2024-05-14T11:16:19.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f782015db30 0x7f78200aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782006ff20 con 0x7f78200d1670 2024-05-14T11:16:19.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7820070470 con 0x7f782006c700 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7826e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f78200ae100 con 0x7f782015db30 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f781fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f782015db30 0x7f78200aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7817400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f78200d1670 0x7f78200d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60702/0 (socket says 172.21.15.116:60702) 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f7817400640 1 -- 172.21.15.116:0/57975906 learned_addr learned my addr 172.21.15.116:0/57975906 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f781fe00640 1 -- 172.21.15.116:0/57975906 >> v1:172.21.15.149:6789/0 conn(0x7f782006c700 legacy=0x7f782006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f781fe00640 1 -- 172.21.15.116:0/57975906 >> v1:172.21.15.116:6789/0 conn(0x7f78200d1670 legacy=0x7f78200d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f781fe00640 1 -- 172.21.15.116:0/57975906 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820070750 con 0x7f782015db30 2024-05-14T11:16:19.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.917+0000 7f781f400640 1 -- 172.21.15.116:0/57975906 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636579497 0 0) 0x7f782006ff20 con 0x7f78200d1670 2024-05-14T11:16:19.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f781fe00640 1 --2- 172.21.15.116:0/57975906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782015db30 0x7f78200aeaa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f78200ae280 tx=0x7f781001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90db08a9f4c3279 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f781f400640 1 -- 172.21.15.116:0/57975906 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100b0070 con 0x7f782015db30 2024-05-14T11:16:19.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f781f400640 1 -- 172.21.15.116:0/57975906 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78100ab600 con 0x7f782015db30 2024-05-14T11:16:19.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f781f400640 1 -- 172.21.15.116:0/57975906 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100ab9d0 con 0x7f782015db30 2024-05-14T11:16:19.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 -- 172.21.15.116:0/57975906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782015db30 msgr2=0x7f78200aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 --2- 172.21.15.116:0/57975906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782015db30 0x7f78200aeaa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f78200ae280 tx=0x7f781001c410 comp rx=0 tx=0).stop 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 -- 172.21.15.116:0/57975906 shutdown_connections 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 --2- 172.21.15.116:0/57975906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782015db30 0x7f78200aeaa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 -- 172.21.15.116:0/57975906 >> 172.21.15.116:0/57975906 conn(0x7f78200b4620 msgr2=0x7f78200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 -- 172.21.15.116:0/57975906 shutdown_connections 2024-05-14T11:16:19.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.921+0000 7f7826e00640 1 -- 172.21.15.116:0/57975906 wait complete. 2024-05-14T11:16:19.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 Processor -- start 2024-05-14T11:16:19.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- start start 2024-05-14T11:16:19.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78200d1670 0x7f782013b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f782015db30 0x7f7820158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f782016bf90 con 0x7f782006c700 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f782015df40 con 0x7f782015db30 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f782016c110 con 0x7f78200d1670 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41284/0 (socket says 172.21.15.116:41284) 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 -- 172.21.15.116:0/1314799915 learned_addr learned my addr 172.21.15.116:0/1314799915 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 -- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78200d1670 msgr2=0x7f782013b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78200d1670 0x7f782013b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 -- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f782015db30 msgr2=0x7f7820158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f782015db30 0x7f7820158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 -- 172.21.15.116:0/1314799915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7820158890 con 0x7f782006c700 2024-05-14T11:16:19.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781fe00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78100ae040 tx=0x7f781001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100b0050 con 0x7f782006c700 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78100bc070 con 0x7f782006c700 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782014fa20 con 0x7f782006c700 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100b5a70 con 0x7f782006c700 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.925+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782014ffe0 con 0x7f782006c700 2024-05-14T11:16:19.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782006a9c0 con 0x7f782006c700 2024-05-14T11:16:19.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f78100ab480 con 0x7f782006c700 2024-05-14T11:16:19.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f781cc00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 0x7f77fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:19.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7810129f20 con 0x7f782006c700 2024-05-14T11:16:19.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f7817400640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 0x7f77fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:19.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.929+0000 7f7817400640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 0x7f77fc071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7820159e40 tx=0x7f7808007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:19.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:19.933+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7810016090 con 0x7f782006c700 2024-05-14T11:16:20.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78200b3ef0 con 0x7f782006c700 2024-05-14T11:16:20.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f781cc00640 1 -- 172.21.15.116:0/1314799915 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f78100f6bf0 con 0x7f782006c700 2024-05-14T11:16:20.127 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T11:16:20.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 msgr2=0x7f77fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:20.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 0x7f77fc071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7820159e40 tx=0x7f7808007cf0 comp rx=0 tx=0).stop 2024-05-14T11:16:20.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 msgr2=0x7f7820157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:20.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.125+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78100ae040 tx=0x7f781001ac50 comp rx=0 tx=0).stop 2024-05-14T11:16:20.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 shutdown_connections 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77fc06f400 0x7f77fc071910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f782015db30 0x7f7820158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78200d1670 0x7f782013b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 --2- 172.21.15.116:0/1314799915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006c700 0x7f7820157c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 >> 172.21.15.116:0/1314799915 conn(0x7f78200b4620 msgr2=0x7f78200b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 shutdown_connections 2024-05-14T11:16:20.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20.129+0000 7f7826e00640 1 -- 172.21.15.116:0/1314799915 wait complete. 2024-05-14T11:16:20.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-14T11:16:20.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:16:20.149 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T11:16:20.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:20.149 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:16:20.158 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:20.159 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42422sec preferred_lft 42422sec 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:16:20.160 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:20.161 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T11:16:20.161 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:20.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:20.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:20.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:20.321 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:16:20.339 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:16:20.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:20.341 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:16:20.422 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:16:20.423 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:16:20.423 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:20.423 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:20.423 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:20.423 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:16:20.423 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:20.423 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:20.423 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:16:20.423 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:16:20.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:16:20.424 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:20.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:20.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:20.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:20.561 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:16:20.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:20.571 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-05-14T11:16:20.624 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:20.625 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:20.625 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:20.625 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-14T11:16:20.625 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:20.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:20.751 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-14T11:16:20.751 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:20.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:20.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:20.915 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:20.915 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:20.915 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:16:20.915 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T11:16:20.915 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:21.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:21.043 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-14T11:16:21.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:21.044 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:21.052 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T11:16:21.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:21.053 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:21.108 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:21.172 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:21.227 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:21.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:21.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:21.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:21.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:21.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:21.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:21.229 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:21.284 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:16:21.284 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-14T11:16:21.331 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:21.352 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:21.402 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:21.403 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:21.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:21.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:21.405 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:21.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:2024-05-14T11:16:21.437+0000 7efe7a247580 -1 init, newargv = 0x558586deda00 newargc=15 2024-05-14T11:16:21.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[33954]: starting ceph client 2024-05-14T11:16:21.450 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[33954]: starting fuse 2024-05-14T11:16:22.549 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:22.575 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:22.630 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:22.631 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:22.632 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:22.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:22.633 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:22.691 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:16:22.691 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:16:22.691 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:16:22.691 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:16:22.691 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:16:22.691 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:16:22.691 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:16:22.692 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:16:22.693 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:16:22.693 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:16:22.693 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:22.693 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.admin")) 2024-05-14T11:16:22.693 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:16:22.821 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.admin.33954.asok 2024-05-14T11:16:22.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:22.834 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33954.asok 2024-05-14T11:16:22.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:22.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33954.asok status 2024-05-14T11:16:23.014 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:16:23.014 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:16:23.014 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "admin", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "33954", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4413, 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4413 2024-05-14T11:16:23.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3732884017 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3732884017 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4413 192.168.0.1:0/3732884017", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/3732884017", 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 7, 2024-05-14T11:16:23.016 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2024-05-14T11:16:23.017 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:16:23.017 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:16:23.017 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:16:23.017 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:16:23.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:23.031 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:16:23.031 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:23.040 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:16:23.040 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:23.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:23.041 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:23.100 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-14T11:16:23.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T11:16:23.209 INFO:teuthology.orchestra.run.smithi116.stdout:# file: . 2024-05-14T11:16:23.209 INFO:teuthology.orchestra.run.smithi116.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-14T11:16:23.210 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:23.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:23.216 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:23.216 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:23.225 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:23.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 Processor -- start 2024-05-14T11:16:23.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 -- start start 2024-05-14T11:16:23.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd77c06f730 0x7fd77c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd77c070080 con 0x7fd77c15fc00 2024-05-14T11:16:23.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd77c06db20 con 0x7fd77c06f730 2024-05-14T11:16:23.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd780e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd77c06cc40 con 0x7fd77c0b0770 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd77c06f730 0x7fd77c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd77a800640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fd77c15fc00 0x7fd77c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:38690/0 (socket says 172.21.15.116:38690) 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd77a800640 1 -- 172.21.15.116:0/1319960496 learned_addr learned my addr 172.21.15.116:0/1319960496 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd778a00640 1 -- 172.21.15.116:0/1319960496 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164336934 0 0) 0x7fd7740014a0 con 0x7fd77c15fc00 2024-05-14T11:16:23.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd778a00640 1 -- 172.21.15.116:0/1319960496 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd760003530 con 0x7fd77c15fc00 2024-05-14T11:16:23.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779400640 1 -- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 msgr2=0x7fd77c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:23.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779400640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779400640 1 -- 172.21.15.116:0/1319960496 >> v1:172.21.15.149:6789/0 conn(0x7fd77c15fc00 legacy=0x7fd77c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779400640 1 -- 172.21.15.116:0/1319960496 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c06cf10 con 0x7fd77c06f730 2024-05-14T11:16:23.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.417+0000 7fd779e00640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:23.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd779400640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c06fb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd76800a960 tx=0x7fd76801d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef04019523fb7b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd778a00640 1 -- 172.21.15.116:0/1319960496 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7680b3070 con 0x7fd77c06f730 2024-05-14T11:16:23.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd778a00640 1 -- 172.21.15.116:0/1319960496 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7680a67e0 con 0x7fd77c06f730 2024-05-14T11:16:23.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd778a00640 1 -- 172.21.15.116:0/1319960496 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800b040 con 0x7fd77c06f730 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 -- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 msgr2=0x7fd77c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c06fb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd76800a960 tx=0x7fd76801d7b0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 -- 172.21.15.116:0/1319960496 shutdown_connections 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c06fb40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 --2- 172.21.15.116:0/1319960496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 -- 172.21.15.116:0/1319960496 >> 172.21.15.116:0/1319960496 conn(0x7fd77c0b4620 msgr2=0x7fd77c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 -- 172.21.15.116:0/1319960496 shutdown_connections 2024-05-14T11:16:23.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.421+0000 7fd780e00640 1 -- 172.21.15.116:0/1319960496 wait complete. 2024-05-14T11:16:23.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 Processor -- start 2024-05-14T11:16:23.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 -- start start 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c1f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c073a40 con 0x7fd77c15fc00 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c06dfb0 con 0x7fd77c06f730 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd780e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd77c168fa0 con 0x7fd77c0b0770 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41288/0 (socket says 172.21.15.116:41288) 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 -- 172.21.15.116:0/3287187662 learned_addr learned my addr 172.21.15.116:0/3287187662 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd779400640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c1f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd779e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 -- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 msgr2=0x7fd77c1f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c1f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 -- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 msgr2=0x7fd77c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.425+0000 7fd77a800640 1 -- 172.21.15.116:0/3287187662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c15da00 con 0x7fd77c15fc00 2024-05-14T11:16:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd77a800640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ca70 tx=0x7fd77400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400abd0 con 0x7fd77c15fc00 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77400ad70 con 0x7fd77c15fc00 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c15dce0 con 0x7fd77c15fc00 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774012620 con 0x7fd77c15fc00 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c15e2a0 con 0x7fd77c15fc00 2024-05-14T11:16:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77c1555c0 con 0x7fd77c15fc00 2024-05-14T11:16:23.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7740127c0 con 0x7fd77c15fc00 2024-05-14T11:16:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 0x7fd75c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.429+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd7740906e0 con 0x7fd77c15fc00 2024-05-14T11:16:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.433+0000 7fd779e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 0x7fd75c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.433+0000 7fd779e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 0x7fd75c071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0d1240 tx=0x7fd76c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.433+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd77c070080 con 0x7fd77c15fc00 2024-05-14T11:16:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd77c06a9c0 con 0x7fd77c15fc00 2024-05-14T11:16:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd772000640 1 -- 172.21.15.116:0/3287187662 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fd77405d300 con 0x7fd77c15fc00 2024-05-14T11:16:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 msgr2=0x7fd75c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 0x7fd75c071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0d1240 tx=0x7fd76c008040 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 msgr2=0x7fd77c15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ca70 tx=0x7fd77400cf30 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 shutdown_connections 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd75c06f430 0x7fd75c071940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd77c15fc00 0x7fd77c15e7c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd77c0b0770 0x7fd77c1f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 --2- 172.21.15.116:0/3287187662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c06f730 0x7fd77c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 >> 172.21.15.116:0/3287187662 conn(0x7fd77c0b4620 msgr2=0x7fd77c0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 shutdown_connections 2024-05-14T11:16:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.605+0000 7fd780e00640 1 -- 172.21.15.116:0/3287187662 wait complete. 2024-05-14T11:16:23.629 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:23.629 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:23.637 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 Processor -- start 2024-05-14T11:16:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 -- start start 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f50a4166750 0x7f50a4168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a406ff20 con 0x7f50a40d1670 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a4070470 con 0x7f50a415db30 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.829+0000 7f50ab800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f50a406d710 con 0x7f50a4166750 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f50a40d1670 0x7f50a40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60708/0 (socket says 172.21.15.116:60708) 2024-05-14T11:16:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a3e00640 1 -- 172.21.15.116:0/2735046981 learned_addr learned my addr 172.21.15.116:0/2735046981 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a9800640 1 --2- 172.21.15.116:0/2735046981 >> v2:172.21.15.149:3300/0 conn(0x7f50a4166750 0x7f50a4168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a3400640 1 -- 172.21.15.116:0/2735046981 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1561081659 0 0) 0x7f50a406ff20 con 0x7f50a40d1670 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a3400640 1 -- 172.21.15.116:0/2735046981 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5088003530 con 0x7f50a40d1670 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a9800640 1 -- 172.21.15.116:0/2735046981 >> v1:172.21.15.149:6789/0 conn(0x7f50a415db30 legacy=0x7f50a40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a9800640 1 -- 172.21.15.116:0/2735046981 >> v1:172.21.15.116:6789/0 conn(0x7f50a40d1670 legacy=0x7f50a40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a9800640 1 -- 172.21.15.116:0/2735046981 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a4070750 con 0x7f50a4166750 2024-05-14T11:16:23.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.833+0000 7f50a9800640 1 --2- 172.21.15.116:0/2735046981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a4168b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097c0 tx=0x7f509c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09624e340498487 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50a3400640 1 -- 172.21.15.116:0/2735046981 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0b0070 con 0x7f50a4166750 2024-05-14T11:16:23.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50a3400640 1 -- 172.21.15.116:0/2735046981 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c0ab600 con 0x7f50a4166750 2024-05-14T11:16:23.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50a3400640 1 -- 172.21.15.116:0/2735046981 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0ab9d0 con 0x7f50a4166750 2024-05-14T11:16:23.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- 172.21.15.116:0/2735046981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 msgr2=0x7f50a4168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 --2- 172.21.15.116:0/2735046981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a4168b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097c0 tx=0x7f509c01c380 comp rx=0 tx=0).stop 2024-05-14T11:16:23.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- 172.21.15.116:0/2735046981 shutdown_connections 2024-05-14T11:16:23.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 --2- 172.21.15.116:0/2735046981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a4168b70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- 172.21.15.116:0/2735046981 >> 172.21.15.116:0/2735046981 conn(0x7f50a40b4620 msgr2=0x7f50a40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:23.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- 172.21.15.116:0/2735046981 shutdown_connections 2024-05-14T11:16:23.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- 172.21.15.116:0/2735046981 wait complete. 2024-05-14T11:16:23.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 Processor -- start 2024-05-14T11:16:23.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.837+0000 7f50ab800640 1 -- start start 2024-05-14T11:16:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50a40d1670 0x7f50a415f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a40b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50a416bae0 con 0x7f50a415db30 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50a40ae870 con 0x7f50a4166750 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50a416bc60 con 0x7f50a40d1670 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41300/0 (socket says 172.21.15.116:41300) 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 -- 172.21.15.116:0/3786928476 learned_addr learned my addr 172.21.15.116:0/3786928476 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a9800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a40b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a8e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50a40d1670 0x7f50a415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 -- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50a40d1670 msgr2=0x7f50a415f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50a40d1670 0x7f50a415f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 -- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 msgr2=0x7f50a40b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a40b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 -- 172.21.15.116:0/3786928476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a40cbc70 con 0x7f50a415db30 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a9800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a40b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a3e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f509000c460 tx=0x7f509000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090017070 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5090005cf0 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50900072d0 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a414f740 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a40cbee0 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.841+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a406a9c0 con 0x7f50a415db30 2024-05-14T11:16:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.845+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f509001a070 con 0x7f50a415db30 2024-05-14T11:16:23.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.845+0000 7f50a0c00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 0x7f5084071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:23.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.845+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f509008e920 con 0x7f50a415db30 2024-05-14T11:16:23.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.845+0000 7f50a8e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 0x7f5084071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:23.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.845+0000 7f50a8e00640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 0x7f5084071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5094001cf0 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:23.849+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f509005ade0 con 0x7f50a415db30 2024-05-14T11:16:24.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.017+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f50a4074630 con 0x7f50a415db30 2024-05-14T11:16:24.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50a0c00640 1 -- 172.21.15.116:0/3786928476 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f509005f0e0 con 0x7f50a415db30 2024-05-14T11:16:24.021 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:24.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 msgr2=0x7f5084071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:24.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 0x7f5084071940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5094001cf0 tx=0x7f5094009040 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 msgr2=0x7f50a414f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f509000c460 tx=0x7f509000c920 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 shutdown_connections 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f508406f430 0x7f5084071940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4166750 0x7f50a40b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50a415db30 0x7f50a414f200 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 --2- 172.21.15.116:0/3786928476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50a40d1670 0x7f50a415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 >> 172.21.15.116:0/3786928476 conn(0x7f50a40b4620 msgr2=0x7f50a415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 shutdown_connections 2024-05-14T11:16:24.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:24.021+0000 7f50ab800640 1 -- 172.21.15.116:0/3786928476 wait complete. 2024-05-14T11:16:24.043 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:16:24.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:24.043 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-14T11:16:24.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[33954]: fuse finished with error 0 and tester_r 0 2024-05-14T11:16:24.070 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:16:30.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:16:30.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:30.172 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-14T11:16:30.265 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:30.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 Processor -- start 2024-05-14T11:16:30.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 -- start start 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8568163fa0 0x7f856806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85680b31e0 0x7f85680b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f856806e7d0 con 0x7f85680d1570 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f856806f570 con 0x7f85680b31e0 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856ea00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f856806f850 con 0x7f8568163fa0 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85680b31e0 0x7f85680b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856d600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8568163fa0 0x7f856806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85680b31e0 0x7f85680b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52036/0 (socket says 172.21.15.116:52036) 2024-05-14T11:16:30.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 -- 172.21.15.116:0/1359013611 learned_addr learned my addr 172.21.15.116:0/1359013611 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:30.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 -- 172.21.15.116:0/1359013611 >> v2:172.21.15.149:3300/0 conn(0x7f8568163fa0 msgr2=0x7f856806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:30.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 --2- 172.21.15.116:0/1359013611 >> v2:172.21.15.149:3300/0 conn(0x7f8568163fa0 0x7f856806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 -- 172.21.15.116:0/1359013611 >> v1:172.21.15.149:6789/0 conn(0x7f85680d1570 legacy=0x7f85680d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f856e000640 1 -- 172.21.15.116:0/1359013611 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856806fb30 con 0x7f85680b31e0 2024-05-14T11:16:30.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.425+0000 7f8567e00640 1 -- 172.21.15.116:0/1359013611 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656179741 0 0) 0x7f856806e7d0 con 0x7f85680d1570 2024-05-14T11:16:30.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f856e000640 1 --2- 172.21.15.116:0/1359013611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85680b35f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8568074010 tx=0x7f856001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3592e871aac57a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f8567e00640 1 -- 172.21.15.116:0/1359013611 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600a5730 con 0x7f85680b31e0 2024-05-14T11:16:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f8567e00640 1 -- 172.21.15.116:0/1359013611 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85600a58d0 con 0x7f85680b31e0 2024-05-14T11:16:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f8567e00640 1 -- 172.21.15.116:0/1359013611 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600a5ca0 con 0x7f85680b31e0 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f856ea00640 1 -- 172.21.15.116:0/1359013611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 msgr2=0x7f85680b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.429+0000 7f856ea00640 1 --2- 172.21.15.116:0/1359013611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85680b35f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8568074010 tx=0x7f856001c960 comp rx=0 tx=0).stop 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- 172.21.15.116:0/1359013611 shutdown_connections 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 --2- 172.21.15.116:0/1359013611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85680b35f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 --2- 172.21.15.116:0/1359013611 >> v2:172.21.15.149:3300/0 conn(0x7f8568163fa0 0x7f856806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- 172.21.15.116:0/1359013611 >> 172.21.15.116:0/1359013611 conn(0x7f85680b55c0 msgr2=0x7f85680d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- 172.21.15.116:0/1359013611 shutdown_connections 2024-05-14T11:16:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- 172.21.15.116:0/1359013611 wait complete. 2024-05-14T11:16:30.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 Processor -- start 2024-05-14T11:16:30.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- start start 2024-05-14T11:16:30.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85680d1570 0x7f85681521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8568163fa0 0x7f85680cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8568171240 con 0x7f85680b31e0 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85681710c0 con 0x7f85680d1570 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.433+0000 7f856ea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f85681713c0 con 0x7f8568163fa0 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52042/0 (socket says 172.21.15.116:52042) 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 -- 172.21.15.116:0/2400571681 learned_addr learned my addr 172.21.15.116:0/2400571681 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856e000640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8568163fa0 0x7f85680cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856cc00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85680d1570 0x7f85681521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 -- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8568163fa0 msgr2=0x7f85680cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8568163fa0 0x7f85680cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 -- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85680d1570 msgr2=0x7f85681521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85680d1570 0x7f85681521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 -- 172.21.15.116:0/2400571681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8568160fd0 con 0x7f85680b31e0 2024-05-14T11:16:30.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856d600640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e870 tx=0x7f855c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c017070 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c00ee60 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85680cf960 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c004240 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8568164ad0 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8568050880 con 0x7f85680b31e0 2024-05-14T11:16:30.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.437+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f855c00c940 con 0x7f85680b31e0 2024-05-14T11:16:30.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.441+0000 7f8565600640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 0x7f8548071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.441+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f855c08f720 con 0x7f85680b31e0 2024-05-14T11:16:30.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.441+0000 7f856cc00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 0x7f8548071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.441+0000 7f856cc00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 0x7f8548071940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8568200e10 tx=0x7f8554009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.445+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f855c05bc70 con 0x7f85680b31e0 2024-05-14T11:16:30.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.605+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85680b93f0 con 0x7f85680b31e0 2024-05-14T11:16:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.605+0000 7f8565600640 1 -- 172.21.15.116:0/2400571681 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f855c05ff70 con 0x7f85680b31e0 2024-05-14T11:16:30.609 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:30.609 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 msgr2=0x7f8548071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 0x7f8548071940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8568200e10 tx=0x7f8554009040 comp rx=0 tx=0).stop 2024-05-14T11:16:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 msgr2=0x7f85681652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e870 tx=0x7f855c00ed30 comp rx=0 tx=0).stop 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 shutdown_connections 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f854806f430 0x7f8548071940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85680d1570 0x7f85681521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8568163fa0 0x7f85680cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 --2- 172.21.15.116:0/2400571681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85680b31e0 0x7f85681652d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 >> 172.21.15.116:0/2400571681 conn(0x7f85680b55c0 msgr2=0x7f8568071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 shutdown_connections 2024-05-14T11:16:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.609+0000 7f856ea00640 1 -- 172.21.15.116:0/2400571681 wait complete. 2024-05-14T11:16:30.631 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:30.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 Processor -- start 2024-05-14T11:16:30.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 -- start start 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4b6c0d1670 0x7f4b6c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6c0d1fc0 con 0x7f4b6c0af380 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6c160510 con 0x7f4b6c156fa0 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b70c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6c0706f0 con 0x7f4b6c0d1670 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4b6c0af380 0x7f4b6c0af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39738/0 (socket says 172.21.15.116:39738) 2024-05-14T11:16:30.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 -- 172.21.15.116:0/648233115 learned_addr learned my addr 172.21.15.116:0/648233115 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:30.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b69400640 1 --2- 172.21.15.116:0/648233115 >> v2:172.21.15.149:3300/0 conn(0x7f4b6c0d1670 0x7f4b6c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830385020 0 0) 0x7f4b640014a0 con 0x7f4b6c0af380 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b50003530 con 0x7f4b6c0af380 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2480830326 0 0) 0x7f4b6c160510 con 0x7f4b6c156fa0 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b640014a0 con 0x7f4b6c156fa0 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2606665207 0 0) 0x7f4b50003530 con 0x7f4b6c0af380 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 >> v2:172.21.15.149:3300/0 conn(0x7f4b6c0d1670 msgr2=0x7f4b6c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 --2- 172.21.15.116:0/648233115 >> v2:172.21.15.149:3300/0 conn(0x7f4b6c0d1670 0x7f4b6c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 >> v1:172.21.15.149:6789/0 conn(0x7f4b6c156fa0 legacy=0x7f4b6c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1607f0 con 0x7f4b6c0af380 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f4b64001c50 con 0x7f4b6c0af380 2024-05-14T11:16:30.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b69400640 1 --2- 172.21.15.116:0/648233115 >> v2:172.21.15.149:3300/0 conn(0x7f4b6c0d1670 0x7f4b6c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 >> v1:172.21.15.116:6789/0 conn(0x7f4b6c0af380 legacy=0x7f4b6c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b50003c40 0x7f4b50004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50004810 0x7f4b50004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 0x7f4b5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c0706f0 con 0x7f4b50005290 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000c240 con 0x7f4b50004810 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5000c440 con 0x7f4b50003c40 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 0x7f4b5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b69400640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50004810 0x7f4b50004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b69e00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b50003c40 0x7f4b50004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 -- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b50003c40 msgr2=0x7f4b50004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:30.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b50003c40 0x7f4b50004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 -- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50004810 msgr2=0x7f4b50004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:30.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50004810 0x7f4b50004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.777+0000 7f4b6a800640 1 -- 172.21.15.116:0/648233115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b50003530 con 0x7f4b50005290 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b6a800640 1 -- 172.21.15.116:0/648233115 reap_dead start 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b6a800640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 0x7f4b5000b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400a980 tx=0x7f4b6401f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f69dd4200e7c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400a5d0 con 0x7f4b50005290 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64007310 con 0x7f4b50005290 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b68a00640 1 -- 172.21.15.116:0/648233115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64007700 con 0x7f4b50005290 2024-05-14T11:16:30.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 -- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 msgr2=0x7f4b5000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 0x7f4b5000b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400a980 tx=0x7f4b6401f980 comp rx=0 tx=0).stop 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 -- 172.21.15.116:0/648233115 shutdown_connections 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 --2- 172.21.15.116:0/648233115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b50005290 0x7f4b5000b9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 -- 172.21.15.116:0/648233115 >> 172.21.15.116:0/648233115 conn(0x7f4b6c0b4620 msgr2=0x7f4b6c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 -- 172.21.15.116:0/648233115 shutdown_connections 2024-05-14T11:16:30.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.781+0000 7f4b70c00640 1 -- 172.21.15.116:0/648233115 wait complete. 2024-05-14T11:16:30.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 Processor -- start 2024-05-14T11:16:30.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 -- start start 2024-05-14T11:16:30.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50005290 0x7f4b6c148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b6c151d50 0x7f4b6c073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c161120 con 0x7f4b6c154630 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c160db0 con 0x7f4b50005290 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b70c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d1a80 con 0x7f4b6c151d50 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b69400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b69400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52058/0 (socket says 172.21.15.116:52058) 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.785+0000 7f4b69400640 1 -- 172.21.15.116:0/1181035737 learned_addr learned my addr 172.21.15.116:0/1181035737 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b6a800640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b6c151d50 0x7f4b6c073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69e00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50005290 0x7f4b6c148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 -- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b6c151d50 msgr2=0x7f4b6c073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b6c151d50 0x7f4b6c073a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 -- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50005290 msgr2=0x7f4b6c148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50005290 0x7f4b6c148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 -- 172.21.15.116:0/1181035737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c152160 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b69400640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b58011d00 tx=0x7f4b5800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800fcc0 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5800fe60 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58004240 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c155b30 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1560f0 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c06a9c0 con 0x7f4b6c154630 2024-05-14T11:16:30.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.789+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b58012040 con 0x7f4b6c154630 2024-05-14T11:16:30.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.793+0000 7f4b62000640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:30.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.793+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b5808f940 con 0x7f4b6c154630 2024-05-14T11:16:30.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.793+0000 7f4b69e00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:30.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.793+0000 7f4b69e00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c000c10 tx=0x7f4b5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:30.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.797+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4b6c152160 con 0x7f4b6c154630 2024-05-14T11:16:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.953+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b6c06fdc0 con 0x7f4b6c154630 2024-05-14T11:16:30.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b62000640 1 -- 172.21.15.116:0/1181035737 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f4b58019030 con 0x7f4b6c154630 2024-05-14T11:16:30.957 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:30.958 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:30.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 msgr2=0x7f4b4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c000c10 tx=0x7f4b5c009040 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 msgr2=0x7f4b6c151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b58011d00 tx=0x7f4b5800f610 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 shutdown_connections 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b6c151d50 0x7f4b6c073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b6c154630 0x7f4b6c151810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 --2- 172.21.15.116:0/1181035737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b50005290 0x7f4b6c148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 >> 172.21.15.116:0/1181035737 conn(0x7f4b6c0b4620 msgr2=0x7f4b6c0b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 shutdown_connections 2024-05-14T11:16:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:30.957+0000 7f4b70c00640 1 -- 172.21.15.116:0/1181035737 wait complete. 2024-05-14T11:16:30.979 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:16:31.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.125+0000 7f0919a00640 1 Processor -- start 2024-05-14T11:16:31.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.125+0000 7f0919a00640 1 -- start start 2024-05-14T11:16:31.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0919a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09140705b0 0x7f09140d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0919a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 0x7f0914160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0919a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140d2080 con 0x7f09140d1c70 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0919a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f091406f7f0 con 0x7f09140705b0 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0919a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f091406ebe0 con 0x7f0914160560 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09140705b0 0x7f09140d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f09140d1c70 0x7f091415fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39740/0 (socket says 172.21.15.116:39740) 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912000640 1 -- 172.21.15.116:0/3379272213 learned_addr learned my addr 172.21.15.116:0/3379272213 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0913400640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 0x7f0914160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912a00640 1 -- 172.21.15.116:0/3379272213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 msgr2=0x7f0914160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912a00640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 0x7f0914160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912a00640 1 -- 172.21.15.116:0/3379272213 >> v1:172.21.15.116:6789/0 conn(0x7f09140d1c70 legacy=0x7f091415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0912a00640 1 -- 172.21.15.116:0/3379272213 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091406eec0 con 0x7f09140705b0 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0911600640 1 -- 172.21.15.116:0/3379272213 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850635242 0 0) 0x7f08fc001120 con 0x7f09140d1c70 2024-05-14T11:16:31.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.129+0000 7f0913400640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 0x7f0914160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:31.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0912a00640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140705b0 0x7f09140d1730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0904009d90 tx=0x7f090401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8c7ef92598ccbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0911600640 1 -- 172.21.15.116:0/3379272213 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040b2070 con 0x7f09140705b0 2024-05-14T11:16:31.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0911600640 1 -- 172.21.15.116:0/3379272213 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09040a5a00 con 0x7f09140705b0 2024-05-14T11:16:31.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0911600640 1 -- 172.21.15.116:0/3379272213 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040a5dd0 con 0x7f09140705b0 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- 172.21.15.116:0/3379272213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140705b0 msgr2=0x7f09140d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140705b0 0x7f09140d1730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0904009d90 tx=0x7f090401c940 comp rx=0 tx=0).stop 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- 172.21.15.116:0/3379272213 shutdown_connections 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0914160560 0x7f0914160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 --2- 172.21.15.116:0/3379272213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140705b0 0x7f09140d1730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- 172.21.15.116:0/3379272213 >> 172.21.15.116:0/3379272213 conn(0x7f09140b4720 msgr2=0x7f09140b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- 172.21.15.116:0/3379272213 shutdown_connections 2024-05-14T11:16:31.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- 172.21.15.116:0/3379272213 wait complete. 2024-05-14T11:16:31.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 Processor -- start 2024-05-14T11:16:31.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.133+0000 7f0919a00640 1 -- start start 2024-05-14T11:16:31.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09140705b0 0x7f09140cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 0x7f09140af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0914163590 con 0x7f09140d1c70 2024-05-14T11:16:31.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0914163410 con 0x7f0914160560 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f091416d0a0 con 0x7f09140705b0 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52078/0 (socket says 172.21.15.116:52078) 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 -- 172.21.15.116:0/2049790590 learned_addr learned my addr 172.21.15.116:0/2049790590 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09140705b0 0x7f09140cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0913400640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 0x7f09140af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 -- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09140705b0 msgr2=0x7f09140cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09140705b0 0x7f09140cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 -- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 msgr2=0x7f09140af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 0x7f09140af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 -- 172.21.15.116:0/2049790590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0914158cc0 con 0x7f09140d1c70 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0913400640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 0x7f09140af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:31.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0912000640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007fd0 tx=0x7f08fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f09140d1c70 2024-05-14T11:16:31.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc005d10 con 0x7f09140d1c70 2024-05-14T11:16:31.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc007310 con 0x7f09140d1c70 2024-05-14T11:16:31.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0914158fa0 con 0x7f09140d1c70 2024-05-14T11:16:31.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0914159560 con 0x7f09140d1c70 2024-05-14T11:16:31.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.137+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09141555c0 con 0x7f09140d1c70 2024-05-14T11:16:31.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.141+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08fc01a070 con 0x7f09140d1c70 2024-05-14T11:16:31.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.141+0000 7f090aa00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 0x7f08f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.141+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f08fc08e9a0 con 0x7f09140d1c70 2024-05-14T11:16:31.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.141+0000 7f0912a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 0x7f08f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.141+0000 7f0912a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 0x7f08f0071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09040af000 tx=0x7f09040a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.145+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f09140d2080 con 0x7f09140d1c70 2024-05-14T11:16:31.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.337+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f091406a9c0 con 0x7f09140d1c70 2024-05-14T11:16:31.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.437+0000 7f090aa00640 1 -- 172.21.15.116:0/2049790590 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f08fc05aee0 con 0x7f09140d1c70 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 msgr2=0x7f08f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 0x7f08f0071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09040af000 tx=0x7f09040a7070 comp rx=0 tx=0).stop 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 msgr2=0x7f09141580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007fd0 tx=0x7f08fc00c890 comp rx=0 tx=0).stop 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 shutdown_connections 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f08f006f400 0x7f08f0071910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0914160560 0x7f09140af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09140d1c70 0x7f09141580b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 --2- 172.21.15.116:0/2049790590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09140705b0 0x7f09140cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 >> 172.21.15.116:0/2049790590 conn(0x7f09140b4720 msgr2=0x7f091415ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:31.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 shutdown_connections 2024-05-14T11:16:31.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.441+0000 7f0919a00640 1 -- 172.21.15.116:0/2049790590 wait complete. 2024-05-14T11:16:31.462 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-14T11:16:31.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 Processor -- start 2024-05-14T11:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 -- start start 2024-05-14T11:16:31.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 0x7f4a5c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c160670 0x7f4a5c160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4a5c06f0a0 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4a5c0cbb60 con 0x7f4a5c0d1730 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a61000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c0d2080 con 0x7f4a5c160670 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a59e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a59e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52090/0 (socket says 172.21.15.116:52090) 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.609+0000 7f4a59e00640 1 -- 172.21.15.116:0/2719191041 learned_addr learned my addr 172.21.15.116:0/2719191041 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a51400640 1 --2- 172.21.15.116:0/2719191041 >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 0x7f4a5c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a5a800640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c160670 0x7f4a5c160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 -- 172.21.15.116:0/2719191041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c160670 msgr2=0x7f4a5c160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c160670 0x7f4a5c160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 -- 172.21.15.116:0/2719191041 >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 msgr2=0x7f4a5c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:31.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2719191041 >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 0x7f4a5c0d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 -- 172.21.15.116:0/2719191041 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c06f420 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a51400640 1 --2- 172.21.15.116:0/2719191041 >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 0x7f4a5c0d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:31.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c070cd0 tx=0x7f4a4c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c028b60c622fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59400640 1 -- 172.21.15.116:0/2719191041 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0a6040 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59400640 1 -- 172.21.15.116:0/2719191041 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4c0af070 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.613+0000 7f4a59400640 1 -- 172.21.15.116:0/2719191041 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0aa860 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- 172.21.15.116:0/2719191041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0cd1c0 msgr2=0x7f4a5c0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c070cd0 tx=0x7f4a4c01c520 comp rx=0 tx=0).stop 2024-05-14T11:16:31.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- 172.21.15.116:0/2719191041 shutdown_connections 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c160670 0x7f4a5c160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- 172.21.15.116:0/2719191041 >> v2:172.21.15.149:3300/0 conn(0x7f4a5c0d1730 0x7f4a5c0d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- 172.21.15.116:0/2719191041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0cd5d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- 172.21.15.116:0/2719191041 >> 172.21.15.116:0/2719191041 conn(0x7f4a5c0b4720 msgr2=0x7f4a5c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- 172.21.15.116:0/2719191041 shutdown_connections 2024-05-14T11:16:31.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- 172.21.15.116:0/2719191041 wait complete. 2024-05-14T11:16:31.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 Processor -- start 2024-05-14T11:16:31.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 -- start start 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.617+0000 7f4a61000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a5c160670 0x7f4a5c0cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c15cf10 con 0x7f4a5c0d1730 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c06ffa0 con 0x7f4a5c160670 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c1613d0 con 0x7f4a5c0cd1c0 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52100/0 (socket says 172.21.15.116:52100) 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 -- 172.21.15.116:0/2289600456 learned_addr learned my addr 172.21.15.116:0/2289600456 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a5a800640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a5c160670 0x7f4a5c0cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 -- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 msgr2=0x7f4a5c0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 -- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a5c160670 msgr2=0x7f4a5c0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a5c160670 0x7f4a5c0cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 -- 172.21.15.116:0/2289600456 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c0c8810 con 0x7f4a5c0d1730 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a51400640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400d800 tx=0x7f4a4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44018070 con 0x7f4a5c0d1730 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44004030 con 0x7f4a5c0d1730 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c158d50 con 0x7f4a5c0d1730 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004400 con 0x7f4a5c0d1730 2024-05-14T11:16:31.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c099860 con 0x7f4a5c0d1730 2024-05-14T11:16:31.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c06a9c0 con 0x7f4a5c0d1730 2024-05-14T11:16:31.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.621+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a44009640 con 0x7f4a5c0d1730 2024-05-14T11:16:31.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.625+0000 7f4a52a00640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 0x7f4a38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:31.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.625+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4a4408fa30 con 0x7f4a5c0d1730 2024-05-14T11:16:31.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.625+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 0x7f4a38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:31.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.625+0000 7f4a59e00640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 0x7f4a38071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0ae000 tx=0x7f4a4c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:31.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.629+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4a4405c540 con 0x7f4a5c0d1730 2024-05-14T11:16:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:31.817+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4a5c069de0 con 0x7f4a5c0d1730 2024-05-14T11:16:32.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.445+0000 7f4a52a00640 1 -- 172.21.15.116:0/2289600456 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f4a4405be40 con 0x7f4a5c0d1730 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 msgr2=0x7f4a38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 0x7f4a38071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0ae000 tx=0x7f4a4c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 msgr2=0x7f4a5c0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400d800 tx=0x7f4a4400dcc0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 shutdown_connections 2024-05-14T11:16:32.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3806f400 0x7f4a38071910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a5c160670 0x7f4a5c0cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c0d1730 0x7f4a5c0ca4b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 --2- 172.21.15.116:0/2289600456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a5c0cd1c0 0x7f4a5c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 >> 172.21.15.116:0/2289600456 conn(0x7f4a5c0b4720 msgr2=0x7f4a5c0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 shutdown_connections 2024-05-14T11:16:32.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.449+0000 7f4a61000640 1 -- 172.21.15.116:0/2289600456 wait complete. 2024-05-14T11:16:32.471 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:16:32.472 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 Processor -- start 2024-05-14T11:16:32.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 -- start start 2024-05-14T11:16:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1d70156fa0 0x7f1d7006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1d700d1670 0x7f1d700d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d7006e9e0 0x7f1d7006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1d701573b0 con 0x7f1d700d1670 2024-05-14T11:16:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1d70161120 con 0x7f1d70156fa0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d76200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1d700d1fc0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d6ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1d700d1670 0x7f1d700d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d6ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1d700d1670 0x7f1d700d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52110/0 (socket says 172.21.15.116:52110) 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d6ea00640 1 -- 172.21.15.116:0/791813120 learned_addr learned my addr 172.21.15.116:0/791813120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.621+0000 7f1d6fe00640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d7006e9e0 0x7f1d7006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6f400640 1 --2- 172.21.15.116:0/791813120 >> v2:172.21.15.149:3300/0 conn(0x7f1d70156fa0 0x7f1d7006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 -- 172.21.15.116:0/791813120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d7006e9e0 msgr2=0x7f1d7006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d7006e9e0 0x7f1d7006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 -- 172.21.15.116:0/791813120 >> v2:172.21.15.149:3300/0 conn(0x7f1d70156fa0 msgr2=0x7f1d7006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/791813120 >> v2:172.21.15.149:3300/0 conn(0x7f1d70156fa0 0x7f1d7006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 -- 172.21.15.116:0/791813120 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d70160510 con 0x7f1d700d1670 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d700d1670 0x7f1d700d1a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d70071f20 tx=0x7f1d6001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6706c58e42ca37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6e000640 1 -- 172.21.15.116:0/791813120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600b4070 con 0x7f1d700d1670 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6e000640 1 -- 172.21.15.116:0/791813120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d600a7b50 con 0x7f1d700d1670 2024-05-14T11:16:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d6e000640 1 -- 172.21.15.116:0/791813120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d600af440 con 0x7f1d700d1670 2024-05-14T11:16:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d76200640 1 -- 172.21.15.116:0/791813120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d700d1670 msgr2=0x7f1d700d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.625+0000 7f1d76200640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d700d1670 0x7f1d700d1a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d70071f20 tx=0x7f1d6001ebc0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- 172.21.15.116:0/791813120 shutdown_connections 2024-05-14T11:16:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d700d1670 0x7f1d700d1a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- 172.21.15.116:0/791813120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d7006e9e0 0x7f1d7006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- 172.21.15.116:0/791813120 >> v2:172.21.15.149:3300/0 conn(0x7f1d70156fa0 0x7f1d7006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- 172.21.15.116:0/791813120 >> 172.21.15.116:0/791813120 conn(0x7f1d700b4620 msgr2=0x7f1d700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- 172.21.15.116:0/791813120 shutdown_connections 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- 172.21.15.116:0/791813120 wait complete. 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 Processor -- start 2024-05-14T11:16:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- start start 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 0x7f1d7014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d70156fa0 0x7f1d7014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70070520 con 0x7f1d7006e9e0 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7006f740 con 0x7f1d70156fa0 2024-05-14T11:16:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.629+0000 7f1d76200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1d700706a0 con 0x7f1d700d1670 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52124/0 (socket says 172.21.15.116:52124) 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 -- 172.21.15.116:0/262470994 learned_addr learned my addr 172.21.15.116:0/262470994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 0x7f1d7014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6fe00640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d70156fa0 0x7f1d7014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 -- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 msgr2=0x7f1d7014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 0x7f1d7014b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 -- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d70156fa0 msgr2=0x7f1d7014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d70156fa0 0x7f1d7014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 -- 172.21.15.116:0/262470994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d70149710 con 0x7f1d7006e9e0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 0x7f1d7014b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d6f400640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800d5d0 tx=0x7f1d5800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004240 con 0x7f1d7006e9e0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d580043e0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58003840 con 0x7f1d7006e9e0 2024-05-14T11:16:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701499f0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d70149fb0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7006a9c0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.633+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d580039e0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.637+0000 7f1d67400640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 0x7f1d4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.637+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d5808f810 con 0x7f1d7006e9e0 2024-05-14T11:16:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.637+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 0x7f1d4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.637+0000 7f1d6ea00640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 0x7f1d4c071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b1000 tx=0x7f1d600a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:32.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.641+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1d5805c3b0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d701511d0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d67400640 1 -- 172.21.15.116:0/262470994 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f1d5805bcb0 con 0x7f1d7006e9e0 2024-05-14T11:16:32.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:32.835 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:32.836 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:16:32.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 msgr2=0x7f1d4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 0x7f1d4c071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b1000 tx=0x7f1d600a9070 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 msgr2=0x7f1d70148900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800d5d0 tx=0x7f1d5800da90 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 shutdown_connections 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4c06f430 0x7f1d4c071940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d70156fa0 0x7f1d7014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d700d1670 0x7f1d7014b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 --2- 172.21.15.116:0/262470994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7006e9e0 0x7f1d70148900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.833+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 >> 172.21.15.116:0/262470994 conn(0x7f1d700b4620 msgr2=0x7f1d700b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.837+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 shutdown_connections 2024-05-14T11:16:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:32.837+0000 7f1d76200640 1 -- 172.21.15.116:0/262470994 wait complete. 2024-05-14T11:16:32.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 Processor -- start 2024-05-14T11:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 -- start start 2024-05-14T11:16:33.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 0x7fd1e406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e406e8f0 con 0x7fd1e4160d20 2024-05-14T11:16:33.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e4160890 con 0x7fd1e406dfa0 2024-05-14T11:16:33.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1ea600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e413fca0 con 0x7fd1e40cfb30 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e3e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 0x7fd1e406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35214/0 (socket says 172.21.15.116:35214) 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9200640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fd1e4160d20 0x7fd1e415cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34778/0 (socket says 172.21.15.116:34778) 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 -- 172.21.15.116:0/306940399 learned_addr learned my addr 172.21.15.116:0/306940399 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 -- 172.21.15.116:0/306940399 >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 msgr2=0x7fd1e406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 --2- 172.21.15.116:0/306940399 >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 0x7fd1e406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 -- 172.21.15.116:0/306940399 >> v1:172.21.15.149:6789/0 conn(0x7fd1e4160d20 legacy=0x7fd1e415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e9c00640 1 -- 172.21.15.116:0/306940399 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e415d870 con 0x7fd1e40cfb30 2024-05-14T11:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.001+0000 7fd1e3e00640 1 --2- 172.21.15.116:0/306940399 >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 0x7fd1e406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:33.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1e9c00640 1 --2- 172.21.15.116:0/306940399 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009e50 tx=0x7fd1dc01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3535175e8c4ed6b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1e3400640 1 -- 172.21.15.116:0/306940399 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0a5860 con 0x7fd1e40cfb30 2024-05-14T11:16:33.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1e3400640 1 -- 172.21.15.116:0/306940399 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0a5a00 con 0x7fd1e40cfb30 2024-05-14T11:16:33.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1e3400640 1 -- 172.21.15.116:0/306940399 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0a5dd0 con 0x7fd1e40cfb30 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1ea600640 1 -- 172.21.15.116:0/306940399 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 msgr2=0x7fd1e40cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1ea600640 1 --2- 172.21.15.116:0/306940399 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009e50 tx=0x7fd1dc01c9a0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.005+0000 7fd1ea600640 1 -- 172.21.15.116:0/306940399 shutdown_connections 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 --2- 172.21.15.116:0/306940399 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e40cfb30 0x7fd1e40cff40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 --2- 172.21.15.116:0/306940399 >> v2:172.21.15.149:3300/0 conn(0x7fd1e406dfa0 0x7fd1e406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 -- 172.21.15.116:0/306940399 >> 172.21.15.116:0/306940399 conn(0x7fd1e40731a0 msgr2=0x7fd1e40735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 -- 172.21.15.116:0/306940399 shutdown_connections 2024-05-14T11:16:33.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 -- 172.21.15.116:0/306940399 wait complete. 2024-05-14T11:16:33.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 Processor -- start 2024-05-14T11:16:33.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 -- start start 2024-05-14T11:16:33.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.009+0000 7fd1ea600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1e40cfb30 0x7fd1e4071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e4160d20 0x7fd1e413fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40d1650 con 0x7fd1e406dfa0 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e40d14d0 con 0x7fd1e40cfb30 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e4161130 con 0x7fd1e4160d20 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52128/0 (socket says 172.21.15.116:52128) 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 -- 172.21.15.116:0/698923762 learned_addr learned my addr 172.21.15.116:0/698923762 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9c00640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e4160d20 0x7fd1e413fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e3e00640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1e40cfb30 0x7fd1e4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 -- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e4160d20 msgr2=0x7fd1e413fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e4160d20 0x7fd1e413fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 -- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1e40cfb30 msgr2=0x7fd1e4071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1e40cfb30 0x7fd1e4071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 -- 172.21.15.116:0/698923762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e4071570 con 0x7fd1e406dfa0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e9200640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c3c0 tx=0x7fd1d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80074e0 con 0x7fd1e406dfa0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d8007680 con 0x7fd1e406dfa0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e406ff90 con 0x7fd1e406dfa0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004a10 con 0x7fd1e406dfa0 2024-05-14T11:16:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e4070550 con 0x7fd1e406dfa0 2024-05-14T11:16:33.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e4050880 con 0x7fd1e406dfa0 2024-05-14T11:16:33.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.013+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd1d8004bb0 con 0x7fd1e406dfa0 2024-05-14T11:16:33.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.017+0000 7fd1e0c00640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 0x7fd1c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.017+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd1d808ea40 con 0x7fd1e406dfa0 2024-05-14T11:16:33.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.017+0000 7fd1e3e00640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 0x7fd1c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.017+0000 7fd1e3e00640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 0x7fd1c4071910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4064500 tx=0x7fd1d0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.021+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd1e406e8f0 con 0x7fd1e406dfa0 2024-05-14T11:16:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.181+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1e40ce5f0 con 0x7fd1e406dfa0 2024-05-14T11:16:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.181+0000 7fd1e0c00640 1 -- 172.21.15.116:0/698923762 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fd1d805af00 con 0x7fd1e406dfa0 2024-05-14T11:16:33.184 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:33.184 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 msgr2=0x7fd1c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 0x7fd1c4071910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4064500 tx=0x7fd1d0009190 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 msgr2=0x7fd1e4142380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c3c0 tx=0x7fd1d800c880 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 shutdown_connections 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1c406f400 0x7fd1c4071910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1e40cfb30 0x7fd1e4071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1e4160d20 0x7fd1e413fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 --2- 172.21.15.116:0/698923762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e406dfa0 0x7fd1e4142380 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 >> 172.21.15.116:0/698923762 conn(0x7fd1e40731a0 msgr2=0x7fd1e41669c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 shutdown_connections 2024-05-14T11:16:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.185+0000 7fd1ea600640 1 -- 172.21.15.116:0/698923762 wait complete. 2024-05-14T11:16:33.207 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:16:33.207 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:16:33.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 Processor -- start 2024-05-14T11:16:33.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 -- start start 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9f6c0b6b60 0x7f9f6c073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9f6c0743c0 0x7f9f6c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b5830 0x7f9f6c160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6c163c30 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6c163f10 con 0x7f9f6c0743c0 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f71000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c1641f0 con 0x7f9f6c0b5830 2024-05-14T11:16:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9f6c0b6b60 0x7f9f6c073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9f6c0b6b60 0x7f9f6c073e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52138/0 (socket says 172.21.15.116:52138) 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 -- 172.21.15.116:0/63151165 learned_addr learned my addr 172.21.15.116:0/63151165 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6aa00640 1 --2- 172.21.15.116:0/63151165 >> v2:172.21.15.149:3300/0 conn(0x7f9f6c0743c0 0x7f9f6c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6be00640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b5830 0x7f9f6c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 -- 172.21.15.116:0/63151165 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b5830 msgr2=0x7f9f6c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b5830 0x7f9f6c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 -- 172.21.15.116:0/63151165 >> v2:172.21.15.149:3300/0 conn(0x7f9f6c0743c0 msgr2=0x7f9f6c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 --2- 172.21.15.116:0/63151165 >> v2:172.21.15.149:3300/0 conn(0x7f9f6c0743c0 0x7f9f6c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.353+0000 7f9f6b400640 1 -- 172.21.15.116:0/63151165 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c1644d0 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f6b400640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b6b60 0x7f9f6c073e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0b1960 tx=0x7f9f5c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22388f7ad60088d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f6a000640 1 -- 172.21.15.116:0/63151165 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0b1070 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f6a000640 1 -- 172.21.15.116:0/63151165 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0ac620 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f6a000640 1 -- 172.21.15.116:0/63151165 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0ac9f0 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 -- 172.21.15.116:0/63151165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b6b60 msgr2=0x7f9f6c073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b6b60 0x7f9f6c073e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0b1960 tx=0x7f9f5c01c4f0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 -- 172.21.15.116:0/63151165 shutdown_connections 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b5830 0x7f9f6c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 --2- 172.21.15.116:0/63151165 >> v2:172.21.15.149:3300/0 conn(0x7f9f6c0743c0 0x7f9f6c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 --2- 172.21.15.116:0/63151165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b6b60 0x7f9f6c073e80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 -- 172.21.15.116:0/63151165 >> 172.21.15.116:0/63151165 conn(0x7f9f6c0d26e0 msgr2=0x7f9f6c0d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 -- 172.21.15.116:0/63151165 shutdown_connections 2024-05-14T11:16:33.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.357+0000 7f9f71000640 1 -- 172.21.15.116:0/63151165 wait complete. 2024-05-14T11:16:33.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 Processor -- start 2024-05-14T11:16:33.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- start start 2024-05-14T11:16:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b6b60 0x7f9f6c147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6c0b0ad0 0x7f9f6c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1568f0 con 0x7f9f6c0b5830 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c156770 con 0x7f9f6c0b0ad0 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c156a70 con 0x7f9f6c0b6b60 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52148/0 (socket says 172.21.15.116:52148) 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 -- 172.21.15.116:0/4006358511 learned_addr learned my addr 172.21.15.116:0/4006358511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6be00640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6c0b0ad0 0x7f9f6c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6aa00640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b6b60 0x7f9f6c147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 -- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b6b60 msgr2=0x7f9f6c147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b6b60 0x7f9f6c147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 -- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6c0b0ad0 msgr2=0x7f9f6c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6c0b0ad0 0x7f9f6c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 -- 172.21.15.116:0/4006358511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c0b1280 con 0x7f9f6c0b5830 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f6b400640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01cd90 tx=0x7f9f5c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0b1040 con 0x7f9f6c0b5830 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c0059a0 con 0x7f9f6c0b5830 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c148710 con 0x7f9f6c0b5830 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c005d50 con 0x7f9f6c0b5830 2024-05-14T11:16:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.361+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c148cd0 con 0x7f9f6c0b5830 2024-05-14T11:16:33.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.365+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c050880 con 0x7f9f6c0b5830 2024-05-14T11:16:33.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.365+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9f5c0bc060 con 0x7f9f6c0b5830 2024-05-14T11:16:33.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.365+0000 7f9f63400640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 0x7f9f4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.365+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f5c128670 con 0x7f9f6c0b5830 2024-05-14T11:16:33.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.365+0000 7f9f6aa00640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 0x7f9f4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.369+0000 7f9f6aa00640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 0x7f9f4c071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c155960 tx=0x7f9f58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.373+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9f5c016110 con 0x7f9f6c0b5830 2024-05-14T11:16:33.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.525+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f6c149a10 con 0x7f9f4c06f400 2024-05-14T11:16:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.537+0000 7f9f63400640 1 -- 172.21.15.116:0/4006358511 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f9f6c149a10 con 0x7f9f4c06f400 2024-05-14T11:16:33.540 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:33.545 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:16:33.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 msgr2=0x7f9f4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 0x7f9f4c071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c155960 tx=0x7f9f58008040 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 msgr2=0x7f9f6c1494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01cd90 tx=0x7f9f5c019290 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 shutdown_connections 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9f4c06f400 0x7f9f4c071910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6c0b0ad0 0x7f9f6c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f6c0b6b60 0x7f9f6c147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 --2- 172.21.15.116:0/4006358511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f6c0b5830 0x7f9f6c1494d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 >> 172.21.15.116:0/4006358511 conn(0x7f9f6c0d26e0 msgr2=0x7f9f6c073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 shutdown_connections 2024-05-14T11:16:33.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.541+0000 7f9f71000640 1 -- 172.21.15.116:0/4006358511 wait complete. 2024-05-14T11:16:33.563 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-14T11:16:32.363008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003634"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807671+0000","last_change":"2024-05-14T11:16:11.249111+0000","last_active":"2024-05-14T11:16:17.807671+0000","last_peered":"2024-05-14T11:16:17.807671+0000","last_clean":"2024-05-14T11:16:17.807671+0000","last_became_active":"2024-05-14T11:16:11.248739+0000","last_became_peered":"2024-05-14T11:16:11.248739+0000","last_unstale":"2024-05-14T11:16:17.807671+0000","last_undegraded":"2024-05-14T11:16:17.807671+0000","last_fullsized":"2024-05-14T11:16:17.807671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808824+0000","last_change":"2024-05-14T11:16:12.266252+0000","last_active":"2024-05-14T11:16:17.808824+0000","last_peered":"2024-05-14T11:16:17.808824+0000","last_clean":"2024-05-14T11:16:17.808824+0000","last_became_active":"2024-05-14T11:16:12.265805+0000","last_became_peered":"2024-05-14T11:16:12.265805+0000","last_unstale":"2024-05-14T11:16:17.808824+0000","last_undegraded":"2024-05-14T11:16:17.808824+0000","last_fullsized":"2024-05-14T11:16:17.808824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808167+0000","last_change":"2024-05-14T11:16:16.339345+0000","last_active":"2024-05-14T11:16:17.808167+0000","last_peered":"2024-05-14T11:16:17.808167+0000","last_clean":"2024-05-14T11:16:17.808167+0000","last_became_active":"2024-05-14T11:16:16.338538+0000","last_became_peered":"2024-05-14T11:16:16.338538+0000","last_unstale":"2024-05-14T11:16:17.808167+0000","last_undegraded":"2024-05-14T11:16:17.808167+0000","last_fullsized":"2024-05-14T11:16:17.808167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809078+0000","last_change":"2024-05-14T11:16:11.246975+0000","last_active":"2024-05-14T11:16:17.809078+0000","last_peered":"2024-05-14T11:16:17.809078+0000","last_clean":"2024-05-14T11:16:17.809078+0000","last_became_active":"2024-05-14T11:16:11.246617+0000","last_became_peered":"2024-05-14T11:16:11.246617+0000","last_unstale":"2024-05-14T11:16:17.809078+0000","last_undegraded":"2024-05-14T11:16:17.809078+0000","last_fullsized":"2024-05-14T11:16:17.809078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807310+0000","last_change":"2024-05-14T11:16:12.261153+0000","last_active":"2024-05-14T11:16:17.807310+0000","last_peered":"2024-05-14T11:16:17.807310+0000","last_clean":"2024-05-14T11:16:17.807310+0000","last_became_active":"2024-05-14T11:16:12.260566+0000","last_became_peered":"2024-05-14T11:16:12.260566+0000","last_unstale":"2024-05-14T11:16:17.807310+0000","last_undegraded":"2024-05-14T11:16:17.807310+0000","last_fullsized":"2024-05-14T11:16:17.807310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:47.069928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300836+0000","last_change":"2024-05-14T11:16:16.298112+0000","last_active":"2024-05-14T11:16:17.300836+0000","last_peered":"2024-05-14T11:16:17.300836+0000","last_clean":"2024-05-14T11:16:17.300836+0000","last_became_active":"2024-05-14T11:16:16.297673+0000","last_became_peered":"2024-05-14T11:16:16.297673+0000","last_unstale":"2024-05-14T11:16:17.300836+0000","last_undegraded":"2024-05-14T11:16:17.300836+0000","last_fullsized":"2024-05-14T11:16:17.300836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:13.339957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807596+0000","last_change":"2024-05-14T11:16:11.253925+0000","last_active":"2024-05-14T11:16:17.807596+0000","last_peered":"2024-05-14T11:16:17.807596+0000","last_clean":"2024-05-14T11:16:17.807596+0000","last_became_active":"2024-05-14T11:16:11.253441+0000","last_became_peered":"2024-05-14T11:16:11.253441+0000","last_unstale":"2024-05-14T11:16:17.807596+0000","last_undegraded":"2024-05-14T11:16:17.807596+0000","last_fullsized":"2024-05-14T11:16:17.807596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806697+0000","last_change":"2024-05-14T11:16:12.255735+0000","last_active":"2024-05-14T11:16:17.806697+0000","last_peered":"2024-05-14T11:16:17.806697+0000","last_clean":"2024-05-14T11:16:17.806697+0000","last_became_active":"2024-05-14T11:16:12.255414+0000","last_became_peered":"2024-05-14T11:16:12.255414+0000","last_unstale":"2024-05-14T11:16:17.806697+0000","last_undegraded":"2024-05-14T11:16:17.806697+0000","last_fullsized":"2024-05-14T11:16:17.806697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300072+0000","last_change":"2024-05-14T11:16:16.297447+0000","last_active":"2024-05-14T11:16:17.300072+0000","last_peered":"2024-05-14T11:16:17.300072+0000","last_clean":"2024-05-14T11:16:17.300072+0000","last_became_active":"2024-05-14T11:16:16.297024+0000","last_became_peered":"2024-05-14T11:16:16.297024+0000","last_unstale":"2024-05-14T11:16:17.300072+0000","last_undegraded":"2024-05-14T11:16:17.300072+0000","last_fullsized":"2024-05-14T11:16:17.300072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:41.465988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806953+0000","last_change":"2024-05-14T11:16:11.263356+0000","last_active":"2024-05-14T11:16:17.806953+0000","last_peered":"2024-05-14T11:16:17.806953+0000","last_clean":"2024-05-14T11:16:17.806953+0000","last_became_active":"2024-05-14T11:16:11.263025+0000","last_became_peered":"2024-05-14T11:16:11.263025+0000","last_unstale":"2024-05-14T11:16:17.806953+0000","last_undegraded":"2024-05-14T11:16:17.806953+0000","last_fullsized":"2024-05-14T11:16:17.806953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806916+0000","last_change":"2024-05-14T11:16:12.253871+0000","last_active":"2024-05-14T11:16:17.806916+0000","last_peered":"2024-05-14T11:16:17.806916+0000","last_clean":"2024-05-14T11:16:17.806916+0000","last_became_active":"2024-05-14T11:16:12.253416+0000","last_became_peered":"2024-05-14T11:16:12.253416+0000","last_unstale":"2024-05-14T11:16:17.806916+0000","last_undegraded":"2024-05-14T11:16:17.806916+0000","last_fullsized":"2024-05-14T11:16:17.806916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300363+0000","last_change":"2024-05-14T11:16:16.289232+0000","last_active":"2024-05-14T11:16:17.300363+0000","last_peered":"2024-05-14T11:16:17.300363+0000","last_clean":"2024-05-14T11:16:17.300363+0000","last_became_active":"2024-05-14T11:16:16.288706+0000","last_became_peered":"2024-05-14T11:16:16.288706+0000","last_unstale":"2024-05-14T11:16:17.300363+0000","last_undegraded":"2024-05-14T11:16:17.300363+0000","last_fullsized":"2024-05-14T11:16:17.300363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:13.765975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299905+0000","last_change":"2024-05-14T11:16:12.237866+0000","last_active":"2024-05-14T11:16:17.299905+0000","last_peered":"2024-05-14T11:16:17.299905+0000","last_clean":"2024-05-14T11:16:17.299905+0000","last_became_active":"2024-05-14T11:16:12.237591+0000","last_became_peered":"2024-05-14T11:16:12.237591+0000","last_unstale":"2024-05-14T11:16:17.299905+0000","last_undegraded":"2024-05-14T11:16:17.299905+0000","last_fullsized":"2024-05-14T11:16:17.299905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807660+0000","last_change":"2024-05-14T11:16:11.250842+0000","last_active":"2024-05-14T11:16:17.807660+0000","last_peered":"2024-05-14T11:16:17.807660+0000","last_clean":"2024-05-14T11:16:17.807660+0000","last_became_active":"2024-05-14T11:16:11.250454+0000","last_became_peered":"2024-05-14T11:16:11.250454+0000","last_unstale":"2024-05-14T11:16:17.807660+0000","last_undegraded":"2024-05-14T11:16:17.807660+0000","last_fullsized":"2024-05-14T11:16:17.807660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302586+0000","last_change":"2024-05-14T11:16:16.299192+0000","last_active":"2024-05-14T11:16:17.302586+0000","last_peered":"2024-05-14T11:16:17.302586+0000","last_clean":"2024-05-14T11:16:17.302586+0000","last_became_active":"2024-05-14T11:16:16.298925+0000","last_became_peered":"2024-05-14T11:16:16.298925+0000","last_unstale":"2024-05-14T11:16:17.302586+0000","last_undegraded":"2024-05-14T11:16:17.302586+0000","last_fullsized":"2024-05-14T11:16:17.302586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808972+0000","last_change":"2024-05-14T11:16:12.266594+0000","last_active":"2024-05-14T11:16:17.808972+0000","last_peered":"2024-05-14T11:16:17.808972+0000","last_clean":"2024-05-14T11:16:17.808972+0000","last_became_active":"2024-05-14T11:16:12.266261+0000","last_became_peered":"2024-05-14T11:16:12.266261+0000","last_unstale":"2024-05-14T11:16:17.808972+0000","last_undegraded":"2024-05-14T11:16:17.808972+0000","last_fullsized":"2024-05-14T11:16:17.808972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808959+0000","last_change":"2024-05-14T11:16:11.251548+0000","last_active":"2024-05-14T11:16:17.808959+0000","last_peered":"2024-05-14T11:16:17.808959+0000","last_clean":"2024-05-14T11:16:17.808959+0000","last_became_active":"2024-05-14T11:16:11.251146+0000","last_became_peered":"2024-05-14T11:16:11.251146+0000","last_unstale":"2024-05-14T11:16:17.808959+0000","last_undegraded":"2024-05-14T11:16:17.808959+0000","last_fullsized":"2024-05-14T11:16:17.808959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300981+0000","last_change":"2024-05-14T11:16:16.296270+0000","last_active":"2024-05-14T11:16:17.300981+0000","last_peered":"2024-05-14T11:16:17.300981+0000","last_clean":"2024-05-14T11:16:17.300981+0000","last_became_active":"2024-05-14T11:16:16.295796+0000","last_became_peered":"2024-05-14T11:16:16.295796+0000","last_unstale":"2024-05-14T11:16:17.300981+0000","last_undegraded":"2024-05-14T11:16:17.300981+0000","last_fullsized":"2024-05-14T11:16:17.300981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808335+0000","last_change":"2024-05-14T11:16:12.259552+0000","last_active":"2024-05-14T11:16:17.808335+0000","last_peered":"2024-05-14T11:16:17.808335+0000","last_clean":"2024-05-14T11:16:17.808335+0000","last_became_active":"2024-05-14T11:16:12.259053+0000","last_became_peered":"2024-05-14T11:16:12.259053+0000","last_unstale":"2024-05-14T11:16:17.808335+0000","last_undegraded":"2024-05-14T11:16:17.808335+0000","last_fullsized":"2024-05-14T11:16:17.808335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806762+0000","last_change":"2024-05-14T11:16:11.251898+0000","last_active":"2024-05-14T11:16:17.806762+0000","last_peered":"2024-05-14T11:16:17.806762+0000","last_clean":"2024-05-14T11:16:17.806762+0000","last_became_active":"2024-05-14T11:16:11.251339+0000","last_became_peered":"2024-05-14T11:16:11.251339+0000","last_unstale":"2024-05-14T11:16:17.806762+0000","last_undegraded":"2024-05-14T11:16:17.806762+0000","last_fullsized":"2024-05-14T11:16:17.806762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809475+0000","last_change":"2024-05-14T11:16:16.331750+0000","last_active":"2024-05-14T11:16:17.809475+0000","last_peered":"2024-05-14T11:16:17.809475+0000","last_clean":"2024-05-14T11:16:17.809475+0000","last_became_active":"2024-05-14T11:16:16.331181+0000","last_became_peered":"2024-05-14T11:16:16.331181+0000","last_unstale":"2024-05-14T11:16:17.809475+0000","last_undegraded":"2024-05-14T11:16:17.809475+0000","last_fullsized":"2024-05-14T11:16:17.809475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301296+0000","last_change":"2024-05-14T11:16:12.245880+0000","last_active":"2024-05-14T11:16:17.301296+0000","last_peered":"2024-05-14T11:16:17.301296+0000","last_clean":"2024-05-14T11:16:17.301296+0000","last_became_active":"2024-05-14T11:16:12.245150+0000","last_became_peered":"2024-05-14T11:16:12.245150+0000","last_unstale":"2024-05-14T11:16:17.301296+0000","last_undegraded":"2024-05-14T11:16:17.301296+0000","last_fullsized":"2024-05-14T11:16:17.301296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806258+0000","last_change":"2024-05-14T11:16:11.252415+0000","last_active":"2024-05-14T11:16:17.806258+0000","last_peered":"2024-05-14T11:16:17.806258+0000","last_clean":"2024-05-14T11:16:17.806258+0000","last_became_active":"2024-05-14T11:16:11.252113+0000","last_became_peered":"2024-05-14T11:16:11.252113+0000","last_unstale":"2024-05-14T11:16:17.806258+0000","last_undegraded":"2024-05-14T11:16:17.806258+0000","last_fullsized":"2024-05-14T11:16:17.806258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302531+0000","last_change":"2024-05-14T11:16:16.298123+0000","last_active":"2024-05-14T11:16:17.302531+0000","last_peered":"2024-05-14T11:16:17.302531+0000","last_clean":"2024-05-14T11:16:17.302531+0000","last_became_active":"2024-05-14T11:16:16.297647+0000","last_became_peered":"2024-05-14T11:16:16.297647+0000","last_unstale":"2024-05-14T11:16:17.302531+0000","last_undegraded":"2024-05-14T11:16:17.302531+0000","last_fullsized":"2024-05-14T11:16:17.302531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807122+0000","last_change":"2024-05-14T11:16:12.247217+0000","last_active":"2024-05-14T11:16:17.807122+0000","last_peered":"2024-05-14T11:16:17.807122+0000","last_clean":"2024-05-14T11:16:17.807122+0000","last_became_active":"2024-05-14T11:16:12.246460+0000","last_became_peered":"2024-05-14T11:16:12.246460+0000","last_unstale":"2024-05-14T11:16:17.807122+0000","last_undegraded":"2024-05-14T11:16:17.807122+0000","last_fullsized":"2024-05-14T11:16:17.807122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301791+0000","last_change":"2024-05-14T11:16:11.246553+0000","last_active":"2024-05-14T11:16:17.301791+0000","last_peered":"2024-05-14T11:16:17.301791+0000","last_clean":"2024-05-14T11:16:17.301791+0000","last_became_active":"2024-05-14T11:16:11.245634+0000","last_became_peered":"2024-05-14T11:16:11.245634+0000","last_unstale":"2024-05-14T11:16:17.301791+0000","last_undegraded":"2024-05-14T11:16:17.301791+0000","last_fullsized":"2024-05-14T11:16:17.301791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300036+0000","last_change":"2024-05-14T11:16:16.331377+0000","last_active":"2024-05-14T11:16:17.300036+0000","last_peered":"2024-05-14T11:16:17.300036+0000","last_clean":"2024-05-14T11:16:17.300036+0000","last_became_active":"2024-05-14T11:16:16.330845+0000","last_became_peered":"2024-05-14T11:16:16.330845+0000","last_unstale":"2024-05-14T11:16:17.300036+0000","last_undegraded":"2024-05-14T11:16:17.300036+0000","last_fullsized":"2024-05-14T11:16:17.300036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:08.942678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809208+0000","last_change":"2024-05-14T11:16:12.265091+0000","last_active":"2024-05-14T11:16:17.809208+0000","last_peered":"2024-05-14T11:16:17.809208+0000","last_clean":"2024-05-14T11:16:17.809208+0000","last_became_active":"2024-05-14T11:16:12.264728+0000","last_became_peered":"2024-05-14T11:16:12.264728+0000","last_unstale":"2024-05-14T11:16:17.809208+0000","last_undegraded":"2024-05-14T11:16:17.809208+0000","last_fullsized":"2024-05-14T11:16:17.809208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807713+0000","last_change":"2024-05-14T11:16:11.253941+0000","last_active":"2024-05-14T11:16:17.807713+0000","last_peered":"2024-05-14T11:16:17.807713+0000","last_clean":"2024-05-14T11:16:17.807713+0000","last_became_active":"2024-05-14T11:16:11.253534+0000","last_became_peered":"2024-05-14T11:16:11.253534+0000","last_unstale":"2024-05-14T11:16:17.807713+0000","last_undegraded":"2024-05-14T11:16:17.807713+0000","last_fullsized":"2024-05-14T11:16:17.807713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808083+0000","last_change":"2024-05-14T11:16:16.338462+0000","last_active":"2024-05-14T11:16:17.808083+0000","last_peered":"2024-05-14T11:16:17.808083+0000","last_clean":"2024-05-14T11:16:17.808083+0000","last_became_active":"2024-05-14T11:16:16.337908+0000","last_became_peered":"2024-05-14T11:16:16.337908+0000","last_unstale":"2024-05-14T11:16:17.808083+0000","last_undegraded":"2024-05-14T11:16:17.808083+0000","last_fullsized":"2024-05-14T11:16:17.808083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:28:29.309169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807237+0000","last_change":"2024-05-14T11:16:12.249280+0000","last_active":"2024-05-14T11:16:17.807237+0000","last_peered":"2024-05-14T11:16:17.807237+0000","last_clean":"2024-05-14T11:16:17.807237+0000","last_became_active":"2024-05-14T11:16:12.248977+0000","last_became_peered":"2024-05-14T11:16:12.248977+0000","last_unstale":"2024-05-14T11:16:17.807237+0000","last_undegraded":"2024-05-14T11:16:17.807237+0000","last_fullsized":"2024-05-14T11:16:17.807237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807419+0000","last_change":"2024-05-14T11:16:11.255839+0000","last_active":"2024-05-14T11:16:17.807419+0000","last_peered":"2024-05-14T11:16:17.807419+0000","last_clean":"2024-05-14T11:16:17.807419+0000","last_became_active":"2024-05-14T11:16:11.255487+0000","last_became_peered":"2024-05-14T11:16:11.255487+0000","last_unstale":"2024-05-14T11:16:17.807419+0000","last_undegraded":"2024-05-14T11:16:17.807419+0000","last_fullsized":"2024-05-14T11:16:17.807419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:59.335122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806362+0000","last_change":"2024-05-14T11:16:16.330365+0000","last_active":"2024-05-14T11:16:17.806362+0000","last_peered":"2024-05-14T11:16:17.806362+0000","last_clean":"2024-05-14T11:16:17.806362+0000","last_became_active":"2024-05-14T11:16:16.329833+0000","last_became_peered":"2024-05-14T11:16:16.329833+0000","last_unstale":"2024-05-14T11:16:17.806362+0000","last_undegraded":"2024-05-14T11:16:17.806362+0000","last_fullsized":"2024-05-14T11:16:17.806362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807044+0000","last_change":"2024-05-14T11:16:12.267992+0000","last_active":"2024-05-14T11:16:17.807044+0000","last_peered":"2024-05-14T11:16:17.807044+0000","last_clean":"2024-05-14T11:16:17.807044+0000","last_became_active":"2024-05-14T11:16:12.267183+0000","last_became_peered":"2024-05-14T11:16:12.267183+0000","last_unstale":"2024-05-14T11:16:17.807044+0000","last_undegraded":"2024-05-14T11:16:17.807044+0000","last_fullsized":"2024-05-14T11:16:17.807044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807300+0000","last_change":"2024-05-14T11:16:11.247765+0000","last_active":"2024-05-14T11:16:17.807300+0000","last_peered":"2024-05-14T11:16:17.807300+0000","last_clean":"2024-05-14T11:16:17.807300+0000","last_became_active":"2024-05-14T11:16:11.247503+0000","last_became_peered":"2024-05-14T11:16:11.247503+0000","last_unstale":"2024-05-14T11:16:17.807300+0000","last_undegraded":"2024-05-14T11:16:17.807300+0000","last_fullsized":"2024-05-14T11:16:17.807300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808159+0000","last_change":"2024-05-14T11:16:16.304314+0000","last_active":"2024-05-14T11:16:17.808159+0000","last_peered":"2024-05-14T11:16:17.808159+0000","last_clean":"2024-05-14T11:16:17.808159+0000","last_became_active":"2024-05-14T11:16:16.303370+0000","last_became_peered":"2024-05-14T11:16:16.303370+0000","last_unstale":"2024-05-14T11:16:17.808159+0000","last_undegraded":"2024-05-14T11:16:17.808159+0000","last_fullsized":"2024-05-14T11:16:17.808159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301326+0000","last_change":"2024-05-14T11:16:12.258856+0000","last_active":"2024-05-14T11:16:17.301326+0000","last_peered":"2024-05-14T11:16:17.301326+0000","last_clean":"2024-05-14T11:16:17.301326+0000","last_became_active":"2024-05-14T11:16:12.257816+0000","last_became_peered":"2024-05-14T11:16:12.257816+0000","last_unstale":"2024-05-14T11:16:17.301326+0000","last_undegraded":"2024-05-14T11:16:17.301326+0000","last_fullsized":"2024-05-14T11:16:17.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:19.338856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807816+0000","last_change":"2024-05-14T11:16:11.257469+0000","last_active":"2024-05-14T11:16:17.807816+0000","last_peered":"2024-05-14T11:16:17.807816+0000","last_clean":"2024-05-14T11:16:17.807816+0000","last_became_active":"2024-05-14T11:16:11.253425+0000","last_became_peered":"2024-05-14T11:16:11.253425+0000","last_unstale":"2024-05-14T11:16:17.807816+0000","last_undegraded":"2024-05-14T11:16:17.807816+0000","last_fullsized":"2024-05-14T11:16:17.807816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807184+0000","last_change":"2024-05-14T11:16:16.305620+0000","last_active":"2024-05-14T11:16:17.807184+0000","last_peered":"2024-05-14T11:16:17.807184+0000","last_clean":"2024-05-14T11:16:17.807184+0000","last_became_active":"2024-05-14T11:16:16.305156+0000","last_became_peered":"2024-05-14T11:16:16.305156+0000","last_unstale":"2024-05-14T11:16:17.807184+0000","last_undegraded":"2024-05-14T11:16:17.807184+0000","last_fullsized":"2024-05-14T11:16:17.807184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806230+0000","last_change":"2024-05-14T11:16:12.245405+0000","last_active":"2024-05-14T11:16:17.806230+0000","last_peered":"2024-05-14T11:16:17.806230+0000","last_clean":"2024-05-14T11:16:17.806230+0000","last_became_active":"2024-05-14T11:16:12.245076+0000","last_became_peered":"2024-05-14T11:16:12.245076+0000","last_unstale":"2024-05-14T11:16:17.806230+0000","last_undegraded":"2024-05-14T11:16:17.806230+0000","last_fullsized":"2024-05-14T11:16:17.806230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297955+0000","last_change":"2024-05-14T11:16:11.245815+0000","last_active":"2024-05-14T11:16:17.297955+0000","last_peered":"2024-05-14T11:16:17.297955+0000","last_clean":"2024-05-14T11:16:17.297955+0000","last_became_active":"2024-05-14T11:16:11.245487+0000","last_became_peered":"2024-05-14T11:16:11.245487+0000","last_unstale":"2024-05-14T11:16:17.297955+0000","last_undegraded":"2024-05-14T11:16:17.297955+0000","last_fullsized":"2024-05-14T11:16:17.297955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809501+0000","last_change":"2024-05-14T11:16:16.304799+0000","last_active":"2024-05-14T11:16:17.809501+0000","last_peered":"2024-05-14T11:16:17.809501+0000","last_clean":"2024-05-14T11:16:17.809501+0000","last_became_active":"2024-05-14T11:16:16.304223+0000","last_became_peered":"2024-05-14T11:16:16.304223+0000","last_unstale":"2024-05-14T11:16:17.809501+0000","last_undegraded":"2024-05-14T11:16:17.809501+0000","last_fullsized":"2024-05-14T11:16:17.809501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.805980+0000","last_change":"2024-05-14T11:16:12.242987+0000","last_active":"2024-05-14T11:16:17.805980+0000","last_peered":"2024-05-14T11:16:17.805980+0000","last_clean":"2024-05-14T11:16:17.805980+0000","last_became_active":"2024-05-14T11:16:12.242648+0000","last_became_peered":"2024-05-14T11:16:12.242648+0000","last_unstale":"2024-05-14T11:16:17.805980+0000","last_undegraded":"2024-05-14T11:16:17.805980+0000","last_fullsized":"2024-05-14T11:16:17.805980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:37.421472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806812+0000","last_change":"2024-05-14T11:16:11.254596+0000","last_active":"2024-05-14T11:16:17.806812+0000","last_peered":"2024-05-14T11:16:17.806812+0000","last_clean":"2024-05-14T11:16:17.806812+0000","last_became_active":"2024-05-14T11:16:11.254304+0000","last_became_peered":"2024-05-14T11:16:11.254304+0000","last_unstale":"2024-05-14T11:16:17.806812+0000","last_undegraded":"2024-05-14T11:16:17.806812+0000","last_fullsized":"2024-05-14T11:16:17.806812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806903+0000","last_change":"2024-05-14T11:16:16.333086+0000","last_active":"2024-05-14T11:16:17.806903+0000","last_peered":"2024-05-14T11:16:17.806903+0000","last_clean":"2024-05-14T11:16:17.806903+0000","last_became_active":"2024-05-14T11:16:16.332633+0000","last_became_peered":"2024-05-14T11:16:16.332633+0000","last_unstale":"2024-05-14T11:16:17.806903+0000","last_undegraded":"2024-05-14T11:16:17.806903+0000","last_fullsized":"2024-05-14T11:16:17.806903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808690+0000","last_change":"2024-05-14T11:16:12.267597+0000","last_active":"2024-05-14T11:16:17.808690+0000","last_peered":"2024-05-14T11:16:17.808690+0000","last_clean":"2024-05-14T11:16:17.808690+0000","last_became_active":"2024-05-14T11:16:12.267050+0000","last_became_peered":"2024-05-14T11:16:12.267050+0000","last_unstale":"2024-05-14T11:16:17.808690+0000","last_undegraded":"2024-05-14T11:16:17.808690+0000","last_fullsized":"2024-05-14T11:16:17.808690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806566+0000","last_change":"2024-05-14T11:16:11.253533+0000","last_active":"2024-05-14T11:16:17.806566+0000","last_peered":"2024-05-14T11:16:17.806566+0000","last_clean":"2024-05-14T11:16:17.806566+0000","last_became_active":"2024-05-14T11:16:11.253104+0000","last_became_peered":"2024-05-14T11:16:11.253104+0000","last_unstale":"2024-05-14T11:16:17.806566+0000","last_undegraded":"2024-05-14T11:16:17.806566+0000","last_fullsized":"2024-05-14T11:16:17.806566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806962+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:16:17.806962+0000","last_peered":"2024-05-14T11:16:17.806962+0000","last_clean":"2024-05-14T11:16:17.806962+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:16:17.806962+0000","last_undegraded":"2024-05-14T11:16:17.806962+0000","last_fullsized":"2024-05-14T11:16:17.806962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299583+0000","last_change":"2024-05-14T11:16:16.330966+0000","last_active":"2024-05-14T11:16:17.299583+0000","last_peered":"2024-05-14T11:16:17.299583+0000","last_clean":"2024-05-14T11:16:17.299583+0000","last_became_active":"2024-05-14T11:16:16.330413+0000","last_became_peered":"2024-05-14T11:16:16.330413+0000","last_unstale":"2024-05-14T11:16:17.299583+0000","last_undegraded":"2024-05-14T11:16:17.299583+0000","last_fullsized":"2024-05-14T11:16:17.299583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807128+0000","last_change":"2024-05-14T11:16:12.263387+0000","last_active":"2024-05-14T11:16:17.807128+0000","last_peered":"2024-05-14T11:16:17.807128+0000","last_clean":"2024-05-14T11:16:17.807128+0000","last_became_active":"2024-05-14T11:16:12.262944+0000","last_became_peered":"2024-05-14T11:16:12.262944+0000","last_unstale":"2024-05-14T11:16:17.807128+0000","last_undegraded":"2024-05-14T11:16:17.807128+0000","last_fullsized":"2024-05-14T11:16:17.807128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298224+0000","last_change":"2024-05-14T11:16:11.259096+0000","last_active":"2024-05-14T11:16:17.298224+0000","last_peered":"2024-05-14T11:16:17.298224+0000","last_clean":"2024-05-14T11:16:17.298224+0000","last_became_active":"2024-05-14T11:16:11.258840+0000","last_became_peered":"2024-05-14T11:16:11.258840+0000","last_unstale":"2024-05-14T11:16:17.298224+0000","last_undegraded":"2024-05-14T11:16:17.298224+0000","last_fullsized":"2024-05-14T11:16:17.298224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300200+0000","last_change":"2024-05-14T11:16:16.333088+0000","last_active":"2024-05-14T11:16:17.300200+0000","last_peered":"2024-05-14T11:16:17.300200+0000","last_clean":"2024-05-14T11:16:17.300200+0000","last_became_active":"2024-05-14T11:16:16.332585+0000","last_became_peered":"2024-05-14T11:16:16.332585+0000","last_unstale":"2024-05-14T11:16:17.300200+0000","last_undegraded":"2024-05-14T11:16:17.300200+0000","last_fullsized":"2024-05-14T11:16:17.300200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806820+0000","last_change":"2024-05-14T11:16:12.270318+0000","last_active":"2024-05-14T11:16:17.806820+0000","last_peered":"2024-05-14T11:16:17.806820+0000","last_clean":"2024-05-14T11:16:17.806820+0000","last_became_active":"2024-05-14T11:16:12.269365+0000","last_became_peered":"2024-05-14T11:16:12.269365+0000","last_unstale":"2024-05-14T11:16:17.806820+0000","last_undegraded":"2024-05-14T11:16:17.806820+0000","last_fullsized":"2024-05-14T11:16:17.806820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807501+0000","last_change":"2024-05-14T11:16:11.252737+0000","last_active":"2024-05-14T11:16:17.807501+0000","last_peered":"2024-05-14T11:16:17.807501+0000","last_clean":"2024-05-14T11:16:17.807501+0000","last_became_active":"2024-05-14T11:16:11.252365+0000","last_became_peered":"2024-05-14T11:16:11.252365+0000","last_unstale":"2024-05-14T11:16:17.807501+0000","last_undegraded":"2024-05-14T11:16:17.807501+0000","last_fullsized":"2024-05-14T11:16:17.807501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806471+0000","last_change":"2024-05-14T11:16:16.302121+0000","last_active":"2024-05-14T11:16:17.806471+0000","last_peered":"2024-05-14T11:16:17.806471+0000","last_clean":"2024-05-14T11:16:17.806471+0000","last_became_active":"2024-05-14T11:16:16.299059+0000","last_became_peered":"2024-05-14T11:16:16.299059+0000","last_unstale":"2024-05-14T11:16:17.806471+0000","last_undegraded":"2024-05-14T11:16:17.806471+0000","last_fullsized":"2024-05-14T11:16:17.806471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807801+0000","last_change":"2024-05-14T11:16:12.247488+0000","last_active":"2024-05-14T11:16:17.807801+0000","last_peered":"2024-05-14T11:16:17.807801+0000","last_clean":"2024-05-14T11:16:17.807801+0000","last_became_active":"2024-05-14T11:16:12.247235+0000","last_became_peered":"2024-05-14T11:16:12.247235+0000","last_unstale":"2024-05-14T11:16:17.807801+0000","last_undegraded":"2024-05-14T11:16:17.807801+0000","last_fullsized":"2024-05-14T11:16:17.807801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:22.549789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298399+0000","last_change":"2024-05-14T11:16:11.246157+0000","last_active":"2024-05-14T11:16:17.298399+0000","last_peered":"2024-05-14T11:16:17.298399+0000","last_clean":"2024-05-14T11:16:17.298399+0000","last_became_active":"2024-05-14T11:16:11.245408+0000","last_became_peered":"2024-05-14T11:16:11.245408+0000","last_unstale":"2024-05-14T11:16:17.298399+0000","last_undegraded":"2024-05-14T11:16:17.298399+0000","last_fullsized":"2024-05-14T11:16:17.298399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806090+0000","last_change":"2024-05-14T11:16:16.330283+0000","last_active":"2024-05-14T11:16:17.806090+0000","last_peered":"2024-05-14T11:16:17.806090+0000","last_clean":"2024-05-14T11:16:17.806090+0000","last_became_active":"2024-05-14T11:16:16.329844+0000","last_became_peered":"2024-05-14T11:16:16.329844+0000","last_unstale":"2024-05-14T11:16:17.806090+0000","last_undegraded":"2024-05-14T11:16:17.806090+0000","last_fullsized":"2024-05-14T11:16:17.806090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807746+0000","last_change":"2024-05-14T11:16:12.248542+0000","last_active":"2024-05-14T11:16:17.807746+0000","last_peered":"2024-05-14T11:16:17.807746+0000","last_clean":"2024-05-14T11:16:17.807746+0000","last_became_active":"2024-05-14T11:16:12.248214+0000","last_became_peered":"2024-05-14T11:16:12.248214+0000","last_unstale":"2024-05-14T11:16:17.807746+0000","last_undegraded":"2024-05-14T11:16:17.807746+0000","last_fullsized":"2024-05-14T11:16:17.807746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302636+0000","last_change":"2024-05-14T11:16:11.241324+0000","last_active":"2024-05-14T11:16:17.302636+0000","last_peered":"2024-05-14T11:16:17.302636+0000","last_clean":"2024-05-14T11:16:17.302636+0000","last_became_active":"2024-05-14T11:16:11.239970+0000","last_became_peered":"2024-05-14T11:16:11.239970+0000","last_unstale":"2024-05-14T11:16:17.302636+0000","last_undegraded":"2024-05-14T11:16:17.302636+0000","last_fullsized":"2024-05-14T11:16:17.302636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:44.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299463+0000","last_change":"2024-05-14T11:16:16.293327+0000","last_active":"2024-05-14T11:16:17.299463+0000","last_peered":"2024-05-14T11:16:17.299463+0000","last_clean":"2024-05-14T11:16:17.299463+0000","last_became_active":"2024-05-14T11:16:16.292800+0000","last_became_peered":"2024-05-14T11:16:16.292800+0000","last_unstale":"2024-05-14T11:16:17.299463+0000","last_undegraded":"2024-05-14T11:16:17.299463+0000","last_fullsized":"2024-05-14T11:16:17.299463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:39.377103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808057+0000","last_change":"2024-05-14T11:16:12.248759+0000","last_active":"2024-05-14T11:16:17.808057+0000","last_peered":"2024-05-14T11:16:17.808057+0000","last_clean":"2024-05-14T11:16:17.808057+0000","last_became_active":"2024-05-14T11:16:12.248407+0000","last_became_peered":"2024-05-14T11:16:12.248407+0000","last_unstale":"2024-05-14T11:16:17.808057+0000","last_undegraded":"2024-05-14T11:16:17.808057+0000","last_fullsized":"2024-05-14T11:16:17.808057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297834+0000","last_change":"2024-05-14T11:16:11.248167+0000","last_active":"2024-05-14T11:16:17.297834+0000","last_peered":"2024-05-14T11:16:17.297834+0000","last_clean":"2024-05-14T11:16:17.297834+0000","last_became_active":"2024-05-14T11:16:11.245949+0000","last_became_peered":"2024-05-14T11:16:11.245949+0000","last_unstale":"2024-05-14T11:16:17.297834+0000","last_undegraded":"2024-05-14T11:16:17.297834+0000","last_fullsized":"2024-05-14T11:16:17.297834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807832+0000","last_change":"2024-05-14T11:16:16.290089+0000","last_active":"2024-05-14T11:16:17.807832+0000","last_peered":"2024-05-14T11:16:17.807832+0000","last_clean":"2024-05-14T11:16:17.807832+0000","last_became_active":"2024-05-14T11:16:16.287987+0000","last_became_peered":"2024-05-14T11:16:16.287987+0000","last_unstale":"2024-05-14T11:16:17.807832+0000","last_undegraded":"2024-05-14T11:16:17.807832+0000","last_fullsized":"2024-05-14T11:16:17.807832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:26.599395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299766+0000","last_change":"2024-05-14T11:16:12.248930+0000","last_active":"2024-05-14T11:16:17.299766+0000","last_peered":"2024-05-14T11:16:17.299766+0000","last_clean":"2024-05-14T11:16:17.299766+0000","last_became_active":"2024-05-14T11:16:12.248159+0000","last_became_peered":"2024-05-14T11:16:12.248159+0000","last_unstale":"2024-05-14T11:16:17.299766+0000","last_undegraded":"2024-05-14T11:16:17.299766+0000","last_fullsized":"2024-05-14T11:16:17.299766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299880+0000","last_change":"2024-05-14T11:16:11.247396+0000","last_active":"2024-05-14T11:16:17.299880+0000","last_peered":"2024-05-14T11:16:17.299880+0000","last_clean":"2024-05-14T11:16:17.299880+0000","last_became_active":"2024-05-14T11:16:11.246855+0000","last_became_peered":"2024-05-14T11:16:11.246855+0000","last_unstale":"2024-05-14T11:16:17.299880+0000","last_undegraded":"2024-05-14T11:16:17.299880+0000","last_fullsized":"2024-05-14T11:16:17.299880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808975+0000","last_change":"2024-05-14T11:16:16.332903+0000","last_active":"2024-05-14T11:16:17.808975+0000","last_peered":"2024-05-14T11:16:17.808975+0000","last_clean":"2024-05-14T11:16:17.808975+0000","last_became_active":"2024-05-14T11:16:16.332436+0000","last_became_peered":"2024-05-14T11:16:16.332436+0000","last_unstale":"2024-05-14T11:16:17.808975+0000","last_undegraded":"2024-05-14T11:16:17.808975+0000","last_fullsized":"2024-05-14T11:16:17.808975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:01.254012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807525+0000","last_change":"2024-05-14T11:16:12.248442+0000","last_active":"2024-05-14T11:16:17.807525+0000","last_peered":"2024-05-14T11:16:17.807525+0000","last_clean":"2024-05-14T11:16:17.807525+0000","last_became_active":"2024-05-14T11:16:12.248082+0000","last_became_peered":"2024-05-14T11:16:12.248082+0000","last_unstale":"2024-05-14T11:16:17.807525+0000","last_undegraded":"2024-05-14T11:16:17.807525+0000","last_fullsized":"2024-05-14T11:16:17.807525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807708+0000","last_change":"2024-05-14T11:16:11.257458+0000","last_active":"2024-05-14T11:16:17.807708+0000","last_peered":"2024-05-14T11:16:17.807708+0000","last_clean":"2024-05-14T11:16:17.807708+0000","last_became_active":"2024-05-14T11:16:11.252978+0000","last_became_peered":"2024-05-14T11:16:11.252978+0000","last_unstale":"2024-05-14T11:16:17.807708+0000","last_undegraded":"2024-05-14T11:16:17.807708+0000","last_fullsized":"2024-05-14T11:16:17.807708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808827+0000","last_change":"2024-05-14T11:16:16.330174+0000","last_active":"2024-05-14T11:16:17.808827+0000","last_peered":"2024-05-14T11:16:17.808827+0000","last_clean":"2024-05-14T11:16:17.808827+0000","last_became_active":"2024-05-14T11:16:16.329696+0000","last_became_peered":"2024-05-14T11:16:16.329696+0000","last_unstale":"2024-05-14T11:16:17.808827+0000","last_undegraded":"2024-05-14T11:16:17.808827+0000","last_fullsized":"2024-05-14T11:16:17.808827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:14.122764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806365+0000","last_change":"2024-05-14T11:16:12.246355+0000","last_active":"2024-05-14T11:16:17.806365+0000","last_peered":"2024-05-14T11:16:17.806365+0000","last_clean":"2024-05-14T11:16:17.806365+0000","last_became_active":"2024-05-14T11:16:12.245693+0000","last_became_peered":"2024-05-14T11:16:12.245693+0000","last_unstale":"2024-05-14T11:16:17.806365+0000","last_undegraded":"2024-05-14T11:16:17.806365+0000","last_fullsized":"2024-05-14T11:16:17.806365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806645+0000","last_change":"2024-05-14T11:16:11.249709+0000","last_active":"2024-05-14T11:16:17.806645+0000","last_peered":"2024-05-14T11:16:17.806645+0000","last_clean":"2024-05-14T11:16:17.806645+0000","last_became_active":"2024-05-14T11:16:11.245550+0000","last_became_peered":"2024-05-14T11:16:11.245550+0000","last_unstale":"2024-05-14T11:16:17.806645+0000","last_undegraded":"2024-05-14T11:16:17.806645+0000","last_fullsized":"2024-05-14T11:16:17.806645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:18.498642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806772+0000","last_change":"2024-05-14T11:16:16.330492+0000","last_active":"2024-05-14T11:16:17.806772+0000","last_peered":"2024-05-14T11:16:17.806772+0000","last_clean":"2024-05-14T11:16:17.806772+0000","last_became_active":"2024-05-14T11:16:16.330027+0000","last_became_peered":"2024-05-14T11:16:16.330027+0000","last_unstale":"2024-05-14T11:16:17.806772+0000","last_undegraded":"2024-05-14T11:16:17.806772+0000","last_fullsized":"2024-05-14T11:16:17.806772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807029+0000","last_change":"2024-05-14T11:16:12.263610+0000","last_active":"2024-05-14T11:16:17.807029+0000","last_peered":"2024-05-14T11:16:17.807029+0000","last_clean":"2024-05-14T11:16:17.807029+0000","last_became_active":"2024-05-14T11:16:12.263072+0000","last_became_peered":"2024-05-14T11:16:12.263072+0000","last_unstale":"2024-05-14T11:16:17.807029+0000","last_undegraded":"2024-05-14T11:16:17.807029+0000","last_fullsized":"2024-05-14T11:16:17.807029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:23.206421+0000","last_change":"2024-05-14T11:16:11.250108+0000","last_active":"2024-05-14T11:16:23.206421+0000","last_peered":"2024-05-14T11:16:23.206421+0000","last_clean":"2024-05-14T11:16:23.206421+0000","last_became_active":"2024-05-14T11:16:11.249460+0000","last_became_peered":"2024-05-14T11:16:11.249460+0000","last_unstale":"2024-05-14T11:16:23.206421+0000","last_undegraded":"2024-05-14T11:16:23.206421+0000","last_fullsized":"2024-05-14T11:16:23.206421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809402+0000","last_change":"2024-05-14T11:16:16.305560+0000","last_active":"2024-05-14T11:16:17.809402+0000","last_peered":"2024-05-14T11:16:17.809402+0000","last_clean":"2024-05-14T11:16:17.809402+0000","last_became_active":"2024-05-14T11:16:16.305166+0000","last_became_peered":"2024-05-14T11:16:16.305166+0000","last_unstale":"2024-05-14T11:16:17.809402+0000","last_undegraded":"2024-05-14T11:16:17.809402+0000","last_fullsized":"2024-05-14T11:16:17.809402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302423+0000","last_change":"2024-05-14T11:16:12.248795+0000","last_active":"2024-05-14T11:16:17.302423+0000","last_peered":"2024-05-14T11:16:17.302423+0000","last_clean":"2024-05-14T11:16:17.302423+0000","last_became_active":"2024-05-14T11:16:12.248478+0000","last_became_peered":"2024-05-14T11:16:12.248478+0000","last_unstale":"2024-05-14T11:16:17.302423+0000","last_undegraded":"2024-05-14T11:16:17.302423+0000","last_fullsized":"2024-05-14T11:16:17.302423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807383+0000","last_change":"2024-05-14T11:16:11.259967+0000","last_active":"2024-05-14T11:16:17.807383+0000","last_peered":"2024-05-14T11:16:17.807383+0000","last_clean":"2024-05-14T11:16:17.807383+0000","last_became_active":"2024-05-14T11:16:11.259703+0000","last_became_peered":"2024-05-14T11:16:11.259703+0000","last_unstale":"2024-05-14T11:16:17.807383+0000","last_undegraded":"2024-05-14T11:16:17.807383+0000","last_fullsized":"2024-05-14T11:16:17.807383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301098+0000","last_change":"2024-05-14T11:16:16.293106+0000","last_active":"2024-05-14T11:16:17.301098+0000","last_peered":"2024-05-14T11:16:17.301098+0000","last_clean":"2024-05-14T11:16:17.301098+0000","last_became_active":"2024-05-14T11:16:16.292666+0000","last_became_peered":"2024-05-14T11:16:16.292666+0000","last_unstale":"2024-05-14T11:16:17.301098+0000","last_undegraded":"2024-05-14T11:16:17.301098+0000","last_fullsized":"2024-05-14T11:16:17.301098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807505+0000","last_change":"2024-05-14T11:16:12.260898+0000","last_active":"2024-05-14T11:16:17.807505+0000","last_peered":"2024-05-14T11:16:17.807505+0000","last_clean":"2024-05-14T11:16:17.807505+0000","last_became_active":"2024-05-14T11:16:12.260114+0000","last_became_peered":"2024-05-14T11:16:12.260114+0000","last_unstale":"2024-05-14T11:16:17.807505+0000","last_undegraded":"2024-05-14T11:16:17.807505+0000","last_fullsized":"2024-05-14T11:16:17.807505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806807+0000","last_change":"2024-05-14T11:16:11.254169+0000","last_active":"2024-05-14T11:16:17.806807+0000","last_peered":"2024-05-14T11:16:17.806807+0000","last_clean":"2024-05-14T11:16:17.806807+0000","last_became_active":"2024-05-14T11:16:11.253658+0000","last_became_peered":"2024-05-14T11:16:11.253658+0000","last_unstale":"2024-05-14T11:16:17.806807+0000","last_undegraded":"2024-05-14T11:16:17.806807+0000","last_fullsized":"2024-05-14T11:16:17.806807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808298+0000","last_change":"2024-05-14T11:16:16.298722+0000","last_active":"2024-05-14T11:16:17.808298+0000","last_peered":"2024-05-14T11:16:17.808298+0000","last_clean":"2024-05-14T11:16:17.808298+0000","last_became_active":"2024-05-14T11:16:16.298312+0000","last_became_peered":"2024-05-14T11:16:16.298312+0000","last_unstale":"2024-05-14T11:16:17.808298+0000","last_undegraded":"2024-05-14T11:16:17.808298+0000","last_fullsized":"2024-05-14T11:16:17.808298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806669+0000","last_change":"2024-05-14T11:16:12.269044+0000","last_active":"2024-05-14T11:16:17.806669+0000","last_peered":"2024-05-14T11:16:17.806669+0000","last_clean":"2024-05-14T11:16:17.806669+0000","last_became_active":"2024-05-14T11:16:12.268428+0000","last_became_peered":"2024-05-14T11:16:12.268428+0000","last_unstale":"2024-05-14T11:16:17.806669+0000","last_undegraded":"2024-05-14T11:16:17.806669+0000","last_fullsized":"2024-05-14T11:16:17.806669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301709+0000","last_change":"2024-05-14T11:16:11.247431+0000","last_active":"2024-05-14T11:16:17.301709+0000","last_peered":"2024-05-14T11:16:17.301709+0000","last_clean":"2024-05-14T11:16:17.301709+0000","last_became_active":"2024-05-14T11:16:11.247003+0000","last_became_peered":"2024-05-14T11:16:11.247003+0000","last_unstale":"2024-05-14T11:16:17.301709+0000","last_undegraded":"2024-05-14T11:16:17.301709+0000","last_fullsized":"2024-05-14T11:16:17.301709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808398+0000","last_change":"2024-05-14T11:16:16.338217+0000","last_active":"2024-05-14T11:16:17.808398+0000","last_peered":"2024-05-14T11:16:17.808398+0000","last_clean":"2024-05-14T11:16:17.808398+0000","last_became_active":"2024-05-14T11:16:16.337645+0000","last_became_peered":"2024-05-14T11:16:16.337645+0000","last_unstale":"2024-05-14T11:16:17.808398+0000","last_undegraded":"2024-05-14T11:16:17.808398+0000","last_fullsized":"2024-05-14T11:16:17.808398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:50:21.563593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806740+0000","last_change":"2024-05-14T11:16:16.337358+0000","last_active":"2024-05-14T11:16:17.806740+0000","last_peered":"2024-05-14T11:16:17.806740+0000","last_clean":"2024-05-14T11:16:17.806740+0000","last_became_active":"2024-05-14T11:16:16.336916+0000","last_became_peered":"2024-05-14T11:16:16.336916+0000","last_unstale":"2024-05-14T11:16:17.806740+0000","last_undegraded":"2024-05-14T11:16:17.806740+0000","last_fullsized":"2024-05-14T11:16:17.806740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806581+0000","last_change":"2024-05-14T11:16:12.267101+0000","last_active":"2024-05-14T11:16:17.806581+0000","last_peered":"2024-05-14T11:16:17.806581+0000","last_clean":"2024-05-14T11:16:17.806581+0000","last_became_active":"2024-05-14T11:16:12.266805+0000","last_became_peered":"2024-05-14T11:16:12.266805+0000","last_unstale":"2024-05-14T11:16:17.806581+0000","last_undegraded":"2024-05-14T11:16:17.806581+0000","last_fullsized":"2024-05-14T11:16:17.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806424+0000","last_change":"2024-05-14T11:16:11.259588+0000","last_active":"2024-05-14T11:16:17.806424+0000","last_peered":"2024-05-14T11:16:17.806424+0000","last_clean":"2024-05-14T11:16:17.806424+0000","last_became_active":"2024-05-14T11:16:11.259246+0000","last_became_peered":"2024-05-14T11:16:11.259246+0000","last_unstale":"2024-05-14T11:16:17.806424+0000","last_undegraded":"2024-05-14T11:16:17.806424+0000","last_fullsized":"2024-05-14T11:16:17.806424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301555+0000","last_change":"2024-05-14T11:16:16.299514+0000","last_active":"2024-05-14T11:16:17.301555+0000","last_peered":"2024-05-14T11:16:17.301555+0000","last_clean":"2024-05-14T11:16:17.301555+0000","last_became_active":"2024-05-14T11:16:16.299075+0000","last_became_peered":"2024-05-14T11:16:16.299075+0000","last_unstale":"2024-05-14T11:16:17.301555+0000","last_undegraded":"2024-05-14T11:16:17.301555+0000","last_fullsized":"2024-05-14T11:16:17.301555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808226+0000","last_change":"2024-05-14T11:16:12.258195+0000","last_active":"2024-05-14T11:16:17.808226+0000","last_peered":"2024-05-14T11:16:17.808226+0000","last_clean":"2024-05-14T11:16:17.808226+0000","last_became_active":"2024-05-14T11:16:12.257876+0000","last_became_peered":"2024-05-14T11:16:12.257876+0000","last_unstale":"2024-05-14T11:16:17.808226+0000","last_undegraded":"2024-05-14T11:16:17.808226+0000","last_fullsized":"2024-05-14T11:16:17.808226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808524+0000","last_change":"2024-05-14T11:16:11.246503+0000","last_active":"2024-05-14T11:16:17.808524+0000","last_peered":"2024-05-14T11:16:17.808524+0000","last_clean":"2024-05-14T11:16:17.808524+0000","last_became_active":"2024-05-14T11:16:11.246056+0000","last_became_peered":"2024-05-14T11:16:11.246056+0000","last_unstale":"2024-05-14T11:16:17.808524+0000","last_undegraded":"2024-05-14T11:16:17.808524+0000","last_fullsized":"2024-05-14T11:16:17.808524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300138+0000","last_change":"2024-05-14T11:16:11.258034+0000","last_active":"2024-05-14T11:16:17.300138+0000","last_peered":"2024-05-14T11:16:17.300138+0000","last_clean":"2024-05-14T11:16:17.300138+0000","last_became_active":"2024-05-14T11:16:11.257706+0000","last_became_peered":"2024-05-14T11:16:11.257706+0000","last_unstale":"2024-05-14T11:16:17.300138+0000","last_undegraded":"2024-05-14T11:16:17.300138+0000","last_fullsized":"2024-05-14T11:16:17.300138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806611+0000","last_change":"2024-05-14T11:16:12.248022+0000","last_active":"2024-05-14T11:16:17.806611+0000","last_peered":"2024-05-14T11:16:17.806611+0000","last_clean":"2024-05-14T11:16:17.806611+0000","last_became_active":"2024-05-14T11:16:12.247640+0000","last_became_peered":"2024-05-14T11:16:12.247640+0000","last_unstale":"2024-05-14T11:16:17.806611+0000","last_undegraded":"2024-05-14T11:16:17.806611+0000","last_fullsized":"2024-05-14T11:16:17.806611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298100+0000","last_change":"2024-05-14T11:16:16.296944+0000","last_active":"2024-05-14T11:16:17.298100+0000","last_peered":"2024-05-14T11:16:17.298100+0000","last_clean":"2024-05-14T11:16:17.298100+0000","last_became_active":"2024-05-14T11:16:16.296515+0000","last_became_peered":"2024-05-14T11:16:16.296515+0000","last_unstale":"2024-05-14T11:16:17.298100+0000","last_undegraded":"2024-05-14T11:16:17.298100+0000","last_fullsized":"2024-05-14T11:16:17.298100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:41.471078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809032+0000","last_change":"2024-05-14T11:16:11.253359+0000","last_active":"2024-05-14T11:16:17.809032+0000","last_peered":"2024-05-14T11:16:17.809032+0000","last_clean":"2024-05-14T11:16:17.809032+0000","last_became_active":"2024-05-14T11:16:11.248847+0000","last_became_peered":"2024-05-14T11:16:11.248847+0000","last_unstale":"2024-05-14T11:16:17.809032+0000","last_undegraded":"2024-05-14T11:16:17.809032+0000","last_fullsized":"2024-05-14T11:16:17.809032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301381+0000","last_change":"2024-05-14T11:16:12.239027+0000","last_active":"2024-05-14T11:16:17.301381+0000","last_peered":"2024-05-14T11:16:17.301381+0000","last_clean":"2024-05-14T11:16:17.301381+0000","last_became_active":"2024-05-14T11:16:12.238687+0000","last_became_peered":"2024-05-14T11:16:12.238687+0000","last_unstale":"2024-05-14T11:16:17.301381+0000","last_undegraded":"2024-05-14T11:16:17.301381+0000","last_fullsized":"2024-05-14T11:16:17.301381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.490412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809262+0000","last_change":"2024-05-14T11:16:16.293565+0000","last_active":"2024-05-14T11:16:17.809262+0000","last_peered":"2024-05-14T11:16:17.809262+0000","last_clean":"2024-05-14T11:16:17.809262+0000","last_became_active":"2024-05-14T11:16:16.293094+0000","last_became_peered":"2024-05-14T11:16:16.293094+0000","last_unstale":"2024-05-14T11:16:17.809262+0000","last_undegraded":"2024-05-14T11:16:17.809262+0000","last_fullsized":"2024-05-14T11:16:17.809262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:03:43.070705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299522+0000","last_change":"2024-05-14T11:16:12.249274+0000","last_active":"2024-05-14T11:16:17.299522+0000","last_peered":"2024-05-14T11:16:17.299522+0000","last_clean":"2024-05-14T11:16:17.299522+0000","last_became_active":"2024-05-14T11:16:12.248560+0000","last_became_peered":"2024-05-14T11:16:12.248560+0000","last_unstale":"2024-05-14T11:16:17.299522+0000","last_undegraded":"2024-05-14T11:16:17.299522+0000","last_fullsized":"2024-05-14T11:16:17.299522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297943+0000","last_change":"2024-05-14T11:16:12.267227+0000","last_active":"2024-05-14T11:16:17.297943+0000","last_peered":"2024-05-14T11:16:17.297943+0000","last_clean":"2024-05-14T11:16:17.297943+0000","last_became_active":"2024-05-14T11:16:12.266918+0000","last_became_peered":"2024-05-14T11:16:12.266918+0000","last_unstale":"2024-05-14T11:16:17.297943+0000","last_undegraded":"2024-05-14T11:16:17.297943+0000","last_fullsized":"2024-05-14T11:16:17.297943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807593+0000","last_change":"2024-05-14T11:16:12.259446+0000","last_active":"2024-05-14T11:16:17.807593+0000","last_peered":"2024-05-14T11:16:17.807593+0000","last_clean":"2024-05-14T11:16:17.807593+0000","last_became_active":"2024-05-14T11:16:12.259111+0000","last_became_peered":"2024-05-14T11:16:12.259111+0000","last_unstale":"2024-05-14T11:16:17.807593+0000","last_undegraded":"2024-05-14T11:16:17.807593+0000","last_fullsized":"2024-05-14T11:16:17.807593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806487+0000","last_change":"2024-05-14T11:16:12.259006+0000","last_active":"2024-05-14T11:16:17.806487+0000","last_peered":"2024-05-14T11:16:17.806487+0000","last_clean":"2024-05-14T11:16:17.806487+0000","last_became_active":"2024-05-14T11:16:12.258686+0000","last_became_peered":"2024-05-14T11:16:12.258686+0000","last_unstale":"2024-05-14T11:16:17.806487+0000","last_undegraded":"2024-05-14T11:16:17.806487+0000","last_fullsized":"2024-05-14T11:16:17.806487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808880+0000","last_change":"2024-05-14T11:16:12.263813+0000","last_active":"2024-05-14T11:16:17.808880+0000","last_peered":"2024-05-14T11:16:17.808880+0000","last_clean":"2024-05-14T11:16:17.808880+0000","last_became_active":"2024-05-14T11:16:12.263020+0000","last_became_peered":"2024-05-14T11:16:12.263020+0000","last_unstale":"2024-05-14T11:16:17.808880+0000","last_undegraded":"2024-05-14T11:16:17.808880+0000","last_fullsized":"2024-05-14T11:16:17.808880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807160+0000","last_change":"2024-05-14T11:16:12.265745+0000","last_active":"2024-05-14T11:16:17.807160+0000","last_peered":"2024-05-14T11:16:17.807160+0000","last_clean":"2024-05-14T11:16:17.807160+0000","last_became_active":"2024-05-14T11:16:12.262165+0000","last_became_peered":"2024-05-14T11:16:12.262165+0000","last_unstale":"2024-05-14T11:16:17.807160+0000","last_undegraded":"2024-05-14T11:16:17.807160+0000","last_fullsized":"2024-05-14T11:16:17.807160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301482+0000","last_change":"2024-05-14T11:16:12.250079+0000","last_active":"2024-05-14T11:16:17.301482+0000","last_peered":"2024-05-14T11:16:17.301482+0000","last_clean":"2024-05-14T11:16:17.301482+0000","last_became_active":"2024-05-14T11:16:12.246835+0000","last_became_peered":"2024-05-14T11:16:12.246835+0000","last_unstale":"2024-05-14T11:16:17.301482+0000","last_undegraded":"2024-05-14T11:16:17.301482+0000","last_fullsized":"2024-05-14T11:16:17.301482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809094+0000","last_change":"2024-05-14T11:16:12.264222+0000","last_active":"2024-05-14T11:16:17.809094+0000","last_peered":"2024-05-14T11:16:17.809094+0000","last_clean":"2024-05-14T11:16:17.809094+0000","last_became_active":"2024-05-14T11:16:12.263773+0000","last_became_peered":"2024-05-14T11:16:12.263773+0000","last_unstale":"2024-05-14T11:16:17.809094+0000","last_undegraded":"2024-05-14T11:16:17.809094+0000","last_fullsized":"2024-05-14T11:16:17.809094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806261+0000","last_change":"2024-05-14T11:16:12.260462+0000","last_active":"2024-05-14T11:16:17.806261+0000","last_peered":"2024-05-14T11:16:17.806261+0000","last_clean":"2024-05-14T11:16:17.806261+0000","last_became_active":"2024-05-14T11:16:12.260089+0000","last_became_peered":"2024-05-14T11:16:12.260089+0000","last_unstale":"2024-05-14T11:16:17.806261+0000","last_undegraded":"2024-05-14T11:16:17.806261+0000","last_fullsized":"2024-05-14T11:16:17.806261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808385+0000","last_change":"2024-05-14T11:16:12.257286+0000","last_active":"2024-05-14T11:16:17.808385+0000","last_peered":"2024-05-14T11:16:17.808385+0000","last_clean":"2024-05-14T11:16:17.808385+0000","last_became_active":"2024-05-14T11:16:12.256949+0000","last_became_peered":"2024-05-14T11:16:12.256949+0000","last_unstale":"2024-05-14T11:16:17.808385+0000","last_undegraded":"2024-05-14T11:16:17.808385+0000","last_fullsized":"2024-05-14T11:16:17.808385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807462+0000","last_change":"2024-05-14T11:16:12.266841+0000","last_active":"2024-05-14T11:16:17.807462+0000","last_peered":"2024-05-14T11:16:17.807462+0000","last_clean":"2024-05-14T11:16:17.807462+0000","last_became_active":"2024-05-14T11:16:12.266472+0000","last_became_peered":"2024-05-14T11:16:12.266472+0000","last_unstale":"2024-05-14T11:16:17.807462+0000","last_undegraded":"2024-05-14T11:16:17.807462+0000","last_fullsized":"2024-05-14T11:16:17.807462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806284+0000","last_change":"2024-05-14T11:16:12.245994+0000","last_active":"2024-05-14T11:16:17.806284+0000","last_peered":"2024-05-14T11:16:17.806284+0000","last_clean":"2024-05-14T11:16:17.806284+0000","last_became_active":"2024-05-14T11:16:12.243237+0000","last_became_peered":"2024-05-14T11:16:12.243237+0000","last_unstale":"2024-05-14T11:16:17.806284+0000","last_undegraded":"2024-05-14T11:16:17.806284+0000","last_fullsized":"2024-05-14T11:16:17.806284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300549+0000","last_change":"2024-05-14T11:16:12.240444+0000","last_active":"2024-05-14T11:16:17.300549+0000","last_peered":"2024-05-14T11:16:17.300549+0000","last_clean":"2024-05-14T11:16:17.300549+0000","last_became_active":"2024-05-14T11:16:12.240076+0000","last_became_peered":"2024-05-14T11:16:12.240076+0000","last_unstale":"2024-05-14T11:16:17.300549+0000","last_undegraded":"2024-05-14T11:16:17.300549+0000","last_fullsized":"2024-05-14T11:16:17.300549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807295+0000","last_change":"2024-05-14T11:16:12.243507+0000","last_active":"2024-05-14T11:16:17.807295+0000","last_peered":"2024-05-14T11:16:17.807295+0000","last_clean":"2024-05-14T11:16:17.807295+0000","last_became_active":"2024-05-14T11:16:12.243180+0000","last_became_peered":"2024-05-14T11:16:12.243180+0000","last_unstale":"2024-05-14T11:16:17.807295+0000","last_undegraded":"2024-05-14T11:16:17.807295+0000","last_fullsized":"2024-05-14T11:16:17.807295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298355+0000","last_change":"2024-05-14T11:16:12.250650+0000","last_active":"2024-05-14T11:16:17.298355+0000","last_peered":"2024-05-14T11:16:17.298355+0000","last_clean":"2024-05-14T11:16:17.298355+0000","last_became_active":"2024-05-14T11:16:12.250259+0000","last_became_peered":"2024-05-14T11:16:12.250259+0000","last_unstale":"2024-05-14T11:16:17.298355+0000","last_undegraded":"2024-05-14T11:16:17.298355+0000","last_fullsized":"2024-05-14T11:16:17.298355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807449+0000","last_change":"2024-05-14T11:16:12.267003+0000","last_active":"2024-05-14T11:16:17.807449+0000","last_peered":"2024-05-14T11:16:17.807449+0000","last_clean":"2024-05-14T11:16:17.807449+0000","last_became_active":"2024-05-14T11:16:12.266655+0000","last_became_peered":"2024-05-14T11:16:12.266655+0000","last_unstale":"2024-05-14T11:16:17.807449+0000","last_undegraded":"2024-05-14T11:16:17.807449+0000","last_fullsized":"2024-05-14T11:16:17.807449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806108+0000","last_change":"2024-05-14T11:16:12.251621+0000","last_active":"2024-05-14T11:16:17.806108+0000","last_peered":"2024-05-14T11:16:17.806108+0000","last_clean":"2024-05-14T11:16:17.806108+0000","last_became_active":"2024-05-14T11:16:12.251284+0000","last_became_peered":"2024-05-14T11:16:12.251284+0000","last_unstale":"2024-05-14T11:16:17.806108+0000","last_undegraded":"2024-05-14T11:16:17.806108+0000","last_fullsized":"2024-05-14T11:16:17.806108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806728+0000","last_change":"2024-05-14T11:16:12.247078+0000","last_active":"2024-05-14T11:16:17.806728+0000","last_peered":"2024-05-14T11:16:17.806728+0000","last_clean":"2024-05-14T11:16:17.806728+0000","last_became_active":"2024-05-14T11:16:12.246785+0000","last_became_peered":"2024-05-14T11:16:12.246785+0000","last_unstale":"2024-05-14T11:16:17.806728+0000","last_undegraded":"2024-05-14T11:16:17.806728+0000","last_fullsized":"2024-05-14T11:16:17.806728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806255+0000","last_change":"2024-05-14T11:16:12.246002+0000","last_active":"2024-05-14T11:16:17.806255+0000","last_peered":"2024-05-14T11:16:17.806255+0000","last_clean":"2024-05-14T11:16:17.806255+0000","last_became_active":"2024-05-14T11:16:12.245378+0000","last_became_peered":"2024-05-14T11:16:12.245378+0000","last_unstale":"2024-05-14T11:16:17.806255+0000","last_undegraded":"2024-05-14T11:16:17.806255+0000","last_fullsized":"2024-05-14T11:16:17.806255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:42.753210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808869+0000","last_change":"2024-05-14T11:16:12.264249+0000","last_active":"2024-05-14T11:16:17.808869+0000","last_peered":"2024-05-14T11:16:17.808869+0000","last_clean":"2024-05-14T11:16:17.808869+0000","last_became_active":"2024-05-14T11:16:12.263604+0000","last_became_peered":"2024-05-14T11:16:12.263604+0000","last_unstale":"2024-05-14T11:16:17.808869+0000","last_undegraded":"2024-05-14T11:16:17.808869+0000","last_fullsized":"2024-05-14T11:16:17.808869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302334+0000","last_change":"2024-05-14T11:16:12.239836+0000","last_active":"2024-05-14T11:16:17.302334+0000","last_peered":"2024-05-14T11:16:17.302334+0000","last_clean":"2024-05-14T11:16:17.302334+0000","last_became_active":"2024-05-14T11:16:12.236431+0000","last_became_peered":"2024-05-14T11:16:12.236431+0000","last_unstale":"2024-05-14T11:16:17.302334+0000","last_undegraded":"2024-05-14T11:16:17.302334+0000","last_fullsized":"2024-05-14T11:16:17.302334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:03.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300974+0000","last_change":"2024-05-14T11:16:12.273349+0000","last_active":"2024-05-14T11:16:17.300974+0000","last_peered":"2024-05-14T11:16:17.300974+0000","last_clean":"2024-05-14T11:16:17.300974+0000","last_became_active":"2024-05-14T11:16:12.272507+0000","last_became_peered":"2024-05-14T11:16:12.272507+0000","last_unstale":"2024-05-14T11:16:17.300974+0000","last_undegraded":"2024-05-14T11:16:17.300974+0000","last_fullsized":"2024-05-14T11:16:17.300974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301473+0000","last_change":"2024-05-14T11:16:12.257302+0000","last_active":"2024-05-14T11:16:17.301473+0000","last_peered":"2024-05-14T11:16:17.301473+0000","last_clean":"2024-05-14T11:16:17.301473+0000","last_became_active":"2024-05-14T11:16:12.256830+0000","last_became_peered":"2024-05-14T11:16:12.256830+0000","last_unstale":"2024-05-14T11:16:17.301473+0000","last_undegraded":"2024-05-14T11:16:17.301473+0000","last_fullsized":"2024-05-14T11:16:17.301473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807763+0000","last_change":"2024-05-14T11:16:12.264613+0000","last_active":"2024-05-14T11:16:17.807763+0000","last_peered":"2024-05-14T11:16:17.807763+0000","last_clean":"2024-05-14T11:16:17.807763+0000","last_became_active":"2024-05-14T11:16:12.264294+0000","last_became_peered":"2024-05-14T11:16:12.264294+0000","last_unstale":"2024-05-14T11:16:17.807763+0000","last_undegraded":"2024-05-14T11:16:17.807763+0000","last_fullsized":"2024-05-14T11:16:17.807763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:42.657656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808085+0000","last_change":"2024-05-14T11:16:12.267492+0000","last_active":"2024-05-14T11:16:17.808085+0000","last_peered":"2024-05-14T11:16:17.808085+0000","last_clean":"2024-05-14T11:16:17.808085+0000","last_became_active":"2024-05-14T11:16:12.267005+0000","last_became_peered":"2024-05-14T11:16:12.267005+0000","last_unstale":"2024-05-14T11:16:17.808085+0000","last_undegraded":"2024-05-14T11:16:17.808085+0000","last_fullsized":"2024-05-14T11:16:17.808085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807982+0000","last_change":"2024-05-14T11:16:12.252241+0000","last_active":"2024-05-14T11:16:17.807982+0000","last_peered":"2024-05-14T11:16:17.807982+0000","last_clean":"2024-05-14T11:16:17.807982+0000","last_became_active":"2024-05-14T11:16:12.251844+0000","last_became_peered":"2024-05-14T11:16:12.251844+0000","last_unstale":"2024-05-14T11:16:17.807982+0000","last_undegraded":"2024-05-14T11:16:17.807982+0000","last_fullsized":"2024-05-14T11:16:17.807982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807948+0000","last_change":"2024-05-14T11:16:12.254254+0000","last_active":"2024-05-14T11:16:17.807948+0000","last_peered":"2024-05-14T11:16:17.807948+0000","last_clean":"2024-05-14T11:16:17.807948+0000","last_became_active":"2024-05-14T11:16:12.253441+0000","last_became_peered":"2024-05-14T11:16:12.253441+0000","last_unstale":"2024-05-14T11:16:17.807948+0000","last_undegraded":"2024-05-14T11:16:17.807948+0000","last_fullsized":"2024-05-14T11:16:17.807948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807243+0000","last_change":"2024-05-14T11:16:12.244922+0000","last_active":"2024-05-14T11:16:17.807243+0000","last_peered":"2024-05-14T11:16:17.807243+0000","last_clean":"2024-05-14T11:16:17.807243+0000","last_became_active":"2024-05-14T11:16:12.244534+0000","last_became_peered":"2024-05-14T11:16:12.244534+0000","last_unstale":"2024-05-14T11:16:17.807243+0000","last_undegraded":"2024-05-14T11:16:17.807243+0000","last_fullsized":"2024-05-14T11:16:17.807243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806550+0000","last_change":"2024-05-14T11:16:12.270470+0000","last_active":"2024-05-14T11:16:17.806550+0000","last_peered":"2024-05-14T11:16:17.806550+0000","last_clean":"2024-05-14T11:16:17.806550+0000","last_became_active":"2024-05-14T11:16:12.269667+0000","last_became_peered":"2024-05-14T11:16:12.269667+0000","last_unstale":"2024-05-14T11:16:17.806550+0000","last_undegraded":"2024-05-14T11:16:17.806550+0000","last_fullsized":"2024-05-14T11:16:17.806550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806567+0000","last_change":"2024-05-14T11:16:12.254558+0000","last_active":"2024-05-14T11:16:17.806567+0000","last_peered":"2024-05-14T11:16:17.806567+0000","last_clean":"2024-05-14T11:16:17.806567+0000","last_became_active":"2024-05-14T11:16:12.254245+0000","last_became_peered":"2024-05-14T11:16:12.254245+0000","last_unstale":"2024-05-14T11:16:17.806567+0000","last_undegraded":"2024-05-14T11:16:17.806567+0000","last_fullsized":"2024-05-14T11:16:17.806567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807656+0000","last_change":"2024-05-14T11:16:12.256145+0000","last_active":"2024-05-14T11:16:17.807656+0000","last_peered":"2024-05-14T11:16:17.807656+0000","last_clean":"2024-05-14T11:16:17.807656+0000","last_became_active":"2024-05-14T11:16:12.255806+0000","last_became_peered":"2024-05-14T11:16:12.255806+0000","last_unstale":"2024-05-14T11:16:17.807656+0000","last_undegraded":"2024-05-14T11:16:17.807656+0000","last_fullsized":"2024-05-14T11:16:17.807656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807528+0000","last_change":"2024-05-14T11:16:12.252502+0000","last_active":"2024-05-14T11:16:17.807528+0000","last_peered":"2024-05-14T11:16:17.807528+0000","last_clean":"2024-05-14T11:16:17.807528+0000","last_became_active":"2024-05-14T11:16:12.251431+0000","last_became_peered":"2024-05-14T11:16:12.251431+0000","last_unstale":"2024-05-14T11:16:17.807528+0000","last_undegraded":"2024-05-14T11:16:17.807528+0000","last_fullsized":"2024-05-14T11:16:17.807528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:16:33.574 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:16:33.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 Processor -- start 2024-05-14T11:16:33.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 -- start start 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fda7806ea50 0x7fda7815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda780cf8c0 0x7fda780c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda780ce110 con 0x7fda780c6fc0 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fda780ce3f0 con 0x7fda780cf8c0 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7ea00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fda780ce6d0 con 0x7fda7806ea50 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda780cf8c0 0x7fda780c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7cc00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fda780c6fc0 0x7fda780cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39750/0 (socket says 172.21.15.116:39750) 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda780cf8c0 0x7fda780c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52152/0 (socket says 172.21.15.116:52152) 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7cc00640 1 -- 172.21.15.116:0/3339089146 learned_addr learned my addr 172.21.15.116:0/3339089146 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7d600640 1 --2- 172.21.15.116:0/3339089146 >> v2:172.21.15.149:3300/0 conn(0x7fda7806ea50 0x7fda7815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 -- 172.21.15.116:0/3339089146 >> v2:172.21.15.149:3300/0 conn(0x7fda7806ea50 msgr2=0x7fda7815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 --2- 172.21.15.116:0/3339089146 >> v2:172.21.15.149:3300/0 conn(0x7fda7806ea50 0x7fda7815faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 -- 172.21.15.116:0/3339089146 >> v1:172.21.15.116:6789/0 conn(0x7fda780c6fc0 legacy=0x7fda780cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda7e000640 1 -- 172.21.15.116:0/3339089146 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda780ce9b0 con 0x7fda780cf8c0 2024-05-14T11:16:33.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.748+0000 7fda77e00640 1 -- 172.21.15.116:0/3339089146 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604321723 0 0) 0x7fda780ce110 con 0x7fda780c6fc0 2024-05-14T11:16:33.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7e000640 1 --2- 172.21.15.116:0/3339089146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda780cf8c0 0x7fda780c73d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fda70009d90 tx=0x7fda7001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c15d2e089399deb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda77e00640 1 -- 172.21.15.116:0/3339089146 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700b2070 con 0x7fda780cf8c0 2024-05-14T11:16:33.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda77e00640 1 -- 172.21.15.116:0/3339089146 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda700a5830 con 0x7fda780cf8c0 2024-05-14T11:16:33.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda77e00640 1 -- 172.21.15.116:0/3339089146 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700a5c00 con 0x7fda780cf8c0 2024-05-14T11:16:33.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 -- 172.21.15.116:0/3339089146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda780cf8c0 msgr2=0x7fda780c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 --2- 172.21.15.116:0/3339089146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda780cf8c0 0x7fda780c73d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fda70009d90 tx=0x7fda7001c8a0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 -- 172.21.15.116:0/3339089146 shutdown_connections 2024-05-14T11:16:33.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 --2- 172.21.15.116:0/3339089146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda780cf8c0 0x7fda780c73d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 --2- 172.21.15.116:0/3339089146 >> v2:172.21.15.149:3300/0 conn(0x7fda7806ea50 0x7fda7815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 -- 172.21.15.116:0/3339089146 >> 172.21.15.116:0/3339089146 conn(0x7fda780d3470 msgr2=0x7fda780d3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 -- 172.21.15.116:0/3339089146 shutdown_connections 2024-05-14T11:16:33.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.752+0000 7fda7ea00640 1 -- 172.21.15.116:0/3339089146 wait complete. 2024-05-14T11:16:33.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 Processor -- start 2024-05-14T11:16:33.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 -- start start 2024-05-14T11:16:33.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 0x7fda781527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda780cf8c0 0x7fda780b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda7816cba0 con 0x7fda7806ea50 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda780cdeb0 con 0x7fda780c6fc0 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7ea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda7816cd20 con 0x7fda780cf8c0 2024-05-14T11:16:33.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52154/0 (socket says 172.21.15.116:52154) 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 -- 172.21.15.116:0/4153242992 learned_addr learned my addr 172.21.15.116:0/4153242992 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7cc00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 0x7fda781527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 -- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda780cf8c0 msgr2=0x7fda780b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7e000640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda780cf8c0 0x7fda780b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda780cf8c0 0x7fda780b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 -- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 msgr2=0x7fda781527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 0x7fda781527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 -- 172.21.15.116:0/4153242992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda781537a0 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7cc00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 0x7fda781527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.756+0000 7fda7d600640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e840 tx=0x7fda6c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c017070 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c00ee90 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00cb60 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda78153a80 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda78154040 con 0x7fda7806ea50 2024-05-14T11:16:33.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda78050880 con 0x7fda7806ea50 2024-05-14T11:16:33.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda6c01a070 con 0x7fda7806ea50 2024-05-14T11:16:33.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 0x7fda58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:33.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda6c08f870 con 0x7fda7806ea50 2024-05-14T11:16:33.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.760+0000 7fda7cc00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 0x7fda58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:33.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.764+0000 7fda7cc00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 0x7fda58071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda7806fb60 tx=0x7fda64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:33.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.764+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda6c05c410 con 0x7fda7806ea50 2024-05-14T11:16:33.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.920+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda781587a0 con 0x7fda5806f400 2024-05-14T11:16:33.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.928+0000 7fda75600640 1 -- 172.21.15.116:0/4153242992 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7fda781587a0 con 0x7fda5806f400 2024-05-14T11:16:33.934 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:33.936 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 msgr2=0x7fda58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 0x7fda58071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda7806fb60 tx=0x7fda64009040 comp rx=0 tx=0).stop 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 msgr2=0x7fda78152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e840 tx=0x7fda6c00ec40 comp rx=0 tx=0).stop 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 shutdown_connections 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda5806f400 0x7fda58071910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda780c6fc0 0x7fda781527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda780cf8c0 0x7fda780b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 --2- 172.21.15.116:0/4153242992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda7806ea50 0x7fda78152f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:33.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 >> 172.21.15.116:0/4153242992 conn(0x7fda780d3470 msgr2=0x7fda7815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:33.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 shutdown_connections 2024-05-14T11:16:33.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:33.932+0000 7fda7ea00640 1 -- 172.21.15.116:0/4153242992 wait complete. 2024-05-14T11:16:33.955 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-14T11:16:32.363008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003634"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807671+0000","last_change":"2024-05-14T11:16:11.249111+0000","last_active":"2024-05-14T11:16:17.807671+0000","last_peered":"2024-05-14T11:16:17.807671+0000","last_clean":"2024-05-14T11:16:17.807671+0000","last_became_active":"2024-05-14T11:16:11.248739+0000","last_became_peered":"2024-05-14T11:16:11.248739+0000","last_unstale":"2024-05-14T11:16:17.807671+0000","last_undegraded":"2024-05-14T11:16:17.807671+0000","last_fullsized":"2024-05-14T11:16:17.807671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808824+0000","last_change":"2024-05-14T11:16:12.266252+0000","last_active":"2024-05-14T11:16:17.808824+0000","last_peered":"2024-05-14T11:16:17.808824+0000","last_clean":"2024-05-14T11:16:17.808824+0000","last_became_active":"2024-05-14T11:16:12.265805+0000","last_became_peered":"2024-05-14T11:16:12.265805+0000","last_unstale":"2024-05-14T11:16:17.808824+0000","last_undegraded":"2024-05-14T11:16:17.808824+0000","last_fullsized":"2024-05-14T11:16:17.808824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808167+0000","last_change":"2024-05-14T11:16:16.339345+0000","last_active":"2024-05-14T11:16:17.808167+0000","last_peered":"2024-05-14T11:16:17.808167+0000","last_clean":"2024-05-14T11:16:17.808167+0000","last_became_active":"2024-05-14T11:16:16.338538+0000","last_became_peered":"2024-05-14T11:16:16.338538+0000","last_unstale":"2024-05-14T11:16:17.808167+0000","last_undegraded":"2024-05-14T11:16:17.808167+0000","last_fullsized":"2024-05-14T11:16:17.808167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809078+0000","last_change":"2024-05-14T11:16:11.246975+0000","last_active":"2024-05-14T11:16:17.809078+0000","last_peered":"2024-05-14T11:16:17.809078+0000","last_clean":"2024-05-14T11:16:17.809078+0000","last_became_active":"2024-05-14T11:16:11.246617+0000","last_became_peered":"2024-05-14T11:16:11.246617+0000","last_unstale":"2024-05-14T11:16:17.809078+0000","last_undegraded":"2024-05-14T11:16:17.809078+0000","last_fullsized":"2024-05-14T11:16:17.809078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807310+0000","last_change":"2024-05-14T11:16:12.261153+0000","last_active":"2024-05-14T11:16:17.807310+0000","last_peered":"2024-05-14T11:16:17.807310+0000","last_clean":"2024-05-14T11:16:17.807310+0000","last_became_active":"2024-05-14T11:16:12.260566+0000","last_became_peered":"2024-05-14T11:16:12.260566+0000","last_unstale":"2024-05-14T11:16:17.807310+0000","last_undegraded":"2024-05-14T11:16:17.807310+0000","last_fullsized":"2024-05-14T11:16:17.807310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:47.069928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300836+0000","last_change":"2024-05-14T11:16:16.298112+0000","last_active":"2024-05-14T11:16:17.300836+0000","last_peered":"2024-05-14T11:16:17.300836+0000","last_clean":"2024-05-14T11:16:17.300836+0000","last_became_active":"2024-05-14T11:16:16.297673+0000","last_became_peered":"2024-05-14T11:16:16.297673+0000","last_unstale":"2024-05-14T11:16:17.300836+0000","last_undegraded":"2024-05-14T11:16:17.300836+0000","last_fullsized":"2024-05-14T11:16:17.300836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:13.339957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807596+0000","last_change":"2024-05-14T11:16:11.253925+0000","last_active":"2024-05-14T11:16:17.807596+0000","last_peered":"2024-05-14T11:16:17.807596+0000","last_clean":"2024-05-14T11:16:17.807596+0000","last_became_active":"2024-05-14T11:16:11.253441+0000","last_became_peered":"2024-05-14T11:16:11.253441+0000","last_unstale":"2024-05-14T11:16:17.807596+0000","last_undegraded":"2024-05-14T11:16:17.807596+0000","last_fullsized":"2024-05-14T11:16:17.807596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806697+0000","last_change":"2024-05-14T11:16:12.255735+0000","last_active":"2024-05-14T11:16:17.806697+0000","last_peered":"2024-05-14T11:16:17.806697+0000","last_clean":"2024-05-14T11:16:17.806697+0000","last_became_active":"2024-05-14T11:16:12.255414+0000","last_became_peered":"2024-05-14T11:16:12.255414+0000","last_unstale":"2024-05-14T11:16:17.806697+0000","last_undegraded":"2024-05-14T11:16:17.806697+0000","last_fullsized":"2024-05-14T11:16:17.806697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300072+0000","last_change":"2024-05-14T11:16:16.297447+0000","last_active":"2024-05-14T11:16:17.300072+0000","last_peered":"2024-05-14T11:16:17.300072+0000","last_clean":"2024-05-14T11:16:17.300072+0000","last_became_active":"2024-05-14T11:16:16.297024+0000","last_became_peered":"2024-05-14T11:16:16.297024+0000","last_unstale":"2024-05-14T11:16:17.300072+0000","last_undegraded":"2024-05-14T11:16:17.300072+0000","last_fullsized":"2024-05-14T11:16:17.300072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:41.465988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806953+0000","last_change":"2024-05-14T11:16:11.263356+0000","last_active":"2024-05-14T11:16:17.806953+0000","last_peered":"2024-05-14T11:16:17.806953+0000","last_clean":"2024-05-14T11:16:17.806953+0000","last_became_active":"2024-05-14T11:16:11.263025+0000","last_became_peered":"2024-05-14T11:16:11.263025+0000","last_unstale":"2024-05-14T11:16:17.806953+0000","last_undegraded":"2024-05-14T11:16:17.806953+0000","last_fullsized":"2024-05-14T11:16:17.806953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806916+0000","last_change":"2024-05-14T11:16:12.253871+0000","last_active":"2024-05-14T11:16:17.806916+0000","last_peered":"2024-05-14T11:16:17.806916+0000","last_clean":"2024-05-14T11:16:17.806916+0000","last_became_active":"2024-05-14T11:16:12.253416+0000","last_became_peered":"2024-05-14T11:16:12.253416+0000","last_unstale":"2024-05-14T11:16:17.806916+0000","last_undegraded":"2024-05-14T11:16:17.806916+0000","last_fullsized":"2024-05-14T11:16:17.806916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300363+0000","last_change":"2024-05-14T11:16:16.289232+0000","last_active":"2024-05-14T11:16:17.300363+0000","last_peered":"2024-05-14T11:16:17.300363+0000","last_clean":"2024-05-14T11:16:17.300363+0000","last_became_active":"2024-05-14T11:16:16.288706+0000","last_became_peered":"2024-05-14T11:16:16.288706+0000","last_unstale":"2024-05-14T11:16:17.300363+0000","last_undegraded":"2024-05-14T11:16:17.300363+0000","last_fullsized":"2024-05-14T11:16:17.300363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:13.765975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299905+0000","last_change":"2024-05-14T11:16:12.237866+0000","last_active":"2024-05-14T11:16:17.299905+0000","last_peered":"2024-05-14T11:16:17.299905+0000","last_clean":"2024-05-14T11:16:17.299905+0000","last_became_active":"2024-05-14T11:16:12.237591+0000","last_became_peered":"2024-05-14T11:16:12.237591+0000","last_unstale":"2024-05-14T11:16:17.299905+0000","last_undegraded":"2024-05-14T11:16:17.299905+0000","last_fullsized":"2024-05-14T11:16:17.299905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807660+0000","last_change":"2024-05-14T11:16:11.250842+0000","last_active":"2024-05-14T11:16:17.807660+0000","last_peered":"2024-05-14T11:16:17.807660+0000","last_clean":"2024-05-14T11:16:17.807660+0000","last_became_active":"2024-05-14T11:16:11.250454+0000","last_became_peered":"2024-05-14T11:16:11.250454+0000","last_unstale":"2024-05-14T11:16:17.807660+0000","last_undegraded":"2024-05-14T11:16:17.807660+0000","last_fullsized":"2024-05-14T11:16:17.807660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302586+0000","last_change":"2024-05-14T11:16:16.299192+0000","last_active":"2024-05-14T11:16:17.302586+0000","last_peered":"2024-05-14T11:16:17.302586+0000","last_clean":"2024-05-14T11:16:17.302586+0000","last_became_active":"2024-05-14T11:16:16.298925+0000","last_became_peered":"2024-05-14T11:16:16.298925+0000","last_unstale":"2024-05-14T11:16:17.302586+0000","last_undegraded":"2024-05-14T11:16:17.302586+0000","last_fullsized":"2024-05-14T11:16:17.302586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808972+0000","last_change":"2024-05-14T11:16:12.266594+0000","last_active":"2024-05-14T11:16:17.808972+0000","last_peered":"2024-05-14T11:16:17.808972+0000","last_clean":"2024-05-14T11:16:17.808972+0000","last_became_active":"2024-05-14T11:16:12.266261+0000","last_became_peered":"2024-05-14T11:16:12.266261+0000","last_unstale":"2024-05-14T11:16:17.808972+0000","last_undegraded":"2024-05-14T11:16:17.808972+0000","last_fullsized":"2024-05-14T11:16:17.808972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808959+0000","last_change":"2024-05-14T11:16:11.251548+0000","last_active":"2024-05-14T11:16:17.808959+0000","last_peered":"2024-05-14T11:16:17.808959+0000","last_clean":"2024-05-14T11:16:17.808959+0000","last_became_active":"2024-05-14T11:16:11.251146+0000","last_became_peered":"2024-05-14T11:16:11.251146+0000","last_unstale":"2024-05-14T11:16:17.808959+0000","last_undegraded":"2024-05-14T11:16:17.808959+0000","last_fullsized":"2024-05-14T11:16:17.808959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300981+0000","last_change":"2024-05-14T11:16:16.296270+0000","last_active":"2024-05-14T11:16:17.300981+0000","last_peered":"2024-05-14T11:16:17.300981+0000","last_clean":"2024-05-14T11:16:17.300981+0000","last_became_active":"2024-05-14T11:16:16.295796+0000","last_became_peered":"2024-05-14T11:16:16.295796+0000","last_unstale":"2024-05-14T11:16:17.300981+0000","last_undegraded":"2024-05-14T11:16:17.300981+0000","last_fullsized":"2024-05-14T11:16:17.300981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808335+0000","last_change":"2024-05-14T11:16:12.259552+0000","last_active":"2024-05-14T11:16:17.808335+0000","last_peered":"2024-05-14T11:16:17.808335+0000","last_clean":"2024-05-14T11:16:17.808335+0000","last_became_active":"2024-05-14T11:16:12.259053+0000","last_became_peered":"2024-05-14T11:16:12.259053+0000","last_unstale":"2024-05-14T11:16:17.808335+0000","last_undegraded":"2024-05-14T11:16:17.808335+0000","last_fullsized":"2024-05-14T11:16:17.808335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806762+0000","last_change":"2024-05-14T11:16:11.251898+0000","last_active":"2024-05-14T11:16:17.806762+0000","last_peered":"2024-05-14T11:16:17.806762+0000","last_clean":"2024-05-14T11:16:17.806762+0000","last_became_active":"2024-05-14T11:16:11.251339+0000","last_became_peered":"2024-05-14T11:16:11.251339+0000","last_unstale":"2024-05-14T11:16:17.806762+0000","last_undegraded":"2024-05-14T11:16:17.806762+0000","last_fullsized":"2024-05-14T11:16:17.806762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809475+0000","last_change":"2024-05-14T11:16:16.331750+0000","last_active":"2024-05-14T11:16:17.809475+0000","last_peered":"2024-05-14T11:16:17.809475+0000","last_clean":"2024-05-14T11:16:17.809475+0000","last_became_active":"2024-05-14T11:16:16.331181+0000","last_became_peered":"2024-05-14T11:16:16.331181+0000","last_unstale":"2024-05-14T11:16:17.809475+0000","last_undegraded":"2024-05-14T11:16:17.809475+0000","last_fullsized":"2024-05-14T11:16:17.809475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301296+0000","last_change":"2024-05-14T11:16:12.245880+0000","last_active":"2024-05-14T11:16:17.301296+0000","last_peered":"2024-05-14T11:16:17.301296+0000","last_clean":"2024-05-14T11:16:17.301296+0000","last_became_active":"2024-05-14T11:16:12.245150+0000","last_became_peered":"2024-05-14T11:16:12.245150+0000","last_unstale":"2024-05-14T11:16:17.301296+0000","last_undegraded":"2024-05-14T11:16:17.301296+0000","last_fullsized":"2024-05-14T11:16:17.301296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806258+0000","last_change":"2024-05-14T11:16:11.252415+0000","last_active":"2024-05-14T11:16:17.806258+0000","last_peered":"2024-05-14T11:16:17.806258+0000","last_clean":"2024-05-14T11:16:17.806258+0000","last_became_active":"2024-05-14T11:16:11.252113+0000","last_became_peered":"2024-05-14T11:16:11.252113+0000","last_unstale":"2024-05-14T11:16:17.806258+0000","last_undegraded":"2024-05-14T11:16:17.806258+0000","last_fullsized":"2024-05-14T11:16:17.806258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302531+0000","last_change":"2024-05-14T11:16:16.298123+0000","last_active":"2024-05-14T11:16:17.302531+0000","last_peered":"2024-05-14T11:16:17.302531+0000","last_clean":"2024-05-14T11:16:17.302531+0000","last_became_active":"2024-05-14T11:16:16.297647+0000","last_became_peered":"2024-05-14T11:16:16.297647+0000","last_unstale":"2024-05-14T11:16:17.302531+0000","last_undegraded":"2024-05-14T11:16:17.302531+0000","last_fullsized":"2024-05-14T11:16:17.302531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807122+0000","last_change":"2024-05-14T11:16:12.247217+0000","last_active":"2024-05-14T11:16:17.807122+0000","last_peered":"2024-05-14T11:16:17.807122+0000","last_clean":"2024-05-14T11:16:17.807122+0000","last_became_active":"2024-05-14T11:16:12.246460+0000","last_became_peered":"2024-05-14T11:16:12.246460+0000","last_unstale":"2024-05-14T11:16:17.807122+0000","last_undegraded":"2024-05-14T11:16:17.807122+0000","last_fullsized":"2024-05-14T11:16:17.807122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301791+0000","last_change":"2024-05-14T11:16:11.246553+0000","last_active":"2024-05-14T11:16:17.301791+0000","last_peered":"2024-05-14T11:16:17.301791+0000","last_clean":"2024-05-14T11:16:17.301791+0000","last_became_active":"2024-05-14T11:16:11.245634+0000","last_became_peered":"2024-05-14T11:16:11.245634+0000","last_unstale":"2024-05-14T11:16:17.301791+0000","last_undegraded":"2024-05-14T11:16:17.301791+0000","last_fullsized":"2024-05-14T11:16:17.301791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300036+0000","last_change":"2024-05-14T11:16:16.331377+0000","last_active":"2024-05-14T11:16:17.300036+0000","last_peered":"2024-05-14T11:16:17.300036+0000","last_clean":"2024-05-14T11:16:17.300036+0000","last_became_active":"2024-05-14T11:16:16.330845+0000","last_became_peered":"2024-05-14T11:16:16.330845+0000","last_unstale":"2024-05-14T11:16:17.300036+0000","last_undegraded":"2024-05-14T11:16:17.300036+0000","last_fullsized":"2024-05-14T11:16:17.300036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:08.942678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809208+0000","last_change":"2024-05-14T11:16:12.265091+0000","last_active":"2024-05-14T11:16:17.809208+0000","last_peered":"2024-05-14T11:16:17.809208+0000","last_clean":"2024-05-14T11:16:17.809208+0000","last_became_active":"2024-05-14T11:16:12.264728+0000","last_became_peered":"2024-05-14T11:16:12.264728+0000","last_unstale":"2024-05-14T11:16:17.809208+0000","last_undegraded":"2024-05-14T11:16:17.809208+0000","last_fullsized":"2024-05-14T11:16:17.809208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807713+0000","last_change":"2024-05-14T11:16:11.253941+0000","last_active":"2024-05-14T11:16:17.807713+0000","last_peered":"2024-05-14T11:16:17.807713+0000","last_clean":"2024-05-14T11:16:17.807713+0000","last_became_active":"2024-05-14T11:16:11.253534+0000","last_became_peered":"2024-05-14T11:16:11.253534+0000","last_unstale":"2024-05-14T11:16:17.807713+0000","last_undegraded":"2024-05-14T11:16:17.807713+0000","last_fullsized":"2024-05-14T11:16:17.807713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808083+0000","last_change":"2024-05-14T11:16:16.338462+0000","last_active":"2024-05-14T11:16:17.808083+0000","last_peered":"2024-05-14T11:16:17.808083+0000","last_clean":"2024-05-14T11:16:17.808083+0000","last_became_active":"2024-05-14T11:16:16.337908+0000","last_became_peered":"2024-05-14T11:16:16.337908+0000","last_unstale":"2024-05-14T11:16:17.808083+0000","last_undegraded":"2024-05-14T11:16:17.808083+0000","last_fullsized":"2024-05-14T11:16:17.808083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:28:29.309169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807237+0000","last_change":"2024-05-14T11:16:12.249280+0000","last_active":"2024-05-14T11:16:17.807237+0000","last_peered":"2024-05-14T11:16:17.807237+0000","last_clean":"2024-05-14T11:16:17.807237+0000","last_became_active":"2024-05-14T11:16:12.248977+0000","last_became_peered":"2024-05-14T11:16:12.248977+0000","last_unstale":"2024-05-14T11:16:17.807237+0000","last_undegraded":"2024-05-14T11:16:17.807237+0000","last_fullsized":"2024-05-14T11:16:17.807237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807419+0000","last_change":"2024-05-14T11:16:11.255839+0000","last_active":"2024-05-14T11:16:17.807419+0000","last_peered":"2024-05-14T11:16:17.807419+0000","last_clean":"2024-05-14T11:16:17.807419+0000","last_became_active":"2024-05-14T11:16:11.255487+0000","last_became_peered":"2024-05-14T11:16:11.255487+0000","last_unstale":"2024-05-14T11:16:17.807419+0000","last_undegraded":"2024-05-14T11:16:17.807419+0000","last_fullsized":"2024-05-14T11:16:17.807419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:59.335122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806362+0000","last_change":"2024-05-14T11:16:16.330365+0000","last_active":"2024-05-14T11:16:17.806362+0000","last_peered":"2024-05-14T11:16:17.806362+0000","last_clean":"2024-05-14T11:16:17.806362+0000","last_became_active":"2024-05-14T11:16:16.329833+0000","last_became_peered":"2024-05-14T11:16:16.329833+0000","last_unstale":"2024-05-14T11:16:17.806362+0000","last_undegraded":"2024-05-14T11:16:17.806362+0000","last_fullsized":"2024-05-14T11:16:17.806362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807044+0000","last_change":"2024-05-14T11:16:12.267992+0000","last_active":"2024-05-14T11:16:17.807044+0000","last_peered":"2024-05-14T11:16:17.807044+0000","last_clean":"2024-05-14T11:16:17.807044+0000","last_became_active":"2024-05-14T11:16:12.267183+0000","last_became_peered":"2024-05-14T11:16:12.267183+0000","last_unstale":"2024-05-14T11:16:17.807044+0000","last_undegraded":"2024-05-14T11:16:17.807044+0000","last_fullsized":"2024-05-14T11:16:17.807044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807300+0000","last_change":"2024-05-14T11:16:11.247765+0000","last_active":"2024-05-14T11:16:17.807300+0000","last_peered":"2024-05-14T11:16:17.807300+0000","last_clean":"2024-05-14T11:16:17.807300+0000","last_became_active":"2024-05-14T11:16:11.247503+0000","last_became_peered":"2024-05-14T11:16:11.247503+0000","last_unstale":"2024-05-14T11:16:17.807300+0000","last_undegraded":"2024-05-14T11:16:17.807300+0000","last_fullsized":"2024-05-14T11:16:17.807300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808159+0000","last_change":"2024-05-14T11:16:16.304314+0000","last_active":"2024-05-14T11:16:17.808159+0000","last_peered":"2024-05-14T11:16:17.808159+0000","last_clean":"2024-05-14T11:16:17.808159+0000","last_became_active":"2024-05-14T11:16:16.303370+0000","last_became_peered":"2024-05-14T11:16:16.303370+0000","last_unstale":"2024-05-14T11:16:17.808159+0000","last_undegraded":"2024-05-14T11:16:17.808159+0000","last_fullsized":"2024-05-14T11:16:17.808159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301326+0000","last_change":"2024-05-14T11:16:12.258856+0000","last_active":"2024-05-14T11:16:17.301326+0000","last_peered":"2024-05-14T11:16:17.301326+0000","last_clean":"2024-05-14T11:16:17.301326+0000","last_became_active":"2024-05-14T11:16:12.257816+0000","last_became_peered":"2024-05-14T11:16:12.257816+0000","last_unstale":"2024-05-14T11:16:17.301326+0000","last_undegraded":"2024-05-14T11:16:17.301326+0000","last_fullsized":"2024-05-14T11:16:17.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:19.338856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807816+0000","last_change":"2024-05-14T11:16:11.257469+0000","last_active":"2024-05-14T11:16:17.807816+0000","last_peered":"2024-05-14T11:16:17.807816+0000","last_clean":"2024-05-14T11:16:17.807816+0000","last_became_active":"2024-05-14T11:16:11.253425+0000","last_became_peered":"2024-05-14T11:16:11.253425+0000","last_unstale":"2024-05-14T11:16:17.807816+0000","last_undegraded":"2024-05-14T11:16:17.807816+0000","last_fullsized":"2024-05-14T11:16:17.807816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807184+0000","last_change":"2024-05-14T11:16:16.305620+0000","last_active":"2024-05-14T11:16:17.807184+0000","last_peered":"2024-05-14T11:16:17.807184+0000","last_clean":"2024-05-14T11:16:17.807184+0000","last_became_active":"2024-05-14T11:16:16.305156+0000","last_became_peered":"2024-05-14T11:16:16.305156+0000","last_unstale":"2024-05-14T11:16:17.807184+0000","last_undegraded":"2024-05-14T11:16:17.807184+0000","last_fullsized":"2024-05-14T11:16:17.807184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806230+0000","last_change":"2024-05-14T11:16:12.245405+0000","last_active":"2024-05-14T11:16:17.806230+0000","last_peered":"2024-05-14T11:16:17.806230+0000","last_clean":"2024-05-14T11:16:17.806230+0000","last_became_active":"2024-05-14T11:16:12.245076+0000","last_became_peered":"2024-05-14T11:16:12.245076+0000","last_unstale":"2024-05-14T11:16:17.806230+0000","last_undegraded":"2024-05-14T11:16:17.806230+0000","last_fullsized":"2024-05-14T11:16:17.806230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297955+0000","last_change":"2024-05-14T11:16:11.245815+0000","last_active":"2024-05-14T11:16:17.297955+0000","last_peered":"2024-05-14T11:16:17.297955+0000","last_clean":"2024-05-14T11:16:17.297955+0000","last_became_active":"2024-05-14T11:16:11.245487+0000","last_became_peered":"2024-05-14T11:16:11.245487+0000","last_unstale":"2024-05-14T11:16:17.297955+0000","last_undegraded":"2024-05-14T11:16:17.297955+0000","last_fullsized":"2024-05-14T11:16:17.297955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809501+0000","last_change":"2024-05-14T11:16:16.304799+0000","last_active":"2024-05-14T11:16:17.809501+0000","last_peered":"2024-05-14T11:16:17.809501+0000","last_clean":"2024-05-14T11:16:17.809501+0000","last_became_active":"2024-05-14T11:16:16.304223+0000","last_became_peered":"2024-05-14T11:16:16.304223+0000","last_unstale":"2024-05-14T11:16:17.809501+0000","last_undegraded":"2024-05-14T11:16:17.809501+0000","last_fullsized":"2024-05-14T11:16:17.809501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.805980+0000","last_change":"2024-05-14T11:16:12.242987+0000","last_active":"2024-05-14T11:16:17.805980+0000","last_peered":"2024-05-14T11:16:17.805980+0000","last_clean":"2024-05-14T11:16:17.805980+0000","last_became_active":"2024-05-14T11:16:12.242648+0000","last_became_peered":"2024-05-14T11:16:12.242648+0000","last_unstale":"2024-05-14T11:16:17.805980+0000","last_undegraded":"2024-05-14T11:16:17.805980+0000","last_fullsized":"2024-05-14T11:16:17.805980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:37.421472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806812+0000","last_change":"2024-05-14T11:16:11.254596+0000","last_active":"2024-05-14T11:16:17.806812+0000","last_peered":"2024-05-14T11:16:17.806812+0000","last_clean":"2024-05-14T11:16:17.806812+0000","last_became_active":"2024-05-14T11:16:11.254304+0000","last_became_peered":"2024-05-14T11:16:11.254304+0000","last_unstale":"2024-05-14T11:16:17.806812+0000","last_undegraded":"2024-05-14T11:16:17.806812+0000","last_fullsized":"2024-05-14T11:16:17.806812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806903+0000","last_change":"2024-05-14T11:16:16.333086+0000","last_active":"2024-05-14T11:16:17.806903+0000","last_peered":"2024-05-14T11:16:17.806903+0000","last_clean":"2024-05-14T11:16:17.806903+0000","last_became_active":"2024-05-14T11:16:16.332633+0000","last_became_peered":"2024-05-14T11:16:16.332633+0000","last_unstale":"2024-05-14T11:16:17.806903+0000","last_undegraded":"2024-05-14T11:16:17.806903+0000","last_fullsized":"2024-05-14T11:16:17.806903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808690+0000","last_change":"2024-05-14T11:16:12.267597+0000","last_active":"2024-05-14T11:16:17.808690+0000","last_peered":"2024-05-14T11:16:17.808690+0000","last_clean":"2024-05-14T11:16:17.808690+0000","last_became_active":"2024-05-14T11:16:12.267050+0000","last_became_peered":"2024-05-14T11:16:12.267050+0000","last_unstale":"2024-05-14T11:16:17.808690+0000","last_undegraded":"2024-05-14T11:16:17.808690+0000","last_fullsized":"2024-05-14T11:16:17.808690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806566+0000","last_change":"2024-05-14T11:16:11.253533+0000","last_active":"2024-05-14T11:16:17.806566+0000","last_peered":"2024-05-14T11:16:17.806566+0000","last_clean":"2024-05-14T11:16:17.806566+0000","last_became_active":"2024-05-14T11:16:11.253104+0000","last_became_peered":"2024-05-14T11:16:11.253104+0000","last_unstale":"2024-05-14T11:16:17.806566+0000","last_undegraded":"2024-05-14T11:16:17.806566+0000","last_fullsized":"2024-05-14T11:16:17.806566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806962+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:16:17.806962+0000","last_peered":"2024-05-14T11:16:17.806962+0000","last_clean":"2024-05-14T11:16:17.806962+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:16:17.806962+0000","last_undegraded":"2024-05-14T11:16:17.806962+0000","last_fullsized":"2024-05-14T11:16:17.806962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299583+0000","last_change":"2024-05-14T11:16:16.330966+0000","last_active":"2024-05-14T11:16:17.299583+0000","last_peered":"2024-05-14T11:16:17.299583+0000","last_clean":"2024-05-14T11:16:17.299583+0000","last_became_active":"2024-05-14T11:16:16.330413+0000","last_became_peered":"2024-05-14T11:16:16.330413+0000","last_unstale":"2024-05-14T11:16:17.299583+0000","last_undegraded":"2024-05-14T11:16:17.299583+0000","last_fullsized":"2024-05-14T11:16:17.299583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807128+0000","last_change":"2024-05-14T11:16:12.263387+0000","last_active":"2024-05-14T11:16:17.807128+0000","last_peered":"2024-05-14T11:16:17.807128+0000","last_clean":"2024-05-14T11:16:17.807128+0000","last_became_active":"2024-05-14T11:16:12.262944+0000","last_became_peered":"2024-05-14T11:16:12.262944+0000","last_unstale":"2024-05-14T11:16:17.807128+0000","last_undegraded":"2024-05-14T11:16:17.807128+0000","last_fullsized":"2024-05-14T11:16:17.807128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298224+0000","last_change":"2024-05-14T11:16:11.259096+0000","last_active":"2024-05-14T11:16:17.298224+0000","last_peered":"2024-05-14T11:16:17.298224+0000","last_clean":"2024-05-14T11:16:17.298224+0000","last_became_active":"2024-05-14T11:16:11.258840+0000","last_became_peered":"2024-05-14T11:16:11.258840+0000","last_unstale":"2024-05-14T11:16:17.298224+0000","last_undegraded":"2024-05-14T11:16:17.298224+0000","last_fullsized":"2024-05-14T11:16:17.298224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300200+0000","last_change":"2024-05-14T11:16:16.333088+0000","last_active":"2024-05-14T11:16:17.300200+0000","last_peered":"2024-05-14T11:16:17.300200+0000","last_clean":"2024-05-14T11:16:17.300200+0000","last_became_active":"2024-05-14T11:16:16.332585+0000","last_became_peered":"2024-05-14T11:16:16.332585+0000","last_unstale":"2024-05-14T11:16:17.300200+0000","last_undegraded":"2024-05-14T11:16:17.300200+0000","last_fullsized":"2024-05-14T11:16:17.300200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806820+0000","last_change":"2024-05-14T11:16:12.270318+0000","last_active":"2024-05-14T11:16:17.806820+0000","last_peered":"2024-05-14T11:16:17.806820+0000","last_clean":"2024-05-14T11:16:17.806820+0000","last_became_active":"2024-05-14T11:16:12.269365+0000","last_became_peered":"2024-05-14T11:16:12.269365+0000","last_unstale":"2024-05-14T11:16:17.806820+0000","last_undegraded":"2024-05-14T11:16:17.806820+0000","last_fullsized":"2024-05-14T11:16:17.806820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807501+0000","last_change":"2024-05-14T11:16:11.252737+0000","last_active":"2024-05-14T11:16:17.807501+0000","last_peered":"2024-05-14T11:16:17.807501+0000","last_clean":"2024-05-14T11:16:17.807501+0000","last_became_active":"2024-05-14T11:16:11.252365+0000","last_became_peered":"2024-05-14T11:16:11.252365+0000","last_unstale":"2024-05-14T11:16:17.807501+0000","last_undegraded":"2024-05-14T11:16:17.807501+0000","last_fullsized":"2024-05-14T11:16:17.807501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806471+0000","last_change":"2024-05-14T11:16:16.302121+0000","last_active":"2024-05-14T11:16:17.806471+0000","last_peered":"2024-05-14T11:16:17.806471+0000","last_clean":"2024-05-14T11:16:17.806471+0000","last_became_active":"2024-05-14T11:16:16.299059+0000","last_became_peered":"2024-05-14T11:16:16.299059+0000","last_unstale":"2024-05-14T11:16:17.806471+0000","last_undegraded":"2024-05-14T11:16:17.806471+0000","last_fullsized":"2024-05-14T11:16:17.806471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807801+0000","last_change":"2024-05-14T11:16:12.247488+0000","last_active":"2024-05-14T11:16:17.807801+0000","last_peered":"2024-05-14T11:16:17.807801+0000","last_clean":"2024-05-14T11:16:17.807801+0000","last_became_active":"2024-05-14T11:16:12.247235+0000","last_became_peered":"2024-05-14T11:16:12.247235+0000","last_unstale":"2024-05-14T11:16:17.807801+0000","last_undegraded":"2024-05-14T11:16:17.807801+0000","last_fullsized":"2024-05-14T11:16:17.807801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:22.549789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298399+0000","last_change":"2024-05-14T11:16:11.246157+0000","last_active":"2024-05-14T11:16:17.298399+0000","last_peered":"2024-05-14T11:16:17.298399+0000","last_clean":"2024-05-14T11:16:17.298399+0000","last_became_active":"2024-05-14T11:16:11.245408+0000","last_became_peered":"2024-05-14T11:16:11.245408+0000","last_unstale":"2024-05-14T11:16:17.298399+0000","last_undegraded":"2024-05-14T11:16:17.298399+0000","last_fullsized":"2024-05-14T11:16:17.298399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806090+0000","last_change":"2024-05-14T11:16:16.330283+0000","last_active":"2024-05-14T11:16:17.806090+0000","last_peered":"2024-05-14T11:16:17.806090+0000","last_clean":"2024-05-14T11:16:17.806090+0000","last_became_active":"2024-05-14T11:16:16.329844+0000","last_became_peered":"2024-05-14T11:16:16.329844+0000","last_unstale":"2024-05-14T11:16:17.806090+0000","last_undegraded":"2024-05-14T11:16:17.806090+0000","last_fullsized":"2024-05-14T11:16:17.806090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807746+0000","last_change":"2024-05-14T11:16:12.248542+0000","last_active":"2024-05-14T11:16:17.807746+0000","last_peered":"2024-05-14T11:16:17.807746+0000","last_clean":"2024-05-14T11:16:17.807746+0000","last_became_active":"2024-05-14T11:16:12.248214+0000","last_became_peered":"2024-05-14T11:16:12.248214+0000","last_unstale":"2024-05-14T11:16:17.807746+0000","last_undegraded":"2024-05-14T11:16:17.807746+0000","last_fullsized":"2024-05-14T11:16:17.807746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302636+0000","last_change":"2024-05-14T11:16:11.241324+0000","last_active":"2024-05-14T11:16:17.302636+0000","last_peered":"2024-05-14T11:16:17.302636+0000","last_clean":"2024-05-14T11:16:17.302636+0000","last_became_active":"2024-05-14T11:16:11.239970+0000","last_became_peered":"2024-05-14T11:16:11.239970+0000","last_unstale":"2024-05-14T11:16:17.302636+0000","last_undegraded":"2024-05-14T11:16:17.302636+0000","last_fullsized":"2024-05-14T11:16:17.302636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:44.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299463+0000","last_change":"2024-05-14T11:16:16.293327+0000","last_active":"2024-05-14T11:16:17.299463+0000","last_peered":"2024-05-14T11:16:17.299463+0000","last_clean":"2024-05-14T11:16:17.299463+0000","last_became_active":"2024-05-14T11:16:16.292800+0000","last_became_peered":"2024-05-14T11:16:16.292800+0000","last_unstale":"2024-05-14T11:16:17.299463+0000","last_undegraded":"2024-05-14T11:16:17.299463+0000","last_fullsized":"2024-05-14T11:16:17.299463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:39.377103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808057+0000","last_change":"2024-05-14T11:16:12.248759+0000","last_active":"2024-05-14T11:16:17.808057+0000","last_peered":"2024-05-14T11:16:17.808057+0000","last_clean":"2024-05-14T11:16:17.808057+0000","last_became_active":"2024-05-14T11:16:12.248407+0000","last_became_peered":"2024-05-14T11:16:12.248407+0000","last_unstale":"2024-05-14T11:16:17.808057+0000","last_undegraded":"2024-05-14T11:16:17.808057+0000","last_fullsized":"2024-05-14T11:16:17.808057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297834+0000","last_change":"2024-05-14T11:16:11.248167+0000","last_active":"2024-05-14T11:16:17.297834+0000","last_peered":"2024-05-14T11:16:17.297834+0000","last_clean":"2024-05-14T11:16:17.297834+0000","last_became_active":"2024-05-14T11:16:11.245949+0000","last_became_peered":"2024-05-14T11:16:11.245949+0000","last_unstale":"2024-05-14T11:16:17.297834+0000","last_undegraded":"2024-05-14T11:16:17.297834+0000","last_fullsized":"2024-05-14T11:16:17.297834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807832+0000","last_change":"2024-05-14T11:16:16.290089+0000","last_active":"2024-05-14T11:16:17.807832+0000","last_peered":"2024-05-14T11:16:17.807832+0000","last_clean":"2024-05-14T11:16:17.807832+0000","last_became_active":"2024-05-14T11:16:16.287987+0000","last_became_peered":"2024-05-14T11:16:16.287987+0000","last_unstale":"2024-05-14T11:16:17.807832+0000","last_undegraded":"2024-05-14T11:16:17.807832+0000","last_fullsized":"2024-05-14T11:16:17.807832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:26.599395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299766+0000","last_change":"2024-05-14T11:16:12.248930+0000","last_active":"2024-05-14T11:16:17.299766+0000","last_peered":"2024-05-14T11:16:17.299766+0000","last_clean":"2024-05-14T11:16:17.299766+0000","last_became_active":"2024-05-14T11:16:12.248159+0000","last_became_peered":"2024-05-14T11:16:12.248159+0000","last_unstale":"2024-05-14T11:16:17.299766+0000","last_undegraded":"2024-05-14T11:16:17.299766+0000","last_fullsized":"2024-05-14T11:16:17.299766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299880+0000","last_change":"2024-05-14T11:16:11.247396+0000","last_active":"2024-05-14T11:16:17.299880+0000","last_peered":"2024-05-14T11:16:17.299880+0000","last_clean":"2024-05-14T11:16:17.299880+0000","last_became_active":"2024-05-14T11:16:11.246855+0000","last_became_peered":"2024-05-14T11:16:11.246855+0000","last_unstale":"2024-05-14T11:16:17.299880+0000","last_undegraded":"2024-05-14T11:16:17.299880+0000","last_fullsized":"2024-05-14T11:16:17.299880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808975+0000","last_change":"2024-05-14T11:16:16.332903+0000","last_active":"2024-05-14T11:16:17.808975+0000","last_peered":"2024-05-14T11:16:17.808975+0000","last_clean":"2024-05-14T11:16:17.808975+0000","last_became_active":"2024-05-14T11:16:16.332436+0000","last_became_peered":"2024-05-14T11:16:16.332436+0000","last_unstale":"2024-05-14T11:16:17.808975+0000","last_undegraded":"2024-05-14T11:16:17.808975+0000","last_fullsized":"2024-05-14T11:16:17.808975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:01.254012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807525+0000","last_change":"2024-05-14T11:16:12.248442+0000","last_active":"2024-05-14T11:16:17.807525+0000","last_peered":"2024-05-14T11:16:17.807525+0000","last_clean":"2024-05-14T11:16:17.807525+0000","last_became_active":"2024-05-14T11:16:12.248082+0000","last_became_peered":"2024-05-14T11:16:12.248082+0000","last_unstale":"2024-05-14T11:16:17.807525+0000","last_undegraded":"2024-05-14T11:16:17.807525+0000","last_fullsized":"2024-05-14T11:16:17.807525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807708+0000","last_change":"2024-05-14T11:16:11.257458+0000","last_active":"2024-05-14T11:16:17.807708+0000","last_peered":"2024-05-14T11:16:17.807708+0000","last_clean":"2024-05-14T11:16:17.807708+0000","last_became_active":"2024-05-14T11:16:11.252978+0000","last_became_peered":"2024-05-14T11:16:11.252978+0000","last_unstale":"2024-05-14T11:16:17.807708+0000","last_undegraded":"2024-05-14T11:16:17.807708+0000","last_fullsized":"2024-05-14T11:16:17.807708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808827+0000","last_change":"2024-05-14T11:16:16.330174+0000","last_active":"2024-05-14T11:16:17.808827+0000","last_peered":"2024-05-14T11:16:17.808827+0000","last_clean":"2024-05-14T11:16:17.808827+0000","last_became_active":"2024-05-14T11:16:16.329696+0000","last_became_peered":"2024-05-14T11:16:16.329696+0000","last_unstale":"2024-05-14T11:16:17.808827+0000","last_undegraded":"2024-05-14T11:16:17.808827+0000","last_fullsized":"2024-05-14T11:16:17.808827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:14.122764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806365+0000","last_change":"2024-05-14T11:16:12.246355+0000","last_active":"2024-05-14T11:16:17.806365+0000","last_peered":"2024-05-14T11:16:17.806365+0000","last_clean":"2024-05-14T11:16:17.806365+0000","last_became_active":"2024-05-14T11:16:12.245693+0000","last_became_peered":"2024-05-14T11:16:12.245693+0000","last_unstale":"2024-05-14T11:16:17.806365+0000","last_undegraded":"2024-05-14T11:16:17.806365+0000","last_fullsized":"2024-05-14T11:16:17.806365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806645+0000","last_change":"2024-05-14T11:16:11.249709+0000","last_active":"2024-05-14T11:16:17.806645+0000","last_peered":"2024-05-14T11:16:17.806645+0000","last_clean":"2024-05-14T11:16:17.806645+0000","last_became_active":"2024-05-14T11:16:11.245550+0000","last_became_peered":"2024-05-14T11:16:11.245550+0000","last_unstale":"2024-05-14T11:16:17.806645+0000","last_undegraded":"2024-05-14T11:16:17.806645+0000","last_fullsized":"2024-05-14T11:16:17.806645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:18.498642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806772+0000","last_change":"2024-05-14T11:16:16.330492+0000","last_active":"2024-05-14T11:16:17.806772+0000","last_peered":"2024-05-14T11:16:17.806772+0000","last_clean":"2024-05-14T11:16:17.806772+0000","last_became_active":"2024-05-14T11:16:16.330027+0000","last_became_peered":"2024-05-14T11:16:16.330027+0000","last_unstale":"2024-05-14T11:16:17.806772+0000","last_undegraded":"2024-05-14T11:16:17.806772+0000","last_fullsized":"2024-05-14T11:16:17.806772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807029+0000","last_change":"2024-05-14T11:16:12.263610+0000","last_active":"2024-05-14T11:16:17.807029+0000","last_peered":"2024-05-14T11:16:17.807029+0000","last_clean":"2024-05-14T11:16:17.807029+0000","last_became_active":"2024-05-14T11:16:12.263072+0000","last_became_peered":"2024-05-14T11:16:12.263072+0000","last_unstale":"2024-05-14T11:16:17.807029+0000","last_undegraded":"2024-05-14T11:16:17.807029+0000","last_fullsized":"2024-05-14T11:16:17.807029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:23.206421+0000","last_change":"2024-05-14T11:16:11.250108+0000","last_active":"2024-05-14T11:16:23.206421+0000","last_peered":"2024-05-14T11:16:23.206421+0000","last_clean":"2024-05-14T11:16:23.206421+0000","last_became_active":"2024-05-14T11:16:11.249460+0000","last_became_peered":"2024-05-14T11:16:11.249460+0000","last_unstale":"2024-05-14T11:16:23.206421+0000","last_undegraded":"2024-05-14T11:16:23.206421+0000","last_fullsized":"2024-05-14T11:16:23.206421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809402+0000","last_change":"2024-05-14T11:16:16.305560+0000","last_active":"2024-05-14T11:16:17.809402+0000","last_peered":"2024-05-14T11:16:17.809402+0000","last_clean":"2024-05-14T11:16:17.809402+0000","last_became_active":"2024-05-14T11:16:16.305166+0000","last_became_peered":"2024-05-14T11:16:16.305166+0000","last_unstale":"2024-05-14T11:16:17.809402+0000","last_undegraded":"2024-05-14T11:16:17.809402+0000","last_fullsized":"2024-05-14T11:16:17.809402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302423+0000","last_change":"2024-05-14T11:16:12.248795+0000","last_active":"2024-05-14T11:16:17.302423+0000","last_peered":"2024-05-14T11:16:17.302423+0000","last_clean":"2024-05-14T11:16:17.302423+0000","last_became_active":"2024-05-14T11:16:12.248478+0000","last_became_peered":"2024-05-14T11:16:12.248478+0000","last_unstale":"2024-05-14T11:16:17.302423+0000","last_undegraded":"2024-05-14T11:16:17.302423+0000","last_fullsized":"2024-05-14T11:16:17.302423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807383+0000","last_change":"2024-05-14T11:16:11.259967+0000","last_active":"2024-05-14T11:16:17.807383+0000","last_peered":"2024-05-14T11:16:17.807383+0000","last_clean":"2024-05-14T11:16:17.807383+0000","last_became_active":"2024-05-14T11:16:11.259703+0000","last_became_peered":"2024-05-14T11:16:11.259703+0000","last_unstale":"2024-05-14T11:16:17.807383+0000","last_undegraded":"2024-05-14T11:16:17.807383+0000","last_fullsized":"2024-05-14T11:16:17.807383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301098+0000","last_change":"2024-05-14T11:16:16.293106+0000","last_active":"2024-05-14T11:16:17.301098+0000","last_peered":"2024-05-14T11:16:17.301098+0000","last_clean":"2024-05-14T11:16:17.301098+0000","last_became_active":"2024-05-14T11:16:16.292666+0000","last_became_peered":"2024-05-14T11:16:16.292666+0000","last_unstale":"2024-05-14T11:16:17.301098+0000","last_undegraded":"2024-05-14T11:16:17.301098+0000","last_fullsized":"2024-05-14T11:16:17.301098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807505+0000","last_change":"2024-05-14T11:16:12.260898+0000","last_active":"2024-05-14T11:16:17.807505+0000","last_peered":"2024-05-14T11:16:17.807505+0000","last_clean":"2024-05-14T11:16:17.807505+0000","last_became_active":"2024-05-14T11:16:12.260114+0000","last_became_peered":"2024-05-14T11:16:12.260114+0000","last_unstale":"2024-05-14T11:16:17.807505+0000","last_undegraded":"2024-05-14T11:16:17.807505+0000","last_fullsized":"2024-05-14T11:16:17.807505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806807+0000","last_change":"2024-05-14T11:16:11.254169+0000","last_active":"2024-05-14T11:16:17.806807+0000","last_peered":"2024-05-14T11:16:17.806807+0000","last_clean":"2024-05-14T11:16:17.806807+0000","last_became_active":"2024-05-14T11:16:11.253658+0000","last_became_peered":"2024-05-14T11:16:11.253658+0000","last_unstale":"2024-05-14T11:16:17.806807+0000","last_undegraded":"2024-05-14T11:16:17.806807+0000","last_fullsized":"2024-05-14T11:16:17.806807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808298+0000","last_change":"2024-05-14T11:16:16.298722+0000","last_active":"2024-05-14T11:16:17.808298+0000","last_peered":"2024-05-14T11:16:17.808298+0000","last_clean":"2024-05-14T11:16:17.808298+0000","last_became_active":"2024-05-14T11:16:16.298312+0000","last_became_peered":"2024-05-14T11:16:16.298312+0000","last_unstale":"2024-05-14T11:16:17.808298+0000","last_undegraded":"2024-05-14T11:16:17.808298+0000","last_fullsized":"2024-05-14T11:16:17.808298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806669+0000","last_change":"2024-05-14T11:16:12.269044+0000","last_active":"2024-05-14T11:16:17.806669+0000","last_peered":"2024-05-14T11:16:17.806669+0000","last_clean":"2024-05-14T11:16:17.806669+0000","last_became_active":"2024-05-14T11:16:12.268428+0000","last_became_peered":"2024-05-14T11:16:12.268428+0000","last_unstale":"2024-05-14T11:16:17.806669+0000","last_undegraded":"2024-05-14T11:16:17.806669+0000","last_fullsized":"2024-05-14T11:16:17.806669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301709+0000","last_change":"2024-05-14T11:16:11.247431+0000","last_active":"2024-05-14T11:16:17.301709+0000","last_peered":"2024-05-14T11:16:17.301709+0000","last_clean":"2024-05-14T11:16:17.301709+0000","last_became_active":"2024-05-14T11:16:11.247003+0000","last_became_peered":"2024-05-14T11:16:11.247003+0000","last_unstale":"2024-05-14T11:16:17.301709+0000","last_undegraded":"2024-05-14T11:16:17.301709+0000","last_fullsized":"2024-05-14T11:16:17.301709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808398+0000","last_change":"2024-05-14T11:16:16.338217+0000","last_active":"2024-05-14T11:16:17.808398+0000","last_peered":"2024-05-14T11:16:17.808398+0000","last_clean":"2024-05-14T11:16:17.808398+0000","last_became_active":"2024-05-14T11:16:16.337645+0000","last_became_peered":"2024-05-14T11:16:16.337645+0000","last_unstale":"2024-05-14T11:16:17.808398+0000","last_undegraded":"2024-05-14T11:16:17.808398+0000","last_fullsized":"2024-05-14T11:16:17.808398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:50:21.563593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806740+0000","last_change":"2024-05-14T11:16:16.337358+0000","last_active":"2024-05-14T11:16:17.806740+0000","last_peered":"2024-05-14T11:16:17.806740+0000","last_clean":"2024-05-14T11:16:17.806740+0000","last_became_active":"2024-05-14T11:16:16.336916+0000","last_became_peered":"2024-05-14T11:16:16.336916+0000","last_unstale":"2024-05-14T11:16:17.806740+0000","last_undegraded":"2024-05-14T11:16:17.806740+0000","last_fullsized":"2024-05-14T11:16:17.806740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806581+0000","last_change":"2024-05-14T11:16:12.267101+0000","last_active":"2024-05-14T11:16:17.806581+0000","last_peered":"2024-05-14T11:16:17.806581+0000","last_clean":"2024-05-14T11:16:17.806581+0000","last_became_active":"2024-05-14T11:16:12.266805+0000","last_became_peered":"2024-05-14T11:16:12.266805+0000","last_unstale":"2024-05-14T11:16:17.806581+0000","last_undegraded":"2024-05-14T11:16:17.806581+0000","last_fullsized":"2024-05-14T11:16:17.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806424+0000","last_change":"2024-05-14T11:16:11.259588+0000","last_active":"2024-05-14T11:16:17.806424+0000","last_peered":"2024-05-14T11:16:17.806424+0000","last_clean":"2024-05-14T11:16:17.806424+0000","last_became_active":"2024-05-14T11:16:11.259246+0000","last_became_peered":"2024-05-14T11:16:11.259246+0000","last_unstale":"2024-05-14T11:16:17.806424+0000","last_undegraded":"2024-05-14T11:16:17.806424+0000","last_fullsized":"2024-05-14T11:16:17.806424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301555+0000","last_change":"2024-05-14T11:16:16.299514+0000","last_active":"2024-05-14T11:16:17.301555+0000","last_peered":"2024-05-14T11:16:17.301555+0000","last_clean":"2024-05-14T11:16:17.301555+0000","last_became_active":"2024-05-14T11:16:16.299075+0000","last_became_peered":"2024-05-14T11:16:16.299075+0000","last_unstale":"2024-05-14T11:16:17.301555+0000","last_undegraded":"2024-05-14T11:16:17.301555+0000","last_fullsized":"2024-05-14T11:16:17.301555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808226+0000","last_change":"2024-05-14T11:16:12.258195+0000","last_active":"2024-05-14T11:16:17.808226+0000","last_peered":"2024-05-14T11:16:17.808226+0000","last_clean":"2024-05-14T11:16:17.808226+0000","last_became_active":"2024-05-14T11:16:12.257876+0000","last_became_peered":"2024-05-14T11:16:12.257876+0000","last_unstale":"2024-05-14T11:16:17.808226+0000","last_undegraded":"2024-05-14T11:16:17.808226+0000","last_fullsized":"2024-05-14T11:16:17.808226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808524+0000","last_change":"2024-05-14T11:16:11.246503+0000","last_active":"2024-05-14T11:16:17.808524+0000","last_peered":"2024-05-14T11:16:17.808524+0000","last_clean":"2024-05-14T11:16:17.808524+0000","last_became_active":"2024-05-14T11:16:11.246056+0000","last_became_peered":"2024-05-14T11:16:11.246056+0000","last_unstale":"2024-05-14T11:16:17.808524+0000","last_undegraded":"2024-05-14T11:16:17.808524+0000","last_fullsized":"2024-05-14T11:16:17.808524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300138+0000","last_change":"2024-05-14T11:16:11.258034+0000","last_active":"2024-05-14T11:16:17.300138+0000","last_peered":"2024-05-14T11:16:17.300138+0000","last_clean":"2024-05-14T11:16:17.300138+0000","last_became_active":"2024-05-14T11:16:11.257706+0000","last_became_peered":"2024-05-14T11:16:11.257706+0000","last_unstale":"2024-05-14T11:16:17.300138+0000","last_undegraded":"2024-05-14T11:16:17.300138+0000","last_fullsized":"2024-05-14T11:16:17.300138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806611+0000","last_change":"2024-05-14T11:16:12.248022+0000","last_active":"2024-05-14T11:16:17.806611+0000","last_peered":"2024-05-14T11:16:17.806611+0000","last_clean":"2024-05-14T11:16:17.806611+0000","last_became_active":"2024-05-14T11:16:12.247640+0000","last_became_peered":"2024-05-14T11:16:12.247640+0000","last_unstale":"2024-05-14T11:16:17.806611+0000","last_undegraded":"2024-05-14T11:16:17.806611+0000","last_fullsized":"2024-05-14T11:16:17.806611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298100+0000","last_change":"2024-05-14T11:16:16.296944+0000","last_active":"2024-05-14T11:16:17.298100+0000","last_peered":"2024-05-14T11:16:17.298100+0000","last_clean":"2024-05-14T11:16:17.298100+0000","last_became_active":"2024-05-14T11:16:16.296515+0000","last_became_peered":"2024-05-14T11:16:16.296515+0000","last_unstale":"2024-05-14T11:16:17.298100+0000","last_undegraded":"2024-05-14T11:16:17.298100+0000","last_fullsized":"2024-05-14T11:16:17.298100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:41.471078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809032+0000","last_change":"2024-05-14T11:16:11.253359+0000","last_active":"2024-05-14T11:16:17.809032+0000","last_peered":"2024-05-14T11:16:17.809032+0000","last_clean":"2024-05-14T11:16:17.809032+0000","last_became_active":"2024-05-14T11:16:11.248847+0000","last_became_peered":"2024-05-14T11:16:11.248847+0000","last_unstale":"2024-05-14T11:16:17.809032+0000","last_undegraded":"2024-05-14T11:16:17.809032+0000","last_fullsized":"2024-05-14T11:16:17.809032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301381+0000","last_change":"2024-05-14T11:16:12.239027+0000","last_active":"2024-05-14T11:16:17.301381+0000","last_peered":"2024-05-14T11:16:17.301381+0000","last_clean":"2024-05-14T11:16:17.301381+0000","last_became_active":"2024-05-14T11:16:12.238687+0000","last_became_peered":"2024-05-14T11:16:12.238687+0000","last_unstale":"2024-05-14T11:16:17.301381+0000","last_undegraded":"2024-05-14T11:16:17.301381+0000","last_fullsized":"2024-05-14T11:16:17.301381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.490412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809262+0000","last_change":"2024-05-14T11:16:16.293565+0000","last_active":"2024-05-14T11:16:17.809262+0000","last_peered":"2024-05-14T11:16:17.809262+0000","last_clean":"2024-05-14T11:16:17.809262+0000","last_became_active":"2024-05-14T11:16:16.293094+0000","last_became_peered":"2024-05-14T11:16:16.293094+0000","last_unstale":"2024-05-14T11:16:17.809262+0000","last_undegraded":"2024-05-14T11:16:17.809262+0000","last_fullsized":"2024-05-14T11:16:17.809262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:03:43.070705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299522+0000","last_change":"2024-05-14T11:16:12.249274+0000","last_active":"2024-05-14T11:16:17.299522+0000","last_peered":"2024-05-14T11:16:17.299522+0000","last_clean":"2024-05-14T11:16:17.299522+0000","last_became_active":"2024-05-14T11:16:12.248560+0000","last_became_peered":"2024-05-14T11:16:12.248560+0000","last_unstale":"2024-05-14T11:16:17.299522+0000","last_undegraded":"2024-05-14T11:16:17.299522+0000","last_fullsized":"2024-05-14T11:16:17.299522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297943+0000","last_change":"2024-05-14T11:16:12.267227+0000","last_active":"2024-05-14T11:16:17.297943+0000","last_peered":"2024-05-14T11:16:17.297943+0000","last_clean":"2024-05-14T11:16:17.297943+0000","last_became_active":"2024-05-14T11:16:12.266918+0000","last_became_peered":"2024-05-14T11:16:12.266918+0000","last_unstale":"2024-05-14T11:16:17.297943+0000","last_undegraded":"2024-05-14T11:16:17.297943+0000","last_fullsized":"2024-05-14T11:16:17.297943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807593+0000","last_change":"2024-05-14T11:16:12.259446+0000","last_active":"2024-05-14T11:16:17.807593+0000","last_peered":"2024-05-14T11:16:17.807593+0000","last_clean":"2024-05-14T11:16:17.807593+0000","last_became_active":"2024-05-14T11:16:12.259111+0000","last_became_peered":"2024-05-14T11:16:12.259111+0000","last_unstale":"2024-05-14T11:16:17.807593+0000","last_undegraded":"2024-05-14T11:16:17.807593+0000","last_fullsized":"2024-05-14T11:16:17.807593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806487+0000","last_change":"2024-05-14T11:16:12.259006+0000","last_active":"2024-05-14T11:16:17.806487+0000","last_peered":"2024-05-14T11:16:17.806487+0000","last_clean":"2024-05-14T11:16:17.806487+0000","last_became_active":"2024-05-14T11:16:12.258686+0000","last_became_peered":"2024-05-14T11:16:12.258686+0000","last_unstale":"2024-05-14T11:16:17.806487+0000","last_undegraded":"2024-05-14T11:16:17.806487+0000","last_fullsized":"2024-05-14T11:16:17.806487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808880+0000","last_change":"2024-05-14T11:16:12.263813+0000","last_active":"2024-05-14T11:16:17.808880+0000","last_peered":"2024-05-14T11:16:17.808880+0000","last_clean":"2024-05-14T11:16:17.808880+0000","last_became_active":"2024-05-14T11:16:12.263020+0000","last_became_peered":"2024-05-14T11:16:12.263020+0000","last_unstale":"2024-05-14T11:16:17.808880+0000","last_undegraded":"2024-05-14T11:16:17.808880+0000","last_fullsized":"2024-05-14T11:16:17.808880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807160+0000","last_change":"2024-05-14T11:16:12.265745+0000","last_active":"2024-05-14T11:16:17.807160+0000","last_peered":"2024-05-14T11:16:17.807160+0000","last_clean":"2024-05-14T11:16:17.807160+0000","last_became_active":"2024-05-14T11:16:12.262165+0000","last_became_peered":"2024-05-14T11:16:12.262165+0000","last_unstale":"2024-05-14T11:16:17.807160+0000","last_undegraded":"2024-05-14T11:16:17.807160+0000","last_fullsized":"2024-05-14T11:16:17.807160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301482+0000","last_change":"2024-05-14T11:16:12.250079+0000","last_active":"2024-05-14T11:16:17.301482+0000","last_peered":"2024-05-14T11:16:17.301482+0000","last_clean":"2024-05-14T11:16:17.301482+0000","last_became_active":"2024-05-14T11:16:12.246835+0000","last_became_peered":"2024-05-14T11:16:12.246835+0000","last_unstale":"2024-05-14T11:16:17.301482+0000","last_undegraded":"2024-05-14T11:16:17.301482+0000","last_fullsized":"2024-05-14T11:16:17.301482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809094+0000","last_change":"2024-05-14T11:16:12.264222+0000","last_active":"2024-05-14T11:16:17.809094+0000","last_peered":"2024-05-14T11:16:17.809094+0000","last_clean":"2024-05-14T11:16:17.809094+0000","last_became_active":"2024-05-14T11:16:12.263773+0000","last_became_peered":"2024-05-14T11:16:12.263773+0000","last_unstale":"2024-05-14T11:16:17.809094+0000","last_undegraded":"2024-05-14T11:16:17.809094+0000","last_fullsized":"2024-05-14T11:16:17.809094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806261+0000","last_change":"2024-05-14T11:16:12.260462+0000","last_active":"2024-05-14T11:16:17.806261+0000","last_peered":"2024-05-14T11:16:17.806261+0000","last_clean":"2024-05-14T11:16:17.806261+0000","last_became_active":"2024-05-14T11:16:12.260089+0000","last_became_peered":"2024-05-14T11:16:12.260089+0000","last_unstale":"2024-05-14T11:16:17.806261+0000","last_undegraded":"2024-05-14T11:16:17.806261+0000","last_fullsized":"2024-05-14T11:16:17.806261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808385+0000","last_change":"2024-05-14T11:16:12.257286+0000","last_active":"2024-05-14T11:16:17.808385+0000","last_peered":"2024-05-14T11:16:17.808385+0000","last_clean":"2024-05-14T11:16:17.808385+0000","last_became_active":"2024-05-14T11:16:12.256949+0000","last_became_peered":"2024-05-14T11:16:12.256949+0000","last_unstale":"2024-05-14T11:16:17.808385+0000","last_undegraded":"2024-05-14T11:16:17.808385+0000","last_fullsized":"2024-05-14T11:16:17.808385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807462+0000","last_change":"2024-05-14T11:16:12.266841+0000","last_active":"2024-05-14T11:16:17.807462+0000","last_peered":"2024-05-14T11:16:17.807462+0000","last_clean":"2024-05-14T11:16:17.807462+0000","last_became_active":"2024-05-14T11:16:12.266472+0000","last_became_peered":"2024-05-14T11:16:12.266472+0000","last_unstale":"2024-05-14T11:16:17.807462+0000","last_undegraded":"2024-05-14T11:16:17.807462+0000","last_fullsized":"2024-05-14T11:16:17.807462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806284+0000","last_change":"2024-05-14T11:16:12.245994+0000","last_active":"2024-05-14T11:16:17.806284+0000","last_peered":"2024-05-14T11:16:17.806284+0000","last_clean":"2024-05-14T11:16:17.806284+0000","last_became_active":"2024-05-14T11:16:12.243237+0000","last_became_peered":"2024-05-14T11:16:12.243237+0000","last_unstale":"2024-05-14T11:16:17.806284+0000","last_undegraded":"2024-05-14T11:16:17.806284+0000","last_fullsized":"2024-05-14T11:16:17.806284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300549+0000","last_change":"2024-05-14T11:16:12.240444+0000","last_active":"2024-05-14T11:16:17.300549+0000","last_peered":"2024-05-14T11:16:17.300549+0000","last_clean":"2024-05-14T11:16:17.300549+0000","last_became_active":"2024-05-14T11:16:12.240076+0000","last_became_peered":"2024-05-14T11:16:12.240076+0000","last_unstale":"2024-05-14T11:16:17.300549+0000","last_undegraded":"2024-05-14T11:16:17.300549+0000","last_fullsized":"2024-05-14T11:16:17.300549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807295+0000","last_change":"2024-05-14T11:16:12.243507+0000","last_active":"2024-05-14T11:16:17.807295+0000","last_peered":"2024-05-14T11:16:17.807295+0000","last_clean":"2024-05-14T11:16:17.807295+0000","last_became_active":"2024-05-14T11:16:12.243180+0000","last_became_peered":"2024-05-14T11:16:12.243180+0000","last_unstale":"2024-05-14T11:16:17.807295+0000","last_undegraded":"2024-05-14T11:16:17.807295+0000","last_fullsized":"2024-05-14T11:16:17.807295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298355+0000","last_change":"2024-05-14T11:16:12.250650+0000","last_active":"2024-05-14T11:16:17.298355+0000","last_peered":"2024-05-14T11:16:17.298355+0000","last_clean":"2024-05-14T11:16:17.298355+0000","last_became_active":"2024-05-14T11:16:12.250259+0000","last_became_peered":"2024-05-14T11:16:12.250259+0000","last_unstale":"2024-05-14T11:16:17.298355+0000","last_undegraded":"2024-05-14T11:16:17.298355+0000","last_fullsized":"2024-05-14T11:16:17.298355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807449+0000","last_change":"2024-05-14T11:16:12.267003+0000","last_active":"2024-05-14T11:16:17.807449+0000","last_peered":"2024-05-14T11:16:17.807449+0000","last_clean":"2024-05-14T11:16:17.807449+0000","last_became_active":"2024-05-14T11:16:12.266655+0000","last_became_peered":"2024-05-14T11:16:12.266655+0000","last_unstale":"2024-05-14T11:16:17.807449+0000","last_undegraded":"2024-05-14T11:16:17.807449+0000","last_fullsized":"2024-05-14T11:16:17.807449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806108+0000","last_change":"2024-05-14T11:16:12.251621+0000","last_active":"2024-05-14T11:16:17.806108+0000","last_peered":"2024-05-14T11:16:17.806108+0000","last_clean":"2024-05-14T11:16:17.806108+0000","last_became_active":"2024-05-14T11:16:12.251284+0000","last_became_peered":"2024-05-14T11:16:12.251284+0000","last_unstale":"2024-05-14T11:16:17.806108+0000","last_undegraded":"2024-05-14T11:16:17.806108+0000","last_fullsized":"2024-05-14T11:16:17.806108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806728+0000","last_change":"2024-05-14T11:16:12.247078+0000","last_active":"2024-05-14T11:16:17.806728+0000","last_peered":"2024-05-14T11:16:17.806728+0000","last_clean":"2024-05-14T11:16:17.806728+0000","last_became_active":"2024-05-14T11:16:12.246785+0000","last_became_peered":"2024-05-14T11:16:12.246785+0000","last_unstale":"2024-05-14T11:16:17.806728+0000","last_undegraded":"2024-05-14T11:16:17.806728+0000","last_fullsized":"2024-05-14T11:16:17.806728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806255+0000","last_change":"2024-05-14T11:16:12.246002+0000","last_active":"2024-05-14T11:16:17.806255+0000","last_peered":"2024-05-14T11:16:17.806255+0000","last_clean":"2024-05-14T11:16:17.806255+0000","last_became_active":"2024-05-14T11:16:12.245378+0000","last_became_peered":"2024-05-14T11:16:12.245378+0000","last_unstale":"2024-05-14T11:16:17.806255+0000","last_undegraded":"2024-05-14T11:16:17.806255+0000","last_fullsized":"2024-05-14T11:16:17.806255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:42.753210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808869+0000","last_change":"2024-05-14T11:16:12.264249+0000","last_active":"2024-05-14T11:16:17.808869+0000","last_peered":"2024-05-14T11:16:17.808869+0000","last_clean":"2024-05-14T11:16:17.808869+0000","last_became_active":"2024-05-14T11:16:12.263604+0000","last_became_peered":"2024-05-14T11:16:12.263604+0000","last_unstale":"2024-05-14T11:16:17.808869+0000","last_undegraded":"2024-05-14T11:16:17.808869+0000","last_fullsized":"2024-05-14T11:16:17.808869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302334+0000","last_change":"2024-05-14T11:16:12.239836+0000","last_active":"2024-05-14T11:16:17.302334+0000","last_peered":"2024-05-14T11:16:17.302334+0000","last_clean":"2024-05-14T11:16:17.302334+0000","last_became_active":"2024-05-14T11:16:12.236431+0000","last_became_peered":"2024-05-14T11:16:12.236431+0000","last_unstale":"2024-05-14T11:16:17.302334+0000","last_undegraded":"2024-05-14T11:16:17.302334+0000","last_fullsized":"2024-05-14T11:16:17.302334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:03.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300974+0000","last_change":"2024-05-14T11:16:12.273349+0000","last_active":"2024-05-14T11:16:17.300974+0000","last_peered":"2024-05-14T11:16:17.300974+0000","last_clean":"2024-05-14T11:16:17.300974+0000","last_became_active":"2024-05-14T11:16:12.272507+0000","last_became_peered":"2024-05-14T11:16:12.272507+0000","last_unstale":"2024-05-14T11:16:17.300974+0000","last_undegraded":"2024-05-14T11:16:17.300974+0000","last_fullsized":"2024-05-14T11:16:17.300974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301473+0000","last_change":"2024-05-14T11:16:12.257302+0000","last_active":"2024-05-14T11:16:17.301473+0000","last_peered":"2024-05-14T11:16:17.301473+0000","last_clean":"2024-05-14T11:16:17.301473+0000","last_became_active":"2024-05-14T11:16:12.256830+0000","last_became_peered":"2024-05-14T11:16:12.256830+0000","last_unstale":"2024-05-14T11:16:17.301473+0000","last_undegraded":"2024-05-14T11:16:17.301473+0000","last_fullsized":"2024-05-14T11:16:17.301473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807763+0000","last_change":"2024-05-14T11:16:12.264613+0000","last_active":"2024-05-14T11:16:17.807763+0000","last_peered":"2024-05-14T11:16:17.807763+0000","last_clean":"2024-05-14T11:16:17.807763+0000","last_became_active":"2024-05-14T11:16:12.264294+0000","last_became_peered":"2024-05-14T11:16:12.264294+0000","last_unstale":"2024-05-14T11:16:17.807763+0000","last_undegraded":"2024-05-14T11:16:17.807763+0000","last_fullsized":"2024-05-14T11:16:17.807763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:42.657656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808085+0000","last_change":"2024-05-14T11:16:12.267492+0000","last_active":"2024-05-14T11:16:17.808085+0000","last_peered":"2024-05-14T11:16:17.808085+0000","last_clean":"2024-05-14T11:16:17.808085+0000","last_became_active":"2024-05-14T11:16:12.267005+0000","last_became_peered":"2024-05-14T11:16:12.267005+0000","last_unstale":"2024-05-14T11:16:17.808085+0000","last_undegraded":"2024-05-14T11:16:17.808085+0000","last_fullsized":"2024-05-14T11:16:17.808085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807982+0000","last_change":"2024-05-14T11:16:12.252241+0000","last_active":"2024-05-14T11:16:17.807982+0000","last_peered":"2024-05-14T11:16:17.807982+0000","last_clean":"2024-05-14T11:16:17.807982+0000","last_became_active":"2024-05-14T11:16:12.251844+0000","last_became_peered":"2024-05-14T11:16:12.251844+0000","last_unstale":"2024-05-14T11:16:17.807982+0000","last_undegraded":"2024-05-14T11:16:17.807982+0000","last_fullsized":"2024-05-14T11:16:17.807982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807948+0000","last_change":"2024-05-14T11:16:12.254254+0000","last_active":"2024-05-14T11:16:17.807948+0000","last_peered":"2024-05-14T11:16:17.807948+0000","last_clean":"2024-05-14T11:16:17.807948+0000","last_became_active":"2024-05-14T11:16:12.253441+0000","last_became_peered":"2024-05-14T11:16:12.253441+0000","last_unstale":"2024-05-14T11:16:17.807948+0000","last_undegraded":"2024-05-14T11:16:17.807948+0000","last_fullsized":"2024-05-14T11:16:17.807948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807243+0000","last_change":"2024-05-14T11:16:12.244922+0000","last_active":"2024-05-14T11:16:17.807243+0000","last_peered":"2024-05-14T11:16:17.807243+0000","last_clean":"2024-05-14T11:16:17.807243+0000","last_became_active":"2024-05-14T11:16:12.244534+0000","last_became_peered":"2024-05-14T11:16:12.244534+0000","last_unstale":"2024-05-14T11:16:17.807243+0000","last_undegraded":"2024-05-14T11:16:17.807243+0000","last_fullsized":"2024-05-14T11:16:17.807243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806550+0000","last_change":"2024-05-14T11:16:12.270470+0000","last_active":"2024-05-14T11:16:17.806550+0000","last_peered":"2024-05-14T11:16:17.806550+0000","last_clean":"2024-05-14T11:16:17.806550+0000","last_became_active":"2024-05-14T11:16:12.269667+0000","last_became_peered":"2024-05-14T11:16:12.269667+0000","last_unstale":"2024-05-14T11:16:17.806550+0000","last_undegraded":"2024-05-14T11:16:17.806550+0000","last_fullsized":"2024-05-14T11:16:17.806550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806567+0000","last_change":"2024-05-14T11:16:12.254558+0000","last_active":"2024-05-14T11:16:17.806567+0000","last_peered":"2024-05-14T11:16:17.806567+0000","last_clean":"2024-05-14T11:16:17.806567+0000","last_became_active":"2024-05-14T11:16:12.254245+0000","last_became_peered":"2024-05-14T11:16:12.254245+0000","last_unstale":"2024-05-14T11:16:17.806567+0000","last_undegraded":"2024-05-14T11:16:17.806567+0000","last_fullsized":"2024-05-14T11:16:17.806567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807656+0000","last_change":"2024-05-14T11:16:12.256145+0000","last_active":"2024-05-14T11:16:17.807656+0000","last_peered":"2024-05-14T11:16:17.807656+0000","last_clean":"2024-05-14T11:16:17.807656+0000","last_became_active":"2024-05-14T11:16:12.255806+0000","last_became_peered":"2024-05-14T11:16:12.255806+0000","last_unstale":"2024-05-14T11:16:17.807656+0000","last_undegraded":"2024-05-14T11:16:17.807656+0000","last_fullsized":"2024-05-14T11:16:17.807656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807528+0000","last_change":"2024-05-14T11:16:12.252502+0000","last_active":"2024-05-14T11:16:17.807528+0000","last_peered":"2024-05-14T11:16:17.807528+0000","last_clean":"2024-05-14T11:16:17.807528+0000","last_became_active":"2024-05-14T11:16:12.251431+0000","last_became_peered":"2024-05-14T11:16:12.251431+0000","last_unstale":"2024-05-14T11:16:17.807528+0000","last_undegraded":"2024-05-14T11:16:17.807528+0000","last_fullsized":"2024-05-14T11:16:17.807528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:16:33.960 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:16:33.960 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T11:16:33.960 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-14T11:16:33.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-14T11:16:33.961 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-14T11:16:34.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 Processor -- start 2024-05-14T11:16:34.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 -- start start 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc156fa0 0x7fd3bc06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3bc0d1fc0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd3bc1573b0 con 0x7fd3bc0d1670 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c4200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd3bc161120 con 0x7fd3bc156fa0 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52170/0 (socket says 172.21.15.116:52170) 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 -- 172.21.15.116:0/1931164767 learned_addr learned my addr 172.21.15.116:0/1931164767 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c1800640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc156fa0 0x7fd3bc06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 -- 172.21.15.116:0/1931164767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc156fa0 msgr2=0x7fd3bc06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc156fa0 0x7fd3bc06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 -- 172.21.15.116:0/1931164767 >> v1:172.21.15.149:6789/0 conn(0x7fd3bc06e9e0 legacy=0x7fd3bc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.140+0000 7fd3c0e00640 1 -- 172.21.15.116:0/1931164767 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc160510 con 0x7fd3bc0d1670 2024-05-14T11:16:34.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c0e00640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009870 tx=0x7fd3a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79df715e91c77e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3bbe00640 1 -- 172.21.15.116:0/1931164767 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80aa430 con 0x7fd3bc0d1670 2024-05-14T11:16:34.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3bbe00640 1 -- 172.21.15.116:0/1931164767 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a80af070 con 0x7fd3bc0d1670 2024-05-14T11:16:34.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3bbe00640 1 -- 172.21.15.116:0/1931164767 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80aa7f0 con 0x7fd3bc0d1670 2024-05-14T11:16:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 -- 172.21.15.116:0/1931164767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc0d1670 msgr2=0x7fd3bc0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009870 tx=0x7fd3a801c4d0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 -- 172.21.15.116:0/1931164767 shutdown_connections 2024-05-14T11:16:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc0d1670 0x7fd3bc0d1a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 --2- 172.21.15.116:0/1931164767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc156fa0 0x7fd3bc06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 -- 172.21.15.116:0/1931164767 >> 172.21.15.116:0/1931164767 conn(0x7fd3bc0b4620 msgr2=0x7fd3bc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:34.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 -- 172.21.15.116:0/1931164767 shutdown_connections 2024-05-14T11:16:34.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.144+0000 7fd3c4200640 1 -- 172.21.15.116:0/1931164767 wait complete. 2024-05-14T11:16:34.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 Processor -- start 2024-05-14T11:16:34.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 -- start start 2024-05-14T11:16:34.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc0d1670 0x7fd3bc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 0x7fd3bc14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc070520 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc0b07a0 con 0x7fd3bc156fa0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c4200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd3bc0706a0 con 0x7fd3bc0d1670 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c1800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c1800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52182/0 (socket says 172.21.15.116:52182) 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c1800640 1 -- 172.21.15.116:0/3536157043 learned_addr learned my addr 172.21.15.116:0/3536157043 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c0e00640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc0d1670 0x7fd3bc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.148+0000 7fd3c2200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 0x7fd3bc14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 -- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc0d1670 msgr2=0x7fd3bc072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc0d1670 0x7fd3bc072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 -- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 msgr2=0x7fd3bc14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 0x7fd3bc14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 -- 172.21.15.116:0/3536157043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc14a1c0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c2200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 0x7fd3bc14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c1800640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d5f0 tx=0x7fd3b000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00125f0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0019070 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00129d0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc1698a0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc073490 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc06a9c0 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3b0012b70 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.152+0000 7fd3b9600640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 0x7fd39c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.156+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd3b008f910 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.156+0000 7fd3c0e00640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 0x7fd39c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.156+0000 7fd3c0e00640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 0x7fd39c071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ae000 tx=0x7fd3a80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.160+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd3b0023020 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.356+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd3bc15a830 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.356+0000 7fd3b9600640 1 -- 172.21.15.116:0/3536157043 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179550 (secure 0 0 0) 0x7fd3b00eed70 con 0x7fd3bc06e9e0 2024-05-14T11:16:34.363 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:34.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 msgr2=0x7fd39c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 0x7fd39c071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ae000 tx=0x7fd3a80a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 msgr2=0x7fd3bc16a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d5f0 tx=0x7fd3b000dab0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 shutdown_connections 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd39c06f400 0x7fd39c071910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc156fa0 0x7fd3bc14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3bc0d1670 0x7fd3bc072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 --2- 172.21.15.116:0/3536157043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3bc06e9e0 0x7fd3bc16a380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 >> 172.21.15.116:0/3536157043 conn(0x7fd3bc0b4620 msgr2=0x7fd3bc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 shutdown_connections 2024-05-14T11:16:34.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.360+0000 7fd3c4200640 1 -- 172.21.15.116:0/3536157043 wait complete. 2024-05-14T11:16:34.384 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1273922728},{"type":"v1","addr":"172.21.15.116:6833","nonce":1273922728}]},"active_addr":"172.21.15.116:6833/1273922728","active_change":"2024-05-14T11:15:52.344089+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":252065595}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":934213060}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2226418868}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2845135842}]}]} 2024-05-14T11:16:34.388 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-14T11:16:34.388 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-14T11:16:34.389 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:34.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 Processor -- start 2024-05-14T11:16:34.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 -- start start 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 0x7fe8ec0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 0x7fe8ec073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8ec074620 con 0x7fe8ec074210 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe8ec0cfeb0 con 0x7fe8ec0b5830 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8f0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec0ceab0 con 0x7fe8ec0d0900 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8ebe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe8ec074210 0x7fe8ec156560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39760/0 (socket says 172.21.15.116:39760) 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8ebe00640 1 -- 172.21.15.116:0/3792456023 learned_addr learned my addr 172.21.15.116:0/3792456023 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8e2a00640 1 --2- 172.21.15.116:0/3792456023 >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 0x7fe8ec073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eb400640 1 --2- 172.21.15.116:0/3792456023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 0x7fe8ec0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eaa00640 1 -- 172.21.15.116:0/3792456023 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2959950740 0 0) 0x7fe8e40014a0 con 0x7fe8ec074210 2024-05-14T11:16:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eaa00640 1 -- 172.21.15.116:0/3792456023 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8d0003530 con 0x7fe8ec074210 2024-05-14T11:16:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eb400640 1 -- 172.21.15.116:0/3792456023 >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 msgr2=0x7fe8ec073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eb400640 1 --2- 172.21.15.116:0/3792456023 >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 0x7fe8ec073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eb400640 1 -- 172.21.15.116:0/3792456023 >> v1:172.21.15.116:6789/0 conn(0x7fe8ec074210 legacy=0x7fe8ec156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8eb400640 1 -- 172.21.15.116:0/3792456023 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec0b5db0 con 0x7fe8ec0d0900 2024-05-14T11:16:34.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.536+0000 7fe8e2a00640 1 --2- 172.21.15.116:0/3792456023 >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 0x7fe8ec073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:34.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8eb400640 1 --2- 172.21.15.116:0/3792456023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 0x7fe8ec0b52f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0097a0 tx=0x7fe8dc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d201e83bb046c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8eaa00640 1 -- 172.21.15.116:0/3792456023 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0b1070 con 0x7fe8ec0d0900 2024-05-14T11:16:34.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8eaa00640 1 -- 172.21.15.116:0/3792456023 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc0ac620 con 0x7fe8ec0d0900 2024-05-14T11:16:34.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8eaa00640 1 -- 172.21.15.116:0/3792456023 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0ac9f0 con 0x7fe8ec0d0900 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 -- 172.21.15.116:0/3792456023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 msgr2=0x7fe8ec0b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/3792456023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 0x7fe8ec0b52f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0097a0 tx=0x7fe8dc01c3c0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 -- 172.21.15.116:0/3792456023 shutdown_connections 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/3792456023 >> v2:172.21.15.149:3300/0 conn(0x7fe8ec0b5830 0x7fe8ec073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/3792456023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0d0900 0x7fe8ec0b52f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 -- 172.21.15.116:0/3792456023 >> 172.21.15.116:0/3792456023 conn(0x7fe8ec0d26e0 msgr2=0x7fe8ec0d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 -- 172.21.15.116:0/3792456023 shutdown_connections 2024-05-14T11:16:34.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.540+0000 7fe8f0c00640 1 -- 172.21.15.116:0/3792456023 wait complete. 2024-05-14T11:16:34.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 Processor -- start 2024-05-14T11:16:34.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 -- start start 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0b5830 0x7fe8ec137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 0x7fe8ec161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec160e80 con 0x7fe8ec074210 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec1603e0 con 0x7fe8ec0d0900 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8f0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ec161000 con 0x7fe8ec0b5830 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8eb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8eb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52198/0 (socket says 172.21.15.116:52198) 2024-05-14T11:16:34.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8eb400640 1 -- 172.21.15.116:0/171206403 learned_addr learned my addr 172.21.15.116:0/171206403 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8e2a00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0b5830 0x7fe8ec137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.544+0000 7fe8ebe00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 0x7fe8ec161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 -- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0b5830 msgr2=0x7fe8ec137b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0b5830 0x7fe8ec137b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 -- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 msgr2=0x7fe8ec161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 0x7fe8ec161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 -- 172.21.15.116:0/171206403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec138410 con 0x7fe8ec074210 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8ebe00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 0x7fe8ec161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8eb400640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0af040 tx=0x7fe8dc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0b1040 con 0x7fe8ec074210 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc0ac420 con 0x7fe8ec074210 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec13a8f0 con 0x7fe8ec074210 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc005c00 con 0x7fe8ec074210 2024-05-14T11:16:34.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec13aeb0 con 0x7fe8ec074210 2024-05-14T11:16:34.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec1385e0 con 0x7fe8ec074210 2024-05-14T11:16:34.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8dc0bc020 con 0x7fe8ec074210 2024-05-14T11:16:34.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 0x7fe8c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.548+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe8dc128c90 con 0x7fe8ec074210 2024-05-14T11:16:34.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.552+0000 7fe8e2a00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 0x7fe8c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.552+0000 7fe8e2a00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 0x7fe8c8071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0b3220 tx=0x7fe8d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.556+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe8dc016070 con 0x7fe8ec074210 2024-05-14T11:16:34.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.712+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8ec050880 con 0x7fe8ec074210 2024-05-14T11:16:34.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.712+0000 7fe8e3e00640 1 -- 172.21.15.116:0/171206403 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fe8dc0f5830 con 0x7fe8ec074210 2024-05-14T11:16:34.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:34.717 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 msgr2=0x7fe8c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 0x7fe8c8071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0b3220 tx=0x7fe8d4009190 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 msgr2=0x7fe8ec1566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0af040 tx=0x7fe8dc019160 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 shutdown_connections 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe8c806f400 0x7fe8c8071910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8ec0d0900 0x7fe8ec161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8ec0b5830 0x7fe8ec137b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 --2- 172.21.15.116:0/171206403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8ec074210 0x7fe8ec1566e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 >> 172.21.15.116:0/171206403 conn(0x7fe8ec0d26e0 msgr2=0x7fe8ec0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:34.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 shutdown_connections 2024-05-14T11:16:34.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.716+0000 7fe8f0c00640 1 -- 172.21.15.116:0/171206403 wait complete. 2024-05-14T11:16:34.739 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-14T11:16:34.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 Processor -- start 2024-05-14T11:16:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 -- start start 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f15f015db30 0x7f15f00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f006ff20 con 0x7f15f006c700 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f0070470 con 0x7f15f00d1670 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f8000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f15f00ae100 con 0x7f15f015db30 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f6000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f15f006c700 0x7f15f006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39768/0 (socket says 172.21.15.116:39768) 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f6000640 1 -- 172.21.15.116:0/3409879357 learned_addr learned my addr 172.21.15.116:0/3409879357 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f5600640 1 --2- 172.21.15.116:0/3409879357 >> v2:172.21.15.149:3300/0 conn(0x7f15f015db30 0x7f15f00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15efe00640 1 -- 172.21.15.116:0/3409879357 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008071538 0 0) 0x7f15f006ff20 con 0x7f15f006c700 2024-05-14T11:16:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15efe00640 1 -- 172.21.15.116:0/3409879357 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15d8003530 con 0x7f15f006c700 2024-05-14T11:16:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f5600640 1 -- 172.21.15.116:0/3409879357 >> v1:172.21.15.149:6789/0 conn(0x7f15f00d1670 legacy=0x7f15f00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f5600640 1 -- 172.21.15.116:0/3409879357 >> v1:172.21.15.116:6789/0 conn(0x7f15f006c700 legacy=0x7f15f006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.884+0000 7f15f5600640 1 -- 172.21.15.116:0/3409879357 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f0070750 con 0x7f15f015db30 2024-05-14T11:16:34.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f5600640 1 --2- 172.21.15.116:0/3409879357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f00aeaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ae280 tx=0x7f15e401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b332fb1ce35a612 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15efe00640 1 -- 172.21.15.116:0/3409879357 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40b0070 con 0x7f15f015db30 2024-05-14T11:16:34.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15efe00640 1 -- 172.21.15.116:0/3409879357 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e40ab600 con 0x7f15f015db30 2024-05-14T11:16:34.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15efe00640 1 -- 172.21.15.116:0/3409879357 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40ab9d0 con 0x7f15f015db30 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 -- 172.21.15.116:0/3409879357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 msgr2=0x7f15f00aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 --2- 172.21.15.116:0/3409879357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f00aeaa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ae280 tx=0x7f15e401c410 comp rx=0 tx=0).stop 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 -- 172.21.15.116:0/3409879357 shutdown_connections 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 --2- 172.21.15.116:0/3409879357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f00aeaa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 -- 172.21.15.116:0/3409879357 >> 172.21.15.116:0/3409879357 conn(0x7f15f00b4620 msgr2=0x7f15f00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 -- 172.21.15.116:0/3409879357 shutdown_connections 2024-05-14T11:16:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.888+0000 7f15f8000640 1 -- 172.21.15.116:0/3409879357 wait complete. 2024-05-14T11:16:34.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 Processor -- start 2024-05-14T11:16:34.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 -- start start 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15f00d1670 0x7f15f0156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f0157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15f016bf90 con 0x7f15f006c700 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f15f015df40 con 0x7f15f015db30 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f15f016c110 con 0x7f15f00d1670 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52210/0 (socket says 172.21.15.116:52210) 2024-05-14T11:16:34.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 -- 172.21.15.116:0/1890900565 learned_addr learned my addr 172.21.15.116:0/1890900565 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f6000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f0157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f4c00640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15f00d1670 0x7f15f0156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 -- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15f00d1670 msgr2=0x7f15f0156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15f00d1670 0x7f15f0156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 -- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 msgr2=0x7f15f0157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f0157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 -- 172.21.15.116:0/1890900565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f00ce560 con 0x7f15f006c700 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f5600640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f15e401a040 tx=0x7f15e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40b0050 con 0x7f15f006c700 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e40bc070 con 0x7f15f006c700 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f00ce840 con 0x7f15f006c700 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.892+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4006e20 con 0x7f15f006c700 2024-05-14T11:16:34.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f00cee00 con 0x7f15f006c700 2024-05-14T11:16:34.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f006a9c0 con 0x7f15f006c700 2024-05-14T11:16:34.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15e40b5830 con 0x7f15f006c700 2024-05-14T11:16:34.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15ed600640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 0x7f15d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:34.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f15e4129c80 con 0x7f15f006c700 2024-05-14T11:16:34.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.896+0000 7f15f4c00640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 0x7f15d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:34.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.900+0000 7f15f4c00640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 0x7f15d0071940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15f0148d70 tx=0x7f15dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:34.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:34.904+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f15e40c6020 con 0x7f15f006c700 2024-05-14T11:16:35.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.060+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15f00b4b60 con 0x7f15f006c700 2024-05-14T11:16:35.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15ed600640 1 -- 172.21.15.116:0/1890900565 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f15e4016090 con 0x7f15f006c700 2024-05-14T11:16:35.066 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:35.066 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:35.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 msgr2=0x7f15d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 0x7f15d0071940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15f0148d70 tx=0x7f15dc009040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 msgr2=0x7f15f0148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f15e401a040 tx=0x7f15e401ac50 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 shutdown_connections 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15d006f430 0x7f15d0071940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15f015db30 0x7f15f0157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15f00d1670 0x7f15f0156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 --2- 172.21.15.116:0/1890900565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15f006c700 0x7f15f0148420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 >> 172.21.15.116:0/1890900565 conn(0x7f15f00b4620 msgr2=0x7f15f00d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 shutdown_connections 2024-05-14T11:16:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.064+0000 7f15f8000640 1 -- 172.21.15.116:0/1890900565 wait complete. 2024-05-14T11:16:35.088 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-14T11:16:35.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-14T11:16:35.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-14T11:16:35.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-14T11:16:35.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-14T11:16:35.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-14T11:16:35.091 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-14T11:16:35.091 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-14T11:16:35.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 Processor -- start 2024-05-14T11:16:35.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f184d000640 1 Processor -- start 2024-05-14T11:16:35.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f3f79c00640 1 Processor -- start 2024-05-14T11:16:35.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 -- start start 2024-05-14T11:16:35.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f3f79c00640 1 -- start start 2024-05-14T11:16:35.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f3f79c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f184d000640 1 -- start start 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f184d000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f184806f550 0x7f18480d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77c0074130 0x7f77c00b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c015d2e0 con 0x7f77c0157050 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c015d5c0 con 0x7f77c00b68b0 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f77bfe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f77c015d8a0 con 0x7f77c0074130 2024-05-14T11:16:35.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.356+0000 7f3f79c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3f74168680 0x7f3f740c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77c0074130 0x7f77c00b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f184d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f18480745c0 0x7f184815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77c0074130 0x7f77c00b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52216/0 (socket says 172.21.15.116:52216) 2024-05-14T11:16:35.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bf400640 1 -- 172.21.15.116:0/3311234605 learned_addr learned my addr 172.21.15.116:0/3311234605 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f79c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740d0f50 con 0x7f3f74168a90 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f79c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3f7409f100 con 0x7f3f74168680 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f79c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3f740d0d00 con 0x7f3f740af3a0 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3f74168680 0x7f3f740c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3f74168680 0x7f3f740c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52042/0 (socket says 172.21.15.116:52042) 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 -- 172.21.15.116:0/1515634710 learned_addr learned my addr 172.21.15.116:0/1515634710 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72a00640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f22f1a00640 1 Processor -- start 2024-05-14T11:16:35.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354691187 0 0) 0x7f77c015d5c0 con 0x7f77c00b68b0 2024-05-14T11:16:35.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f22f1a00640 1 -- start start 2024-05-14T11:16:35.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 -- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 msgr2=0x7f3f741640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 -- 172.21.15.116:0/1515634710 >> v1:172.21.15.116:6789/0 conn(0x7f3f74168a90 legacy=0x7f3f740cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a4003530 con 0x7f77c00b68b0 2024-05-14T11:16:35.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f3f72000640 1 -- 172.21.15.116:0/1515634710 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740d07b0 con 0x7f3f74168680 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4253549339 0 0) 0x7f77a4003530 con 0x7f77c00b68b0 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 >> v2:172.21.15.116:3300/0 conn(0x7f77c0074130 msgr2=0x7f77c00b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 --2- 172.21.15.116:0/3311234605 >> v2:172.21.15.116:3300/0 conn(0x7f77c0074130 0x7f77c00b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 >> v1:172.21.15.116:6789/0 conn(0x7f77c0157050 legacy=0x7f77c006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c015db80 con 0x7f77c00b68b0 2024-05-14T11:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f77ac001700 con 0x7f77c00b68b0 2024-05-14T11:16:35.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 >> v1:172.21.15.149:6789/0 conn(0x7f77c00b68b0 legacy=0x7f77c0073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f56c6c00640 1 Processor -- start 2024-05-14T11:16:35.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4003a60 0x7f77a4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 0x7f77a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f56c6c00640 1 -- start start 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a4005280 0x7f77a40141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77c015d8a0 con 0x7f77a4004810 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77a40149e0 con 0x7f77a4005280 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f77a4014be0 con 0x7f77a4003a60 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bf400640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a4005280 0x7f77a40141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 0x7f77a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 -- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4003a60 msgr2=0x7f77a4003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4003a60 0x7f77a4003eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 -- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a4005280 msgr2=0x7f77a40141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a4005280 0x7f77a40141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77be000640 1 -- 172.21.15.116:0/3311234605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4003530 con 0x7f77a4004810 2024-05-14T11:16:35.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f77bf400640 1 -- 172.21.15.116:0/3311234605 reap_dead start 2024-05-14T11:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 Processor -- start 2024-05-14T11:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 -- start start 2024-05-14T11:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22f1a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f22ec06dfa0 0x7f22ec06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22f1a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 0x7f22ec0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56c6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56c6c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56c00d1670 0x7f56c00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56c6c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c0161f30 con 0x7f56c006c700 2024-05-14T11:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56c6c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f56c00d1fc0 con 0x7f56c00d1670 2024-05-14T11:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56c6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56c0073550 con 0x7f56c0161b20 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22eaa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f22ec06dfa0 0x7f22ec06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22f1a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec06e8f0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22f1a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec06f250 con 0x7f22ec06dfa0 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22f1a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec153930 con 0x7f22ec0b5c90 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22e3e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f22ec0b4c30 0x7f22ec0b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34818/0 (socket says 172.21.15.116:34818) 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22e3e00640 1 -- 172.21.15.116:0/3396331877 learned_addr learned my addr 172.21.15.116:0/3396331877 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35340/0 (socket says 172.21.15.116:35340) 2024-05-14T11:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 -- 172.21.15.116:0/1025476589 learned_addr learned my addr 172.21.15.116:0/1025476589 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22eaa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f22ec06dfa0 0x7f22ec06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52062/0 (socket says 172.21.15.116:52062) 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f184d000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184806f960 con 0x7f184815c790 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f184d000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1848161110 con 0x7f18480745c0 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f184d000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1848160500 con 0x7f184806f550 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f18480745c0 0x7f184815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f18480745c0 0x7f184815cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52226/0 (socket says 172.21.15.116:52226) 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 -- 172.21.15.116:0/562715688 learned_addr learned my addr 172.21.15.116:0/562715688 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f1847400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f184806f550 0x7f18480d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22ea000640 1 -- 172.21.15.116:0/3396331877 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762660733 0 0) 0x7f22d4001120 con 0x7f22ec0b4c30 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.360+0000 7f1846a00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f184815c790 0x7f1848073eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34804/0 (socket says 172.21.15.116:34804) 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 -- 172.21.15.116:0/562715688 >> v2:172.21.15.149:3300/0 conn(0x7f184806f550 msgr2=0x7f18480d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 --2- 172.21.15.116:0/562715688 >> v2:172.21.15.149:3300/0 conn(0x7f184806f550 0x7f18480d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 -- 172.21.15.116:0/562715688 >> v1:172.21.15.149:6789/0 conn(0x7f184815c790 legacy=0x7f1848073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f1847e00640 1 -- 172.21.15.116:0/562715688 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481607e0 con 0x7f18480745c0 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9ef803a590 0x7f9ef8038520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ef80e25f0 0x7f9ef8038a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef80e2a00 con 0x7f9ef803a9a0 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef80510c0 con 0x7f9ef80e25f0 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9effe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef8050020 con 0x7f9ef803a590 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f3f72000640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168680 0x7f3f740c4200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f6001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e6cdce180843c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 -- 172.21.15.116:0/1025476589 >> v2:172.21.15.116:3300/0 conn(0x7f56c00d1670 msgr2=0x7f56c00d1a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T11:16:35.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 --2- 172.21.15.116:0/1025476589 >> v2:172.21.15.116:3300/0 conn(0x7f56c00d1670 0x7f56c00d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 -- 172.21.15.116:0/1025476589 >> v1:172.21.15.149:6789/0 conn(0x7f56c006c700 legacy=0x7f56c006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f56bfe00640 1 -- 172.21.15.116:0/1025476589 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c0164ad0 con 0x7f56c0161b20 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9efe000640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f9ef803a9a0 0x7f9ef80060a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34834/0 (socket says 172.21.15.116:34834) 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f9efe000640 1 -- 172.21.15.116:0/1963879940 learned_addr learned my addr 172.21.15.116:0/1963879940 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f3f71600640 1 -- 172.21.15.116:0/1515634710 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600b0070 con 0x7f3f74168680 2024-05-14T11:16:35.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77be000640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 0x7f77a4004c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c110 tx=0x7f77ac01f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b51fad0b9302ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f3f71600640 1 -- 172.21.15.116:0/1515634710 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f600ab600 con 0x7f3f74168680 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f3f71600640 1 -- 172.21.15.116:0/1515634710 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600ab9d0 con 0x7f3f74168680 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 --2- 172.21.15.116:0/1963879940 >> v2:172.21.15.149:3300/0 conn(0x7f9ef803a590 0x7f9ef8038520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9eff400640 1 --2- 172.21.15.116:0/1963879940 >> v2:172.21.15.116:3300/0 conn(0x7f9ef80e25f0 0x7f9ef8038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625656665 0 0) 0x7f9ef0001120 con 0x7f9ef803a9a0 2024-05-14T11:16:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee4003530 con 0x7f9ef803a9a0 2024-05-14T11:16:35.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0b5070 con 0x7f77a4004810 2024-05-14T11:16:35.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac0b0440 con 0x7f77a4004810 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 954321001 0 0) 0x7f9ee4003530 con 0x7f9ef803a9a0 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 >> v2:172.21.15.149:3300/0 conn(0x7f9ef803a590 msgr2=0x7f9ef8038520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 --2- 172.21.15.116:0/1963879940 >> v2:172.21.15.149:3300/0 conn(0x7f9ef803a590 0x7f9ef8038520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 >> v2:172.21.15.116:3300/0 conn(0x7f9ef80e25f0 msgr2=0x7f9ef8038a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 --2- 172.21.15.116:0/1963879940 >> v2:172.21.15.116:3300/0 conn(0x7f9ef80e25f0 0x7f9ef8038a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef8050300 con 0x7f9ef803a9a0 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9ef00018d0 con 0x7f9ef803a9a0 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 >> v1:172.21.15.149:6789/0 conn(0x7f9ef803a9a0 legacy=0x7f9ef80060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 0x7f9ee4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4004810 0x7f9ee4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4005280 0x7f9ee400fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80510c0 con 0x7f9ee4004810 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8050020 con 0x7f9ee4005280 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee4010630 con 0x7f9ee4003a60 2024-05-14T11:16:35.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 0x7f9ee4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f3f79c00640 1 -- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168680 msgr2=0x7f3f740c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f3f79c00640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168680 0x7f3f740c4200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f6001c4d0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bd600640 1 -- 172.21.15.116:0/3311234605 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007c10 con 0x7f77a4004810 2024-05-14T11:16:35.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f22ea000640 1 -- 172.21.15.116:0/3396331877 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d0003530 con 0x7f22ec0b4c30 2024-05-14T11:16:35.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f22eaa00640 1 -- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 msgr2=0x7f22ec0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.364+0000 7f22eb400640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 0x7f22ec0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 -- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4005280 msgr2=0x7f9ee400fde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4005280 0x7f9ee400fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f22eaa00640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 0x7f22ec0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f22eaa00640 1 -- 172.21.15.116:0/3396331877 >> v1:172.21.15.149:6789/0 conn(0x7f22ec0b4c30 legacy=0x7f22ec0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f22eaa00640 1 -- 172.21.15.116:0/3396331877 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0b5750 con 0x7f22ec06dfa0 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 -- 172.21.15.116:0/1963879940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4004810 msgr2=0x7f9ee4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4004810 0x7f9ee4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 -- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 msgr2=0x7f77a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 0x7f77a4004c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c110 tx=0x7f77ac01f1d0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9efea00640 1 -- 172.21.15.116:0/1963879940 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef8050300 con 0x7f9ee4003a60 2024-05-14T11:16:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f9eff400640 1 -- 172.21.15.116:0/1963879940 reap_dead start 2024-05-14T11:16:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 -- 172.21.15.116:0/3311234605 shutdown_connections 2024-05-14T11:16:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 --2- 172.21.15.116:0/3311234605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a4004810 0x7f77a4004c20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 -- 172.21.15.116:0/3311234605 >> 172.21.15.116:0/3311234605 conn(0x7f77c00d2380 msgr2=0x7f77c00d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 -- 172.21.15.116:0/3311234605 shutdown_connections 2024-05-14T11:16:35.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f1847e00640 1 --2- 172.21.15.116:0/562715688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18480745c0 0x7f184815cf90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1848074c80 tx=0x7f1840020d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cdff5b5f3a0c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f56bfe00640 1 --2- 172.21.15.116:0/1025476589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009870 tx=0x7f56ac01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1177c927588faf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 -- 172.21.15.116:0/1515634710 shutdown_connections 2024-05-14T11:16:35.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168680 0x7f3f740c4200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 --2- 172.21.15.116:0/1515634710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 -- 172.21.15.116:0/1515634710 >> 172.21.15.116:0/1515634710 conn(0x7f3f740cde80 msgr2=0x7f3f740ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f1846000640 1 -- 172.21.15.116:0/562715688 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400a9d40 con 0x7f18480745c0 2024-05-14T11:16:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f22eb400640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 0x7f22ec0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56bea00640 1 -- 172.21.15.116:0/1025476589 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b0070 con 0x7f56c0161b20 2024-05-14T11:16:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56bea00640 1 -- 172.21.15.116:0/1025476589 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac0ab600 con 0x7f56c0161b20 2024-05-14T11:16:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56bea00640 1 -- 172.21.15.116:0/1025476589 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0ab9d0 con 0x7f56c0161b20 2024-05-14T11:16:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f1846000640 1 -- 172.21.15.116:0/562715688 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184000d0d0 con 0x7f18480745c0 2024-05-14T11:16:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7fe58c400640 1 Processor -- start 2024-05-14T11:16:35.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f1846000640 1 -- 172.21.15.116:0/562715688 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000d4a0 con 0x7f18480745c0 2024-05-14T11:16:35.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 -- 172.21.15.116:0/1025476589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 msgr2=0x7f56c0168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1025476589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009870 tx=0x7f56ac01c380 comp rx=0 tx=0).stop 2024-05-14T11:16:35.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.368+0000 7f77bfe00640 1 -- 172.21.15.116:0/3311234605 wait complete. 2024-05-14T11:16:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 -- 172.21.15.116:0/1515634710 shutdown_connections 2024-05-14T11:16:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f3f79c00640 1 -- 172.21.15.116:0/1515634710 wait complete. 2024-05-14T11:16:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 -- 172.21.15.116:0/562715688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18480745c0 msgr2=0x7f184815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 --2- 172.21.15.116:0/562715688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18480745c0 0x7f184815cf90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1848074c80 tx=0x7f1840020d60 comp rx=0 tx=0).stop 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 -- 172.21.15.116:0/1025476589 shutdown_connections 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1025476589 >> v2:172.21.15.116:3300/0 conn(0x7f56c00d1670 0x7f56c00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1025476589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c0161b20 0x7f56c0168580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 -- 172.21.15.116:0/1025476589 >> 172.21.15.116:0/1025476589 conn(0x7f56c00b4620 msgr2=0x7f56c00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 -- 172.21.15.116:0/562715688 shutdown_connections 2024-05-14T11:16:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7fe58c400640 1 -- start start 2024-05-14T11:16:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 --2- 172.21.15.116:0/562715688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18480745c0 0x7f184815cf90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 --2- 172.21.15.116:0/562715688 >> v2:172.21.15.149:3300/0 conn(0x7f184806f550 0x7f18480d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 -- 172.21.15.116:0/1025476589 shutdown_connections 2024-05-14T11:16:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f56c6c00640 1 -- 172.21.15.116:0/1025476589 wait complete. 2024-05-14T11:16:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 -- 172.21.15.116:0/562715688 >> 172.21.15.116:0/562715688 conn(0x7f18480d28e0 msgr2=0x7f18480d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f77bfe00640 1 Processor -- start 2024-05-14T11:16:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 -- 172.21.15.116:0/562715688 shutdown_connections 2024-05-14T11:16:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f184d000640 1 -- 172.21.15.116:0/562715688 wait complete. 2024-05-14T11:16:35.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.372+0000 7f77bfe00640 1 -- start start 2024-05-14T11:16:35.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9efea00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 0x7f9ee4003ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009ab0 tx=0x7f9eec01ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99bc8d006caa6ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0b2070 con 0x7f9ee4003a60 2024-05-14T11:16:35.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec0a5530 con 0x7f9ee4003a60 2024-05-14T11:16:35.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9efd600640 1 -- 172.21.15.116:0/1963879940 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0096b0 con 0x7f9ee4003a60 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f3f79c00640 1 Processor -- start 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f22eaa00640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec06e3b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0097c0 tx=0x7f22dc01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce52789cd7929882 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f22ea000640 1 -- 172.21.15.116:0/3396331877 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0b0070 con 0x7f22ec06dfa0 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58c400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe5840d1d80 0x7fe584168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58c400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 0x7fe58406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58c400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe584161f30 con 0x7fe584161b20 2024-05-14T11:16:35.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58c400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe584073550 con 0x7fe5840d1d80 2024-05-14T11:16:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58c400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe584164ad0 con 0x7fe58406f620 2024-05-14T11:16:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58a400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 0x7fe58406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f3f79c00640 1 -- start start 2024-05-14T11:16:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f22ea000640 1 -- 172.21.15.116:0/3396331877 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc0ab600 con 0x7f22ec06dfa0 2024-05-14T11:16:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f22ea000640 1 -- 172.21.15.116:0/3396331877 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0ab9d0 con 0x7f22ec06dfa0 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4004810 0x7f77c0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 0x7f77c0201f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77c015b6d0 0x7f77c015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b7050 con 0x7f77c00b3890 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b6ed0 con 0x7f77c015b6d0 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bfe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f77c006ecd0 con 0x7f77a4004810 2024-05-14T11:16:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f56c6c00640 1 Processor -- start 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9effe00640 1 -- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 msgr2=0x7f9ee4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9effe00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 0x7f9ee4003ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009ab0 tx=0x7f9eec01ca50 comp rx=0 tx=0).stop 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f56c6c00640 1 -- start start 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe5840d1d80 0x7fe584168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe584161b20 0x7fe5840d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39804/0 (socket says 172.21.15.116:39804) 2024-05-14T11:16:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589a00640 1 -- 172.21.15.116:0/249580802 learned_addr learned my addr 172.21.15.116:0/249580802 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 0x7f77c0201f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 0x7f77c0201f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52274/0 (socket says 172.21.15.116:52274) 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 -- 172.21.15.116:0/2807947863 learned_addr learned my addr 172.21.15.116:0/2807947863 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bf400640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77c015b6d0 0x7f77c015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bea00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4004810 0x7f77c0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 -- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4004810 msgr2=0x7f77c0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4004810 0x7f77c0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 -- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77c015b6d0 msgr2=0x7f77c015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77c015b6d0 0x7f77c015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c015c020 con 0x7f77c00b3890 2024-05-14T11:16:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589000640 1 -- 172.21.15.116:0/249580802 >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 msgr2=0x7fe58406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77bea00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77a4004810 0x7f77c0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fd80f200640 1 Processor -- start 2024-05-14T11:16:35.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9effe00640 1 -- 172.21.15.116:0/1963879940 shutdown_connections 2024-05-14T11:16:35.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9effe00640 1 --2- 172.21.15.116:0/1963879940 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4003a60 0x7f9ee4003ed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f9effe00640 1 -- 172.21.15.116:0/1963879940 >> 172.21.15.116:0/1963879940 conn(0x7f9ef8054d00 msgr2=0x7f9ef8055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7f77be000640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 0x7f77c0201f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c200 tx=0x7f77ac01c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 -- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 msgr2=0x7f22ec06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f184d000640 1 Processor -- start 2024-05-14T11:16:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec06e3b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0097c0 tx=0x7f22dc01c430 comp rx=0 tx=0).stop 2024-05-14T11:16:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589000640 1 --2- 172.21.15.116:0/249580802 >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 0x7fe58406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589000640 1 -- 172.21.15.116:0/249580802 >> v1:172.21.15.116:6789/0 conn(0x7fe584161b20 legacy=0x7fe5840d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe589000640 1 -- 172.21.15.116:0/249580802 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584164db0 con 0x7fe5840d1d80 2024-05-14T11:16:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fe58a400640 1 --2- 172.21.15.116:0/249580802 >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 0x7fe58406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0b5040 con 0x7f77c00b3890 2024-05-14T11:16:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac005110 con 0x7f77c00b3890 2024-05-14T11:16:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac003600 con 0x7f77c00b3890 2024-05-14T11:16:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f184d000640 1 -- start start 2024-05-14T11:16:35.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77bfe00640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c01599f0 con 0x7f77c00b3890 2024-05-14T11:16:35.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.376+0000 7fd80f200640 1 -- start start 2024-05-14T11:16:35.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77bfe00640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c0159fb0 con 0x7f77c00b3890 2024-05-14T11:16:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f9effe00640 1 -- 172.21.15.116:0/1963879940 shutdown_connections 2024-05-14T11:16:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c006c700 0x7f56c014b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c00d1670 0x7f56c015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f9effe00640 1 -- 172.21.15.116:0/1963879940 wait complete. 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c0161b20 0x7f56c015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 -- 172.21.15.116:0/3396331877 shutdown_connections 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b5c90 0x7f22ec0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 --2- 172.21.15.116:0/3396331877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec06e3b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 -- 172.21.15.116:0/3396331877 >> 172.21.15.116:0/3396331877 conn(0x7f22ec073210 msgr2=0x7f22ec073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0165370 con 0x7f56c006c700 2024-05-14T11:16:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0165090 con 0x7f56c0161b20 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56c0070630 con 0x7f56c00d1670 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56bf400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c00d1670 0x7f56c015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56bf400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c00d1670 0x7f56c015a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35372/0 (socket says 172.21.15.116:35372) 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56bf400640 1 -- 172.21.15.116:0/1410428737 learned_addr learned my addr 172.21.15.116:0/1410428737 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c0161b20 0x7f56c015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77c00b3ca0 con 0x7f77c00b3890 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 -- 172.21.15.116:0/3396331877 shutdown_connections 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f22f1a00640 1 -- 172.21.15.116:0/3396331877 wait complete. 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f74168680 0x7f3f7415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168a90 0x7f3f74164880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740c4b50 con 0x7f3f74168680 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741681f0 con 0x7f3f74168a90 2024-05-14T11:16:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f79c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3f74171990 con 0x7f3f740af3a0 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 -- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c00d1670 msgr2=0x7f56c015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56c00d1670 0x7f56c015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 -- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c006c700 msgr2=0x7f56c014b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c006c700 0x7f56c014b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c015b4b0 con 0x7f56c0161b20 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f72000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f74168680 0x7f3f7415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f72000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f74168680 0x7f3f7415f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52294/0 (socket says 172.21.15.116:52294) 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f3f72000640 1 -- 172.21.15.116:0/3487342575 learned_addr learned my addr 172.21.15.116:0/3487342575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.380+0000 7f56c4c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c0161b20 0x7f56c015aa30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56b800fc50 tx=0x7f56b800d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 Processor -- start 2024-05-14T11:16:35.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b801a070 con 0x7f56c0161b20 2024-05-14T11:16:35.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f73400640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168a90 0x7f3f74164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c00b1a90 con 0x7f56c0161b20 2024-05-14T11:16:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 -- start start 2024-05-14T11:16:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 -- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168a90 msgr2=0x7f3f74164880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f74168a90 0x7f3f74164880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 -- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f74168680 msgr2=0x7f3f7415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f74168680 0x7f3f7415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7415fc50 con 0x7f3f740af3a0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f72a00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741fecd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400e870 tx=0x7f3f6400ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64017070 con 0x7f3f740af3a0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6400c940 con 0x7f3f740af3a0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7415b540 con 0x7f3f740af3a0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400ccf0 con 0x7f3f740af3a0 2024-05-14T11:16:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77ac0037a0 con 0x7f77c00b3890 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7415bb00 con 0x7f3f740af3a0 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f184d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184806f550 0x7f18481559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c00b2050 con 0x7f56c0161b20 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b80069e0 con 0x7f56c0161b20 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800fe60 con 0x7f56c0161b20 2024-05-14T11:16:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f22f1a00640 1 Processor -- start 2024-05-14T11:16:35.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5698001040 con 0x7f56c0161b20 2024-05-14T11:16:35.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f184d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 0x7f1848150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7fd80f200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd808161b20 0x7fd808168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4003a60 0x7f9ef80cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f22f1a00640 1 -- start start 2024-05-14T11:16:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7fd80f200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd808161f30 con 0x7fd80806f620 2024-05-14T11:16:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7fd80f200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd808073550 con 0x7fd8080d1670 2024-05-14T11:16:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7fd80f200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd808164ad0 con 0x7fd808161b20 2024-05-14T11:16:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f4c001040 con 0x7f3f740af3a0 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 0x7f9ef80c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ef80c93c0 0x7f9ef80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8050ba0 con 0x7f9ee4003a60 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80508c0 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f184d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1848153f70 0x7f184814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f9effe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef8038520 con 0x7f9ef80c93c0 2024-05-14T11:16:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f184d000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f184806d110 con 0x7f184806f550 2024-05-14T11:16:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f1846a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 0x7f1848150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.384+0000 7f184d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f184806cf90 con 0x7f1848153f70 2024-05-14T11:16:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 0x7f1848150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35398/0 (socket says 172.21.15.116:35398) 2024-05-14T11:16:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 -- 172.21.15.116:0/4107214107 learned_addr learned my addr 172.21.15.116:0/4107214107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1847e00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1848153f70 0x7f184814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77a006f430 0x7f77a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f184d000640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f184806d290 con 0x7f184815c790 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 0x7f9ef80c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 0x7f9ef80c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52134/0 (socket says 172.21.15.116:52134) 2024-05-14T11:16:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 -- 172.21.15.116:0/3182533602 learned_addr learned my addr 172.21.15.116:0/3182533602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f77ac12cd50 con 0x7f77c00b3890 2024-05-14T11:16:35.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1847400640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184806f550 0x7f18481559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f77a0078c00 0x7f77a007b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fe589000640 1 --2- 172.21.15.116:0/249580802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5840d1d80 0x7fe584168580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe584070c70 tx=0x7fe57401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d48422970c2535 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd807e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd808161b20 0x7fd808168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 -- 172.21.15.116:0/3182533602 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ef80c93c0 msgr2=0x7f9ef80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ef80c93c0 0x7f9ef80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 -- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4003a60 msgr2=0x7f9ef80cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee4003a60 0x7f9ef80cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77ac0aaea0 con 0x7f77a0078c00 2024-05-14T11:16:35.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f77ac12d140 con 0x7f77c00b3890 2024-05-14T11:16:35.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef80cce90 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd807400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fd8080d1670 0x7fd8080d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34840/0 (socket says 172.21.15.116:34840) 2024-05-14T11:16:35.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd807400640 1 -- 172.21.15.116:0/565227285 learned_addr learned my addr 172.21.15.116:0/565227285 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77bf400640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f77a0078c00 0x7f77a007b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9efe000640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 0x7f9ef80c9900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000f390 tx=0x7f9ef000f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec1f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 0x7f22ec162970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22ec0b5c90 0x7f22ec14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0b65e0 con 0x7f22ec0b5c90 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec15c930 con 0x7f22ec06dfa0 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22f1a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0b6760 con 0x7f22ec0b4c30 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446042620 0 0) 0x7fd808073550 con 0x7fd8080d1670 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7ec003530 con 0x7fd8080d1670 2024-05-14T11:16:35.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77bea00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77a006f430 0x7f77a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56b801d070 con 0x7f56c0161b20 2024-05-14T11:16:35.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77bf400640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f77a0078c00 0x7f77a007b090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001a070 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 0x7f22ec162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 0x7f22ec162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35406/0 (socket says 172.21.15.116:35406) 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 -- 172.21.15.116:0/2532797346 learned_addr learned my addr 172.21.15.116:0/2532797346 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec1f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef0008d10 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000a2f0 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244694883 0 0) 0x7fd7ec003530 con 0x7fd8080d1670 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 >> v2:172.21.15.116:3300/0 conn(0x7fd808161b20 msgr2=0x7fd808168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 --2- 172.21.15.116:0/565227285 >> v2:172.21.15.116:3300/0 conn(0x7fd808161b20 0x7fd808168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 >> v1:172.21.15.116:6789/0 conn(0x7fd80806f620 legacy=0x7fd80806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd808164db0 con 0x7fd8080d1670 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fd7f8001700 con 0x7fd8080d1670 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 >> v1:172.21.15.149:6789/0 conn(0x7fd8080d1670 legacy=0x7fd8080d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7ec003ab0 0x7fd7ec003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77ac0aaea0 con 0x7f77a0078c00 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 -- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1848153f70 msgr2=0x7f184814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1848153f70 0x7f184814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 -- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184806f550 msgr2=0x7f18481559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184806f550 0x7f18481559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f1846a00640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1848156ae0 con 0x7f184815c790 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef80dc340 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef80cd410 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 -- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 msgr2=0x7f22ec1f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec06dfa0 0x7f22ec1f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 -- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22ec0b5c90 msgr2=0x7f22ec14e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22ec0b5c90 0x7f22ec14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1f8cd0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f22e3e00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 0x7f22ec162970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c700 tx=0x7f22d400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4017070 con 0x7f22ec0b4c30 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d4001d40 con 0x7f22ec0b4c30 2024-05-14T11:16:35.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4005cc0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f56b7e00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56a006f430 0x7f56a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f56b8092770 con 0x7f56c0161b20 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56b7e00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f56a0078bf0 0x7f56a007b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56b801de50 con 0x7f56a0078bf0 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f56b8092b60 con 0x7f56c0161b20 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7ec004810 0x7fd7ec004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd7ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd808164ad0 con 0x7fd7ec004810 2024-05-14T11:16:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00c2f0 con 0x7fd7ec005360 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ec00c4f0 con 0x7fd7ec003ab0 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.388+0000 7fd80d200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd7ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd807e00640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7ec003ab0 0x7fd7ec003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd80d200640 1 -- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7ec003ab0 msgr2=0x7fd7ec003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd80d200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7ec003ab0 0x7fd7ec003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd80d200640 1 -- 172.21.15.116:0/565227285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7ec004810 msgr2=0x7fd7ec004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd80d200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7ec004810 0x7fd7ec004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd80d200640 1 -- 172.21.15.116:0/565227285 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec003530 con 0x7fd7ec005360 2024-05-14T11:16:35.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f1846a00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 0x7f1848150cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f183800c3f0 tx=0x7f183800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22f1a00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1f91e0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22f1a00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1f97a0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56bfe00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56a006f430 0x7f56a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56bf400640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f56a0078bf0 0x7f56a007b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7fd807e00640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7ec003ab0 0x7fd7ec003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f22f1a00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22ec06a9c0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380074e0 con 0x7f184815c790 2024-05-14T11:16:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838007680 con 0x7f184815c790 2024-05-14T11:16:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380049f0 con 0x7f184815c790 2024-05-14T11:16:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f56bf400640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f56a0078bf0 0x7f56a007b080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f184d000640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1848156240 con 0x7f184815c790 2024-05-14T11:16:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f184d000640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1848156800 con 0x7f184815c790 2024-05-14T11:16:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ef001d070 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f77bea00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77a006f430 0x7f77a0071940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77c006a000 tx=0x7f77b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.392+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ef8005580 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56b801de50 con 0x7f56a0078bf0 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f56bfe00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56a006f430 0x7f56a0071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ad000 tx=0x7f56ac004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fe583e00640 1 -- 172.21.15.116:0/249580802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740b0070 con 0x7fe5840d1d80 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fe583e00640 1 -- 172.21.15.116:0/249580802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5740ab600 con 0x7fe5840d1d80 2024-05-14T11:16:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fe583e00640 1 -- 172.21.15.116:0/249580802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740ab9d0 con 0x7fe5840d1d80 2024-05-14T11:16:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f6401a070 con 0x7f3f740af3a0 2024-05-14T11:16:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9ef6a00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ee006f430 0x7f9ee0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1818001040 con 0x7f184815c790 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9ef0091820 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9ef6a00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7f9ee0078af0 0x7f9ee007af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ee007b650 con 0x7f9ee0078af0 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9ef0098050 con 0x7f9ef80c8fb0 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9efea00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ee006f430 0x7f9ee0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fd80d200640 1 -- 172.21.15.116:0/565227285 reap_dead start 2024-05-14T11:16:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9eff400640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7f9ee0078af0 0x7f9ee007af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22d401a070 con 0x7f22ec0b4c30 2024-05-14T11:16:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22c806f450 0x7f22c8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fd80d200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd7ec00ba10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd800001dc0 tx=0x7fd80001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d81ade6e39f8c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3f5406f4d0 0x7f3f540719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f183801a020 con 0x7f184815c790 2024-05-14T11:16:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9eff400640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7f9ee0078af0 0x7f9ee007af80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f22d408ecb0 con 0x7f22ec0b4c30 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7f22c8078c10 0x7f22c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22d401dec0 con 0x7f22c8078c10 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f22d408f060 con 0x7f22ec0b4c30 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f6408fb50 con 0x7f3f740af3a0 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f72000640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3f5406f4d0 0x7f3f540719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22c806f450 0x7f22c8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800001a10 con 0x7fd7ec005360 2024-05-14T11:16:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f3f54078c90 0x7f3f5407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f64023e70 con 0x7f3f54078c90 2024-05-14T11:16:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f6408ff40 con 0x7f3f740af3a0 2024-05-14T11:16:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.396+0000 7f9efea00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ee006f430 0x7f9ee0071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0af000 tx=0x7f9eec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f22eb400640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7f22c8078c10 0x7f22c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f183f400640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f182406f500 0x7f1824071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f3f73400640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f3f54078c90 0x7f3f5407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f183808ebb0 con 0x7f184815c790 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f183f400640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f1824078cc0 0x7f182407b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1838017e80 con 0x7f1824078cc0 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f18380932a0 con 0x7f184815c790 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd800003b80 con 0x7fd7ec005360 2024-05-14T11:16:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fd806a00640 1 -- 172.21.15.116:0/565227285 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800001a10 con 0x7fd7ec005360 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 -- 172.21.15.116:0/249580802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5840d1d80 msgr2=0x7fe584168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 --2- 172.21.15.116:0/249580802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5840d1d80 0x7fe584168580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe584070c70 tx=0x7fe57401c210 comp rx=0 tx=0).stop 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f3f72000640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3f5406f4d0 0x7f3f540719e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f3f73400640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f3f54078c90 0x7f3f5407b120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 -- 172.21.15.116:0/249580802 shutdown_connections 2024-05-14T11:16:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 --2- 172.21.15.116:0/249580802 >> v2:172.21.15.149:3300/0 conn(0x7fe58406f620 0x7fe58406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 --2- 172.21.15.116:0/249580802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5840d1d80 0x7fe584168580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 -- 172.21.15.116:0/249580802 >> 172.21.15.116:0/249580802 conn(0x7fe5840b4620 msgr2=0x7fe5840b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22c806f450 0x7f22c8071960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0ad000 tx=0x7f22dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 -- 172.21.15.116:0/249580802 shutdown_connections 2024-05-14T11:16:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f22eb400640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7f22c8078c10 0x7f22c807b0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f1847400640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f182406f500 0x7f1824071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f1847e00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f1824078cc0 0x7f182407b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fe58c400640 1 -- 172.21.15.116:0/249580802 wait complete. 2024-05-14T11:16:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ee007b650 con 0x7f9ee0078af0 2024-05-14T11:16:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fd80f200640 1 -- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 msgr2=0x7fd7ec00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7fd80f200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd7ec00ba10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd800001dc0 tx=0x7fd80001c4d0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22d401dec0 con 0x7f22c8078c10 2024-05-14T11:16:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.400+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f64023e70 con 0x7f3f54078c90 2024-05-14T11:16:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7f1847400640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f182406f500 0x7f1824071a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18480b5660 tx=0x7f1830008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7f1847e00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f1824078cc0 0x7f182407b150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1838017e80 con 0x7f1824078cc0 2024-05-14T11:16:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fe58c400640 1 Processor -- start 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fd80f200640 1 -- 172.21.15.116:0/565227285 shutdown_connections 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fd80f200640 1 --2- 172.21.15.116:0/565227285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd7ec00ba10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fd80f200640 1 -- 172.21.15.116:0/565227285 >> 172.21.15.116:0/565227285 conn(0x7fd8080b4620 msgr2=0x7fd8080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fd80f200640 1 -- 172.21.15.116:0/565227285 shutdown_connections 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fd80f200640 1 -- 172.21.15.116:0/565227285 wait complete. 2024-05-14T11:16:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.404+0000 7fe58c400640 1 -- start start 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 0x7fe584156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe584161b20 0x7fe5840b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe58414f600 0x7fe5840b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841712a0 con 0x7fe58406f620 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe584171120 con 0x7fe58414f600 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe58c400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe584171420 con 0x7fe584161b20 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe589a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 0x7fe584156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe589a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 0x7fe584156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52350/0 (socket says 172.21.15.116:52350) 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.408+0000 7fe589a00640 1 -- 172.21.15.116:0/1043753519 learned_addr learned my addr 172.21.15.116:0/1043753519 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fd80f200640 1 Processor -- start 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fd80f200640 1 -- start start 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe58414f600 0x7fe5840b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe589000640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe584161b20 0x7fe5840b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fd80f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd808205ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80f200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd808160d90 0x7fd80809e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80f200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80809ec30 0x7fd8080b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80f200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168910 con 0x7fd80809ec30 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 -- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe584161b20 msgr2=0x7fe5840b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe584161b20 0x7fe5840b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 -- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 msgr2=0x7fe584156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 0x7fe584156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe58a400640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584161310 con 0x7fe58414f600 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.412+0000 7fe589a00640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe58406f620 0x7fe584156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168790 con 0x7fd7ec005360 2024-05-14T11:16:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80f200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd80806fa30 con 0x7fd808160d90 2024-05-14T11:16:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe58a400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe58414f600 0x7fe5840b12d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00ccf0 tx=0x7fe57c00a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00f680 con 0x7fe58414f600 2024-05-14T11:16:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd808160d90 0x7fd80809e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd808205ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd808205ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52170/0 (socket says 172.21.15.116:52170) 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 -- 172.21.15.116:0/2981399642 learned_addr learned my addr 172.21.15.116:0/2981399642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c00f840 con 0x7fe58414f600 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00fc30 con 0x7fe58414f600 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80d200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80809ec30 0x7fd8080b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 -- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd808160d90 msgr2=0x7fd80809e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841552c0 con 0x7fe58414f600 2024-05-14T11:16:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd808160d90 0x7fd80809e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 -- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80809ec30 msgr2=0x7fd8080b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80809ec30 0x7fd8080b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe584155880 con 0x7fe58414f600 2024-05-14T11:16:35.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807e00640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80809f040 con 0x7fd7ec005360 2024-05-14T11:16:35.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd80d200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80809ec30 0x7fd8080b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.416+0000 7fd807400640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd808160d90 0x7fd80809e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd807e00640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd808205ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009fd0 tx=0x7fd7f4010d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400f040 con 0x7fd7ec005360 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80809b280 con 0x7fd7ec005360 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8080b3b70 con 0x7fd7ec005360 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f400eb00 con 0x7fd7ec005360 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400f040 con 0x7fd7ec005360 2024-05-14T11:16:35.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f4c002de0 con 0x7f3f54078c90 2024-05-14T11:16:35.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe57c00e3e0 con 0x7fe58414f600 2024-05-14T11:16:35.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.420+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe584155f40 con 0x7fe58414f600 2024-05-14T11:16:35.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe56806f400 0x7fe568071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77c0050880 con 0x7f77a0078c00 2024-05-14T11:16:35.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe57c090120 con 0x7fe58414f600 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7e0001040 con 0x7fd7ec005360 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7fe568078b50 0x7fe56807afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe57c013e50 con 0x7fe568078b50 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe57c097050 con 0x7fe58414f600 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe589a00640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe56806f400 0x7fe568071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77b6a00640 1 -- 172.21.15.116:0/2807947863 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f77c0050880 con 0x7f77a0078c00 2024-05-14T11:16:35.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe589000640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7fe568078b50 0x7fe56807afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f77a0078c00 msgr2=0x7f77a007b090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f77a0078c00 0x7f77a007b090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe589a00640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe56806f400 0x7fe568071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe578000f70 tx=0x7fe578009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7f400ecb0 con 0x7fd7ec005360 2024-05-14T11:16:35.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe589000640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7fe568078b50 0x7fe56807afe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77a006f430 msgr2=0x7f77a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f77a006f430 0x7f77a0071940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77c006a000 tx=0x7f77b0009040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 msgr2=0x7f77c0201f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 --2- 172.21.15.116:0/2807947863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77c00b3890 0x7f77c0201f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c200 tx=0x7f77ac01c010 comp rx=0 tx=0).stop 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77bf400640 1 -- 172.21.15.116:0/2807947863 reap_dead start 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 shutdown_connections 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 >> 172.21.15.116:0/2807947863 conn(0x7f77c00d2380 msgr2=0x7f77c00b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 shutdown_connections 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7f77abe00640 1 -- 172.21.15.116:0/2807947863 wait complete. 2024-05-14T11:16:35.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.424+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe57c013e50 con 0x7fe568078b50 2024-05-14T11:16:35.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f6aa00640 1 -- 172.21.15.116:0/3487342575 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f4c002de0 con 0x7f3f54078c90 2024-05-14T11:16:35.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7e806f630 0x7fd7e8071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd7f408f860 con 0x7fd7ec005360 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7fd7e8078df0 0x7fd7e807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7f400ce50 con 0x7fd7e8078df0 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd7f40942a0 con 0x7fd7ec005360 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd807400640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7e806f630 0x7fd7e8071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f3f54078c90 msgr2=0x7f3f5407b120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f3f54078c90 0x7f3f5407b120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3f5406f4d0 msgr2=0x7f3f540719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3f5406f4d0 0x7f3f540719e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f60008040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 msgr2=0x7f3f741fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 --2- 172.21.15.116:0/3487342575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3f740af3a0 0x7f3f741fecd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400e870 tx=0x7f3f6400ec70 comp rx=0 tx=0).stop 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd80d200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7fd7e8078df0 0x7fd7e807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f73400640 1 -- 172.21.15.116:0/3487342575 reap_dead start 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 shutdown_connections 2024-05-14T11:16:35.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 >> 172.21.15.116:0/3487342575 conn(0x7f3f740cde80 msgr2=0x7f3f7415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd80d200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7fd7e8078df0 0x7fd7e807b280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7f400ce50 con 0x7fd7e8078df0 2024-05-14T11:16:35.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.428+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 shutdown_connections 2024-05-14T11:16:35.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f3f79c00640 1 -- 172.21.15.116:0/3487342575 wait complete. 2024-05-14T11:16:35.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22f1a00640 1 -- 172.21.15.116:0/2532797346 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22ec1f8f90 con 0x7f22c8078c10 2024-05-14T11:16:35.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7fd807400640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7e806f630 0x7fd7e8071b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd80809a1a0 tx=0x7fd7f8001b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5698002de0 con 0x7f56a0078bf0 2024-05-14T11:16:35.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22e2a00640 1 -- 172.21.15.116:0/2532797346 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f22ec1f8f90 con 0x7f22c8078c10 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56b7e00640 1 -- 172.21.15.116:0/1410428737 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5698002de0 con 0x7f56a0078bf0 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7f22c8078c10 msgr2=0x7f22c807b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7f22c8078c10 0x7f22c807b0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22c806f450 msgr2=0x7f22c8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22c806f450 0x7f22c8071960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0ad000 tx=0x7f22dc008040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 msgr2=0x7f22ec162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 --2- 172.21.15.116:0/2532797346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec0b4c30 0x7f22ec162970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c700 tx=0x7f22d400cbc0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f56a0078bf0 msgr2=0x7f56a007b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f56a0078bf0 0x7f56a007b080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22eb400640 1 -- 172.21.15.116:0/2532797346 reap_dead start 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 shutdown_connections 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 >> 172.21.15.116:0/2532797346 conn(0x7f22ec073210 msgr2=0x7f22ec0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 shutdown_connections 2024-05-14T11:16:35.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56a006f430 msgr2=0x7f56a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56a006f430 0x7f56a0071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ad000 tx=0x7f56ac004740 comp rx=0 tx=0).stop 2024-05-14T11:16:35.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c0161b20 msgr2=0x7f56c015aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 --2- 172.21.15.116:0/1410428737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c0161b20 0x7f56c015aa30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56b800fc50 tx=0x7f56b800d430 comp rx=0 tx=0).stop 2024-05-14T11:16:35.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f22dbe00640 1 -- 172.21.15.116:0/2532797346 wait complete. 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c4c00640 1 -- 172.21.15.116:0/1410428737 reap_dead start 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 shutdown_connections 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.432+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 >> 172.21.15.116:0/1410428737 conn(0x7f56c00b4620 msgr2=0x7f56c0163890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 shutdown_connections 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f56c6c00640 1 -- 172.21.15.116:0/1410428737 wait complete. 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ef80dc0f0 con 0x7f9ee0078af0 2024-05-14T11:16:35.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1818002de0 con 0x7f1824078cc0 2024-05-14T11:16:35.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9ef6a00640 1 -- 172.21.15.116:0/3182533602 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9ef80dc0f0 con 0x7f9ee0078af0 2024-05-14T11:16:35.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183f400640 1 -- 172.21.15.116:0/4107214107 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1818002de0 con 0x7f1824078cc0 2024-05-14T11:16:35.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7f9ee0078af0 msgr2=0x7f9ee007af80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7f9ee0078af0 0x7f9ee007af80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f1824078cc0 msgr2=0x7f182407b150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f1824078cc0 0x7f182407b150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f182406f500 msgr2=0x7f1824071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f182406f500 0x7f1824071a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f18480b5660 tx=0x7f1830008040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 msgr2=0x7f1848150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f183cc00640 1 --2- 172.21.15.116:0/4107214107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f184815c790 0x7f1848150cd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f183800c3f0 tx=0x7f183800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f1847e00640 1 -- 172.21.15.116:0/4107214107 reap_dead start 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ee006f430 msgr2=0x7f9ee0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ee006f430 0x7f9ee0071940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0af000 tx=0x7f9eec008040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 msgr2=0x7f9ef80c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.436+0000 7f9effe00640 1 --2- 172.21.15.116:0/3182533602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ef80c8fb0 0x7f9ef80c9900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000f390 tx=0x7f9ef000f850 comp rx=0 tx=0).stop 2024-05-14T11:16:35.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 shutdown_connections 2024-05-14T11:16:35.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 >> 172.21.15.116:0/4107214107 conn(0x7f18480d28e0 msgr2=0x7f18480b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f9eff400640 1 -- 172.21.15.116:0/3182533602 reap_dead start 2024-05-14T11:16:35.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 shutdown_connections 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 >> 172.21.15.116:0/3182533602 conn(0x7f9ef8054d00 msgr2=0x7f9ef8006d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 shutdown_connections 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f9effe00640 1 -- 172.21.15.116:0/3182533602 wait complete. 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 shutdown_connections 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.440+0000 7f183cc00640 1 -- 172.21.15.116:0/4107214107 wait complete. 2024-05-14T11:16:35.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.452+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe584155070 con 0x7fe568078b50 2024-05-14T11:16:35.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.452+0000 7fe581600640 1 -- 172.21.15.116:0/1043753519 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe584155070 con 0x7fe568078b50 2024-05-14T11:16:35.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd7e0002de0 con 0x7fd7e8078df0 2024-05-14T11:16:35.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7fe568078b50 msgr2=0x7fe56807afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7fe568078b50 0x7fe56807afe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe56806f400 msgr2=0x7fe568071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe56806f400 0x7fe568071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe578000f70 tx=0x7fe578009040 comp rx=0 tx=0).stop 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe58414f600 msgr2=0x7fe5840b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 --2- 172.21.15.116:0/1043753519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe58414f600 0x7fe5840b12d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00ccf0 tx=0x7fe57c00a650 comp rx=0 tx=0).stop 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58a400640 1 -- 172.21.15.116:0/1043753519 reap_dead start 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 shutdown_connections 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.456+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 >> 172.21.15.116:0/1043753519 conn(0x7fe5840b4620 msgr2=0x7fe58406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd7ffe00640 1 -- 172.21.15.116:0/2981399642 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd7e0002de0 con 0x7fd7e8078df0 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7fd7e8078df0 msgr2=0x7fd7e807b280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7fd7e8078df0 0x7fd7e807b280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7e806f630 msgr2=0x7fd7e8071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7e806f630 0x7fd7e8071b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd80809a1a0 tx=0x7fd7f8001b30 comp rx=0 tx=0).stop 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 msgr2=0x7fd808205ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 --2- 172.21.15.116:0/2981399642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7ec005360 0x7fd808205ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009fd0 tx=0x7fd7f4010d60 comp rx=0 tx=0).stop 2024-05-14T11:16:35.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80d200640 1 -- 172.21.15.116:0/2981399642 reap_dead start 2024-05-14T11:16:35.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 shutdown_connections 2024-05-14T11:16:35.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 shutdown_connections 2024-05-14T11:16:35.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 >> 172.21.15.116:0/2981399642 conn(0x7fd8080b4620 msgr2=0x7fd808072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fe58c400640 1 -- 172.21.15.116:0/1043753519 wait complete. 2024-05-14T11:16:35.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 shutdown_connections 2024-05-14T11:16:35.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.460+0000 7fd80f200640 1 -- 172.21.15.116:0/2981399642 wait complete. 2024-05-14T11:16:35.478 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:35.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-14T11:16:35.480 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-05-14T11:16:35.480 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-14T11:16:35.481 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:35.481 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-14T11:16:35.482 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:35.486 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-14T11:16:35.488 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:35.488 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-14T11:16:35.491 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:35.491 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-14T11:16:35.497 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-05-14T11:16:35.497 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-14T11:16:35.498 INFO:teuthology.orchestra.run.smithi116.stdout:55834574860 2024-05-14T11:16:35.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-14T11:16:35.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.708+0000 7f3b77e00640 1 Processor -- start 2024-05-14T11:16:35.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.708+0000 7f3b77e00640 1 -- start start 2024-05-14T11:16:35.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.712+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.712+0000 7f3b77e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b70038350 con 0x7f3b70033a80 2024-05-14T11:16:35.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.712+0000 7f3b77e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700ec0d0 con 0x7f3b70006810 2024-05-14T11:16:35.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.712+0000 7f3b77e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3b700060a0 con 0x7f3b700eb8f0 2024-05-14T11:16:35.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35438/0 (socket says 172.21.15.116:35438) 2024-05-14T11:16:35.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 -- 172.21.15.116:0/222102325 learned_addr learned my addr 172.21.15.116:0/222102325 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f0a00800640 1 Processor -- start 2024-05-14T11:16:35.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f0a00800640 1 -- start start 2024-05-14T11:16:35.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 -- 172.21.15.116:0/222102325 >> v1:172.21.15.149:6789/0 conn(0x7f3b70006810 legacy=0x7f3b70006c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T11:16:35.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 -- 172.21.15.116:0/222102325 >> v1:172.21.15.116:6789/0 conn(0x7f3b70033a80 legacy=0x7f3b70033e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7f3b76a00640 1 -- 172.21.15.116:0/222102325 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700ec3b0 con 0x7f3b700eb8f0 2024-05-14T11:16:35.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7fcc78800640 1 Processor -- start 2024-05-14T11:16:35.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.716+0000 7fcc78800640 1 -- start start 2024-05-14T11:16:35.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f0a00800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 0x7f09f80b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f3b76a00640 1 --2- 172.21.15.116:0/222102325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098a0 tx=0x7f3b6801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6b151e78c583c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f0a00800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f09f80d3af0 0x7f09f80d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc78800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcc70073e20 0x7fcc70074230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc78800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcc700aec70 0x7fcc700af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f09fde00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 0x7f09f80b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f09fde00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 0x7f09f80b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35454/0 (socket says 172.21.15.116:35454) 2024-05-14T11:16:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f09fde00640 1 -- 172.21.15.116:0/1585672581 learned_addr learned my addr 172.21.15.116:0/1585672581 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f09fe800640 1 --2- 172.21.15.116:0/1585672581 >> v2:172.21.15.149:3300/0 conn(0x7f09f80d3af0 0x7f09f80d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc75e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fcc700b0a90 0x7fcc7006d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34848/0 (socket says 172.21.15.116:34848) 2024-05-14T11:16:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc75e00640 1 -- 172.21.15.116:0/1420445401 learned_addr learned my addr 172.21.15.116:0/1420445401 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc70074770 con 0x7fcc700b0a90 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006d940 con 0x7fcc70073e20 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006dc20 con 0x7fcc700aec70 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc76800640 1 --2- 172.21.15.116:0/1420445401 >> v2:172.21.15.149:3300/0 conn(0x7fcc700aec70 0x7fcc700af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7fcc75400640 1 --2- 172.21.15.116:0/1420445401 >> v2:172.21.15.116:3300/0 conn(0x7fcc70073e20 0x7fcc70074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09f80b80e0 con 0x7f09f80b2400 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f09f80b4890 con 0x7f09f80d3af0 2024-05-14T11:16:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09f80d52c0 con 0x7f09f80d23b0 2024-05-14T11:16:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f3b75600640 1 -- 172.21.15.116:0/222102325 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680b0070 con 0x7f3b700eb8f0 2024-05-14T11:16:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f3b75600640 1 -- 172.21.15.116:0/222102325 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b680ab600 con 0x7f3b700eb8f0 2024-05-14T11:16:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.720+0000 7f3b75600640 1 -- 172.21.15.116:0/222102325 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680ab9d0 con 0x7f3b700eb8f0 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755018336 0 0) 0x7fcc6c0014a0 con 0x7fcc700b0a90 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc58003530 con 0x7fcc700b0a90 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1339191272 0 0) 0x7fcc58003530 con 0x7fcc700b0a90 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 >> v2:172.21.15.149:3300/0 conn(0x7fcc700aec70 msgr2=0x7fcc700af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 --2- 172.21.15.116:0/1420445401 >> v2:172.21.15.149:3300/0 conn(0x7fcc700aec70 0x7fcc700af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 >> v2:172.21.15.116:3300/0 conn(0x7fcc70073e20 msgr2=0x7fcc70074230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 --2- 172.21.15.116:0/1420445401 >> v2:172.21.15.116:3300/0 conn(0x7fcc70073e20 0x7fcc70074230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7006df00 con 0x7fcc700b0a90 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fcc6c001c50 con 0x7fcc700b0a90 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 >> v1:172.21.15.149:6789/0 conn(0x7fcc700b0a90 legacy=0x7fcc7006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 0x7fcc58003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc58004810 0x7fcc58004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc58005320 0x7fcc5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7006d940 con 0x7fcc58003ab0 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7006dc20 con 0x7fcc58004810 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5800c330 con 0x7fcc58005320 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 0x7fcc58003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 -- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc58005320 msgr2=0x7fcc5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc76800640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc58005320 0x7fcc5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc58005320 0x7fcc5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 -- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc58004810 msgr2=0x7fcc58004c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc58004810 0x7fcc58004c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc75e00640 1 -- 172.21.15.116:0/1420445401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58003530 con 0x7fcc58003ab0 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc76800640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc58005320 0x7fcc5800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7fcc76800640 1 -- 172.21.15.116:0/1420445401 reap_dead start 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f3b77e00640 1 -- 172.21.15.116:0/222102325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 msgr2=0x7f3b700e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f3b77e00640 1 --2- 172.21.15.116:0/222102325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098a0 tx=0x7f3b6801c3a0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f3b77e00640 1 -- 172.21.15.116:0/222102325 shutdown_connections 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f3b77e00640 1 --2- 172.21.15.116:0/222102325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b700eb8f0 0x7f3b700e7c10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f3b77e00640 1 -- 172.21.15.116:0/222102325 >> 172.21.15.116:0/222102325 conn(0x7f3b70054d00 msgr2=0x7f3b70055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f09fe800640 1 -- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 msgr2=0x7f09f80b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f09fe800640 1 --2- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 0x7f09f80b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f09fe800640 1 -- 172.21.15.116:0/1585672581 >> v1:172.21.15.149:6789/0 conn(0x7f09f80b2400 legacy=0x7f09f80d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.724+0000 7f09fe800640 1 -- 172.21.15.116:0/1585672581 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f80d46a0 con 0x7f09f80d3af0 2024-05-14T11:16:35.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f3b77e00640 1 -- 172.21.15.116:0/222102325 shutdown_connections 2024-05-14T11:16:35.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f1af4e00640 1 Processor -- start 2024-05-14T11:16:35.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f3b77e00640 1 -- 172.21.15.116:0/222102325 wait complete. 2024-05-14T11:16:35.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f1af4e00640 1 -- start start 2024-05-14T11:16:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f09fe800640 1 --2- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d3af0 0x7f09f80d5750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f09e80097c0 tx=0x7f09e801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c30f32812961ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f09fca00640 1 -- 172.21.15.116:0/1585672581 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e80b0070 con 0x7f09f80d3af0 2024-05-14T11:16:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f09fca00640 1 -- 172.21.15.116:0/1585672581 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e80a4e50 con 0x7f09f80d3af0 2024-05-14T11:16:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.728+0000 7f09fca00640 1 -- 172.21.15.116:0/1585672581 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e80ab820 con 0x7f09f80d3af0 2024-05-14T11:16:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1af4e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1ae8033a80 0x7f1ae8033e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f3b77e00640 1 Processor -- start 2024-05-14T11:16:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aef400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f1ae80eb8f0 0x7f1ae80e7c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34860/0 (socket says 172.21.15.116:34860) 2024-05-14T11:16:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aef400640 1 -- 172.21.15.116:0/13849863 learned_addr learned my addr 172.21.15.116:0/13849863 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f3b77e00640 1 -- start start 2024-05-14T11:16:35.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae8038350 con 0x7f1ae8006810 2024-05-14T11:16:35.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae80343d0 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae80060a0 con 0x7f1ae8033a80 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc75e00640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 0x7fcc58003f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00a9d0 tx=0x7fcc6c01d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7407999a6dce356c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00a620 con 0x7fcc58003ab0 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0ad630 con 0x7fcc58003ab0 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc74a00640 1 -- 172.21.15.116:0/1420445401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0ada60 con 0x7fcc58003ab0 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 msgr2=0x7fcc58003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7fcc78800640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 0x7fcc58003f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00a9d0 tx=0x7fcc6c01d220 comp rx=0 tx=0).stop 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d3af0 msgr2=0x7f09f80d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 --2- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d3af0 0x7f09f80d5750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f09e80097c0 tx=0x7f09e801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 shutdown_connections 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 --2- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d3af0 0x7f09f80d5750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 --2- 172.21.15.116:0/1585672581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80d23b0 0x7f09f80b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 >> 172.21.15.116:0/1585672581 conn(0x7f09f809ea30 msgr2=0x7f09f808cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 shutdown_connections 2024-05-14T11:16:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f0a00800640 1 -- 172.21.15.116:0/1585672581 wait complete. 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aeea00640 1 --2- 172.21.15.116:0/13849863 >> v2:172.21.15.116:3300/0 conn(0x7f1ae8033a80 0x7f1ae8033e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923576144 0 0) 0x7f1ae4001120 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ad8003530 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 552876646 0 0) 0x7f1ad8003530 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 >> v2:172.21.15.116:3300/0 conn(0x7f1ae8033a80 msgr2=0x7f1ae8033e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 --2- 172.21.15.116:0/13849863 >> v2:172.21.15.116:3300/0 conn(0x7f1ae8033a80 0x7f1ae8033e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 >> v1:172.21.15.116:6789/0 conn(0x7f1ae8006810 legacy=0x7f1ae8006c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae80ec0d0 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f1ae40018d0 con 0x7f1ae80eb8f0 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 >> v1:172.21.15.149:6789/0 conn(0x7f1ae80eb8f0 legacy=0x7f1ae80e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ad8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad8005360 0x7f1ad800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae80060a0 con 0x7f1ad8005360 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad800c290 con 0x7f1ad8003a10 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad800c490 con 0x7f1ad8004810 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aef400640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ad8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.732+0000 7f1aefe00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad8005360 0x7f1ad800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aeea00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 shutdown_connections 2024-05-14T11:16:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7fcc78800640 1 --2- 172.21.15.116:0/1420445401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc58003ab0 0x7fcc58003f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 >> 172.21.15.116:0/1420445401 conn(0x7fcc70072c60 msgr2=0x7fcc700b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 shutdown_connections 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7fcc78800640 1 -- 172.21.15.116:0/1420445401 wait complete. 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aef400640 1 -- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ad8004810 msgr2=0x7f1ad8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aef400640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aef400640 1 -- 172.21.15.116:0/13849863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad8005360 msgr2=0x7f1ad800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aef400640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad8005360 0x7f1ad800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aef400640 1 -- 172.21.15.116:0/13849863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad8003530 con 0x7f1ad8003a10 2024-05-14T11:16:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aeea00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ad8004810 0x7f1ad8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b70006810 0x7f3b700dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aefe00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad8005360 0x7f1ad800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f1aefe00640 1 -- 172.21.15.116:0/13849863 reap_dead start 2024-05-14T11:16:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f0a00800640 1 Processor -- start 2024-05-14T11:16:35.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f0a00800640 1 -- start start 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b76000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b76000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35476/0 (socket says 172.21.15.116:35476) 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b700eb8f0 0x7f3b700e2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b76000640 1 -- 172.21.15.116:0/2188332654 learned_addr learned my addr 172.21.15.116:0/2188332654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77400640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b700eb8f0 0x7f3b700e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700f5dc0 con 0x7f3b70006810 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700f5c40 con 0x7f3b700eb8f0 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.736+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3b700f5f40 con 0x7f3b70033a80 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 -- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b700eb8f0 msgr2=0x7f3b700e2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b700eb8f0 0x7f3b700e2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 -- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b70006810 msgr2=0x7f3b700dd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b70006810 0x7f3b700dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b70049dd0 con 0x7f3b70033a80 2024-05-14T11:16:35.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 Processor -- start 2024-05-14T11:16:35.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 -- start start 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b76000640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b440 tx=0x7f3b6400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015020 con 0x7f3b70033a80 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64011830 con 0x7f3b70033a80 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011c00 con 0x7f3b70033a80 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7004a200 con 0x7f3b70033a80 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b700e2bd0 con 0x7f3b70033a80 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 0x7f09f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 0x7fcc70162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fde00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fde00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35482/0 (socket says 172.21.15.116:35482) 2024-05-14T11:16:35.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 0x7fcc7009e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52212/0 (socket says 172.21.15.116:52212) 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70164330 con 0x7fcc7009e420 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70160290 con 0x7fcc7009ed70 2024-05-14T11:16:35.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7fcc78800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcc701644b0 con 0x7fcc701627a0 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80b2990 con 0x7f09f815a350 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80b2810 con 0x7f09f80d23b0 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fde00640 1 -- 172.21.15.116:0/3617024572 learned_addr learned my addr 172.21.15.116:0/3617024572 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f09fe800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 0x7f09f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.740+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09f816cb10 con 0x7f09f80b2400 2024-05-14T11:16:35.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1aef400640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ad8003ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c490 tx=0x7f1ae401cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf117bf48db5d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc76800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 0x7fcc70162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc76800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52224/0 (socket says 172.21.15.116:52224) 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc76800640 1 -- 172.21.15.116:0/2849721446 learned_addr learned my addr 172.21.15.116:0/2849721446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75400640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 0x7fcc7009e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 -- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 msgr2=0x7f09f8157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 -- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 msgr2=0x7f09f80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 0x7f09f80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f815aa60 con 0x7f09f80b2400 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fe800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 0x7f09f80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fd400640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400c0e0 con 0x7f1ad8003a10 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae40a9070 con 0x7f1ad8003a10 2024-05-14T11:16:35.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1aee000640 1 -- 172.21.15.116:0/13849863 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40a5de0 con 0x7f1ad8003a10 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 -- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 msgr2=0x7fcc701f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09fde00640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009cd0 tx=0x7f09f400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f400cda0 con 0x7f09f80b2400 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f4019070 con 0x7f09f80b2400 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40146c0 con 0x7f09f80b2400 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f815ad40 con 0x7f09f80b2400 2024-05-14T11:16:35.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f815b300 con 0x7f09f80b2400 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 msgr2=0x7f1ad8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ad8003ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c490 tx=0x7f1ae401cc20 comp rx=0 tx=0).stop 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 -- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 msgr2=0x7fcc7009e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 0x7fcc7009e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 -- 172.21.15.116:0/2849721446 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc701630f0 con 0x7fcc701627a0 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc76800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75400640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 0x7fcc7009e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 shutdown_connections 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 --2- 172.21.15.116:0/13849863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ad8003ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 >> 172.21.15.116:0/13849863 conn(0x7f1ae8054d00 msgr2=0x7f1ae8055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 shutdown_connections 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f1af4e00640 1 -- 172.21.15.116:0/13849863 wait complete. 2024-05-14T11:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7fcc75e00640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 0x7fcc70162bb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0aa000 tx=0x7fcc58005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b64018020 con 0x7f3b70033a80 2024-05-14T11:16:35.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.744+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b500052a0 con 0x7f3b70033a80 2024-05-14T11:16:35.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7f3b6f400640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 0x7f3b58071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0bc070 con 0x7fcc701627a0 2024-05-14T11:16:35.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0014a0 con 0x7fcc701627a0 2024-05-14T11:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0ad7e0 con 0x7fcc701627a0 2024-05-14T11:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7015d5e0 con 0x7fcc701627a0 2024-05-14T11:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f806a9c0 con 0x7f09f80b2400 2024-05-14T11:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3b6408edb0 con 0x7f3b70033a80 2024-05-14T11:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7015dba0 con 0x7fcc701627a0 2024-05-14T11:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.748+0000 7f3b76a00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 0x7f3b58071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09f4012050 con 0x7f09f80b2400 2024-05-14T11:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f1af4e00640 1 Processor -- start 2024-05-14T11:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f3b76a00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 0x7f3b58071a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ad000 tx=0x7f3b68004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f1af4e00640 1 -- start start 2024-05-14T11:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7006a9c0 con 0x7fcc701627a0 2024-05-14T11:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f09f2000640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 0x7f09dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f09f408fad0 con 0x7f09f80b2400 2024-05-14T11:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f09fd400640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 0x7f09dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3b6401f070 con 0x7f3b70033a80 2024-05-14T11:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.752+0000 7f09fd400640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 0x7f09dc071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09f806cdc0 tx=0x7f09ec009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ae80e7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 0x7f1ae80e2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcc6c0bf070 con 0x7fcc701627a0 2024-05-14T11:16:35.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae80ec690 con 0x7f1ae80df780 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8006380 con 0x7f1ad8003a10 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1af4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae80ebd00 con 0x7f1ae80e27a0 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35494/0 (socket says 172.21.15.116:35494) 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 -- 172.21.15.116:0/1370994037 learned_addr learned my addr 172.21.15.116:0/1370994037 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aeea00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 0x7f1ae80e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 -- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 msgr2=0x7f1ae80e7290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ae80e7290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 -- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 msgr2=0x7f1ae80e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 0x7f1ae80e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7f1aefe00640 1 -- 172.21.15.116:0/1370994037 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae8005ea0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.756+0000 7fcc6a000640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 0x7fcc54071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7f1aefe00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1af00663a0 tx=0x7f1af0069d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7f1aeea00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 0x7f1ae80e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcc6c0ad4b0 con 0x7fcc701627a0 2024-05-14T11:16:35.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7fcc75400640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 0x7fcc54071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f09f4010040 con 0x7f09f80b2400 2024-05-14T11:16:35.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.760+0000 7fcc75400640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 0x7fcc54071960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006aa0 tx=0x7fcc60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcc6c0c6080 con 0x7fcc701627a0 2024-05-14T11:16:35.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af009a070 con 0x7f1ae80e27a0 2024-05-14T11:16:35.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af00754a0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0075890 con 0x7f1ae80e27a0 2024-05-14T11:16:35.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae80e2bb0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.764+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae80e44c0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.768+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae80e42b0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.768+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1af009d070 con 0x7f1ae80e27a0 2024-05-14T11:16:35.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.772+0000 7f1ae3400640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 0x7f1ad00719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.772+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1af010ae20 con 0x7f1ae80e27a0 2024-05-14T11:16:35.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.772+0000 7f1aef400640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 0x7f1ad00719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.772+0000 7f1aef400640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 0x7f1ad00719e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80531f0 tx=0x7f1ae40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.776+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1ae80e2bb0 con 0x7f1ae80e27a0 2024-05-14T11:16:35.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.804+0000 7f49b2800640 1 Processor -- start 2024-05-14T11:16:35.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.804+0000 7f49b2800640 1 -- start start 2024-05-14T11:16:35.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fc223e00640 1 Processor -- start 2024-05-14T11:16:35.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.804+0000 7f49b2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac070610 0x7f49ac160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.804+0000 7f49b2800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac06f5c0 con 0x7f49ac070130 2024-05-14T11:16:35.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.804+0000 7f49ab400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f49ac070130 0x7f49ac1605b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39838/0 (socket says 172.21.15.116:39838) 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49b2800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac161450 con 0x7f49ac0b54b0 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49ab400640 1 -- 172.21.15.116:0/213044910 learned_addr learned my addr 172.21.15.116:0/213044910 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac161200 con 0x7f49ac070610 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423429422 0 0) 0x7f49ac06f5c0 con 0x7f49ac070130 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4990003530 con 0x7f49ac070130 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 283771624 0 0) 0x7f4990003530 con 0x7f49ac070130 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac070610 msgr2=0x7f49ac160cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac070610 0x7f49ac160cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 >> v1:172.21.15.149:6789/0 conn(0x7f49ac0b54b0 legacy=0x7f49ac0b58c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac0d1fb0 con 0x7f49ac070130 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f4994001700 con 0x7f49ac070130 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 >> v1:172.21.15.116:6789/0 conn(0x7f49ac070130 legacy=0x7f49ac1605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4990003a90 0x7f4990003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4990004810 0x7f4990004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49900099c0 0x7f499000bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac161200 con 0x7f4990003a90 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f499000c350 con 0x7f4990004810 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f499000c550 con 0x7f49900099c0 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fc223e00640 1 -- start start 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fc223e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc224074400 0x7fc2240d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fc223e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49ab400640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4990003a90 0x7f4990003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fe6b6600640 1 Processor -- start 2024-05-14T11:16:35.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49abe00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49900099c0 0x7f499000bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fe6b6600640 1 -- start start 2024-05-14T11:16:35.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7f49aaa00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4990004810 0x7f4990004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.808+0000 7fc222000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35520/0 (socket says 172.21.15.116:35520) 2024-05-14T11:16:35.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 -- 172.21.15.116:0/1692160363 learned_addr learned my addr 172.21.15.116:0/1692160363 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2240cde50 con 0x7fc22415bd50 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc2240ce130 con 0x7fc224074400 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc2240ce410 con 0x7fc2240d1d90 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222a00640 1 --2- 172.21.15.116:0/1692160363 >> v2:172.21.15.149:3300/0 conn(0x7fc224074400 0x7fc2240d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49abe00640 1 -- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4990004810 msgr2=0x7f4990004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49abe00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4990004810 0x7f4990004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49abe00640 1 -- 172.21.15.116:0/213044910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4990003a90 msgr2=0x7f4990003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49abe00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4990003a90 0x7f4990003f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49abe00640 1 -- 172.21.15.116:0/213044910 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac161450 con 0x7f49900099c0 2024-05-14T11:16:35.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7f49aaa00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4990004810 0x7f4990004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:35.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 -- 172.21.15.116:0/1692160363 >> v2:172.21.15.149:3300/0 conn(0x7fc224074400 msgr2=0x7fc2240d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 --2- 172.21.15.116:0/1692160363 >> v2:172.21.15.149:3300/0 conn(0x7fc224074400 0x7fc2240d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 -- 172.21.15.116:0/1692160363 >> v1:172.21.15.116:6789/0 conn(0x7fc22415bd50 legacy=0x7fc2240b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fc222000640 1 -- 172.21.15.116:0/1692160363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2240ce6f0 con 0x7fc2240d1d90 2024-05-14T11:16:35.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fbca9000640 1 Processor -- start 2024-05-14T11:16:35.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.812+0000 7fbca9000640 1 -- start start 2024-05-14T11:16:35.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6b6600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe6b00732b0 0x7fe6b00736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49abe00640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49900099c0 0x7f499000bde0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009e30 tx=0x7f49a401ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaccdfcab5c1131 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6b6600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6b6600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b0070150 con 0x7fe6b006f3e0 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6b6600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b006ff00 con 0x7fe6b00732b0 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6b6600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b0070430 con 0x7fe6b00aeb50 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fc222000640 1 --2- 172.21.15.116:0/1692160363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc210007360 tx=0x7fc21001e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2a140db66fc546 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6af400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe6b00732b0 0x7fe6b00736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40b2070 con 0x7f49900099c0 2024-05-14T11:16:35.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40a5b00 con 0x7f49900099c0 2024-05-14T11:16:35.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49aa000640 1 -- 172.21.15.116:0/213044910 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4009a30 con 0x7f49900099c0 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35522/0 (socket says 172.21.15.116:35522) 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 -- 172.21.15.116:0/1903674550 learned_addr learned my addr 172.21.15.116:0/1903674550 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 -- 172.21.15.116:0/1903674550 >> v2:172.21.15.149:3300/0 conn(0x7fe6b00732b0 msgr2=0x7fe6b00736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 --2- 172.21.15.116:0/1903674550 >> v2:172.21.15.149:3300/0 conn(0x7fe6b00732b0 0x7fe6b00736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fc221600640 1 -- 172.21.15.116:0/1692160363 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210008be0 con 0x7fc2240d1d90 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fc221600640 1 -- 172.21.15.116:0/1692160363 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2100b0070 con 0x7fc2240d1d90 2024-05-14T11:16:35.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fc221600640 1 -- 172.21.15.116:0/1692160363 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21000a0d0 con 0x7fc2240d1d90 2024-05-14T11:16:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 -- 172.21.15.116:0/1903674550 >> v1:172.21.15.116:6789/0 conn(0x7fe6b006f3e0 legacy=0x7fe6b006f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fe6afe00640 1 -- 172.21.15.116:0/1903674550 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b0070710 con 0x7fe6b00aeb50 2024-05-14T11:16:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fbca9000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbca40ae8a0 0x7fbca40cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 msgr2=0x7fc22415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 --2- 172.21.15.116:0/1692160363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc210007360 tx=0x7fc21001e5a0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49900099c0 msgr2=0x7f499000bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49b2800640 1 --2- 172.21.15.116:0/213044910 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49900099c0 0x7f499000bde0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009e30 tx=0x7f49a401ce10 comp rx=0 tx=0).stop 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49abe00640 1 -- 172.21.15.116:0/213044910 reap_dead start 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 shutdown_connections 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 >> 172.21.15.116:0/213044910 conn(0x7f49ac071370 msgr2=0x7f49ac071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.816+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 shutdown_connections 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7f49b2800640 1 -- 172.21.15.116:0/213044910 wait complete. 2024-05-14T11:16:35.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 shutdown_connections 2024-05-14T11:16:35.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca9000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 0x7fbca406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca9000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca40cd450 con 0x7fbca40cd040 2024-05-14T11:16:35.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca9000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbca40af6b0 con 0x7fbca40ae8a0 2024-05-14T11:16:35.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca9000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbca40ccbb0 con 0x7fbca40aef10 2024-05-14T11:16:35.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 --2- 172.21.15.116:0/1692160363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2240d1d90 0x7fc22415b810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fbca40cd040 0x7fbca40ae190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34880/0 (socket says 172.21.15.116:34880) 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1e00640 1 -- 172.21.15.116:0/1962207422 learned_addr learned my addr 172.21.15.116:0/1962207422 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> v2:172.21.15.116:3300/0 conn(0x7fbca40ae8a0 0x7fbca40cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 --2- 172.21.15.116:0/1692160363 >> v2:172.21.15.149:3300/0 conn(0x7fc224074400 0x7fc2240d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 >> 172.21.15.116:0/1692160363 conn(0x7fc2240b6100 msgr2=0x7fc2240b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 shutdown_connections 2024-05-14T11:16:35.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fc223e00640 1 -- 172.21.15.116:0/1692160363 wait complete. 2024-05-14T11:16:35.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557943175 0 0) 0x7fbca40cd450 con 0x7fbca40cd040 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc88003530 con 0x7fbca40cd040 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095399628 0 0) 0x7fbc88003530 con 0x7fbca40cd040 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 msgr2=0x7fbca406e970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 0x7fbca406e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 >> v2:172.21.15.116:3300/0 conn(0x7fbca40ae8a0 msgr2=0x7fbca40cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 --2- 172.21.15.116:0/1962207422 >> v2:172.21.15.116:3300/0 conn(0x7fbca40ae8a0 0x7fbca40cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca40cc280 con 0x7fbca40cd040 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fbc9c001a80 con 0x7fbca40cd040 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> v2:172.21.15.116:3300/0 conn(0x7fbca40ae8a0 0x7fbca40cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 >> v1:172.21.15.149:6789/0 conn(0x7fbca40cd040 legacy=0x7fbca40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 0x7fbc88003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc88004050 0x7fbc88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88004da0 0x7fbc88005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbca40af6b0 con 0x7fbc88004da0 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbca40ccbb0 con 0x7fbc88004050 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbc88014560 con 0x7fbca40aef10 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc88004050 0x7fbc88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca2800640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88004da0 0x7fbc88005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 -- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 msgr2=0x7fbc88003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca40aef10 0x7fbc88003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 -- 172.21.15.116:0/1962207422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88004da0 msgr2=0x7fbc88005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 Processor -- start 2024-05-14T11:16:35.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 -- start start 2024-05-14T11:16:35.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88004da0 0x7fbc88005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.820+0000 7fbca1400640 1 -- 172.21.15.116:0/1962207422 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc88003530 con 0x7fbc88004050 2024-05-14T11:16:35.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fbca2800640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88004da0 0x7fbc88005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6afe00640 1 --2- 172.21.15.116:0/1903674550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009e50 tx=0x7fe6a801d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607d6b7b9242ef6b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6ae000640 1 -- 172.21.15.116:0/1903674550 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80a58f0 con 0x7fe6b00aeb50 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6ae000640 1 -- 172.21.15.116:0/1903674550 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a80a5a90 con 0x7fe6b00aeb50 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6ae000640 1 -- 172.21.15.116:0/1903674550 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80a5e60 con 0x7fe6b00aeb50 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fc223e00640 1 Processor -- start 2024-05-14T11:16:35.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fc223e00640 1 -- start start 2024-05-14T11:16:35.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 0x7f49ac168160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6b6600640 1 -- 172.21.15.116:0/1903674550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 msgr2=0x7fe6b00aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7fe6b6600640 1 --2- 172.21.15.116:0/1903674550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009e50 tx=0x7fe6a801d460 comp rx=0 tx=0).stop 2024-05-14T11:16:35.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 0x7f49ac1667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac0b2c10 0x7f49ac0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0b6560 con 0x7f49900099c0 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac0b63e0 con 0x7f49ac165720 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.824+0000 7f49b2800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac06d7a0 con 0x7f49ac0b2c10 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 -- 172.21.15.116:0/1903674550 shutdown_connections 2024-05-14T11:16:35.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49ab400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 0x7f49ac168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 --2- 172.21.15.116:0/1903674550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b00aeb50 0x7fe6b00aef60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 --2- 172.21.15.116:0/1903674550 >> v2:172.21.15.149:3300/0 conn(0x7fe6b00732b0 0x7fe6b00736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 -- 172.21.15.116:0/1903674550 >> 172.21.15.116:0/1903674550 conn(0x7fe6b00b4620 msgr2=0x7fe6b00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 0x7f49ac1667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49ab400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 0x7f49ac168160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52480/0 (socket says 172.21.15.116:52480) 2024-05-14T11:16:35.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49ab400640 1 -- 172.21.15.116:0/3040449713 learned_addr learned my addr 172.21.15.116:0/3040449713 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 -- 172.21.15.116:0/1903674550 shutdown_connections 2024-05-14T11:16:35.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49abe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac0b2c10 0x7f49ac0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fe6b6600640 1 -- 172.21.15.116:0/1903674550 wait complete. 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22415bd50 0x7fc22415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc222a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc222a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35566/0 (socket says 172.21.15.116:35566) 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 -- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac0b2c10 msgr2=0x7f49ac0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac0b2c10 0x7f49ac0b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 -- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 msgr2=0x7f49ac168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 0x7f49ac168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 -- 172.21.15.116:0/3040449713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac0b3560 con 0x7f49ac165720 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fbca1400640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc88004050 0x7fbc88004810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009cb0 tx=0x7fbc9001cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5f3416012da445 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b2070 con 0x7fbc88004050 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc900a5950 con 0x7fbc88004050 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fbca0a00640 1 -- 172.21.15.116:0/1962207422 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90009880 con 0x7fbc88004050 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc222a00640 1 -- 172.21.15.116:0/1100054455 learned_addr learned my addr 172.21.15.116:0/1100054455 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc222000640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22415bd50 0x7fc22415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49aaa00640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 0x7f49ac1667a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49ac071eb0 tx=0x7f499c00d8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc22415d530 0x7fc22415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc22415c2e0 con 0x7fc22415d530 2024-05-14T11:16:35.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc22415c160 con 0x7fc22415bd50 2024-05-14T11:16:35.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc22406d150 con 0x7fc224074400 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7fc223400640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc22415d530 0x7fc22415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0125f0 con 0x7f49ac165720 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c019070 con 0x7f49ac165720 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0129d0 con 0x7f49ac165720 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 -- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22415bd50 msgr2=0x7fc22415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22415bd50 0x7fc22415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 -- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc22415d530 msgr2=0x7fc22415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc22415d530 0x7fc22415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22415c500 con 0x7fc224074400 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc222a00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009d10 tx=0x7fc21c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac1fa0e0 con 0x7f49ac165720 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.828+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1fa6a0 con 0x7f49ac165720 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c018020 con 0x7fc224074400 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c014480 con 0x7fc224074400 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c014830 con 0x7fc224074400 2024-05-14T11:16:35.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22415c770 con 0x7fc224074400 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fbca9000640 1 -- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc88004050 msgr2=0x7fbc88004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fbca9000640 1 --2- 172.21.15.116:0/1962207422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc88004050 0x7fbc88004810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009cb0 tx=0x7fbc9001cc90 comp rx=0 tx=0).stop 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fbca2800640 1 -- 172.21.15.116:0/1962207422 reap_dead start 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fbca9000640 1 -- 172.21.15.116:0/1962207422 shutdown_connections 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fbca9000640 1 -- 172.21.15.116:0/1962207422 >> 172.21.15.116:0/1962207422 conn(0x7fbca40d10f0 msgr2=0x7fbca40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22415cd30 con 0x7fc224074400 2024-05-14T11:16:35.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fe6b6600640 1 Processor -- start 2024-05-14T11:16:35.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.832+0000 7fe6b6600640 1 -- start start 2024-05-14T11:16:35.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac06a9c0 con 0x7f49ac165720 2024-05-14T11:16:35.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc224050880 con 0x7fc224074400 2024-05-14T11:16:35.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f499c012b70 con 0x7f49ac165720 2024-05-14T11:16:35.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b00732b0 0x7fe6b01533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6b00aeb50 0x7fe6b0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0164510 con 0x7fe6b00732b0 2024-05-14T11:16:35.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0164390 con 0x7fe6b00aeb50 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6b6600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b0171010 con 0x7fe6b006f3e0 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35572/0 (socket says 172.21.15.116:35572) 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 -- 172.21.15.116:0/2308361957 learned_addr learned my addr 172.21.15.116:0/2308361957 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6aea00640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b00732b0 0x7fe6b01533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6afe00640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6b00aeb50 0x7fe6b0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc21c01b020 con 0x7fc224074400 2024-05-14T11:16:35.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 -- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6b00aeb50 msgr2=0x7fe6b0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6b00aeb50 0x7fe6b0156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 -- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b00732b0 msgr2=0x7fe6b01533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b00732b0 0x7fe6b01533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 -- 172.21.15.116:0/2308361957 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b0153910 con 0x7fe6b006f3e0 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fe6af400640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000d620 tx=0x7fe6a000dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7f49a3400640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 0x7f49880719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fbca9000640 1 -- 172.21.15.116:0/1962207422 shutdown_connections 2024-05-14T11:16:35.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fc21aa00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 0x7fc2040719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f499c08f6c0 con 0x7f49ac165720 2024-05-14T11:16:35.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc21c08fcb0 con 0x7fc224074400 2024-05-14T11:16:35.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7f49ab400640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 0x7f49880719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fc222000640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 0x7fc2040719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.836+0000 7fbca9000640 1 -- 172.21.15.116:0/1962207422 wait complete. 2024-05-14T11:16:35.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.840+0000 7fc222000640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 0x7fc2040719e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2100af000 tx=0x7fc2100a9000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.840+0000 7f49ab400640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 0x7f49880719e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4994000f70 tx=0x7f4994009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.840+0000 7fbca9000640 1 Processor -- start 2024-05-14T11:16:35.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.840+0000 7fbca9000640 1 -- start start 2024-05-14T11:16:35.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.840+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f49ac0b3560 con 0x7f49ac165720 2024-05-14T11:16:35.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbca415e7c0 0x7fbca415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52318/0 (socket says 172.21.15.116:52318) 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35584/0 (socket says 172.21.15.116:35584) 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbca40af320 con 0x7fbca415e7c0 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1e00640 1 -- 172.21.15.116:0/1627239216 learned_addr learned my addr 172.21.15.116:0/1627239216 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbca40cc840 con 0x7fbca40b5120 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbca406e970 con 0x7fbca415e3b0 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca2800640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbca415e7c0 0x7fbca415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 -- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 msgr2=0x7fbca4155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 -- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbca415e7c0 msgr2=0x7fbca415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbca415e7c0 0x7fbca415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca41557b0 con 0x7fbca415e3b0 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1e00640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:35.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.844+0000 7fbca1400640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ac000 tx=0x7fbc88005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000b620 con 0x7fe6b006f3e0 2024-05-14T11:16:35.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b2040 con 0x7fbca415e3b0 2024-05-14T11:16:35.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9000f000 con 0x7fbca415e3b0 2024-05-14T11:16:35.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ad6a0 con 0x7fbca415e3b0 2024-05-14T11:16:35.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc21c010040 con 0x7fc224074400 2024-05-14T11:16:35.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a0019070 con 0x7fe6b006f3e0 2024-05-14T11:16:35.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000b9c0 con 0x7fe6b006f3e0 2024-05-14T11:16:35.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b0153ca0 con 0x7fe6b006f3e0 2024-05-14T11:16:35.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b0154260 con 0x7fe6b006f3e0 2024-05-14T11:16:35.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41f6130 con 0x7fbca415e3b0 2024-05-14T11:16:35.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.860+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41f55b0 con 0x7fbca415e3b0 2024-05-14T11:16:35.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.864+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b006a9c0 con 0x7fe6b006f3e0 2024-05-14T11:16:35.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.864+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c0052a0 con 0x7fbca415e3b0 2024-05-14T11:16:35.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.864+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6a000bb60 con 0x7fe6b006f3e0 2024-05-14T11:16:35.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.864+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbc900ad840 con 0x7fbca415e3b0 2024-05-14T11:16:35.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fe6a7400640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 0x7fe690071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe6a008fdc0 con 0x7fe6b006f3e0 2024-05-14T11:16:35.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fbc9a000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 0x7fbc84071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:35.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbc90128a20 con 0x7fbca415e3b0 2024-05-14T11:16:35.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe6a005c950 con 0x7fe6b006f3e0 2024-05-14T11:16:35.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fbca1e00640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 0x7fbc84071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.868+0000 7fe6aea00640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 0x7fe690071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:35.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.872+0000 7fe6aea00640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 0x7fe690071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0163260 tx=0x7fe69c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.872+0000 7fbca1e00640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 0x7fbc84071a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005c30 tx=0x7fbc9c001320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:35.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:35.872+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbc900169c0 con 0x7fbca415e3b0 2024-05-14T11:16:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.064+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcc7009f180 con 0x7fcc701627a0 2024-05-14T11:16:36.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.064+0000 7fcc6a000640 1 -- 172.21.15.116:0/2849721446 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc701630f0 con 0x7fcc701627a0 2024-05-14T11:16:36.067 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:36.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 msgr2=0x7fcc54071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 0x7fcc54071960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006aa0 tx=0x7fcc60008040 comp rx=0 tx=0).stop 2024-05-14T11:16:36.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 msgr2=0x7fcc70162bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 0x7fcc70162bb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0aa000 tx=0x7fcc58005840 comp rx=0 tx=0).stop 2024-05-14T11:16:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 shutdown_connections 2024-05-14T11:16:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5406f450 0x7fcc54071960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7009ed70 0x7fcc701f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7009e420 0x7fcc7009e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 --2- 172.21.15.116:0/2849721446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc701627a0 0x7fcc70162bb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 >> 172.21.15.116:0/2849721446 conn(0x7fcc70072c60 msgr2=0x7fcc701ed750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 shutdown_connections 2024-05-14T11:16:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.068+0000 7fcc78800640 1 -- 172.21.15.116:0/2849721446 wait complete. 2024-05-14T11:16:36.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.080+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f09f80d1ec0 con 0x7f09f80b2400 2024-05-14T11:16:36.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.080+0000 7f09f2000640 1 -- 172.21.15.116:0/3617024572 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09f405c5e0 con 0x7f09f80b2400 2024-05-14T11:16:36.086 INFO:teuthology.orchestra.run.smithi116.stdout:55834574856 2024-05-14T11:16:36.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 msgr2=0x7f09dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 0x7f09dc071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09f806cdc0 tx=0x7f09ec009060 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 msgr2=0x7f09f8145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009cd0 tx=0x7f09f400c490 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 shutdown_connections 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09dc06f400 0x7f09dc071910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f80d23b0 0x7f09f8157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09f815a350 0x7f09f80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 --2- 172.21.15.116:0/3617024572 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09f80b2400 0x7f09f8145890 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 >> 172.21.15.116:0/3617024572 conn(0x7f09f809ea30 msgr2=0x7f09f80b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 shutdown_connections 2024-05-14T11:16:36.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.084+0000 7f0a00800640 1 -- 172.21.15.116:0/3617024572 wait complete. 2024-05-14T11:16:36.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.088+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3b50005590 con 0x7f3b70033a80 2024-05-14T11:16:36.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.088+0000 7f3b6f400640 1 -- 172.21.15.116:0/2188332654 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b6405b9d0 con 0x7f3b70033a80 2024-05-14T11:16:36.093 INFO:teuthology.orchestra.run.smithi116.stdout:55834574855 2024-05-14T11:16:36.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 msgr2=0x7f3b58071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 0x7f3b58071a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ad000 tx=0x7f3b68004740 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 msgr2=0x7f3b700e2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b440 tx=0x7f3b6400b900 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 shutdown_connections 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b5806f500 0x7f3b58071a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b700eb8f0 0x7f3b700e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b70033a80 0x7f3b700e2150 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b440 tx=0x7f3b6400b900 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 --2- 172.21.15.116:0/2188332654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b70006810 0x7f3b700dd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 >> 172.21.15.116:0/2188332654 conn(0x7f3b70054d00 msgr2=0x7f3b70038530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 shutdown_connections 2024-05-14T11:16:36.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.100+0000 7f3b77e00640 1 -- 172.21.15.116:0/2188332654 wait complete. 2024-05-14T11:16:36.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.104+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1ae8005580 con 0x7f1ae80e27a0 2024-05-14T11:16:36.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.4 2024-05-14T11:16:36.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.104+0000 7f1ae3400640 1 -- 172.21.15.116:0/1370994037 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1af0092070 con 0x7f1ae80e27a0 2024-05-14T11:16:36.108 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:36.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 msgr2=0x7f1ad00719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 0x7f1ad00719e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80531f0 tx=0x7f1ae40a7070 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 msgr2=0x7f1ae81716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1af00663a0 tx=0x7f1af0069d20 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 shutdown_connections 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1ad006f4d0 0x7f1ad00719e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ae80e27a0 0x7f1ae81716f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ae80df780 0x7f1ae80e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 --2- 172.21.15.116:0/1370994037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad8003a10 0x7f1ae80e7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.108+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 >> 172.21.15.116:0/1370994037 conn(0x7f1ae8054d00 msgr2=0x7f1ae80e9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.112+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 shutdown_connections 2024-05-14T11:16:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.112+0000 7f1af4e00640 1 -- 172.21.15.116:0/1370994037 wait complete. 2024-05-14T11:16:36.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-05-14T11:16:36.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2024-05-14T11:16:36.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.5 2024-05-14T11:16:36.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.144+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f49ac1f9e70 con 0x7f49ac165720 2024-05-14T11:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.144+0000 7f49a3400640 1 -- 172.21.15.116:0/3040449713 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f499c05bc90 con 0x7f49ac165720 2024-05-14T11:16:36.148 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 msgr2=0x7f49880719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 0x7f49880719e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4994000f70 tx=0x7f4994009040 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 msgr2=0x7f49ac1667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 0x7f49ac1667a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49ac071eb0 tx=0x7f499c00d8a0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 shutdown_connections 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f498806f4d0 0x7f49880719e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49ac165720 0x7f49ac1667a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49ac0b2c10 0x7f49ac0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 --2- 172.21.15.116:0/3040449713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49900099c0 0x7f49ac168160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 >> 172.21.15.116:0/3040449713 conn(0x7f49ac071370 msgr2=0x7f49ac0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 shutdown_connections 2024-05-14T11:16:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.148+0000 7f49b2800640 1 -- 172.21.15.116:0/3040449713 wait complete. 2024-05-14T11:16:36.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.3 2024-05-14T11:16:36.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.172+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbc7c005590 con 0x7fbca415e3b0 2024-05-14T11:16:36.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.172+0000 7fbc9a000640 1 -- 172.21.15.116:0/1627239216 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc900f4ff0 con 0x7fbca415e3b0 2024-05-14T11:16:36.176 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 msgr2=0x7fbc84071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 0x7fbc84071a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005c30 tx=0x7fbc9c001320 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 msgr2=0x7fbca415ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ac000 tx=0x7fbc88005840 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 shutdown_connections 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc8406f500 0x7fbc84071a10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbca415e7c0 0x7fbca415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbca415e3b0 0x7fbca415ae10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 --2- 172.21.15.116:0/1627239216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca40b5120 0x7fbca4155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 >> 172.21.15.116:0/1627239216 conn(0x7fbca40d10f0 msgr2=0x7fbca40b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 shutdown_connections 2024-05-14T11:16:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.176+0000 7fbca9000640 1 -- 172.21.15.116:0/1627239216 wait complete. 2024-05-14T11:16:36.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.6 2024-05-14T11:16:36.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.200+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc2240b6cc0 con 0x7fc224074400 2024-05-14T11:16:36.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.200+0000 7fc21aa00640 1 -- 172.21.15.116:0/1100054455 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc21c05c840 con 0x7fc224074400 2024-05-14T11:16:36.203 INFO:teuthology.orchestra.run.smithi116.stdout:55834574856 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 msgr2=0x7fc2040719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 0x7fc2040719e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2100af000 tx=0x7fc2100a9000 comp rx=0 tx=0).stop 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 msgr2=0x7fc224158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009d10 tx=0x7fc21c00c490 comp rx=0 tx=0).stop 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 shutdown_connections 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc20406f4d0 0x7fc2040719e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc22415bd50 0x7fc22415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc22415d530 0x7fc22415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 --2- 172.21.15.116:0/1100054455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc224074400 0x7fc224158d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 >> 172.21.15.116:0/1100054455 conn(0x7fc2240b6100 msgr2=0x7fc2240b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 shutdown_connections 2024-05-14T11:16:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.204+0000 7fc223e00640 1 -- 172.21.15.116:0/1100054455 wait complete. 2024-05-14T11:16:36.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.216+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe6b006ecf0 con 0x7fe6b006f3e0 2024-05-14T11:16:36.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.216+0000 7fe6a7400640 1 -- 172.21.15.116:0/2308361957 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6a005c250 con 0x7fe6b006f3e0 2024-05-14T11:16:36.219 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 msgr2=0x7fe690071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 0x7fe690071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0163260 tx=0x7fe69c009040 comp rx=0 tx=0).stop 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 msgr2=0x7fe6b0154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000d620 tx=0x7fe6a000dae0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 shutdown_connections 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe69006f430 0x7fe690071940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6b00aeb50 0x7fe6b0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b00732b0 0x7fe6b01533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 --2- 172.21.15.116:0/2308361957 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6b006f3e0 0x7fe6b0154a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:36.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 >> 172.21.15.116:0/2308361957 conn(0x7fe6b00b4620 msgr2=0x7fe6b015d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:36.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 shutdown_connections 2024-05-14T11:16:36.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:36.220+0000 7fe6b6600640 1 -- 172.21.15.116:0/2308361957 wait complete. 2024-05-14T11:16:36.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.7 2024-05-14T11:16:36.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2024-05-14T11:16:37.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-14T11:16:37.130 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-14T11:16:37.138 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-14T11:16:37.143 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-14T11:16:37.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-14T11:16:37.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-14T11:16:37.232 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-14T11:16:37.245 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-14T11:16:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.320+0000 7fd574a00640 1 Processor -- start 2024-05-14T11:16:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.320+0000 7fd574a00640 1 -- start start 2024-05-14T11:16:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.320+0000 7fd574a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd5700d0520 0x7fd57015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.320+0000 7fd574a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5700cdc10 0x7fd5700ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd574a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5700ce560 con 0x7fd570073dd0 2024-05-14T11:16:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd574a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd57006da40 con 0x7fd5700d0520 2024-05-14T11:16:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd574a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5700b2c20 con 0x7fd5700cdc10 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd5700d0520 0x7fd57015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd5700d0520 0x7fd57015fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52322/0 (socket says 172.21.15.116:52322) 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 -- 172.21.15.116:0/2140074037 learned_addr learned my addr 172.21.15.116:0/2140074037 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 -- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5700cdc10 msgr2=0x7fd5700ce020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 --2- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5700cdc10 0x7fd5700ce020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 -- 172.21.15.116:0/2140074037 >> v1:172.21.15.116:6789/0 conn(0x7fd570073dd0 legacy=0x7fd5700741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.324+0000 7fd56f400640 1 -- 172.21.15.116:0/2140074037 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5700d10f0 con 0x7fd5700d0520 2024-05-14T11:16:37.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd56f400640 1 --2- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700d0520 0x7fd57015fe00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564d507f828be27a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd56e000640 1 -- 172.21.15.116:0/2140074037 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600aa650 con 0x7fd5700d0520 2024-05-14T11:16:37.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd56e000640 1 -- 172.21.15.116:0/2140074037 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5600af070 con 0x7fd5700d0520 2024-05-14T11:16:37.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd56e000640 1 -- 172.21.15.116:0/2140074037 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600aaa10 con 0x7fd5700d0520 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 -- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700d0520 msgr2=0x7fd57015fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 --2- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700d0520 0x7fd57015fe00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56001c4f0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 -- 172.21.15.116:0/2140074037 shutdown_connections 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 --2- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5700cdc10 0x7fd5700ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 --2- 172.21.15.116:0/2140074037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700d0520 0x7fd57015fe00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 -- 172.21.15.116:0/2140074037 >> 172.21.15.116:0/2140074037 conn(0x7fd5700b6100 msgr2=0x7fd5700b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.332+0000 7fd574a00640 1 -- 172.21.15.116:0/2140074037 shutdown_connections 2024-05-14T11:16:37.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.336+0000 7fd574a00640 1 -- 172.21.15.116:0/2140074037 wait complete. 2024-05-14T11:16:37.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.336+0000 7fd574a00640 1 Processor -- start 2024-05-14T11:16:37.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.336+0000 7fd574a00640 1 -- start start 2024-05-14T11:16:37.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 0x7fd5700c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5700d0520 0x7fd5700c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700d1eb0 con 0x7fd5700d0520 2024-05-14T11:16:37.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700b2840 con 0x7fd5700cdc10 2024-05-14T11:16:37.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd574a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5700c5c80 con 0x7fd570073dd0 2024-05-14T11:16:37.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52336/0 (socket says 172.21.15.116:52336) 2024-05-14T11:16:37.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 -- 172.21.15.116:0/2108387630 learned_addr learned my addr 172.21.15.116:0/2108387630 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56f400640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 0x7fd5700c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56fe00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5700d0520 0x7fd5700c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 -- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 msgr2=0x7fd5700c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 0x7fd5700c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 -- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5700d0520 msgr2=0x7fd5700c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5700d0520 0x7fd5700c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56ea00640 1 -- 172.21.15.116:0/2108387630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd570157700 con 0x7fd5700cdc10 2024-05-14T11:16:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.340+0000 7fd56f400640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 0x7fd5700c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:37.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd56ea00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00b3e0 tx=0x7fd55c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c015020 con 0x7fd5700cdc10 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd55c011850 con 0x7fd5700cdc10 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c011c40 con 0x7fd5700cdc10 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd570161e60 con 0x7fd5700cdc10 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd570162420 con 0x7fd5700cdc10 2024-05-14T11:16:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.344+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd570050880 con 0x7fd5700cdc10 2024-05-14T11:16:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.352+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd55c018020 con 0x7fd5700cdc10 2024-05-14T11:16:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.352+0000 7fd567400640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 0x7fd550071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.352+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd55c08f4b0 con 0x7fd5700cdc10 2024-05-14T11:16:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.352+0000 7fd56f400640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 0x7fd550071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.356+0000 7fd56f400640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 0x7fd550071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ae000 tx=0x7fd5600a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.356+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd55c01f020 con 0x7fd5700cdc10 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.364+0000 7feec3e00640 1 Processor -- start 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.364+0000 7feec3e00640 1 -- start start 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec3e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 0x7feebc073e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 0x7feebc0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec3e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc074770 con 0x7feebc0b1890 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec3e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7feebc06dfc0 con 0x7feebc0b1480 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec3e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feebc06d110 con 0x7feebc074360 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec0a00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7feebc0b1890 0x7feebc0ae6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34890/0 (socket says 172.21.15.116:34890) 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec0a00640 1 -- 172.21.15.116:0/3561638854 learned_addr learned my addr 172.21.15.116:0/3561638854 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec1e00640 1 --2- 172.21.15.116:0/3561638854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 0x7feebc0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec1400640 1 --2- 172.21.15.116:0/3561638854 >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 0x7feebc073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feebbe00640 1 -- 172.21.15.116:0/3561638854 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811321682 0 0) 0x7feea8001120 con 0x7feebc0b1890 2024-05-14T11:16:37.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feebbe00640 1 -- 172.21.15.116:0/3561638854 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feea4003530 con 0x7feebc0b1890 2024-05-14T11:16:37.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec1e00640 1 -- 172.21.15.116:0/3561638854 >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 msgr2=0x7feebc073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.368+0000 7feec1e00640 1 --2- 172.21.15.116:0/3561638854 >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 0x7feebc073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.372+0000 7feec1e00640 1 -- 172.21.15.116:0/3561638854 >> v1:172.21.15.149:6789/0 conn(0x7feebc0b1890 legacy=0x7feebc0ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.372+0000 7feec1e00640 1 -- 172.21.15.116:0/3561638854 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc06d3b0 con 0x7feebc074360 2024-05-14T11:16:37.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.372+0000 7feec1400640 1 --2- 172.21.15.116:0/3561638854 >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 0x7feebc073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:37.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7f358e000640 1 Processor -- start 2024-05-14T11:16:37.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7f358e000640 1 -- start start 2024-05-14T11:16:37.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec1e00640 1 --2- 172.21.15.116:0/3561638854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 0x7feebc0aedc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733ef8c58982b80c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feebbe00640 1 -- 172.21.15.116:0/3561638854 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b2070 con 0x7feebc074360 2024-05-14T11:16:37.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feebbe00640 1 -- 172.21.15.116:0/3561638854 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40a5a00 con 0x7feebc074360 2024-05-14T11:16:37.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feebbe00640 1 -- 172.21.15.116:0/3561638854 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40a5db0 con 0x7feebc074360 2024-05-14T11:16:37.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec3e00640 1 -- 172.21.15.116:0/3561638854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 msgr2=0x7feebc0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec3e00640 1 --2- 172.21.15.116:0/3561638854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 0x7feebc0aedc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c960 comp rx=0 tx=0).stop 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec3e00640 1 -- 172.21.15.116:0/3561638854 shutdown_connections 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7f358e000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec3e00640 1 --2- 172.21.15.116:0/3561638854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc074360 0x7feebc0aedc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7feec3e00640 1 --2- 172.21.15.116:0/3561638854 >> v2:172.21.15.149:3300/0 conn(0x7feebc0b1480 0x7feebc073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7feec3e00640 1 -- 172.21.15.116:0/3561638854 >> 172.21.15.116:0/3561638854 conn(0x7feebc072c60 msgr2=0x7feebc0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.376+0000 7f358e000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3588073e20 0x7f3588074230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f358e000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3588074770 con 0x7f3588163910 2024-05-14T11:16:37.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f358e000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f358806d940 con 0x7f3588073e20 2024-05-14T11:16:37.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f358e000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f358806dc20 con 0x7f35880b0a90 2024-05-14T11:16:37.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7feec3e00640 1 -- 172.21.15.116:0/3561638854 shutdown_connections 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35616/0 (socket says 172.21.15.116:35616) 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 -- 172.21.15.116:0/3704200041 learned_addr learned my addr 172.21.15.116:0/3704200041 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3586a00640 1 --2- 172.21.15.116:0/3704200041 >> v2:172.21.15.149:3300/0 conn(0x7f3588073e20 0x7f3588074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 -- 172.21.15.116:0/3704200041 >> v2:172.21.15.149:3300/0 conn(0x7f3588073e20 msgr2=0x7f3588074230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 --2- 172.21.15.116:0/3704200041 >> v2:172.21.15.149:3300/0 conn(0x7f3588073e20 0x7f3588074230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 -- 172.21.15.116:0/3704200041 >> v1:172.21.15.116:6789/0 conn(0x7f3588163910 legacy=0x7f3588163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7feec3e00640 1 -- 172.21.15.116:0/3561638854 wait complete. 2024-05-14T11:16:37.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3586000640 1 -- 172.21.15.116:0/3704200041 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3949691351 0 0) 0x7f3578001120 con 0x7f3588163910 2024-05-14T11:16:37.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.380+0000 7f3587400640 1 -- 172.21.15.116:0/3704200041 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358806df00 con 0x7f35880b0a90 2024-05-14T11:16:37.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7feec3e00640 1 Processor -- start 2024-05-14T11:16:37.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7feec3e00640 1 -- start start 2024-05-14T11:16:37.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f8136600640 1 Processor -- start 2024-05-14T11:16:37.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f8136600640 1 -- start start 2024-05-14T11:16:37.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f3587400640 1 --2- 172.21.15.116:0/3704200041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3580009e50 tx=0x7f358001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3f9dca4e2454c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f3586000640 1 -- 172.21.15.116:0/3704200041 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800a5770 con 0x7f35880b0a90 2024-05-14T11:16:37.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f3586000640 1 -- 172.21.15.116:0/3704200041 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35800a5910 con 0x7f35880b0a90 2024-05-14T11:16:37.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f3586000640 1 -- 172.21.15.116:0/3704200041 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800a5ce0 con 0x7f35880b0a90 2024-05-14T11:16:37.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7feec3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 0x7feebc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f358e000640 1 -- 172.21.15.116:0/3704200041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 msgr2=0x7f358806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.384+0000 7f358e000640 1 --2- 172.21.15.116:0/3704200041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3580009e50 tx=0x7f358001c9a0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 0x7feebc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 -- 172.21.15.116:0/3704200041 shutdown_connections 2024-05-14T11:16:37.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f8136600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8130073e20 0x7f8130074230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f8136600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f81300aec70 0x7f81300af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 0x7feebc1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 --2- 172.21.15.116:0/3704200041 >> v2:172.21.15.149:3300/0 conn(0x7f3588073e20 0x7f3588074230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 --2- 172.21.15.116:0/3704200041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35880b0a90 0x7f358806d110 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 -- 172.21.15.116:0/3704200041 >> 172.21.15.116:0/3704200041 conn(0x7f3588072c60 msgr2=0x7f35880b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 -- 172.21.15.116:0/3704200041 shutdown_connections 2024-05-14T11:16:37.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f358e000640 1 -- 172.21.15.116:0/3704200041 wait complete. 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 0x7feebc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 0x7feebc0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52370/0 (socket says 172.21.15.116:52370) 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec0a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 0x7feebc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec0a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 0x7feebc1530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35628/0 (socket says 172.21.15.116:35628) 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec0a00640 1 -- 172.21.15.116:0/516565160 learned_addr learned my addr 172.21.15.116:0/516565160 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feebc170dc0 con 0x7feebc0b1890 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0cd510 con 0x7feebc074360 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feebc170f40 con 0x7feebc0b1480 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f8136600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8130074770 con 0x7f81300b0a90 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f8136600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f813006d940 con 0x7f81300aec70 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f8136600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f813006dc20 con 0x7f8130073e20 2024-05-14T11:16:37.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812f400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f81300b0a90 0x7f813006d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34900/0 (socket says 172.21.15.116:34900) 2024-05-14T11:16:37.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812f400640 1 -- 172.21.15.116:0/1592939771 learned_addr learned my addr 172.21.15.116:0/1592939771 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:37.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 0x7feebc1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812ea00640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8130073e20 0x7f8130074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812fe00640 1 --2- 172.21.15.116:0/1592939771 >> v2:172.21.15.149:3300/0 conn(0x7f81300aec70 0x7f81300af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812e000640 1 -- 172.21.15.116:0/1592939771 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582602770 0 0) 0x7f81280014a0 con 0x7f81300b0a90 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812e000640 1 -- 172.21.15.116:0/1592939771 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8114003530 con 0x7f81300b0a90 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812fe00640 1 -- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8130073e20 msgr2=0x7f8130074230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812fe00640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8130073e20 0x7f8130074230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812fe00640 1 -- 172.21.15.116:0/1592939771 >> v1:172.21.15.149:6789/0 conn(0x7f81300b0a90 legacy=0x7f813006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7f812fe00640 1 -- 172.21.15.116:0/1592939771 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813006df00 con 0x7f81300aec70 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 -- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 msgr2=0x7feebc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 0x7feebc1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 -- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 msgr2=0x7feebc0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 0x7feebc0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc158330 con 0x7feebc0b1890 2024-05-14T11:16:37.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.388+0000 7feec1e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 0x7feebc1630c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feeb40ac000 tx=0x7feebc0afb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7f358e000640 1 Processor -- start 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b2050 con 0x7feebc0b1890 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc158610 con 0x7feebc0b1890 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc158bd0 con 0x7feebc0b1890 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40be070 con 0x7feebc0b1890 2024-05-14T11:16:37.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4004240 con 0x7feebc0b1890 2024-05-14T11:16:37.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.392+0000 7f358e000640 1 -- start start 2024-05-14T11:16:37.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35880b0a90 0x7f358809c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 0x7f358809cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f812fe00640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300aec70 0x7f81300af080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097c0 tx=0x7f811c024c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7321ee6324cd4ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f358816d600 con 0x7f35880b0a90 2024-05-14T11:16:37.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f358816d480 con 0x7f3588073e20 2024-05-14T11:16:37.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f358e000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f358816d780 con 0x7f3588163910 2024-05-14T11:16:37.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feeb40043e0 con 0x7feebc0b1890 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3586a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35880b0a90 0x7f358809c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52400/0 (socket says 172.21.15.116:52400) 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 -- 172.21.15.116:0/3154676118 learned_addr learned my addr 172.21.15.116:0/3154676118 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587e00640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 0x7f358809cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 -- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 msgr2=0x7f358809cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 0x7f358809cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 -- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35880b0a90 msgr2=0x7f358809c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35880b0a90 0x7f358809c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 -- 172.21.15.116:0/3154676118 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358809b660 con 0x7f3588073e20 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587e00640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 0x7f358809cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f812e000640 1 -- 172.21.15.116:0/1592939771 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0ba070 con 0x7f81300aec70 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f812e000640 1 -- 172.21.15.116:0/1592939771 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c0adaf0 con 0x7f81300aec70 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f812e000640 1 -- 172.21.15.116:0/1592939771 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0b5440 con 0x7f81300aec70 2024-05-14T11:16:37.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f3587400640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35800af040 tx=0x7f3580019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 -- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300aec70 msgr2=0x7f81300af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300aec70 0x7f81300af080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097c0 tx=0x7f811c024c60 comp rx=0 tx=0).stop 2024-05-14T11:16:37.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800b8670 con 0x7f3588073e20 2024-05-14T11:16:37.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35800be070 con 0x7f3588073e20 2024-05-14T11:16:37.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800b8a70 con 0x7f3588073e20 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7feeb9600640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 0x7fee9c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 -- 172.21.15.116:0/1592939771 shutdown_connections 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300aec70 0x7f81300af080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 --2- 172.21.15.116:0/1592939771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8130073e20 0x7f8130074230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.396+0000 7f8136600640 1 -- 172.21.15.116:0/1592939771 >> 172.21.15.116:0/1592939771 conn(0x7f8130072c60 msgr2=0x7f81300b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee980052a0 con 0x7feebc0b1890 2024-05-14T11:16:37.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358809b940 con 0x7f3588073e20 2024-05-14T11:16:37.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358809bf00 con 0x7f3588073e20 2024-05-14T11:16:37.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f8136600640 1 -- 172.21.15.116:0/1592939771 shutdown_connections 2024-05-14T11:16:37.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f8136600640 1 -- 172.21.15.116:0/1592939771 wait complete. 2024-05-14T11:16:37.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358806a9c0 con 0x7f3588073e20 2024-05-14T11:16:37.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feeb412a250 con 0x7feebc0b1890 2024-05-14T11:16:37.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f8136600640 1 Processor -- start 2024-05-14T11:16:37.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f8136600640 1 -- start start 2024-05-14T11:16:37.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7feec1400640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 0x7fee9c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.400+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35800b8c10 con 0x7f3588073e20 2024-05-14T11:16:37.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f357f400640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 0x7f3564071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7feec1400640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 0x7fee9c0719e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feeb00045f0 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f8136600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f358012a590 con 0x7f3588073e20 2024-05-14T11:16:37.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f3586a00640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 0x7f3564071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f812f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f812f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52540/0 (socket says 172.21.15.116:52540) 2024-05-14T11:16:37.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f812f400640 1 -- 172.21.15.116:0/794795539 learned_addr learned my addr 172.21.15.116:0/794795539 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.404+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 0x7f81300cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 0x7f813009e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 0x7f81300cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81300c8c40 con 0x7f8130073e20 2024-05-14T11:16:37.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812fe00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 0x7f813009e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81300c8ac0 con 0x7f81300b0a90 2024-05-14T11:16:37.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f81300c8dc0 con 0x7f81300aec70 2024-05-14T11:16:37.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 -- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 msgr2=0x7f813009e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 0x7f813009e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 -- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 msgr2=0x7f81300d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f3586a00640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 0x7f3564071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f358815be30 tx=0x7f3570008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81300cfa70 con 0x7f81300aec70 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812f400640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812fe00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 0x7f813009e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:37.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f812ea00640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 0x7f81300cc0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f812000e870 tx=0x7f812000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120017070 con 0x7f81300aec70 2024-05-14T11:16:37.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.408+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81300cfd50 con 0x7f81300aec70 2024-05-14T11:16:37.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81300d07d0 con 0x7f81300aec70 2024-05-14T11:16:37.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3580016620 con 0x7f3588073e20 2024-05-14T11:16:37.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812000ee60 con 0x7f81300aec70 2024-05-14T11:16:37.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120017070 con 0x7f81300aec70 2024-05-14T11:16:37.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feeb4016580 con 0x7feebc0b1890 2024-05-14T11:16:37.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.412+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81080052a0 con 0x7f81300aec70 2024-05-14T11:16:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.416+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f812001a070 con 0x7f81300aec70 2024-05-14T11:16:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.416+0000 7f8127400640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 0x7f8110071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.420+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f812008f370 con 0x7f81300aec70 2024-05-14T11:16:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.420+0000 7f812f400640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 0x7f8110071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.420+0000 7f812f400640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 0x7f8110071b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8128007f20 tx=0x7f8128007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.420+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f812005b8d0 con 0x7f81300aec70 2024-05-14T11:16:37.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.472+0000 7fae02000640 1 Processor -- start 2024-05-14T11:16:37.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.472+0000 7fae02000640 1 -- start start 2024-05-14T11:16:37.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.472+0000 7f44ea200640 1 Processor -- start 2024-05-14T11:16:37.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.472+0000 7f44ea200640 1 -- start start 2024-05-14T11:16:37.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fae02000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fadfc0d1aa0 0x7fadfc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44ea200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f44e4070320 0x7f44e40af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44ea200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfbe00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fadfc0d1690 0x7fadfc0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:34902/0 (socket says 172.21.15.116:34902) 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfbe00640 1 -- 172.21.15.116:0/1530254986 learned_addr learned my addr 172.21.15.116:0/1530254986 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadfc1648d0 con 0x7fadfc164400 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadfc164bb0 con 0x7fadfc0d1690 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fadfc164e90 con 0x7fadfc0d1aa0 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44ea200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e40cf730 con 0x7f44e40cbe40 2024-05-14T11:16:37.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44ea200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f44e4070730 con 0x7f44e4070320 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44ea200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40cf200 con 0x7f44e406eb40 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776921062 0 0) 0x7fadfc164bb0 con 0x7fadfc0d1690 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade0003530 con 0x7fadfc0d1690 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fae01600640 1 --2- 172.21.15.116:0/1530254986 >> v2:172.21.15.116:3300/0 conn(0x7fadfc0d1aa0 0x7fadfc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44e3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f44e40cbe40 0x7f44e40cc250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39898/0 (socket says 172.21.15.116:39898) 2024-05-14T11:16:37.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44e3e00640 1 -- 172.21.15.116:0/4131656831 learned_addr learned my addr 172.21.15.116:0/4131656831 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44e3400640 1 --2- 172.21.15.116:0/4131656831 >> v2:172.21.15.116:3300/0 conn(0x7f44e4070320 0x7f44e40af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.1 v1:172.21.15.149:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 107492440 0 0) 0x7fade0003530 con 0x7fadfc0d1690 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 >> v2:172.21.15.116:3300/0 conn(0x7fadfc0d1aa0 msgr2=0x7fadfc0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 --2- 172.21.15.116:0/1530254986 >> v2:172.21.15.116:3300/0 conn(0x7fadfc0d1aa0 0x7fadfc0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 >> v1:172.21.15.116:6789/0 conn(0x7fadfc164400 legacy=0x7fadfc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 --> v1:172.21.15.149:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc165170 con 0x7fadfc0d1690 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.1 v1:172.21.15.149:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fade8001700 con 0x7fadfc0d1690 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 >> v1:172.21.15.149:6789/0 conn(0x7fadfc0d1690 legacy=0x7fadfc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0003a10 0x7fade0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fade0005280 0x7fade000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc164e90 con 0x7fade0004810 2024-05-14T11:16:37.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fade000c1d0 con 0x7fade0003a10 2024-05-14T11:16:37.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fade000c3d0 con 0x7fade0005280 2024-05-14T11:16:37.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.476+0000 7f44e2a00640 1 --2- 172.21.15.116:0/4131656831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fae00c00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0003a10 0x7fade0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fae01600640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fade0005280 0x7fade000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2000640 1 -- 172.21.15.116:0/4131656831 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127007979 0 0) 0x7f44e40cf730 con 0x7f44e40cbe40 2024-05-14T11:16:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2000640 1 -- 172.21.15.116:0/4131656831 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44c8005740 con 0x7f44e40cbe40 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 -- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fade0005280 msgr2=0x7fade000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fade0005280 0x7fade000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 -- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0003a10 msgr2=0x7fade0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0003a10 0x7fade0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7fadfbe00640 1 -- 172.21.15.116:0/1530254986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc165170 con 0x7fade0004810 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2a00640 1 -- 172.21.15.116:0/4131656831 >> v2:172.21.15.116:3300/0 conn(0x7f44e4070320 msgr2=0x7f44e40af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2a00640 1 --2- 172.21.15.116:0/4131656831 >> v2:172.21.15.116:3300/0 conn(0x7f44e4070320 0x7f44e40af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2a00640 1 -- 172.21.15.116:0/4131656831 >> v1:172.21.15.116:6789/0 conn(0x7f44e40cbe40 legacy=0x7f44e40cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.480+0000 7f44e2a00640 1 -- 172.21.15.116:0/4131656831 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40cf450 con 0x7f44e406eb40 2024-05-14T11:16:37.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7fae00c00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0003a10 0x7fade0003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:37.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7fae01600640 1 -- 172.21.15.116:0/1530254986 reap_dead start 2024-05-14T11:16:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7f44e2a00640 1 --2- 172.21.15.116:0/4131656831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e406ef50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cf380 tx=0x7f44d401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe1b87f30ac15cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7f44e2000640 1 -- 172.21.15.116:0/4131656831 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b1070 con 0x7f44e406eb40 2024-05-14T11:16:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7f44e2000640 1 -- 172.21.15.116:0/4131656831 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d40ac410 con 0x7f44e406eb40 2024-05-14T11:16:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.484+0000 7f44e2000640 1 -- 172.21.15.116:0/4131656831 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40ac7c0 con 0x7f44e406eb40 2024-05-14T11:16:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 -- 172.21.15.116:0/4131656831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 msgr2=0x7f44e406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 --2- 172.21.15.116:0/4131656831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e406ef50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cf380 tx=0x7f44d401c4c0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fadfbe00640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 0x7fade0004c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fade800c320 tx=0x7fade801ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdaacc893a348ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 -- 172.21.15.116:0/4131656831 shutdown_connections 2024-05-14T11:16:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 --2- 172.21.15.116:0/4131656831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e406ef50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 --2- 172.21.15.116:0/4131656831 >> v2:172.21.15.116:3300/0 conn(0x7f44e4070320 0x7f44e40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 -- 172.21.15.116:0/4131656831 >> 172.21.15.116:0/4131656831 conn(0x7f44e40d10f0 msgr2=0x7f44e40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8007da0 con 0x7fade0004810 2024-05-14T11:16:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 -- 172.21.15.116:0/4131656831 shutdown_connections 2024-05-14T11:16:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7f44ea200640 1 -- 172.21.15.116:0/4131656831 wait complete. 2024-05-14T11:16:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade80a9020 con 0x7fade0004810 2024-05-14T11:16:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fadfb400640 1 -- 172.21.15.116:0/1530254986 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8005b00 con 0x7fade0004810 2024-05-14T11:16:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 msgr2=0x7fade0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.488+0000 7fae02000640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 0x7fade0004c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fade800c320 tx=0x7fade801ccd0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 shutdown_connections 2024-05-14T11:16:37.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7fae02000640 1 --2- 172.21.15.116:0/1530254986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 >> 172.21.15.116:0/1530254986 conn(0x7fadfc0b4510 msgr2=0x7fadfc0b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 shutdown_connections 2024-05-14T11:16:37.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7fae02000640 1 -- 172.21.15.116:0/1530254986 wait complete. 2024-05-14T11:16:37.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7f44ea200640 1 Processor -- start 2024-05-14T11:16:37.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.492+0000 7f44ea200640 1 -- start start 2024-05-14T11:16:37.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7fae02000640 1 Processor -- start 2024-05-14T11:16:37.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7fae02000640 1 -- start start 2024-05-14T11:16:37.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44ea200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44ea200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44e4070320 0x7f44e40bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44ea200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 0x7f44e406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44e3400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44e3400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:40092/0 (socket says 172.21.15.116:40092) 2024-05-14T11:16:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44e3400640 1 -- 172.21.15.116:0/3549758512 learned_addr learned my addr 172.21.15.116:0/3549758512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44e2a00640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44e4070320 0x7f44e40bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44e3e00640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 0x7f44e406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.496+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40af540 con 0x7f44e4070320 2024-05-14T11:16:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44e409f040 con 0x7f44e40cbe40 2024-05-14T11:16:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40af6c0 con 0x7f44e406eb40 2024-05-14T11:16:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 -- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 msgr2=0x7f44e406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 0x7f44e406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 -- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44e4070320 msgr2=0x7f44e40bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44e4070320 0x7f44e40bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e409e850 con 0x7f44e406eb40 2024-05-14T11:16:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3e00640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 0x7f44e406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44e3400640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00efc0 tx=0x7f44dc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 0x7fadfc15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc018020 con 0x7f44e406eb40 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc15e7b0 0x7fadfc0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae00c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae00c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36152/0 (socket says 172.21.15.116:36152) 2024-05-14T11:16:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 0x7fadfc15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 0x7fadfc15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:40106/0 (socket says 172.21.15.116:40106) 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae00c00640 1 -- 172.21.15.116:0/913598647 learned_addr learned my addr 172.21.15.116:0/913598647 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae01600640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc15e7b0 0x7fadfc0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc09acc0 con 0x7fadfc15e7b0 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc09ab40 con 0x7fade0004810 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7fae02000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fadfc09ae40 con 0x7fadfc0ca120 2024-05-14T11:16:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc014480 con 0x7f44e406eb40 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc014850 con 0x7f44e406eb40 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.500+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e415dfc0 con 0x7f44e406eb40 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e415e580 con 0x7f44e406eb40 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 -- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 msgr2=0x7fadfc15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 0x7fadfc15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 -- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc15e7b0 msgr2=0x7fadfc0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc15e7b0 0x7fadfc0c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc15ebc0 con 0x7fade0004810 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae00c00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e500 tx=0x7fadec00e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e406a9c0 con 0x7f44e406eb40 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec0052e0 con 0x7fade0004810 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec018070 con 0x7fade0004810 2024-05-14T11:16:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec003820 con 0x7fade0004810 2024-05-14T11:16:37.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae02000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc0ce4f0 con 0x7fade0004810 2024-05-14T11:16:37.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7fae02000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc0ccac0 con 0x7fade0004810 2024-05-14T11:16:37.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd00052a0 con 0x7fade0004810 2024-05-14T11:16:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.504+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44dc01b020 con 0x7f44e406eb40 2024-05-14T11:16:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7f44db400640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 0x7f44c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fadec0039c0 con 0x7fade0004810 2024-05-14T11:16:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7f44e2a00640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 0x7f44c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f44dc08fed0 con 0x7f44e406eb40 2024-05-14T11:16:37.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7fadf8c00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fadec08f1b0 con 0x7fade0004810 2024-05-14T11:16:37.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.508+0000 7f44e2a00640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 0x7f44c4071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae000 tx=0x7f44d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.512+0000 7fadfbe00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.512+0000 7fadfbe00640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fade800c2f0 tx=0x7fade8001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.512+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f44dc022070 con 0x7f44e406eb40 2024-05-14T11:16:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.512+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fadfc15ebc0 con 0x7fade0004810 2024-05-14T11:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 Processor -- start 2024-05-14T11:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 -- start start 2024-05-14T11:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0f00af3a0 0x7fe0f01640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe0f00c4910 0x7fe0f00cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f00d0f50 con 0x7fe0f0168680 2024-05-14T11:16:37.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f009f100 con 0x7fe0f00af3a0 2024-05-14T11:16:37.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0f6400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f00d0d00 con 0x7fe0f00c4910 2024-05-14T11:16:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.532+0000 7fe0efe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe0f00c4910 0x7fe0f00cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe0f00c4910 0x7fe0f00cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36154/0 (socket says 172.21.15.116:36154) 2024-05-14T11:16:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 -- 172.21.15.116:0/2664140602 learned_addr learned my addr 172.21.15.116:0/2664140602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0ef400640 1 --2- 172.21.15.116:0/2664140602 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00af3a0 0x7fe0f01640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 -- 172.21.15.116:0/2664140602 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00af3a0 msgr2=0x7fe0f01640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 --2- 172.21.15.116:0/2664140602 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00af3a0 0x7fe0f01640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 -- 172.21.15.116:0/2664140602 >> v1:172.21.15.149:6789/0 conn(0x7fe0f0168680 legacy=0x7fe0f00c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fe0efe00640 1 -- 172.21.15.116:0/2664140602 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f00d07b0 con 0x7fe0f00c4910 2024-05-14T11:16:37.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.536+0000 7fc299800640 1 Processor -- start 2024-05-14T11:16:37.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 -- start start 2024-05-14T11:16:37.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0efe00640 1 --2- 172.21.15.116:0/2664140602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00c4910 0x7fe0f00cf130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00727a0 tx=0x7fe0dc01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6edf7d3e69b2e98a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0ee000640 1 -- 172.21.15.116:0/2664140602 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0b0070 con 0x7fe0f00c4910 2024-05-14T11:16:37.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0ee000640 1 -- 172.21.15.116:0/2664140602 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc0ab600 con 0x7fe0f00c4910 2024-05-14T11:16:37.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0ee000640 1 -- 172.21.15.116:0/2664140602 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0ab9d0 con 0x7fe0f00c4910 2024-05-14T11:16:37.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 -- 172.21.15.116:0/2664140602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00c4910 msgr2=0x7fe0f00cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 --2- 172.21.15.116:0/2664140602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00c4910 0x7fe0f00cf130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00727a0 tx=0x7fe0dc01c540 comp rx=0 tx=0).stop 2024-05-14T11:16:37.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 -- 172.21.15.116:0/2664140602 shutdown_connections 2024-05-14T11:16:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 0x7fc2940d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2940d1d90 0x7fc29415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2940cde50 con 0x7fc29415bd50 2024-05-14T11:16:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc2940ce130 con 0x7fc2940d1d90 2024-05-14T11:16:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc299800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc2940ce410 con 0x7fc294074400 2024-05-14T11:16:37.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 --2- 172.21.15.116:0/2664140602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00c4910 0x7fe0f00cf130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 --2- 172.21.15.116:0/2664140602 >> v2:172.21.15.116:3300/0 conn(0x7fe0f00af3a0 0x7fe0f01640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 -- 172.21.15.116:0/2664140602 >> 172.21.15.116:0/2664140602 conn(0x7fe0f00cde80 msgr2=0x7fe0f00ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 -- 172.21.15.116:0/2664140602 shutdown_connections 2024-05-14T11:16:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc293e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 0x7fc2940d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc293400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2940d1d90 0x7fc29415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc293e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 0x7fc2940d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36160/0 (socket says 172.21.15.116:36160) 2024-05-14T11:16:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fc293e00640 1 -- 172.21.15.116:0/3679547771 learned_addr learned my addr 172.21.15.116:0/3679547771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.540+0000 7fe0f6400640 1 -- 172.21.15.116:0/2664140602 wait complete. 2024-05-14T11:16:37.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fe0f6400640 1 Processor -- start 2024-05-14T11:16:37.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fe0f6400640 1 -- start start 2024-05-14T11:16:37.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fc293400640 1 -- 172.21.15.116:0/3679547771 >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 msgr2=0x7fc2940d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:37.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fc293400640 1 --2- 172.21.15.116:0/3679547771 >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 0x7fc2940d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fc293400640 1 -- 172.21.15.116:0/3679547771 >> v1:172.21.15.149:6789/0 conn(0x7fc29415bd50 legacy=0x7fc2940b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fc293400640 1 -- 172.21.15.116:0/3679547771 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2940ce6f0 con 0x7fc2940d1d90 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.544+0000 7fc292a00640 1 -- 172.21.15.116:0/3679547771 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548595848 0 0) 0x7fc2940cde50 con 0x7fc29415bd50 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fc293400640 1 --2- 172.21.15.116:0/3679547771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940d1d90 0x7fc29415b810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc294073ca0 tx=0x7fc28c020a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec11fddb722f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fc292a00640 1 -- 172.21.15.116:0/3679547771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00e2d0 con 0x7fc2940d1d90 2024-05-14T11:16:37.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fc292a00640 1 -- 172.21.15.116:0/3679547771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc28c007020 con 0x7fc2940d1d90 2024-05-14T11:16:37.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00af3a0 0x7fe0f0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fc292a00640 1 -- 172.21.15.116:0/3679547771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00e2d0 con 0x7fc2940d1d90 2024-05-14T11:16:37.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 0x7fe0f00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00cfc00 con 0x7fe0f0168680 2024-05-14T11:16:37.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00cfa80 con 0x7fe0f00af3a0 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0f6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f0171990 con 0x7fe0f00b2140 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0eea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0eea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58576/0 (socket says 172.21.15.116:58576) 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.548+0000 7fe0eea00640 1 -- 172.21.15.116:0/3560189726 learned_addr learned my addr 172.21.15.116:0/3560189726 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0efe00640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 0x7fe0f00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 -- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 msgr2=0x7fe0f00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 -- 172.21.15.116:0/3679547771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940d1d90 msgr2=0x7fc29415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 --2- 172.21.15.116:0/3679547771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940d1d90 0x7fc29415b810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc294073ca0 tx=0x7fc28c020a70 comp rx=0 tx=0).stop 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 0x7fe0f00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 -- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00af3a0 msgr2=0x7fe0f0160820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00af3a0 0x7fe0f0160820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 -- 172.21.15.116:0/3560189726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f0160d60 con 0x7fe0f0168680 2024-05-14T11:16:37.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0efe00640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 0x7fe0f00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 -- 172.21.15.116:0/3679547771 shutdown_connections 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 --2- 172.21.15.116:0/3679547771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940d1d90 0x7fc29415b810 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 --2- 172.21.15.116:0/3679547771 >> v2:172.21.15.149:3300/0 conn(0x7fc294074400 0x7fc2940d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 -- 172.21.15.116:0/3679547771 >> 172.21.15.116:0/3679547771 conn(0x7fc2940b6100 msgr2=0x7fc2940b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0eea00640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c870 tx=0x7fe0e800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8003bb0 con 0x7fe0f0168680 2024-05-14T11:16:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 -- 172.21.15.116:0/3679547771 shutdown_connections 2024-05-14T11:16:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f00b0c30 con 0x7fe0f0168680 2024-05-14T11:16:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f00b1380 con 0x7fe0f0168680 2024-05-14T11:16:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f00b0e00 con 0x7fe0f0168680 2024-05-14T11:16:37.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.552+0000 7fc299800640 1 -- 172.21.15.116:0/3679547771 wait complete. 2024-05-14T11:16:37.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.556+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8003d50 con 0x7fe0f0168680 2024-05-14T11:16:37.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.556+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8003bb0 con 0x7fe0f0168680 2024-05-14T11:16:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.560+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe0e800a850 con 0x7fe0f0168680 2024-05-14T11:16:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.564+0000 7fe0e3400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 0x7fe0d0071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.564+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe0e808fb80 con 0x7fe0f0168680 2024-05-14T11:16:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.564+0000 7fc299800640 1 Processor -- start 2024-05-14T11:16:37.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.564+0000 7fc299800640 1 -- start start 2024-05-14T11:16:37.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fe0ef400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 0x7fe0d0071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29415bd50 0x7fc2941590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940c5fb0 0x7fc2940af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc29415c2e0 con 0x7fc2940c5fb0 2024-05-14T11:16:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc29415c160 con 0x7fc29415bd50 2024-05-14T11:16:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc299800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc29406d150 con 0x7fc294074400 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:40134/0 (socket says 172.21.15.116:40134) 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 -- 172.21.15.116:0/2474769611 learned_addr learned my addr 172.21.15.116:0/2474769611 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc298e00640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940c5fb0 0x7fc2940af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 -- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29415bd50 msgr2=0x7fc2941590c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29415bd50 0x7fc2941590c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 -- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940c5fb0 msgr2=0x7fc2940af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940c5fb0 0x7fc2940af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 -- 172.21.15.116:0/2474769611 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29415d070 con 0x7fc294074400 2024-05-14T11:16:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fe0ef400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 0x7fe0d0071b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc293e00640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc28400da80 tx=0x7fc28400df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe0f00b0e00 con 0x7fe0f0168680 2024-05-14T11:16:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.568+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284015400 con 0x7fc294074400 2024-05-14T11:16:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.572+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2840155a0 con 0x7fc294074400 2024-05-14T11:16:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.572+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284015950 con 0x7fc294074400 2024-05-14T11:16:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.572+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29415d350 con 0x7fc294074400 2024-05-14T11:16:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.572+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29415d910 con 0x7fc294074400 2024-05-14T11:16:37.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.576+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc284015af0 con 0x7fc294074400 2024-05-14T11:16:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.576+0000 7fc28be00640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 0x7fc274071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.576+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc28408fd80 con 0x7fc294074400 2024-05-14T11:16:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.576+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc294050880 con 0x7fc294074400 2024-05-14T11:16:37.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.576+0000 7fc293400640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 0x7fc274071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:37.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.584+0000 7fc293400640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 0x7fc274071940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b1000 tx=0x7fc28c0ab070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:37.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.584+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc28405c890 con 0x7fc294074400 2024-05-14T11:16:37.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.716+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd570161bf0 con 0x7fd5700cdc10 2024-05-14T11:16:37.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.720+0000 7fd567400640 1 -- 172.21.15.116:0/2108387630 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd55c05c050 con 0x7fd5700cdc10 2024-05-14T11:16:37.724 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:37.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 msgr2=0x7fd550071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 0x7fd550071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ae000 tx=0x7fd5600a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:37.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 msgr2=0x7fd57015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00b3e0 tx=0x7fd55c00b8a0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 shutdown_connections 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55006f400 0x7fd550071910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5700cdc10 0x7fd57015e3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5700d0520 0x7fd5700c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 --2- 172.21.15.116:0/2108387630 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd570073dd0 0x7fd5700c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 >> 172.21.15.116:0/2108387630 conn(0x7fd5700b6100 msgr2=0x7fd570157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 shutdown_connections 2024-05-14T11:16:37.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.724+0000 7fd574a00640 1 -- 172.21.15.116:0/2108387630 wait complete. 2024-05-14T11:16:37.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-05-14T11:16:37.758 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.756+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fee98005590 con 0x7feebc0b1890 2024-05-14T11:16:37.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.756+0000 7feeb9600640 1 -- 172.21.15.116:0/516565160 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feeb4016580 con 0x7feebc0b1890 2024-05-14T11:16:37.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.756+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f35880b5670 con 0x7f3588073e20 2024-05-14T11:16:37.759 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-05-14T11:16:37.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.756+0000 7f357f400640 1 -- 172.21.15.116:0/3154676118 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35800f7140 con 0x7f3588073e20 2024-05-14T11:16:37.760 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-05-14T11:16:37.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8108005860 con 0x7f81300aec70 2024-05-14T11:16:37.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f8127400640 1 -- 172.21.15.116:0/794795539 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f812005fbd0 con 0x7f81300aec70 2024-05-14T11:16:37.762 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:37.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 msgr2=0x7fee9c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 0x7fee9c0719e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feeb00045f0 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2024-05-14T11:16:37.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 msgr2=0x7feebc1630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 0x7feebc1630c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feeb40ac000 tx=0x7feebc0afb10 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 msgr2=0x7f3564071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 0x7f3564071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f358815be30 tx=0x7f3570008040 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 msgr2=0x7f35880b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35800af040 tx=0x7f3580019740 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 shutdown_connections 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f356406f400 0x7f3564071910 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f358815be30 tx=0x7f3570008040 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3588163910 0x7f358809cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35880b0a90 0x7f358809c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 --2- 172.21.15.116:0/3154676118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3588073e20 0x7f35880b2fb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 >> 172.21.15.116:0/3154676118 conn(0x7f3588072c60 msgr2=0x7f3588162b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 shutdown_connections 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee9c06f4d0 0x7fee9c0719e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feebc0b1890 0x7feebc1630c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feebc0b1480 0x7feebc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 --2- 172.21.15.116:0/516565160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feebc074360 0x7feebc0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 >> 172.21.15.116:0/516565160 conn(0x7feebc072c60 msgr2=0x7feebc1597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 shutdown_connections 2024-05-14T11:16:37.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 shutdown_connections 2024-05-14T11:16:37.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7f358e000640 1 -- 172.21.15.116:0/3154676118 wait complete. 2024-05-14T11:16:37.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.760+0000 7feec3e00640 1 -- 172.21.15.116:0/516565160 wait complete. 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 msgr2=0x7f8110071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 0x7f8110071b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8128007f20 tx=0x7f8128007e70 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 msgr2=0x7f81300cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 0x7f81300cc0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f812000e870 tx=0x7f812000ed30 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 shutdown_connections 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811006f630 0x7f8110071b40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81300b0a90 0x7f813009e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81300aec70 0x7f81300cc0a0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f812000e870 tx=0x7f812000ed30 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 --2- 172.21.15.116:0/794795539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8130073e20 0x7f81300d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 >> 172.21.15.116:0/794795539 conn(0x7f8130072c60 msgr2=0x7f813006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 shutdown_connections 2024-05-14T11:16:37.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.764+0000 7f8136600640 1 -- 172.21.15.116:0/794795539 wait complete. 2024-05-14T11:16:37.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-05-14T11:16:37.800 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-05-14T11:16:37.801 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.808+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f44e406f040 con 0x7f44e406eb40 2024-05-14T11:16:37.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.808+0000 7f44db400640 1 -- 172.21.15.116:0/3549758512 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44e409e850 con 0x7f44e406eb40 2024-05-14T11:16:37.812 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:37.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-05-14T11:16:37.813 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 msgr2=0x7f44c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 0x7f44c4071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae000 tx=0x7f44d4008040 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 msgr2=0x7f44e415c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00efc0 tx=0x7f44dc00c490 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 shutdown_connections 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f44c406f400 0x7f44c4071910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44e40cbe40 0x7f44e406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44e4070320 0x7f44e40bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 --2- 172.21.15.116:0/3549758512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44e406eb40 0x7f44e415c130 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 >> 172.21.15.116:0/3549758512 conn(0x7f44e40d10f0 msgr2=0x7f44e40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 shutdown_connections 2024-05-14T11:16:37.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.812+0000 7f44ea200640 1 -- 172.21.15.116:0/3549758512 wait complete. 2024-05-14T11:16:37.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-05-14T11:16:37.835 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.848+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fadd0005590 con 0x7fade0004810 2024-05-14T11:16:37.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.848+0000 7fadf8c00640 1 -- 172.21.15.116:0/913598647 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fadec05bd60 con 0x7fade0004810 2024-05-14T11:16:37.852 INFO:teuthology.orchestra.run.smithi116.stdout:55834574860 2024-05-14T11:16:37.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 msgr2=0x7faddc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fade800c2f0 tx=0x7fade8001a00 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 msgr2=0x7fadfc0cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadec00e500 tx=0x7fadec00e9c0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 shutdown_connections 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc15e7b0 0x7fadfc0c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fadfc0ca120 0x7fadfc15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 --2- 172.21.15.116:0/913598647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fade0004810 0x7fadfc0cda30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 >> 172.21.15.116:0/913598647 conn(0x7fadfc0b4510 msgr2=0x7fadfc072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 shutdown_connections 2024-05-14T11:16:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fadf2000640 1 -- 172.21.15.116:0/913598647 wait complete. 2024-05-14T11:16:37.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe0f006a9c0 con 0x7fe0f0168680 2024-05-14T11:16:37.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.852+0000 7fe0e3400640 1 -- 172.21.15.116:0/3560189726 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe0e805c830 con 0x7fe0f0168680 2024-05-14T11:16:37.856 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 msgr2=0x7fe0d0071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 0x7fe0d0071b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4008040 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 msgr2=0x7fe0f00ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c870 tx=0x7fe0e800cd30 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 shutdown_connections 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe0d006f600 0x7fe0d0071b10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0f00b2140 0x7fe0f00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0f0168680 0x7fe0f00ca870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 --2- 172.21.15.116:0/3560189726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0f00af3a0 0x7fe0f0160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 >> 172.21.15.116:0/3560189726 conn(0x7fe0f00cde80 msgr2=0x7fe0f006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 shutdown_connections 2024-05-14T11:16:37.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.856+0000 7fe0f6400640 1 -- 172.21.15.116:0/3560189726 wait complete. 2024-05-14T11:16:37.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-05-14T11:16:37.878 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-05-14T11:16:37.881 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.884+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc2940c9130 con 0x7fc294074400 2024-05-14T11:16:37.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.884+0000 7fc28be00640 1 -- 172.21.15.116:0/2474769611 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc28405c190 con 0x7fc294074400 2024-05-14T11:16:37.888 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 msgr2=0x7fc274071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 0x7fc274071940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b1000 tx=0x7fc28c0ab070 comp rx=0 tx=0).stop 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 msgr2=0x7fc29415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc28400da80 tx=0x7fc28400df40 comp rx=0 tx=0).stop 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 shutdown_connections 2024-05-14T11:16:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27406f430 0x7fc274071940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29415bd50 0x7fc2941590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2940c5fb0 0x7fc2940af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 --2- 172.21.15.116:0/2474769611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc294074400 0x7fc29415c700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 >> 172.21.15.116:0/2474769611 conn(0x7fc2940b6100 msgr2=0x7fc2940cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 shutdown_connections 2024-05-14T11:16:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:37.888+0000 7fc299800640 1 -- 172.21.15.116:0/2474769611 wait complete. 2024-05-14T11:16:37.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-05-14T11:16:37.909 DEBUG:teuthology.parallel:result is None 2024-05-14T11:16:37.909 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:16:37.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:16:38.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 Processor -- start 2024-05-14T11:16:38.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 -- start start 2024-05-14T11:16:38.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00cdf80 0x7f55f00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b40c0 0x7f55f00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f00b4a10 con 0x7f55f00d1510 2024-05-14T11:16:38.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f55f00b3060 con 0x7f55f00b40c0 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f7a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00b3300 con 0x7f55f00cdf80 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b40c0 0x7f55f00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b40c0 0x7f55f00b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58590/0 (socket says 172.21.15.116:58590) 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 -- 172.21.15.116:0/1616366676 learned_addr learned my addr 172.21.15.116:0/1616366676 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5000640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00cdf80 0x7f55f00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 -- 172.21.15.116:0/1616366676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00cdf80 msgr2=0x7f55f00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00cdf80 0x7f55f00ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 -- 172.21.15.116:0/1616366676 >> v1:172.21.15.149:6789/0 conn(0x7f55f00d1510 legacy=0x7f55f00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.100+0000 7f55f5a00640 1 -- 172.21.15.116:0/1616366676 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00b35e0 con 0x7f55f00b40c0 2024-05-14T11:16:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f5a00640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00b40c0 0x7f55f00b44d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009e50 tx=0x7f55e801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862ab1a1a13f5259 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55efe00640 1 -- 172.21.15.116:0/1616366676 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e80a58d0 con 0x7f55f00b40c0 2024-05-14T11:16:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55efe00640 1 -- 172.21.15.116:0/1616366676 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e80a5a70 con 0x7f55f00b40c0 2024-05-14T11:16:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55efe00640 1 -- 172.21.15.116:0/1616366676 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e80a5e40 con 0x7f55f00b40c0 2024-05-14T11:16:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 -- 172.21.15.116:0/1616366676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00b40c0 msgr2=0x7f55f00b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00b40c0 0x7f55f00b44d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009e50 tx=0x7f55e801ca00 comp rx=0 tx=0).stop 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 -- 172.21.15.116:0/1616366676 shutdown_connections 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00b40c0 0x7f55f00b44d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 --2- 172.21.15.116:0/1616366676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00cdf80 0x7f55f00ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 -- 172.21.15.116:0/1616366676 >> 172.21.15.116:0/1616366676 conn(0x7f55f00d0770 msgr2=0x7f55f00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 -- 172.21.15.116:0/1616366676 shutdown_connections 2024-05-14T11:16:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.104+0000 7f55f7a00640 1 -- 172.21.15.116:0/1616366676 wait complete. 2024-05-14T11:16:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 Processor -- start 2024-05-14T11:16:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 -- start start 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00b40c0 0x7f55f01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f00d1510 0x7f55f0151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f55f006ff40 con 0x7f55f00cdf80 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55f006fdc0 con 0x7f55f00d1510 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f7a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f55f016cd60 con 0x7f55f00b40c0 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55e7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55e7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58606/0 (socket says 172.21.15.116:58606) 2024-05-14T11:16:38.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55e7e00640 1 -- 172.21.15.116:0/3634908864 learned_addr learned my addr 172.21.15.116:0/3634908864 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.108+0000 7f55f5a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f00d1510 0x7f55f0151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55f5000640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00b40c0 0x7f55f01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 -- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00b40c0 msgr2=0x7f55f01524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00b40c0 0x7f55f01524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 -- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f00d1510 msgr2=0x7f55f0151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f00d1510 0x7f55f0151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 -- 172.21.15.116:0/3634908864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00b6f70 con 0x7f55f00cdf80 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55e7e00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c3c0 tx=0x7f55d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8017070 con 0x7f55f00cdf80 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d80052d0 con 0x7f55f00cdf80 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8003800 con 0x7f55f00cdf80 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f00cb7b0 con 0x7f55f00cdf80 2024-05-14T11:16:38.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f00cbd70 con 0x7f55f00cdf80 2024-05-14T11:16:38.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f006a9c0 con 0x7f55f00cdf80 2024-05-14T11:16:38.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.112+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f55d801a070 con 0x7f55f00cdf80 2024-05-14T11:16:38.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.116+0000 7f55ed600640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.116+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f55d808e8f0 con 0x7f55f00cdf80 2024-05-14T11:16:38.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.116+0000 7f55f5000640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.116+0000 7f55f5000640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 0x7f55cc071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009ac0 tx=0x7f55e0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.120+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f55d8005480 con 0x7f55f00cdf80 2024-05-14T11:16:38.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.272+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55f00b9590 con 0x7f55cc06f400 2024-05-14T11:16:38.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.284+0000 7f55ed600640 1 -- 172.21.15.116:0/3634908864 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f55f00b9590 con 0x7f55cc06f400 2024-05-14T11:16:38.289 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:16:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 msgr2=0x7f55cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 0x7f55cc071910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009ac0 tx=0x7f55e0009190 comp rx=0 tx=0).stop 2024-05-14T11:16:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 msgr2=0x7f55f00b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c3c0 tx=0x7f55d800c880 comp rx=0 tx=0).stop 2024-05-14T11:16:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 shutdown_connections 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f00d1510 0x7f55f0151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f00cdf80 0x7f55f00b6a30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 --2- 172.21.15.116:0/3634908864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f00b40c0 0x7f55f01524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 >> 172.21.15.116:0/3634908864 conn(0x7f55f00d0770 msgr2=0x7f55f0154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 shutdown_connections 2024-05-14T11:16:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.288+0000 7f55f7a00640 1 -- 172.21.15.116:0/3634908864 wait complete. 2024-05-14T11:16:38.310 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-05-14T11:16:36.364009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003391"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807671+0000","last_change":"2024-05-14T11:16:11.249111+0000","last_active":"2024-05-14T11:16:17.807671+0000","last_peered":"2024-05-14T11:16:17.807671+0000","last_clean":"2024-05-14T11:16:17.807671+0000","last_became_active":"2024-05-14T11:16:11.248739+0000","last_became_peered":"2024-05-14T11:16:11.248739+0000","last_unstale":"2024-05-14T11:16:17.807671+0000","last_undegraded":"2024-05-14T11:16:17.807671+0000","last_fullsized":"2024-05-14T11:16:17.807671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808824+0000","last_change":"2024-05-14T11:16:12.266252+0000","last_active":"2024-05-14T11:16:17.808824+0000","last_peered":"2024-05-14T11:16:17.808824+0000","last_clean":"2024-05-14T11:16:17.808824+0000","last_became_active":"2024-05-14T11:16:12.265805+0000","last_became_peered":"2024-05-14T11:16:12.265805+0000","last_unstale":"2024-05-14T11:16:17.808824+0000","last_undegraded":"2024-05-14T11:16:17.808824+0000","last_fullsized":"2024-05-14T11:16:17.808824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808167+0000","last_change":"2024-05-14T11:16:16.339345+0000","last_active":"2024-05-14T11:16:17.808167+0000","last_peered":"2024-05-14T11:16:17.808167+0000","last_clean":"2024-05-14T11:16:17.808167+0000","last_became_active":"2024-05-14T11:16:16.338538+0000","last_became_peered":"2024-05-14T11:16:16.338538+0000","last_unstale":"2024-05-14T11:16:17.808167+0000","last_undegraded":"2024-05-14T11:16:17.808167+0000","last_fullsized":"2024-05-14T11:16:17.808167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809078+0000","last_change":"2024-05-14T11:16:11.246975+0000","last_active":"2024-05-14T11:16:17.809078+0000","last_peered":"2024-05-14T11:16:17.809078+0000","last_clean":"2024-05-14T11:16:17.809078+0000","last_became_active":"2024-05-14T11:16:11.246617+0000","last_became_peered":"2024-05-14T11:16:11.246617+0000","last_unstale":"2024-05-14T11:16:17.809078+0000","last_undegraded":"2024-05-14T11:16:17.809078+0000","last_fullsized":"2024-05-14T11:16:17.809078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807310+0000","last_change":"2024-05-14T11:16:12.261153+0000","last_active":"2024-05-14T11:16:17.807310+0000","last_peered":"2024-05-14T11:16:17.807310+0000","last_clean":"2024-05-14T11:16:17.807310+0000","last_became_active":"2024-05-14T11:16:12.260566+0000","last_became_peered":"2024-05-14T11:16:12.260566+0000","last_unstale":"2024-05-14T11:16:17.807310+0000","last_undegraded":"2024-05-14T11:16:17.807310+0000","last_fullsized":"2024-05-14T11:16:17.807310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:47.069928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300836+0000","last_change":"2024-05-14T11:16:16.298112+0000","last_active":"2024-05-14T11:16:17.300836+0000","last_peered":"2024-05-14T11:16:17.300836+0000","last_clean":"2024-05-14T11:16:17.300836+0000","last_became_active":"2024-05-14T11:16:16.297673+0000","last_became_peered":"2024-05-14T11:16:16.297673+0000","last_unstale":"2024-05-14T11:16:17.300836+0000","last_undegraded":"2024-05-14T11:16:17.300836+0000","last_fullsized":"2024-05-14T11:16:17.300836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:13.339957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807596+0000","last_change":"2024-05-14T11:16:11.253925+0000","last_active":"2024-05-14T11:16:17.807596+0000","last_peered":"2024-05-14T11:16:17.807596+0000","last_clean":"2024-05-14T11:16:17.807596+0000","last_became_active":"2024-05-14T11:16:11.253441+0000","last_became_peered":"2024-05-14T11:16:11.253441+0000","last_unstale":"2024-05-14T11:16:17.807596+0000","last_undegraded":"2024-05-14T11:16:17.807596+0000","last_fullsized":"2024-05-14T11:16:17.807596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806697+0000","last_change":"2024-05-14T11:16:12.255735+0000","last_active":"2024-05-14T11:16:17.806697+0000","last_peered":"2024-05-14T11:16:17.806697+0000","last_clean":"2024-05-14T11:16:17.806697+0000","last_became_active":"2024-05-14T11:16:12.255414+0000","last_became_peered":"2024-05-14T11:16:12.255414+0000","last_unstale":"2024-05-14T11:16:17.806697+0000","last_undegraded":"2024-05-14T11:16:17.806697+0000","last_fullsized":"2024-05-14T11:16:17.806697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300072+0000","last_change":"2024-05-14T11:16:16.297447+0000","last_active":"2024-05-14T11:16:17.300072+0000","last_peered":"2024-05-14T11:16:17.300072+0000","last_clean":"2024-05-14T11:16:17.300072+0000","last_became_active":"2024-05-14T11:16:16.297024+0000","last_became_peered":"2024-05-14T11:16:16.297024+0000","last_unstale":"2024-05-14T11:16:17.300072+0000","last_undegraded":"2024-05-14T11:16:17.300072+0000","last_fullsized":"2024-05-14T11:16:17.300072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:41.465988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806953+0000","last_change":"2024-05-14T11:16:11.263356+0000","last_active":"2024-05-14T11:16:17.806953+0000","last_peered":"2024-05-14T11:16:17.806953+0000","last_clean":"2024-05-14T11:16:17.806953+0000","last_became_active":"2024-05-14T11:16:11.263025+0000","last_became_peered":"2024-05-14T11:16:11.263025+0000","last_unstale":"2024-05-14T11:16:17.806953+0000","last_undegraded":"2024-05-14T11:16:17.806953+0000","last_fullsized":"2024-05-14T11:16:17.806953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806916+0000","last_change":"2024-05-14T11:16:12.253871+0000","last_active":"2024-05-14T11:16:17.806916+0000","last_peered":"2024-05-14T11:16:17.806916+0000","last_clean":"2024-05-14T11:16:17.806916+0000","last_became_active":"2024-05-14T11:16:12.253416+0000","last_became_peered":"2024-05-14T11:16:12.253416+0000","last_unstale":"2024-05-14T11:16:17.806916+0000","last_undegraded":"2024-05-14T11:16:17.806916+0000","last_fullsized":"2024-05-14T11:16:17.806916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300363+0000","last_change":"2024-05-14T11:16:16.289232+0000","last_active":"2024-05-14T11:16:17.300363+0000","last_peered":"2024-05-14T11:16:17.300363+0000","last_clean":"2024-05-14T11:16:17.300363+0000","last_became_active":"2024-05-14T11:16:16.288706+0000","last_became_peered":"2024-05-14T11:16:16.288706+0000","last_unstale":"2024-05-14T11:16:17.300363+0000","last_undegraded":"2024-05-14T11:16:17.300363+0000","last_fullsized":"2024-05-14T11:16:17.300363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:13.765975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299905+0000","last_change":"2024-05-14T11:16:12.237866+0000","last_active":"2024-05-14T11:16:17.299905+0000","last_peered":"2024-05-14T11:16:17.299905+0000","last_clean":"2024-05-14T11:16:17.299905+0000","last_became_active":"2024-05-14T11:16:12.237591+0000","last_became_peered":"2024-05-14T11:16:12.237591+0000","last_unstale":"2024-05-14T11:16:17.299905+0000","last_undegraded":"2024-05-14T11:16:17.299905+0000","last_fullsized":"2024-05-14T11:16:17.299905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807660+0000","last_change":"2024-05-14T11:16:11.250842+0000","last_active":"2024-05-14T11:16:17.807660+0000","last_peered":"2024-05-14T11:16:17.807660+0000","last_clean":"2024-05-14T11:16:17.807660+0000","last_became_active":"2024-05-14T11:16:11.250454+0000","last_became_peered":"2024-05-14T11:16:11.250454+0000","last_unstale":"2024-05-14T11:16:17.807660+0000","last_undegraded":"2024-05-14T11:16:17.807660+0000","last_fullsized":"2024-05-14T11:16:17.807660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302586+0000","last_change":"2024-05-14T11:16:16.299192+0000","last_active":"2024-05-14T11:16:17.302586+0000","last_peered":"2024-05-14T11:16:17.302586+0000","last_clean":"2024-05-14T11:16:17.302586+0000","last_became_active":"2024-05-14T11:16:16.298925+0000","last_became_peered":"2024-05-14T11:16:16.298925+0000","last_unstale":"2024-05-14T11:16:17.302586+0000","last_undegraded":"2024-05-14T11:16:17.302586+0000","last_fullsized":"2024-05-14T11:16:17.302586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808972+0000","last_change":"2024-05-14T11:16:12.266594+0000","last_active":"2024-05-14T11:16:17.808972+0000","last_peered":"2024-05-14T11:16:17.808972+0000","last_clean":"2024-05-14T11:16:17.808972+0000","last_became_active":"2024-05-14T11:16:12.266261+0000","last_became_peered":"2024-05-14T11:16:12.266261+0000","last_unstale":"2024-05-14T11:16:17.808972+0000","last_undegraded":"2024-05-14T11:16:17.808972+0000","last_fullsized":"2024-05-14T11:16:17.808972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808959+0000","last_change":"2024-05-14T11:16:11.251548+0000","last_active":"2024-05-14T11:16:17.808959+0000","last_peered":"2024-05-14T11:16:17.808959+0000","last_clean":"2024-05-14T11:16:17.808959+0000","last_became_active":"2024-05-14T11:16:11.251146+0000","last_became_peered":"2024-05-14T11:16:11.251146+0000","last_unstale":"2024-05-14T11:16:17.808959+0000","last_undegraded":"2024-05-14T11:16:17.808959+0000","last_fullsized":"2024-05-14T11:16:17.808959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300981+0000","last_change":"2024-05-14T11:16:16.296270+0000","last_active":"2024-05-14T11:16:17.300981+0000","last_peered":"2024-05-14T11:16:17.300981+0000","last_clean":"2024-05-14T11:16:17.300981+0000","last_became_active":"2024-05-14T11:16:16.295796+0000","last_became_peered":"2024-05-14T11:16:16.295796+0000","last_unstale":"2024-05-14T11:16:17.300981+0000","last_undegraded":"2024-05-14T11:16:17.300981+0000","last_fullsized":"2024-05-14T11:16:17.300981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808335+0000","last_change":"2024-05-14T11:16:12.259552+0000","last_active":"2024-05-14T11:16:17.808335+0000","last_peered":"2024-05-14T11:16:17.808335+0000","last_clean":"2024-05-14T11:16:17.808335+0000","last_became_active":"2024-05-14T11:16:12.259053+0000","last_became_peered":"2024-05-14T11:16:12.259053+0000","last_unstale":"2024-05-14T11:16:17.808335+0000","last_undegraded":"2024-05-14T11:16:17.808335+0000","last_fullsized":"2024-05-14T11:16:17.808335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806762+0000","last_change":"2024-05-14T11:16:11.251898+0000","last_active":"2024-05-14T11:16:17.806762+0000","last_peered":"2024-05-14T11:16:17.806762+0000","last_clean":"2024-05-14T11:16:17.806762+0000","last_became_active":"2024-05-14T11:16:11.251339+0000","last_became_peered":"2024-05-14T11:16:11.251339+0000","last_unstale":"2024-05-14T11:16:17.806762+0000","last_undegraded":"2024-05-14T11:16:17.806762+0000","last_fullsized":"2024-05-14T11:16:17.806762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809475+0000","last_change":"2024-05-14T11:16:16.331750+0000","last_active":"2024-05-14T11:16:17.809475+0000","last_peered":"2024-05-14T11:16:17.809475+0000","last_clean":"2024-05-14T11:16:17.809475+0000","last_became_active":"2024-05-14T11:16:16.331181+0000","last_became_peered":"2024-05-14T11:16:16.331181+0000","last_unstale":"2024-05-14T11:16:17.809475+0000","last_undegraded":"2024-05-14T11:16:17.809475+0000","last_fullsized":"2024-05-14T11:16:17.809475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301296+0000","last_change":"2024-05-14T11:16:12.245880+0000","last_active":"2024-05-14T11:16:17.301296+0000","last_peered":"2024-05-14T11:16:17.301296+0000","last_clean":"2024-05-14T11:16:17.301296+0000","last_became_active":"2024-05-14T11:16:12.245150+0000","last_became_peered":"2024-05-14T11:16:12.245150+0000","last_unstale":"2024-05-14T11:16:17.301296+0000","last_undegraded":"2024-05-14T11:16:17.301296+0000","last_fullsized":"2024-05-14T11:16:17.301296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806258+0000","last_change":"2024-05-14T11:16:11.252415+0000","last_active":"2024-05-14T11:16:17.806258+0000","last_peered":"2024-05-14T11:16:17.806258+0000","last_clean":"2024-05-14T11:16:17.806258+0000","last_became_active":"2024-05-14T11:16:11.252113+0000","last_became_peered":"2024-05-14T11:16:11.252113+0000","last_unstale":"2024-05-14T11:16:17.806258+0000","last_undegraded":"2024-05-14T11:16:17.806258+0000","last_fullsized":"2024-05-14T11:16:17.806258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302531+0000","last_change":"2024-05-14T11:16:16.298123+0000","last_active":"2024-05-14T11:16:17.302531+0000","last_peered":"2024-05-14T11:16:17.302531+0000","last_clean":"2024-05-14T11:16:17.302531+0000","last_became_active":"2024-05-14T11:16:16.297647+0000","last_became_peered":"2024-05-14T11:16:16.297647+0000","last_unstale":"2024-05-14T11:16:17.302531+0000","last_undegraded":"2024-05-14T11:16:17.302531+0000","last_fullsized":"2024-05-14T11:16:17.302531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807122+0000","last_change":"2024-05-14T11:16:12.247217+0000","last_active":"2024-05-14T11:16:17.807122+0000","last_peered":"2024-05-14T11:16:17.807122+0000","last_clean":"2024-05-14T11:16:17.807122+0000","last_became_active":"2024-05-14T11:16:12.246460+0000","last_became_peered":"2024-05-14T11:16:12.246460+0000","last_unstale":"2024-05-14T11:16:17.807122+0000","last_undegraded":"2024-05-14T11:16:17.807122+0000","last_fullsized":"2024-05-14T11:16:17.807122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301791+0000","last_change":"2024-05-14T11:16:11.246553+0000","last_active":"2024-05-14T11:16:17.301791+0000","last_peered":"2024-05-14T11:16:17.301791+0000","last_clean":"2024-05-14T11:16:17.301791+0000","last_became_active":"2024-05-14T11:16:11.245634+0000","last_became_peered":"2024-05-14T11:16:11.245634+0000","last_unstale":"2024-05-14T11:16:17.301791+0000","last_undegraded":"2024-05-14T11:16:17.301791+0000","last_fullsized":"2024-05-14T11:16:17.301791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300036+0000","last_change":"2024-05-14T11:16:16.331377+0000","last_active":"2024-05-14T11:16:17.300036+0000","last_peered":"2024-05-14T11:16:17.300036+0000","last_clean":"2024-05-14T11:16:17.300036+0000","last_became_active":"2024-05-14T11:16:16.330845+0000","last_became_peered":"2024-05-14T11:16:16.330845+0000","last_unstale":"2024-05-14T11:16:17.300036+0000","last_undegraded":"2024-05-14T11:16:17.300036+0000","last_fullsized":"2024-05-14T11:16:17.300036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:08.942678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809208+0000","last_change":"2024-05-14T11:16:12.265091+0000","last_active":"2024-05-14T11:16:17.809208+0000","last_peered":"2024-05-14T11:16:17.809208+0000","last_clean":"2024-05-14T11:16:17.809208+0000","last_became_active":"2024-05-14T11:16:12.264728+0000","last_became_peered":"2024-05-14T11:16:12.264728+0000","last_unstale":"2024-05-14T11:16:17.809208+0000","last_undegraded":"2024-05-14T11:16:17.809208+0000","last_fullsized":"2024-05-14T11:16:17.809208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807713+0000","last_change":"2024-05-14T11:16:11.253941+0000","last_active":"2024-05-14T11:16:17.807713+0000","last_peered":"2024-05-14T11:16:17.807713+0000","last_clean":"2024-05-14T11:16:17.807713+0000","last_became_active":"2024-05-14T11:16:11.253534+0000","last_became_peered":"2024-05-14T11:16:11.253534+0000","last_unstale":"2024-05-14T11:16:17.807713+0000","last_undegraded":"2024-05-14T11:16:17.807713+0000","last_fullsized":"2024-05-14T11:16:17.807713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808083+0000","last_change":"2024-05-14T11:16:16.338462+0000","last_active":"2024-05-14T11:16:17.808083+0000","last_peered":"2024-05-14T11:16:17.808083+0000","last_clean":"2024-05-14T11:16:17.808083+0000","last_became_active":"2024-05-14T11:16:16.337908+0000","last_became_peered":"2024-05-14T11:16:16.337908+0000","last_unstale":"2024-05-14T11:16:17.808083+0000","last_undegraded":"2024-05-14T11:16:17.808083+0000","last_fullsized":"2024-05-14T11:16:17.808083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:28:29.309169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807237+0000","last_change":"2024-05-14T11:16:12.249280+0000","last_active":"2024-05-14T11:16:17.807237+0000","last_peered":"2024-05-14T11:16:17.807237+0000","last_clean":"2024-05-14T11:16:17.807237+0000","last_became_active":"2024-05-14T11:16:12.248977+0000","last_became_peered":"2024-05-14T11:16:12.248977+0000","last_unstale":"2024-05-14T11:16:17.807237+0000","last_undegraded":"2024-05-14T11:16:17.807237+0000","last_fullsized":"2024-05-14T11:16:17.807237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807419+0000","last_change":"2024-05-14T11:16:11.255839+0000","last_active":"2024-05-14T11:16:17.807419+0000","last_peered":"2024-05-14T11:16:17.807419+0000","last_clean":"2024-05-14T11:16:17.807419+0000","last_became_active":"2024-05-14T11:16:11.255487+0000","last_became_peered":"2024-05-14T11:16:11.255487+0000","last_unstale":"2024-05-14T11:16:17.807419+0000","last_undegraded":"2024-05-14T11:16:17.807419+0000","last_fullsized":"2024-05-14T11:16:17.807419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:59.335122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806362+0000","last_change":"2024-05-14T11:16:16.330365+0000","last_active":"2024-05-14T11:16:17.806362+0000","last_peered":"2024-05-14T11:16:17.806362+0000","last_clean":"2024-05-14T11:16:17.806362+0000","last_became_active":"2024-05-14T11:16:16.329833+0000","last_became_peered":"2024-05-14T11:16:16.329833+0000","last_unstale":"2024-05-14T11:16:17.806362+0000","last_undegraded":"2024-05-14T11:16:17.806362+0000","last_fullsized":"2024-05-14T11:16:17.806362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807044+0000","last_change":"2024-05-14T11:16:12.267992+0000","last_active":"2024-05-14T11:16:17.807044+0000","last_peered":"2024-05-14T11:16:17.807044+0000","last_clean":"2024-05-14T11:16:17.807044+0000","last_became_active":"2024-05-14T11:16:12.267183+0000","last_became_peered":"2024-05-14T11:16:12.267183+0000","last_unstale":"2024-05-14T11:16:17.807044+0000","last_undegraded":"2024-05-14T11:16:17.807044+0000","last_fullsized":"2024-05-14T11:16:17.807044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807300+0000","last_change":"2024-05-14T11:16:11.247765+0000","last_active":"2024-05-14T11:16:17.807300+0000","last_peered":"2024-05-14T11:16:17.807300+0000","last_clean":"2024-05-14T11:16:17.807300+0000","last_became_active":"2024-05-14T11:16:11.247503+0000","last_became_peered":"2024-05-14T11:16:11.247503+0000","last_unstale":"2024-05-14T11:16:17.807300+0000","last_undegraded":"2024-05-14T11:16:17.807300+0000","last_fullsized":"2024-05-14T11:16:17.807300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808159+0000","last_change":"2024-05-14T11:16:16.304314+0000","last_active":"2024-05-14T11:16:17.808159+0000","last_peered":"2024-05-14T11:16:17.808159+0000","last_clean":"2024-05-14T11:16:17.808159+0000","last_became_active":"2024-05-14T11:16:16.303370+0000","last_became_peered":"2024-05-14T11:16:16.303370+0000","last_unstale":"2024-05-14T11:16:17.808159+0000","last_undegraded":"2024-05-14T11:16:17.808159+0000","last_fullsized":"2024-05-14T11:16:17.808159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301326+0000","last_change":"2024-05-14T11:16:12.258856+0000","last_active":"2024-05-14T11:16:17.301326+0000","last_peered":"2024-05-14T11:16:17.301326+0000","last_clean":"2024-05-14T11:16:17.301326+0000","last_became_active":"2024-05-14T11:16:12.257816+0000","last_became_peered":"2024-05-14T11:16:12.257816+0000","last_unstale":"2024-05-14T11:16:17.301326+0000","last_undegraded":"2024-05-14T11:16:17.301326+0000","last_fullsized":"2024-05-14T11:16:17.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:19.338856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807816+0000","last_change":"2024-05-14T11:16:11.257469+0000","last_active":"2024-05-14T11:16:17.807816+0000","last_peered":"2024-05-14T11:16:17.807816+0000","last_clean":"2024-05-14T11:16:17.807816+0000","last_became_active":"2024-05-14T11:16:11.253425+0000","last_became_peered":"2024-05-14T11:16:11.253425+0000","last_unstale":"2024-05-14T11:16:17.807816+0000","last_undegraded":"2024-05-14T11:16:17.807816+0000","last_fullsized":"2024-05-14T11:16:17.807816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807184+0000","last_change":"2024-05-14T11:16:16.305620+0000","last_active":"2024-05-14T11:16:17.807184+0000","last_peered":"2024-05-14T11:16:17.807184+0000","last_clean":"2024-05-14T11:16:17.807184+0000","last_became_active":"2024-05-14T11:16:16.305156+0000","last_became_peered":"2024-05-14T11:16:16.305156+0000","last_unstale":"2024-05-14T11:16:17.807184+0000","last_undegraded":"2024-05-14T11:16:17.807184+0000","last_fullsized":"2024-05-14T11:16:17.807184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806230+0000","last_change":"2024-05-14T11:16:12.245405+0000","last_active":"2024-05-14T11:16:17.806230+0000","last_peered":"2024-05-14T11:16:17.806230+0000","last_clean":"2024-05-14T11:16:17.806230+0000","last_became_active":"2024-05-14T11:16:12.245076+0000","last_became_peered":"2024-05-14T11:16:12.245076+0000","last_unstale":"2024-05-14T11:16:17.806230+0000","last_undegraded":"2024-05-14T11:16:17.806230+0000","last_fullsized":"2024-05-14T11:16:17.806230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297955+0000","last_change":"2024-05-14T11:16:11.245815+0000","last_active":"2024-05-14T11:16:17.297955+0000","last_peered":"2024-05-14T11:16:17.297955+0000","last_clean":"2024-05-14T11:16:17.297955+0000","last_became_active":"2024-05-14T11:16:11.245487+0000","last_became_peered":"2024-05-14T11:16:11.245487+0000","last_unstale":"2024-05-14T11:16:17.297955+0000","last_undegraded":"2024-05-14T11:16:17.297955+0000","last_fullsized":"2024-05-14T11:16:17.297955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809501+0000","last_change":"2024-05-14T11:16:16.304799+0000","last_active":"2024-05-14T11:16:17.809501+0000","last_peered":"2024-05-14T11:16:17.809501+0000","last_clean":"2024-05-14T11:16:17.809501+0000","last_became_active":"2024-05-14T11:16:16.304223+0000","last_became_peered":"2024-05-14T11:16:16.304223+0000","last_unstale":"2024-05-14T11:16:17.809501+0000","last_undegraded":"2024-05-14T11:16:17.809501+0000","last_fullsized":"2024-05-14T11:16:17.809501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.805980+0000","last_change":"2024-05-14T11:16:12.242987+0000","last_active":"2024-05-14T11:16:17.805980+0000","last_peered":"2024-05-14T11:16:17.805980+0000","last_clean":"2024-05-14T11:16:17.805980+0000","last_became_active":"2024-05-14T11:16:12.242648+0000","last_became_peered":"2024-05-14T11:16:12.242648+0000","last_unstale":"2024-05-14T11:16:17.805980+0000","last_undegraded":"2024-05-14T11:16:17.805980+0000","last_fullsized":"2024-05-14T11:16:17.805980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:37.421472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806812+0000","last_change":"2024-05-14T11:16:11.254596+0000","last_active":"2024-05-14T11:16:17.806812+0000","last_peered":"2024-05-14T11:16:17.806812+0000","last_clean":"2024-05-14T11:16:17.806812+0000","last_became_active":"2024-05-14T11:16:11.254304+0000","last_became_peered":"2024-05-14T11:16:11.254304+0000","last_unstale":"2024-05-14T11:16:17.806812+0000","last_undegraded":"2024-05-14T11:16:17.806812+0000","last_fullsized":"2024-05-14T11:16:17.806812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806903+0000","last_change":"2024-05-14T11:16:16.333086+0000","last_active":"2024-05-14T11:16:17.806903+0000","last_peered":"2024-05-14T11:16:17.806903+0000","last_clean":"2024-05-14T11:16:17.806903+0000","last_became_active":"2024-05-14T11:16:16.332633+0000","last_became_peered":"2024-05-14T11:16:16.332633+0000","last_unstale":"2024-05-14T11:16:17.806903+0000","last_undegraded":"2024-05-14T11:16:17.806903+0000","last_fullsized":"2024-05-14T11:16:17.806903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808690+0000","last_change":"2024-05-14T11:16:12.267597+0000","last_active":"2024-05-14T11:16:17.808690+0000","last_peered":"2024-05-14T11:16:17.808690+0000","last_clean":"2024-05-14T11:16:17.808690+0000","last_became_active":"2024-05-14T11:16:12.267050+0000","last_became_peered":"2024-05-14T11:16:12.267050+0000","last_unstale":"2024-05-14T11:16:17.808690+0000","last_undegraded":"2024-05-14T11:16:17.808690+0000","last_fullsized":"2024-05-14T11:16:17.808690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806566+0000","last_change":"2024-05-14T11:16:11.253533+0000","last_active":"2024-05-14T11:16:17.806566+0000","last_peered":"2024-05-14T11:16:17.806566+0000","last_clean":"2024-05-14T11:16:17.806566+0000","last_became_active":"2024-05-14T11:16:11.253104+0000","last_became_peered":"2024-05-14T11:16:11.253104+0000","last_unstale":"2024-05-14T11:16:17.806566+0000","last_undegraded":"2024-05-14T11:16:17.806566+0000","last_fullsized":"2024-05-14T11:16:17.806566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806962+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:16:17.806962+0000","last_peered":"2024-05-14T11:16:17.806962+0000","last_clean":"2024-05-14T11:16:17.806962+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:16:17.806962+0000","last_undegraded":"2024-05-14T11:16:17.806962+0000","last_fullsized":"2024-05-14T11:16:17.806962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299583+0000","last_change":"2024-05-14T11:16:16.330966+0000","last_active":"2024-05-14T11:16:17.299583+0000","last_peered":"2024-05-14T11:16:17.299583+0000","last_clean":"2024-05-14T11:16:17.299583+0000","last_became_active":"2024-05-14T11:16:16.330413+0000","last_became_peered":"2024-05-14T11:16:16.330413+0000","last_unstale":"2024-05-14T11:16:17.299583+0000","last_undegraded":"2024-05-14T11:16:17.299583+0000","last_fullsized":"2024-05-14T11:16:17.299583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807128+0000","last_change":"2024-05-14T11:16:12.263387+0000","last_active":"2024-05-14T11:16:17.807128+0000","last_peered":"2024-05-14T11:16:17.807128+0000","last_clean":"2024-05-14T11:16:17.807128+0000","last_became_active":"2024-05-14T11:16:12.262944+0000","last_became_peered":"2024-05-14T11:16:12.262944+0000","last_unstale":"2024-05-14T11:16:17.807128+0000","last_undegraded":"2024-05-14T11:16:17.807128+0000","last_fullsized":"2024-05-14T11:16:17.807128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298224+0000","last_change":"2024-05-14T11:16:11.259096+0000","last_active":"2024-05-14T11:16:17.298224+0000","last_peered":"2024-05-14T11:16:17.298224+0000","last_clean":"2024-05-14T11:16:17.298224+0000","last_became_active":"2024-05-14T11:16:11.258840+0000","last_became_peered":"2024-05-14T11:16:11.258840+0000","last_unstale":"2024-05-14T11:16:17.298224+0000","last_undegraded":"2024-05-14T11:16:17.298224+0000","last_fullsized":"2024-05-14T11:16:17.298224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300200+0000","last_change":"2024-05-14T11:16:16.333088+0000","last_active":"2024-05-14T11:16:17.300200+0000","last_peered":"2024-05-14T11:16:17.300200+0000","last_clean":"2024-05-14T11:16:17.300200+0000","last_became_active":"2024-05-14T11:16:16.332585+0000","last_became_peered":"2024-05-14T11:16:16.332585+0000","last_unstale":"2024-05-14T11:16:17.300200+0000","last_undegraded":"2024-05-14T11:16:17.300200+0000","last_fullsized":"2024-05-14T11:16:17.300200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806820+0000","last_change":"2024-05-14T11:16:12.270318+0000","last_active":"2024-05-14T11:16:17.806820+0000","last_peered":"2024-05-14T11:16:17.806820+0000","last_clean":"2024-05-14T11:16:17.806820+0000","last_became_active":"2024-05-14T11:16:12.269365+0000","last_became_peered":"2024-05-14T11:16:12.269365+0000","last_unstale":"2024-05-14T11:16:17.806820+0000","last_undegraded":"2024-05-14T11:16:17.806820+0000","last_fullsized":"2024-05-14T11:16:17.806820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807501+0000","last_change":"2024-05-14T11:16:11.252737+0000","last_active":"2024-05-14T11:16:17.807501+0000","last_peered":"2024-05-14T11:16:17.807501+0000","last_clean":"2024-05-14T11:16:17.807501+0000","last_became_active":"2024-05-14T11:16:11.252365+0000","last_became_peered":"2024-05-14T11:16:11.252365+0000","last_unstale":"2024-05-14T11:16:17.807501+0000","last_undegraded":"2024-05-14T11:16:17.807501+0000","last_fullsized":"2024-05-14T11:16:17.807501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806471+0000","last_change":"2024-05-14T11:16:16.302121+0000","last_active":"2024-05-14T11:16:17.806471+0000","last_peered":"2024-05-14T11:16:17.806471+0000","last_clean":"2024-05-14T11:16:17.806471+0000","last_became_active":"2024-05-14T11:16:16.299059+0000","last_became_peered":"2024-05-14T11:16:16.299059+0000","last_unstale":"2024-05-14T11:16:17.806471+0000","last_undegraded":"2024-05-14T11:16:17.806471+0000","last_fullsized":"2024-05-14T11:16:17.806471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807801+0000","last_change":"2024-05-14T11:16:12.247488+0000","last_active":"2024-05-14T11:16:17.807801+0000","last_peered":"2024-05-14T11:16:17.807801+0000","last_clean":"2024-05-14T11:16:17.807801+0000","last_became_active":"2024-05-14T11:16:12.247235+0000","last_became_peered":"2024-05-14T11:16:12.247235+0000","last_unstale":"2024-05-14T11:16:17.807801+0000","last_undegraded":"2024-05-14T11:16:17.807801+0000","last_fullsized":"2024-05-14T11:16:17.807801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:22.549789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298399+0000","last_change":"2024-05-14T11:16:11.246157+0000","last_active":"2024-05-14T11:16:17.298399+0000","last_peered":"2024-05-14T11:16:17.298399+0000","last_clean":"2024-05-14T11:16:17.298399+0000","last_became_active":"2024-05-14T11:16:11.245408+0000","last_became_peered":"2024-05-14T11:16:11.245408+0000","last_unstale":"2024-05-14T11:16:17.298399+0000","last_undegraded":"2024-05-14T11:16:17.298399+0000","last_fullsized":"2024-05-14T11:16:17.298399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806090+0000","last_change":"2024-05-14T11:16:16.330283+0000","last_active":"2024-05-14T11:16:17.806090+0000","last_peered":"2024-05-14T11:16:17.806090+0000","last_clean":"2024-05-14T11:16:17.806090+0000","last_became_active":"2024-05-14T11:16:16.329844+0000","last_became_peered":"2024-05-14T11:16:16.329844+0000","last_unstale":"2024-05-14T11:16:17.806090+0000","last_undegraded":"2024-05-14T11:16:17.806090+0000","last_fullsized":"2024-05-14T11:16:17.806090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807746+0000","last_change":"2024-05-14T11:16:12.248542+0000","last_active":"2024-05-14T11:16:17.807746+0000","last_peered":"2024-05-14T11:16:17.807746+0000","last_clean":"2024-05-14T11:16:17.807746+0000","last_became_active":"2024-05-14T11:16:12.248214+0000","last_became_peered":"2024-05-14T11:16:12.248214+0000","last_unstale":"2024-05-14T11:16:17.807746+0000","last_undegraded":"2024-05-14T11:16:17.807746+0000","last_fullsized":"2024-05-14T11:16:17.807746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302636+0000","last_change":"2024-05-14T11:16:11.241324+0000","last_active":"2024-05-14T11:16:17.302636+0000","last_peered":"2024-05-14T11:16:17.302636+0000","last_clean":"2024-05-14T11:16:17.302636+0000","last_became_active":"2024-05-14T11:16:11.239970+0000","last_became_peered":"2024-05-14T11:16:11.239970+0000","last_unstale":"2024-05-14T11:16:17.302636+0000","last_undegraded":"2024-05-14T11:16:17.302636+0000","last_fullsized":"2024-05-14T11:16:17.302636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:44.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299463+0000","last_change":"2024-05-14T11:16:16.293327+0000","last_active":"2024-05-14T11:16:17.299463+0000","last_peered":"2024-05-14T11:16:17.299463+0000","last_clean":"2024-05-14T11:16:17.299463+0000","last_became_active":"2024-05-14T11:16:16.292800+0000","last_became_peered":"2024-05-14T11:16:16.292800+0000","last_unstale":"2024-05-14T11:16:17.299463+0000","last_undegraded":"2024-05-14T11:16:17.299463+0000","last_fullsized":"2024-05-14T11:16:17.299463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:39.377103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808057+0000","last_change":"2024-05-14T11:16:12.248759+0000","last_active":"2024-05-14T11:16:17.808057+0000","last_peered":"2024-05-14T11:16:17.808057+0000","last_clean":"2024-05-14T11:16:17.808057+0000","last_became_active":"2024-05-14T11:16:12.248407+0000","last_became_peered":"2024-05-14T11:16:12.248407+0000","last_unstale":"2024-05-14T11:16:17.808057+0000","last_undegraded":"2024-05-14T11:16:17.808057+0000","last_fullsized":"2024-05-14T11:16:17.808057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297834+0000","last_change":"2024-05-14T11:16:11.248167+0000","last_active":"2024-05-14T11:16:17.297834+0000","last_peered":"2024-05-14T11:16:17.297834+0000","last_clean":"2024-05-14T11:16:17.297834+0000","last_became_active":"2024-05-14T11:16:11.245949+0000","last_became_peered":"2024-05-14T11:16:11.245949+0000","last_unstale":"2024-05-14T11:16:17.297834+0000","last_undegraded":"2024-05-14T11:16:17.297834+0000","last_fullsized":"2024-05-14T11:16:17.297834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807832+0000","last_change":"2024-05-14T11:16:16.290089+0000","last_active":"2024-05-14T11:16:17.807832+0000","last_peered":"2024-05-14T11:16:17.807832+0000","last_clean":"2024-05-14T11:16:17.807832+0000","last_became_active":"2024-05-14T11:16:16.287987+0000","last_became_peered":"2024-05-14T11:16:16.287987+0000","last_unstale":"2024-05-14T11:16:17.807832+0000","last_undegraded":"2024-05-14T11:16:17.807832+0000","last_fullsized":"2024-05-14T11:16:17.807832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:26.599395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299766+0000","last_change":"2024-05-14T11:16:12.248930+0000","last_active":"2024-05-14T11:16:17.299766+0000","last_peered":"2024-05-14T11:16:17.299766+0000","last_clean":"2024-05-14T11:16:17.299766+0000","last_became_active":"2024-05-14T11:16:12.248159+0000","last_became_peered":"2024-05-14T11:16:12.248159+0000","last_unstale":"2024-05-14T11:16:17.299766+0000","last_undegraded":"2024-05-14T11:16:17.299766+0000","last_fullsized":"2024-05-14T11:16:17.299766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299880+0000","last_change":"2024-05-14T11:16:11.247396+0000","last_active":"2024-05-14T11:16:17.299880+0000","last_peered":"2024-05-14T11:16:17.299880+0000","last_clean":"2024-05-14T11:16:17.299880+0000","last_became_active":"2024-05-14T11:16:11.246855+0000","last_became_peered":"2024-05-14T11:16:11.246855+0000","last_unstale":"2024-05-14T11:16:17.299880+0000","last_undegraded":"2024-05-14T11:16:17.299880+0000","last_fullsized":"2024-05-14T11:16:17.299880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808975+0000","last_change":"2024-05-14T11:16:16.332903+0000","last_active":"2024-05-14T11:16:17.808975+0000","last_peered":"2024-05-14T11:16:17.808975+0000","last_clean":"2024-05-14T11:16:17.808975+0000","last_became_active":"2024-05-14T11:16:16.332436+0000","last_became_peered":"2024-05-14T11:16:16.332436+0000","last_unstale":"2024-05-14T11:16:17.808975+0000","last_undegraded":"2024-05-14T11:16:17.808975+0000","last_fullsized":"2024-05-14T11:16:17.808975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:01.254012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807525+0000","last_change":"2024-05-14T11:16:12.248442+0000","last_active":"2024-05-14T11:16:17.807525+0000","last_peered":"2024-05-14T11:16:17.807525+0000","last_clean":"2024-05-14T11:16:17.807525+0000","last_became_active":"2024-05-14T11:16:12.248082+0000","last_became_peered":"2024-05-14T11:16:12.248082+0000","last_unstale":"2024-05-14T11:16:17.807525+0000","last_undegraded":"2024-05-14T11:16:17.807525+0000","last_fullsized":"2024-05-14T11:16:17.807525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807708+0000","last_change":"2024-05-14T11:16:11.257458+0000","last_active":"2024-05-14T11:16:17.807708+0000","last_peered":"2024-05-14T11:16:17.807708+0000","last_clean":"2024-05-14T11:16:17.807708+0000","last_became_active":"2024-05-14T11:16:11.252978+0000","last_became_peered":"2024-05-14T11:16:11.252978+0000","last_unstale":"2024-05-14T11:16:17.807708+0000","last_undegraded":"2024-05-14T11:16:17.807708+0000","last_fullsized":"2024-05-14T11:16:17.807708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808827+0000","last_change":"2024-05-14T11:16:16.330174+0000","last_active":"2024-05-14T11:16:17.808827+0000","last_peered":"2024-05-14T11:16:17.808827+0000","last_clean":"2024-05-14T11:16:17.808827+0000","last_became_active":"2024-05-14T11:16:16.329696+0000","last_became_peered":"2024-05-14T11:16:16.329696+0000","last_unstale":"2024-05-14T11:16:17.808827+0000","last_undegraded":"2024-05-14T11:16:17.808827+0000","last_fullsized":"2024-05-14T11:16:17.808827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:14.122764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806365+0000","last_change":"2024-05-14T11:16:12.246355+0000","last_active":"2024-05-14T11:16:17.806365+0000","last_peered":"2024-05-14T11:16:17.806365+0000","last_clean":"2024-05-14T11:16:17.806365+0000","last_became_active":"2024-05-14T11:16:12.245693+0000","last_became_peered":"2024-05-14T11:16:12.245693+0000","last_unstale":"2024-05-14T11:16:17.806365+0000","last_undegraded":"2024-05-14T11:16:17.806365+0000","last_fullsized":"2024-05-14T11:16:17.806365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806645+0000","last_change":"2024-05-14T11:16:11.249709+0000","last_active":"2024-05-14T11:16:17.806645+0000","last_peered":"2024-05-14T11:16:17.806645+0000","last_clean":"2024-05-14T11:16:17.806645+0000","last_became_active":"2024-05-14T11:16:11.245550+0000","last_became_peered":"2024-05-14T11:16:11.245550+0000","last_unstale":"2024-05-14T11:16:17.806645+0000","last_undegraded":"2024-05-14T11:16:17.806645+0000","last_fullsized":"2024-05-14T11:16:17.806645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:18.498642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806772+0000","last_change":"2024-05-14T11:16:16.330492+0000","last_active":"2024-05-14T11:16:17.806772+0000","last_peered":"2024-05-14T11:16:17.806772+0000","last_clean":"2024-05-14T11:16:17.806772+0000","last_became_active":"2024-05-14T11:16:16.330027+0000","last_became_peered":"2024-05-14T11:16:16.330027+0000","last_unstale":"2024-05-14T11:16:17.806772+0000","last_undegraded":"2024-05-14T11:16:17.806772+0000","last_fullsized":"2024-05-14T11:16:17.806772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807029+0000","last_change":"2024-05-14T11:16:12.263610+0000","last_active":"2024-05-14T11:16:17.807029+0000","last_peered":"2024-05-14T11:16:17.807029+0000","last_clean":"2024-05-14T11:16:17.807029+0000","last_became_active":"2024-05-14T11:16:12.263072+0000","last_became_peered":"2024-05-14T11:16:12.263072+0000","last_unstale":"2024-05-14T11:16:17.807029+0000","last_undegraded":"2024-05-14T11:16:17.807029+0000","last_fullsized":"2024-05-14T11:16:17.807029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:23.206421+0000","last_change":"2024-05-14T11:16:11.250108+0000","last_active":"2024-05-14T11:16:23.206421+0000","last_peered":"2024-05-14T11:16:23.206421+0000","last_clean":"2024-05-14T11:16:23.206421+0000","last_became_active":"2024-05-14T11:16:11.249460+0000","last_became_peered":"2024-05-14T11:16:11.249460+0000","last_unstale":"2024-05-14T11:16:23.206421+0000","last_undegraded":"2024-05-14T11:16:23.206421+0000","last_fullsized":"2024-05-14T11:16:23.206421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809402+0000","last_change":"2024-05-14T11:16:16.305560+0000","last_active":"2024-05-14T11:16:17.809402+0000","last_peered":"2024-05-14T11:16:17.809402+0000","last_clean":"2024-05-14T11:16:17.809402+0000","last_became_active":"2024-05-14T11:16:16.305166+0000","last_became_peered":"2024-05-14T11:16:16.305166+0000","last_unstale":"2024-05-14T11:16:17.809402+0000","last_undegraded":"2024-05-14T11:16:17.809402+0000","last_fullsized":"2024-05-14T11:16:17.809402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302423+0000","last_change":"2024-05-14T11:16:12.248795+0000","last_active":"2024-05-14T11:16:17.302423+0000","last_peered":"2024-05-14T11:16:17.302423+0000","last_clean":"2024-05-14T11:16:17.302423+0000","last_became_active":"2024-05-14T11:16:12.248478+0000","last_became_peered":"2024-05-14T11:16:12.248478+0000","last_unstale":"2024-05-14T11:16:17.302423+0000","last_undegraded":"2024-05-14T11:16:17.302423+0000","last_fullsized":"2024-05-14T11:16:17.302423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807383+0000","last_change":"2024-05-14T11:16:11.259967+0000","last_active":"2024-05-14T11:16:17.807383+0000","last_peered":"2024-05-14T11:16:17.807383+0000","last_clean":"2024-05-14T11:16:17.807383+0000","last_became_active":"2024-05-14T11:16:11.259703+0000","last_became_peered":"2024-05-14T11:16:11.259703+0000","last_unstale":"2024-05-14T11:16:17.807383+0000","last_undegraded":"2024-05-14T11:16:17.807383+0000","last_fullsized":"2024-05-14T11:16:17.807383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301098+0000","last_change":"2024-05-14T11:16:16.293106+0000","last_active":"2024-05-14T11:16:17.301098+0000","last_peered":"2024-05-14T11:16:17.301098+0000","last_clean":"2024-05-14T11:16:17.301098+0000","last_became_active":"2024-05-14T11:16:16.292666+0000","last_became_peered":"2024-05-14T11:16:16.292666+0000","last_unstale":"2024-05-14T11:16:17.301098+0000","last_undegraded":"2024-05-14T11:16:17.301098+0000","last_fullsized":"2024-05-14T11:16:17.301098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807505+0000","last_change":"2024-05-14T11:16:12.260898+0000","last_active":"2024-05-14T11:16:17.807505+0000","last_peered":"2024-05-14T11:16:17.807505+0000","last_clean":"2024-05-14T11:16:17.807505+0000","last_became_active":"2024-05-14T11:16:12.260114+0000","last_became_peered":"2024-05-14T11:16:12.260114+0000","last_unstale":"2024-05-14T11:16:17.807505+0000","last_undegraded":"2024-05-14T11:16:17.807505+0000","last_fullsized":"2024-05-14T11:16:17.807505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806807+0000","last_change":"2024-05-14T11:16:11.254169+0000","last_active":"2024-05-14T11:16:17.806807+0000","last_peered":"2024-05-14T11:16:17.806807+0000","last_clean":"2024-05-14T11:16:17.806807+0000","last_became_active":"2024-05-14T11:16:11.253658+0000","last_became_peered":"2024-05-14T11:16:11.253658+0000","last_unstale":"2024-05-14T11:16:17.806807+0000","last_undegraded":"2024-05-14T11:16:17.806807+0000","last_fullsized":"2024-05-14T11:16:17.806807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808298+0000","last_change":"2024-05-14T11:16:16.298722+0000","last_active":"2024-05-14T11:16:17.808298+0000","last_peered":"2024-05-14T11:16:17.808298+0000","last_clean":"2024-05-14T11:16:17.808298+0000","last_became_active":"2024-05-14T11:16:16.298312+0000","last_became_peered":"2024-05-14T11:16:16.298312+0000","last_unstale":"2024-05-14T11:16:17.808298+0000","last_undegraded":"2024-05-14T11:16:17.808298+0000","last_fullsized":"2024-05-14T11:16:17.808298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806669+0000","last_change":"2024-05-14T11:16:12.269044+0000","last_active":"2024-05-14T11:16:17.806669+0000","last_peered":"2024-05-14T11:16:17.806669+0000","last_clean":"2024-05-14T11:16:17.806669+0000","last_became_active":"2024-05-14T11:16:12.268428+0000","last_became_peered":"2024-05-14T11:16:12.268428+0000","last_unstale":"2024-05-14T11:16:17.806669+0000","last_undegraded":"2024-05-14T11:16:17.806669+0000","last_fullsized":"2024-05-14T11:16:17.806669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301709+0000","last_change":"2024-05-14T11:16:11.247431+0000","last_active":"2024-05-14T11:16:17.301709+0000","last_peered":"2024-05-14T11:16:17.301709+0000","last_clean":"2024-05-14T11:16:17.301709+0000","last_became_active":"2024-05-14T11:16:11.247003+0000","last_became_peered":"2024-05-14T11:16:11.247003+0000","last_unstale":"2024-05-14T11:16:17.301709+0000","last_undegraded":"2024-05-14T11:16:17.301709+0000","last_fullsized":"2024-05-14T11:16:17.301709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808398+0000","last_change":"2024-05-14T11:16:16.338217+0000","last_active":"2024-05-14T11:16:17.808398+0000","last_peered":"2024-05-14T11:16:17.808398+0000","last_clean":"2024-05-14T11:16:17.808398+0000","last_became_active":"2024-05-14T11:16:16.337645+0000","last_became_peered":"2024-05-14T11:16:16.337645+0000","last_unstale":"2024-05-14T11:16:17.808398+0000","last_undegraded":"2024-05-14T11:16:17.808398+0000","last_fullsized":"2024-05-14T11:16:17.808398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:50:21.563593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806740+0000","last_change":"2024-05-14T11:16:16.337358+0000","last_active":"2024-05-14T11:16:17.806740+0000","last_peered":"2024-05-14T11:16:17.806740+0000","last_clean":"2024-05-14T11:16:17.806740+0000","last_became_active":"2024-05-14T11:16:16.336916+0000","last_became_peered":"2024-05-14T11:16:16.336916+0000","last_unstale":"2024-05-14T11:16:17.806740+0000","last_undegraded":"2024-05-14T11:16:17.806740+0000","last_fullsized":"2024-05-14T11:16:17.806740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806581+0000","last_change":"2024-05-14T11:16:12.267101+0000","last_active":"2024-05-14T11:16:17.806581+0000","last_peered":"2024-05-14T11:16:17.806581+0000","last_clean":"2024-05-14T11:16:17.806581+0000","last_became_active":"2024-05-14T11:16:12.266805+0000","last_became_peered":"2024-05-14T11:16:12.266805+0000","last_unstale":"2024-05-14T11:16:17.806581+0000","last_undegraded":"2024-05-14T11:16:17.806581+0000","last_fullsized":"2024-05-14T11:16:17.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806424+0000","last_change":"2024-05-14T11:16:11.259588+0000","last_active":"2024-05-14T11:16:17.806424+0000","last_peered":"2024-05-14T11:16:17.806424+0000","last_clean":"2024-05-14T11:16:17.806424+0000","last_became_active":"2024-05-14T11:16:11.259246+0000","last_became_peered":"2024-05-14T11:16:11.259246+0000","last_unstale":"2024-05-14T11:16:17.806424+0000","last_undegraded":"2024-05-14T11:16:17.806424+0000","last_fullsized":"2024-05-14T11:16:17.806424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301555+0000","last_change":"2024-05-14T11:16:16.299514+0000","last_active":"2024-05-14T11:16:17.301555+0000","last_peered":"2024-05-14T11:16:17.301555+0000","last_clean":"2024-05-14T11:16:17.301555+0000","last_became_active":"2024-05-14T11:16:16.299075+0000","last_became_peered":"2024-05-14T11:16:16.299075+0000","last_unstale":"2024-05-14T11:16:17.301555+0000","last_undegraded":"2024-05-14T11:16:17.301555+0000","last_fullsized":"2024-05-14T11:16:17.301555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808226+0000","last_change":"2024-05-14T11:16:12.258195+0000","last_active":"2024-05-14T11:16:17.808226+0000","last_peered":"2024-05-14T11:16:17.808226+0000","last_clean":"2024-05-14T11:16:17.808226+0000","last_became_active":"2024-05-14T11:16:12.257876+0000","last_became_peered":"2024-05-14T11:16:12.257876+0000","last_unstale":"2024-05-14T11:16:17.808226+0000","last_undegraded":"2024-05-14T11:16:17.808226+0000","last_fullsized":"2024-05-14T11:16:17.808226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808524+0000","last_change":"2024-05-14T11:16:11.246503+0000","last_active":"2024-05-14T11:16:17.808524+0000","last_peered":"2024-05-14T11:16:17.808524+0000","last_clean":"2024-05-14T11:16:17.808524+0000","last_became_active":"2024-05-14T11:16:11.246056+0000","last_became_peered":"2024-05-14T11:16:11.246056+0000","last_unstale":"2024-05-14T11:16:17.808524+0000","last_undegraded":"2024-05-14T11:16:17.808524+0000","last_fullsized":"2024-05-14T11:16:17.808524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300138+0000","last_change":"2024-05-14T11:16:11.258034+0000","last_active":"2024-05-14T11:16:17.300138+0000","last_peered":"2024-05-14T11:16:17.300138+0000","last_clean":"2024-05-14T11:16:17.300138+0000","last_became_active":"2024-05-14T11:16:11.257706+0000","last_became_peered":"2024-05-14T11:16:11.257706+0000","last_unstale":"2024-05-14T11:16:17.300138+0000","last_undegraded":"2024-05-14T11:16:17.300138+0000","last_fullsized":"2024-05-14T11:16:17.300138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806611+0000","last_change":"2024-05-14T11:16:12.248022+0000","last_active":"2024-05-14T11:16:17.806611+0000","last_peered":"2024-05-14T11:16:17.806611+0000","last_clean":"2024-05-14T11:16:17.806611+0000","last_became_active":"2024-05-14T11:16:12.247640+0000","last_became_peered":"2024-05-14T11:16:12.247640+0000","last_unstale":"2024-05-14T11:16:17.806611+0000","last_undegraded":"2024-05-14T11:16:17.806611+0000","last_fullsized":"2024-05-14T11:16:17.806611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298100+0000","last_change":"2024-05-14T11:16:16.296944+0000","last_active":"2024-05-14T11:16:17.298100+0000","last_peered":"2024-05-14T11:16:17.298100+0000","last_clean":"2024-05-14T11:16:17.298100+0000","last_became_active":"2024-05-14T11:16:16.296515+0000","last_became_peered":"2024-05-14T11:16:16.296515+0000","last_unstale":"2024-05-14T11:16:17.298100+0000","last_undegraded":"2024-05-14T11:16:17.298100+0000","last_fullsized":"2024-05-14T11:16:17.298100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:41.471078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809032+0000","last_change":"2024-05-14T11:16:11.253359+0000","last_active":"2024-05-14T11:16:17.809032+0000","last_peered":"2024-05-14T11:16:17.809032+0000","last_clean":"2024-05-14T11:16:17.809032+0000","last_became_active":"2024-05-14T11:16:11.248847+0000","last_became_peered":"2024-05-14T11:16:11.248847+0000","last_unstale":"2024-05-14T11:16:17.809032+0000","last_undegraded":"2024-05-14T11:16:17.809032+0000","last_fullsized":"2024-05-14T11:16:17.809032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301381+0000","last_change":"2024-05-14T11:16:12.239027+0000","last_active":"2024-05-14T11:16:17.301381+0000","last_peered":"2024-05-14T11:16:17.301381+0000","last_clean":"2024-05-14T11:16:17.301381+0000","last_became_active":"2024-05-14T11:16:12.238687+0000","last_became_peered":"2024-05-14T11:16:12.238687+0000","last_unstale":"2024-05-14T11:16:17.301381+0000","last_undegraded":"2024-05-14T11:16:17.301381+0000","last_fullsized":"2024-05-14T11:16:17.301381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.490412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809262+0000","last_change":"2024-05-14T11:16:16.293565+0000","last_active":"2024-05-14T11:16:17.809262+0000","last_peered":"2024-05-14T11:16:17.809262+0000","last_clean":"2024-05-14T11:16:17.809262+0000","last_became_active":"2024-05-14T11:16:16.293094+0000","last_became_peered":"2024-05-14T11:16:16.293094+0000","last_unstale":"2024-05-14T11:16:17.809262+0000","last_undegraded":"2024-05-14T11:16:17.809262+0000","last_fullsized":"2024-05-14T11:16:17.809262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:03:43.070705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299522+0000","last_change":"2024-05-14T11:16:12.249274+0000","last_active":"2024-05-14T11:16:17.299522+0000","last_peered":"2024-05-14T11:16:17.299522+0000","last_clean":"2024-05-14T11:16:17.299522+0000","last_became_active":"2024-05-14T11:16:12.248560+0000","last_became_peered":"2024-05-14T11:16:12.248560+0000","last_unstale":"2024-05-14T11:16:17.299522+0000","last_undegraded":"2024-05-14T11:16:17.299522+0000","last_fullsized":"2024-05-14T11:16:17.299522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297943+0000","last_change":"2024-05-14T11:16:12.267227+0000","last_active":"2024-05-14T11:16:17.297943+0000","last_peered":"2024-05-14T11:16:17.297943+0000","last_clean":"2024-05-14T11:16:17.297943+0000","last_became_active":"2024-05-14T11:16:12.266918+0000","last_became_peered":"2024-05-14T11:16:12.266918+0000","last_unstale":"2024-05-14T11:16:17.297943+0000","last_undegraded":"2024-05-14T11:16:17.297943+0000","last_fullsized":"2024-05-14T11:16:17.297943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807593+0000","last_change":"2024-05-14T11:16:12.259446+0000","last_active":"2024-05-14T11:16:17.807593+0000","last_peered":"2024-05-14T11:16:17.807593+0000","last_clean":"2024-05-14T11:16:17.807593+0000","last_became_active":"2024-05-14T11:16:12.259111+0000","last_became_peered":"2024-05-14T11:16:12.259111+0000","last_unstale":"2024-05-14T11:16:17.807593+0000","last_undegraded":"2024-05-14T11:16:17.807593+0000","last_fullsized":"2024-05-14T11:16:17.807593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806487+0000","last_change":"2024-05-14T11:16:12.259006+0000","last_active":"2024-05-14T11:16:17.806487+0000","last_peered":"2024-05-14T11:16:17.806487+0000","last_clean":"2024-05-14T11:16:17.806487+0000","last_became_active":"2024-05-14T11:16:12.258686+0000","last_became_peered":"2024-05-14T11:16:12.258686+0000","last_unstale":"2024-05-14T11:16:17.806487+0000","last_undegraded":"2024-05-14T11:16:17.806487+0000","last_fullsized":"2024-05-14T11:16:17.806487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808880+0000","last_change":"2024-05-14T11:16:12.263813+0000","last_active":"2024-05-14T11:16:17.808880+0000","last_peered":"2024-05-14T11:16:17.808880+0000","last_clean":"2024-05-14T11:16:17.808880+0000","last_became_active":"2024-05-14T11:16:12.263020+0000","last_became_peered":"2024-05-14T11:16:12.263020+0000","last_unstale":"2024-05-14T11:16:17.808880+0000","last_undegraded":"2024-05-14T11:16:17.808880+0000","last_fullsized":"2024-05-14T11:16:17.808880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807160+0000","last_change":"2024-05-14T11:16:12.265745+0000","last_active":"2024-05-14T11:16:17.807160+0000","last_peered":"2024-05-14T11:16:17.807160+0000","last_clean":"2024-05-14T11:16:17.807160+0000","last_became_active":"2024-05-14T11:16:12.262165+0000","last_became_peered":"2024-05-14T11:16:12.262165+0000","last_unstale":"2024-05-14T11:16:17.807160+0000","last_undegraded":"2024-05-14T11:16:17.807160+0000","last_fullsized":"2024-05-14T11:16:17.807160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301482+0000","last_change":"2024-05-14T11:16:12.250079+0000","last_active":"2024-05-14T11:16:17.301482+0000","last_peered":"2024-05-14T11:16:17.301482+0000","last_clean":"2024-05-14T11:16:17.301482+0000","last_became_active":"2024-05-14T11:16:12.246835+0000","last_became_peered":"2024-05-14T11:16:12.246835+0000","last_unstale":"2024-05-14T11:16:17.301482+0000","last_undegraded":"2024-05-14T11:16:17.301482+0000","last_fullsized":"2024-05-14T11:16:17.301482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809094+0000","last_change":"2024-05-14T11:16:12.264222+0000","last_active":"2024-05-14T11:16:17.809094+0000","last_peered":"2024-05-14T11:16:17.809094+0000","last_clean":"2024-05-14T11:16:17.809094+0000","last_became_active":"2024-05-14T11:16:12.263773+0000","last_became_peered":"2024-05-14T11:16:12.263773+0000","last_unstale":"2024-05-14T11:16:17.809094+0000","last_undegraded":"2024-05-14T11:16:17.809094+0000","last_fullsized":"2024-05-14T11:16:17.809094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806261+0000","last_change":"2024-05-14T11:16:12.260462+0000","last_active":"2024-05-14T11:16:17.806261+0000","last_peered":"2024-05-14T11:16:17.806261+0000","last_clean":"2024-05-14T11:16:17.806261+0000","last_became_active":"2024-05-14T11:16:12.260089+0000","last_became_peered":"2024-05-14T11:16:12.260089+0000","last_unstale":"2024-05-14T11:16:17.806261+0000","last_undegraded":"2024-05-14T11:16:17.806261+0000","last_fullsized":"2024-05-14T11:16:17.806261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808385+0000","last_change":"2024-05-14T11:16:12.257286+0000","last_active":"2024-05-14T11:16:17.808385+0000","last_peered":"2024-05-14T11:16:17.808385+0000","last_clean":"2024-05-14T11:16:17.808385+0000","last_became_active":"2024-05-14T11:16:12.256949+0000","last_became_peered":"2024-05-14T11:16:12.256949+0000","last_unstale":"2024-05-14T11:16:17.808385+0000","last_undegraded":"2024-05-14T11:16:17.808385+0000","last_fullsized":"2024-05-14T11:16:17.808385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807462+0000","last_change":"2024-05-14T11:16:12.266841+0000","last_active":"2024-05-14T11:16:17.807462+0000","last_peered":"2024-05-14T11:16:17.807462+0000","last_clean":"2024-05-14T11:16:17.807462+0000","last_became_active":"2024-05-14T11:16:12.266472+0000","last_became_peered":"2024-05-14T11:16:12.266472+0000","last_unstale":"2024-05-14T11:16:17.807462+0000","last_undegraded":"2024-05-14T11:16:17.807462+0000","last_fullsized":"2024-05-14T11:16:17.807462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806284+0000","last_change":"2024-05-14T11:16:12.245994+0000","last_active":"2024-05-14T11:16:17.806284+0000","last_peered":"2024-05-14T11:16:17.806284+0000","last_clean":"2024-05-14T11:16:17.806284+0000","last_became_active":"2024-05-14T11:16:12.243237+0000","last_became_peered":"2024-05-14T11:16:12.243237+0000","last_unstale":"2024-05-14T11:16:17.806284+0000","last_undegraded":"2024-05-14T11:16:17.806284+0000","last_fullsized":"2024-05-14T11:16:17.806284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300549+0000","last_change":"2024-05-14T11:16:12.240444+0000","last_active":"2024-05-14T11:16:17.300549+0000","last_peered":"2024-05-14T11:16:17.300549+0000","last_clean":"2024-05-14T11:16:17.300549+0000","last_became_active":"2024-05-14T11:16:12.240076+0000","last_became_peered":"2024-05-14T11:16:12.240076+0000","last_unstale":"2024-05-14T11:16:17.300549+0000","last_undegraded":"2024-05-14T11:16:17.300549+0000","last_fullsized":"2024-05-14T11:16:17.300549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807295+0000","last_change":"2024-05-14T11:16:12.243507+0000","last_active":"2024-05-14T11:16:17.807295+0000","last_peered":"2024-05-14T11:16:17.807295+0000","last_clean":"2024-05-14T11:16:17.807295+0000","last_became_active":"2024-05-14T11:16:12.243180+0000","last_became_peered":"2024-05-14T11:16:12.243180+0000","last_unstale":"2024-05-14T11:16:17.807295+0000","last_undegraded":"2024-05-14T11:16:17.807295+0000","last_fullsized":"2024-05-14T11:16:17.807295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298355+0000","last_change":"2024-05-14T11:16:12.250650+0000","last_active":"2024-05-14T11:16:17.298355+0000","last_peered":"2024-05-14T11:16:17.298355+0000","last_clean":"2024-05-14T11:16:17.298355+0000","last_became_active":"2024-05-14T11:16:12.250259+0000","last_became_peered":"2024-05-14T11:16:12.250259+0000","last_unstale":"2024-05-14T11:16:17.298355+0000","last_undegraded":"2024-05-14T11:16:17.298355+0000","last_fullsized":"2024-05-14T11:16:17.298355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807449+0000","last_change":"2024-05-14T11:16:12.267003+0000","last_active":"2024-05-14T11:16:17.807449+0000","last_peered":"2024-05-14T11:16:17.807449+0000","last_clean":"2024-05-14T11:16:17.807449+0000","last_became_active":"2024-05-14T11:16:12.266655+0000","last_became_peered":"2024-05-14T11:16:12.266655+0000","last_unstale":"2024-05-14T11:16:17.807449+0000","last_undegraded":"2024-05-14T11:16:17.807449+0000","last_fullsized":"2024-05-14T11:16:17.807449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806108+0000","last_change":"2024-05-14T11:16:12.251621+0000","last_active":"2024-05-14T11:16:17.806108+0000","last_peered":"2024-05-14T11:16:17.806108+0000","last_clean":"2024-05-14T11:16:17.806108+0000","last_became_active":"2024-05-14T11:16:12.251284+0000","last_became_peered":"2024-05-14T11:16:12.251284+0000","last_unstale":"2024-05-14T11:16:17.806108+0000","last_undegraded":"2024-05-14T11:16:17.806108+0000","last_fullsized":"2024-05-14T11:16:17.806108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806728+0000","last_change":"2024-05-14T11:16:12.247078+0000","last_active":"2024-05-14T11:16:17.806728+0000","last_peered":"2024-05-14T11:16:17.806728+0000","last_clean":"2024-05-14T11:16:17.806728+0000","last_became_active":"2024-05-14T11:16:12.246785+0000","last_became_peered":"2024-05-14T11:16:12.246785+0000","last_unstale":"2024-05-14T11:16:17.806728+0000","last_undegraded":"2024-05-14T11:16:17.806728+0000","last_fullsized":"2024-05-14T11:16:17.806728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806255+0000","last_change":"2024-05-14T11:16:12.246002+0000","last_active":"2024-05-14T11:16:17.806255+0000","last_peered":"2024-05-14T11:16:17.806255+0000","last_clean":"2024-05-14T11:16:17.806255+0000","last_became_active":"2024-05-14T11:16:12.245378+0000","last_became_peered":"2024-05-14T11:16:12.245378+0000","last_unstale":"2024-05-14T11:16:17.806255+0000","last_undegraded":"2024-05-14T11:16:17.806255+0000","last_fullsized":"2024-05-14T11:16:17.806255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:42.753210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808869+0000","last_change":"2024-05-14T11:16:12.264249+0000","last_active":"2024-05-14T11:16:17.808869+0000","last_peered":"2024-05-14T11:16:17.808869+0000","last_clean":"2024-05-14T11:16:17.808869+0000","last_became_active":"2024-05-14T11:16:12.263604+0000","last_became_peered":"2024-05-14T11:16:12.263604+0000","last_unstale":"2024-05-14T11:16:17.808869+0000","last_undegraded":"2024-05-14T11:16:17.808869+0000","last_fullsized":"2024-05-14T11:16:17.808869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302334+0000","last_change":"2024-05-14T11:16:12.239836+0000","last_active":"2024-05-14T11:16:17.302334+0000","last_peered":"2024-05-14T11:16:17.302334+0000","last_clean":"2024-05-14T11:16:17.302334+0000","last_became_active":"2024-05-14T11:16:12.236431+0000","last_became_peered":"2024-05-14T11:16:12.236431+0000","last_unstale":"2024-05-14T11:16:17.302334+0000","last_undegraded":"2024-05-14T11:16:17.302334+0000","last_fullsized":"2024-05-14T11:16:17.302334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:03.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300974+0000","last_change":"2024-05-14T11:16:12.273349+0000","last_active":"2024-05-14T11:16:17.300974+0000","last_peered":"2024-05-14T11:16:17.300974+0000","last_clean":"2024-05-14T11:16:17.300974+0000","last_became_active":"2024-05-14T11:16:12.272507+0000","last_became_peered":"2024-05-14T11:16:12.272507+0000","last_unstale":"2024-05-14T11:16:17.300974+0000","last_undegraded":"2024-05-14T11:16:17.300974+0000","last_fullsized":"2024-05-14T11:16:17.300974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301473+0000","last_change":"2024-05-14T11:16:12.257302+0000","last_active":"2024-05-14T11:16:17.301473+0000","last_peered":"2024-05-14T11:16:17.301473+0000","last_clean":"2024-05-14T11:16:17.301473+0000","last_became_active":"2024-05-14T11:16:12.256830+0000","last_became_peered":"2024-05-14T11:16:12.256830+0000","last_unstale":"2024-05-14T11:16:17.301473+0000","last_undegraded":"2024-05-14T11:16:17.301473+0000","last_fullsized":"2024-05-14T11:16:17.301473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807763+0000","last_change":"2024-05-14T11:16:12.264613+0000","last_active":"2024-05-14T11:16:17.807763+0000","last_peered":"2024-05-14T11:16:17.807763+0000","last_clean":"2024-05-14T11:16:17.807763+0000","last_became_active":"2024-05-14T11:16:12.264294+0000","last_became_peered":"2024-05-14T11:16:12.264294+0000","last_unstale":"2024-05-14T11:16:17.807763+0000","last_undegraded":"2024-05-14T11:16:17.807763+0000","last_fullsized":"2024-05-14T11:16:17.807763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:42.657656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808085+0000","last_change":"2024-05-14T11:16:12.267492+0000","last_active":"2024-05-14T11:16:17.808085+0000","last_peered":"2024-05-14T11:16:17.808085+0000","last_clean":"2024-05-14T11:16:17.808085+0000","last_became_active":"2024-05-14T11:16:12.267005+0000","last_became_peered":"2024-05-14T11:16:12.267005+0000","last_unstale":"2024-05-14T11:16:17.808085+0000","last_undegraded":"2024-05-14T11:16:17.808085+0000","last_fullsized":"2024-05-14T11:16:17.808085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807982+0000","last_change":"2024-05-14T11:16:12.252241+0000","last_active":"2024-05-14T11:16:17.807982+0000","last_peered":"2024-05-14T11:16:17.807982+0000","last_clean":"2024-05-14T11:16:17.807982+0000","last_became_active":"2024-05-14T11:16:12.251844+0000","last_became_peered":"2024-05-14T11:16:12.251844+0000","last_unstale":"2024-05-14T11:16:17.807982+0000","last_undegraded":"2024-05-14T11:16:17.807982+0000","last_fullsized":"2024-05-14T11:16:17.807982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807948+0000","last_change":"2024-05-14T11:16:12.254254+0000","last_active":"2024-05-14T11:16:17.807948+0000","last_peered":"2024-05-14T11:16:17.807948+0000","last_clean":"2024-05-14T11:16:17.807948+0000","last_became_active":"2024-05-14T11:16:12.253441+0000","last_became_peered":"2024-05-14T11:16:12.253441+0000","last_unstale":"2024-05-14T11:16:17.807948+0000","last_undegraded":"2024-05-14T11:16:17.807948+0000","last_fullsized":"2024-05-14T11:16:17.807948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807243+0000","last_change":"2024-05-14T11:16:12.244922+0000","last_active":"2024-05-14T11:16:17.807243+0000","last_peered":"2024-05-14T11:16:17.807243+0000","last_clean":"2024-05-14T11:16:17.807243+0000","last_became_active":"2024-05-14T11:16:12.244534+0000","last_became_peered":"2024-05-14T11:16:12.244534+0000","last_unstale":"2024-05-14T11:16:17.807243+0000","last_undegraded":"2024-05-14T11:16:17.807243+0000","last_fullsized":"2024-05-14T11:16:17.807243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806550+0000","last_change":"2024-05-14T11:16:12.270470+0000","last_active":"2024-05-14T11:16:17.806550+0000","last_peered":"2024-05-14T11:16:17.806550+0000","last_clean":"2024-05-14T11:16:17.806550+0000","last_became_active":"2024-05-14T11:16:12.269667+0000","last_became_peered":"2024-05-14T11:16:12.269667+0000","last_unstale":"2024-05-14T11:16:17.806550+0000","last_undegraded":"2024-05-14T11:16:17.806550+0000","last_fullsized":"2024-05-14T11:16:17.806550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806567+0000","last_change":"2024-05-14T11:16:12.254558+0000","last_active":"2024-05-14T11:16:17.806567+0000","last_peered":"2024-05-14T11:16:17.806567+0000","last_clean":"2024-05-14T11:16:17.806567+0000","last_became_active":"2024-05-14T11:16:12.254245+0000","last_became_peered":"2024-05-14T11:16:12.254245+0000","last_unstale":"2024-05-14T11:16:17.806567+0000","last_undegraded":"2024-05-14T11:16:17.806567+0000","last_fullsized":"2024-05-14T11:16:17.806567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807656+0000","last_change":"2024-05-14T11:16:12.256145+0000","last_active":"2024-05-14T11:16:17.807656+0000","last_peered":"2024-05-14T11:16:17.807656+0000","last_clean":"2024-05-14T11:16:17.807656+0000","last_became_active":"2024-05-14T11:16:12.255806+0000","last_became_peered":"2024-05-14T11:16:12.255806+0000","last_unstale":"2024-05-14T11:16:17.807656+0000","last_undegraded":"2024-05-14T11:16:17.807656+0000","last_fullsized":"2024-05-14T11:16:17.807656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807528+0000","last_change":"2024-05-14T11:16:12.252502+0000","last_active":"2024-05-14T11:16:17.807528+0000","last_peered":"2024-05-14T11:16:17.807528+0000","last_clean":"2024-05-14T11:16:17.807528+0000","last_became_active":"2024-05-14T11:16:12.251431+0000","last_became_peered":"2024-05-14T11:16:12.251431+0000","last_unstale":"2024-05-14T11:16:17.807528+0000","last_undegraded":"2024-05-14T11:16:17.807528+0000","last_fullsized":"2024-05-14T11:16:17.807528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:16:38.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:16:38.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.500+0000 7f6b03600640 1 Processor -- start 2024-05-14T11:16:38.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.500+0000 7f6b03600640 1 -- start start 2024-05-14T11:16:38.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b03600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6afc0d1c20 0x7f6afc0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b03600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc0b3060 con 0x7f6afc0cdf80 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b03600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc0b3300 con 0x7f6afc0b40c0 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b03600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6afc0b35e0 con 0x7f6afc0d1c20 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b00c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6afc0cdf80 0x7f6afc0d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51740/0 (socket says 172.21.15.116:51740) 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6b00c00640 1 -- 172.21.15.116:0/1777011213 learned_addr learned my addr 172.21.15.116:0/1777011213 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afbe00640 1 --2- 172.21.15.116:0/1777011213 >> v2:172.21.15.149:3300/0 conn(0x7f6afc0d1c20 0x7f6afc0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afb400640 1 -- 172.21.15.116:0/1777011213 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076835684 0 0) 0x7f6afc0b3060 con 0x7f6afc0cdf80 2024-05-14T11:16:38.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afb400640 1 -- 172.21.15.116:0/1777011213 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ae0003530 con 0x7f6afc0cdf80 2024-05-14T11:16:38.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afbe00640 1 -- 172.21.15.116:0/1777011213 >> v1:172.21.15.149:6789/0 conn(0x7f6afc0b40c0 legacy=0x7f6afc0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afbe00640 1 -- 172.21.15.116:0/1777011213 >> v1:172.21.15.116:6789/0 conn(0x7f6afc0cdf80 legacy=0x7f6afc0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.504+0000 7f6afbe00640 1 -- 172.21.15.116:0/1777011213 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0b38c0 con 0x7f6afc0d1c20 2024-05-14T11:16:38.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6afbe00640 1 --2- 172.21.15.116:0/1777011213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0d1c20 0x7f6afc0ce390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80097c0 tx=0x7f6ae801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e3cdbf1224fbec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6afb400640 1 -- 172.21.15.116:0/1777011213 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80aa5f0 con 0x7f6afc0d1c20 2024-05-14T11:16:38.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6afb400640 1 -- 172.21.15.116:0/1777011213 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae80af070 con 0x7f6afc0d1c20 2024-05-14T11:16:38.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6afb400640 1 -- 172.21.15.116:0/1777011213 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80aa9b0 con 0x7f6afc0d1c20 2024-05-14T11:16:38.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- 172.21.15.116:0/1777011213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0d1c20 msgr2=0x7f6afc0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 --2- 172.21.15.116:0/1777011213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0d1c20 0x7f6afc0ce390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80097c0 tx=0x7f6ae801c470 comp rx=0 tx=0).stop 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- 172.21.15.116:0/1777011213 shutdown_connections 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 --2- 172.21.15.116:0/1777011213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0d1c20 0x7f6afc0ce390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- 172.21.15.116:0/1777011213 >> 172.21.15.116:0/1777011213 conn(0x7f6afc0d0770 msgr2=0x7f6afc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- 172.21.15.116:0/1777011213 shutdown_connections 2024-05-14T11:16:38.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- 172.21.15.116:0/1777011213 wait complete. 2024-05-14T11:16:38.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 Processor -- start 2024-05-14T11:16:38.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.508+0000 7f6b03600640 1 -- start start 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 0x7f6afc0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0cdf80 0x7f6afc0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc16dd70 con 0x7f6afc0b0e20 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc06ffd0 con 0x7f6afc0cdf80 2024-05-14T11:16:38.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6afc16def0 con 0x7f6afc0b40c0 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58610/0 (socket says 172.21.15.116:58610) 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 -- 172.21.15.116:0/3408053547 learned_addr learned my addr 172.21.15.116:0/3408053547 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6afbe00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0cdf80 0x7f6afc0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b00c00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 0x7f6afc0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 -- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 msgr2=0x7f6afc0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 0x7f6afc0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 -- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0cdf80 msgr2=0x7f6afc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0cdf80 0x7f6afc0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 -- 172.21.15.116:0/3408053547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0b5d40 con 0x7f6afc0b0e20 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b00c00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 0x7f6afc0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:38.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b01600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c3f0 tx=0x7f6af400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4017070 con 0x7f6afc0b0e20 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af40052d0 con 0x7f6afc0b0e20 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4003800 con 0x7f6afc0b0e20 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6afc0b1560 con 0x7f6afc0b0e20 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6afc0b1b20 con 0x7f6afc0b0e20 2024-05-14T11:16:38.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.512+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6afc06a9c0 con 0x7f6afc0b0e20 2024-05-14T11:16:38.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.516+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6af401a070 con 0x7f6afc0b0e20 2024-05-14T11:16:38.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.516+0000 7f6af8c00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 0x7f6adc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.516+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6af408e950 con 0x7f6afc0b0e20 2024-05-14T11:16:38.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.516+0000 7f6b00c00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 0x7f6adc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.516+0000 7f6b00c00640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 0x7f6adc071940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005c10 tx=0x7f6aec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.520+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6afc0b5d40 con 0x7f6afc0b0e20 2024-05-14T11:16:38.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.672+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6afc0b9590 con 0x7f6adc06f430 2024-05-14T11:16:38.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.680+0000 7f6af8c00640 1 -- 172.21.15.116:0/3408053547 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332520 (secure 0 0 0) 0x7f6afc0b9590 con 0x7f6adc06f430 2024-05-14T11:16:38.688 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 msgr2=0x7f6adc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 0x7f6adc071940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005c10 tx=0x7f6aec009040 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 msgr2=0x7f6afc0b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c3f0 tx=0x7f6af400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 shutdown_connections 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6adc06f430 0x7f6adc071940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6afc0cdf80 0x7f6afc0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc0b0e20 0x7f6afc0b6a20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 --2- 172.21.15.116:0/3408053547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6afc0b40c0 0x7f6afc0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.684+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 >> 172.21.15.116:0/3408053547 conn(0x7f6afc0d0770 msgr2=0x7f6afc070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.688+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 shutdown_connections 2024-05-14T11:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.688+0000 7f6b03600640 1 -- 172.21.15.116:0/3408053547 wait complete. 2024-05-14T11:16:38.708 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-14T11:16:38.364618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003477"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807671+0000","last_change":"2024-05-14T11:16:11.249111+0000","last_active":"2024-05-14T11:16:17.807671+0000","last_peered":"2024-05-14T11:16:17.807671+0000","last_clean":"2024-05-14T11:16:17.807671+0000","last_became_active":"2024-05-14T11:16:11.248739+0000","last_became_peered":"2024-05-14T11:16:11.248739+0000","last_unstale":"2024-05-14T11:16:17.807671+0000","last_undegraded":"2024-05-14T11:16:17.807671+0000","last_fullsized":"2024-05-14T11:16:17.807671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808824+0000","last_change":"2024-05-14T11:16:12.266252+0000","last_active":"2024-05-14T11:16:17.808824+0000","last_peered":"2024-05-14T11:16:17.808824+0000","last_clean":"2024-05-14T11:16:17.808824+0000","last_became_active":"2024-05-14T11:16:12.265805+0000","last_became_peered":"2024-05-14T11:16:12.265805+0000","last_unstale":"2024-05-14T11:16:17.808824+0000","last_undegraded":"2024-05-14T11:16:17.808824+0000","last_fullsized":"2024-05-14T11:16:17.808824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808167+0000","last_change":"2024-05-14T11:16:16.339345+0000","last_active":"2024-05-14T11:16:17.808167+0000","last_peered":"2024-05-14T11:16:17.808167+0000","last_clean":"2024-05-14T11:16:17.808167+0000","last_became_active":"2024-05-14T11:16:16.338538+0000","last_became_peered":"2024-05-14T11:16:16.338538+0000","last_unstale":"2024-05-14T11:16:17.808167+0000","last_undegraded":"2024-05-14T11:16:17.808167+0000","last_fullsized":"2024-05-14T11:16:17.808167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809078+0000","last_change":"2024-05-14T11:16:11.246975+0000","last_active":"2024-05-14T11:16:17.809078+0000","last_peered":"2024-05-14T11:16:17.809078+0000","last_clean":"2024-05-14T11:16:17.809078+0000","last_became_active":"2024-05-14T11:16:11.246617+0000","last_became_peered":"2024-05-14T11:16:11.246617+0000","last_unstale":"2024-05-14T11:16:17.809078+0000","last_undegraded":"2024-05-14T11:16:17.809078+0000","last_fullsized":"2024-05-14T11:16:17.809078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807310+0000","last_change":"2024-05-14T11:16:12.261153+0000","last_active":"2024-05-14T11:16:17.807310+0000","last_peered":"2024-05-14T11:16:17.807310+0000","last_clean":"2024-05-14T11:16:17.807310+0000","last_became_active":"2024-05-14T11:16:12.260566+0000","last_became_peered":"2024-05-14T11:16:12.260566+0000","last_unstale":"2024-05-14T11:16:17.807310+0000","last_undegraded":"2024-05-14T11:16:17.807310+0000","last_fullsized":"2024-05-14T11:16:17.807310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:47.069928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300836+0000","last_change":"2024-05-14T11:16:16.298112+0000","last_active":"2024-05-14T11:16:17.300836+0000","last_peered":"2024-05-14T11:16:17.300836+0000","last_clean":"2024-05-14T11:16:17.300836+0000","last_became_active":"2024-05-14T11:16:16.297673+0000","last_became_peered":"2024-05-14T11:16:16.297673+0000","last_unstale":"2024-05-14T11:16:17.300836+0000","last_undegraded":"2024-05-14T11:16:17.300836+0000","last_fullsized":"2024-05-14T11:16:17.300836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:13.339957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807596+0000","last_change":"2024-05-14T11:16:11.253925+0000","last_active":"2024-05-14T11:16:17.807596+0000","last_peered":"2024-05-14T11:16:17.807596+0000","last_clean":"2024-05-14T11:16:17.807596+0000","last_became_active":"2024-05-14T11:16:11.253441+0000","last_became_peered":"2024-05-14T11:16:11.253441+0000","last_unstale":"2024-05-14T11:16:17.807596+0000","last_undegraded":"2024-05-14T11:16:17.807596+0000","last_fullsized":"2024-05-14T11:16:17.807596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806697+0000","last_change":"2024-05-14T11:16:12.255735+0000","last_active":"2024-05-14T11:16:17.806697+0000","last_peered":"2024-05-14T11:16:17.806697+0000","last_clean":"2024-05-14T11:16:17.806697+0000","last_became_active":"2024-05-14T11:16:12.255414+0000","last_became_peered":"2024-05-14T11:16:12.255414+0000","last_unstale":"2024-05-14T11:16:17.806697+0000","last_undegraded":"2024-05-14T11:16:17.806697+0000","last_fullsized":"2024-05-14T11:16:17.806697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300072+0000","last_change":"2024-05-14T11:16:16.297447+0000","last_active":"2024-05-14T11:16:17.300072+0000","last_peered":"2024-05-14T11:16:17.300072+0000","last_clean":"2024-05-14T11:16:17.300072+0000","last_became_active":"2024-05-14T11:16:16.297024+0000","last_became_peered":"2024-05-14T11:16:16.297024+0000","last_unstale":"2024-05-14T11:16:17.300072+0000","last_undegraded":"2024-05-14T11:16:17.300072+0000","last_fullsized":"2024-05-14T11:16:17.300072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:41.465988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806953+0000","last_change":"2024-05-14T11:16:11.263356+0000","last_active":"2024-05-14T11:16:17.806953+0000","last_peered":"2024-05-14T11:16:17.806953+0000","last_clean":"2024-05-14T11:16:17.806953+0000","last_became_active":"2024-05-14T11:16:11.263025+0000","last_became_peered":"2024-05-14T11:16:11.263025+0000","last_unstale":"2024-05-14T11:16:17.806953+0000","last_undegraded":"2024-05-14T11:16:17.806953+0000","last_fullsized":"2024-05-14T11:16:17.806953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806916+0000","last_change":"2024-05-14T11:16:12.253871+0000","last_active":"2024-05-14T11:16:17.806916+0000","last_peered":"2024-05-14T11:16:17.806916+0000","last_clean":"2024-05-14T11:16:17.806916+0000","last_became_active":"2024-05-14T11:16:12.253416+0000","last_became_peered":"2024-05-14T11:16:12.253416+0000","last_unstale":"2024-05-14T11:16:17.806916+0000","last_undegraded":"2024-05-14T11:16:17.806916+0000","last_fullsized":"2024-05-14T11:16:17.806916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300363+0000","last_change":"2024-05-14T11:16:16.289232+0000","last_active":"2024-05-14T11:16:17.300363+0000","last_peered":"2024-05-14T11:16:17.300363+0000","last_clean":"2024-05-14T11:16:17.300363+0000","last_became_active":"2024-05-14T11:16:16.288706+0000","last_became_peered":"2024-05-14T11:16:16.288706+0000","last_unstale":"2024-05-14T11:16:17.300363+0000","last_undegraded":"2024-05-14T11:16:17.300363+0000","last_fullsized":"2024-05-14T11:16:17.300363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:13.765975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299905+0000","last_change":"2024-05-14T11:16:12.237866+0000","last_active":"2024-05-14T11:16:17.299905+0000","last_peered":"2024-05-14T11:16:17.299905+0000","last_clean":"2024-05-14T11:16:17.299905+0000","last_became_active":"2024-05-14T11:16:12.237591+0000","last_became_peered":"2024-05-14T11:16:12.237591+0000","last_unstale":"2024-05-14T11:16:17.299905+0000","last_undegraded":"2024-05-14T11:16:17.299905+0000","last_fullsized":"2024-05-14T11:16:17.299905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807660+0000","last_change":"2024-05-14T11:16:11.250842+0000","last_active":"2024-05-14T11:16:17.807660+0000","last_peered":"2024-05-14T11:16:17.807660+0000","last_clean":"2024-05-14T11:16:17.807660+0000","last_became_active":"2024-05-14T11:16:11.250454+0000","last_became_peered":"2024-05-14T11:16:11.250454+0000","last_unstale":"2024-05-14T11:16:17.807660+0000","last_undegraded":"2024-05-14T11:16:17.807660+0000","last_fullsized":"2024-05-14T11:16:17.807660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302586+0000","last_change":"2024-05-14T11:16:16.299192+0000","last_active":"2024-05-14T11:16:17.302586+0000","last_peered":"2024-05-14T11:16:17.302586+0000","last_clean":"2024-05-14T11:16:17.302586+0000","last_became_active":"2024-05-14T11:16:16.298925+0000","last_became_peered":"2024-05-14T11:16:16.298925+0000","last_unstale":"2024-05-14T11:16:17.302586+0000","last_undegraded":"2024-05-14T11:16:17.302586+0000","last_fullsized":"2024-05-14T11:16:17.302586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808972+0000","last_change":"2024-05-14T11:16:12.266594+0000","last_active":"2024-05-14T11:16:17.808972+0000","last_peered":"2024-05-14T11:16:17.808972+0000","last_clean":"2024-05-14T11:16:17.808972+0000","last_became_active":"2024-05-14T11:16:12.266261+0000","last_became_peered":"2024-05-14T11:16:12.266261+0000","last_unstale":"2024-05-14T11:16:17.808972+0000","last_undegraded":"2024-05-14T11:16:17.808972+0000","last_fullsized":"2024-05-14T11:16:17.808972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808959+0000","last_change":"2024-05-14T11:16:11.251548+0000","last_active":"2024-05-14T11:16:17.808959+0000","last_peered":"2024-05-14T11:16:17.808959+0000","last_clean":"2024-05-14T11:16:17.808959+0000","last_became_active":"2024-05-14T11:16:11.251146+0000","last_became_peered":"2024-05-14T11:16:11.251146+0000","last_unstale":"2024-05-14T11:16:17.808959+0000","last_undegraded":"2024-05-14T11:16:17.808959+0000","last_fullsized":"2024-05-14T11:16:17.808959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300981+0000","last_change":"2024-05-14T11:16:16.296270+0000","last_active":"2024-05-14T11:16:17.300981+0000","last_peered":"2024-05-14T11:16:17.300981+0000","last_clean":"2024-05-14T11:16:17.300981+0000","last_became_active":"2024-05-14T11:16:16.295796+0000","last_became_peered":"2024-05-14T11:16:16.295796+0000","last_unstale":"2024-05-14T11:16:17.300981+0000","last_undegraded":"2024-05-14T11:16:17.300981+0000","last_fullsized":"2024-05-14T11:16:17.300981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808335+0000","last_change":"2024-05-14T11:16:12.259552+0000","last_active":"2024-05-14T11:16:17.808335+0000","last_peered":"2024-05-14T11:16:17.808335+0000","last_clean":"2024-05-14T11:16:17.808335+0000","last_became_active":"2024-05-14T11:16:12.259053+0000","last_became_peered":"2024-05-14T11:16:12.259053+0000","last_unstale":"2024-05-14T11:16:17.808335+0000","last_undegraded":"2024-05-14T11:16:17.808335+0000","last_fullsized":"2024-05-14T11:16:17.808335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806762+0000","last_change":"2024-05-14T11:16:11.251898+0000","last_active":"2024-05-14T11:16:17.806762+0000","last_peered":"2024-05-14T11:16:17.806762+0000","last_clean":"2024-05-14T11:16:17.806762+0000","last_became_active":"2024-05-14T11:16:11.251339+0000","last_became_peered":"2024-05-14T11:16:11.251339+0000","last_unstale":"2024-05-14T11:16:17.806762+0000","last_undegraded":"2024-05-14T11:16:17.806762+0000","last_fullsized":"2024-05-14T11:16:17.806762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809475+0000","last_change":"2024-05-14T11:16:16.331750+0000","last_active":"2024-05-14T11:16:17.809475+0000","last_peered":"2024-05-14T11:16:17.809475+0000","last_clean":"2024-05-14T11:16:17.809475+0000","last_became_active":"2024-05-14T11:16:16.331181+0000","last_became_peered":"2024-05-14T11:16:16.331181+0000","last_unstale":"2024-05-14T11:16:17.809475+0000","last_undegraded":"2024-05-14T11:16:17.809475+0000","last_fullsized":"2024-05-14T11:16:17.809475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301296+0000","last_change":"2024-05-14T11:16:12.245880+0000","last_active":"2024-05-14T11:16:17.301296+0000","last_peered":"2024-05-14T11:16:17.301296+0000","last_clean":"2024-05-14T11:16:17.301296+0000","last_became_active":"2024-05-14T11:16:12.245150+0000","last_became_peered":"2024-05-14T11:16:12.245150+0000","last_unstale":"2024-05-14T11:16:17.301296+0000","last_undegraded":"2024-05-14T11:16:17.301296+0000","last_fullsized":"2024-05-14T11:16:17.301296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806258+0000","last_change":"2024-05-14T11:16:11.252415+0000","last_active":"2024-05-14T11:16:17.806258+0000","last_peered":"2024-05-14T11:16:17.806258+0000","last_clean":"2024-05-14T11:16:17.806258+0000","last_became_active":"2024-05-14T11:16:11.252113+0000","last_became_peered":"2024-05-14T11:16:11.252113+0000","last_unstale":"2024-05-14T11:16:17.806258+0000","last_undegraded":"2024-05-14T11:16:17.806258+0000","last_fullsized":"2024-05-14T11:16:17.806258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302531+0000","last_change":"2024-05-14T11:16:16.298123+0000","last_active":"2024-05-14T11:16:17.302531+0000","last_peered":"2024-05-14T11:16:17.302531+0000","last_clean":"2024-05-14T11:16:17.302531+0000","last_became_active":"2024-05-14T11:16:16.297647+0000","last_became_peered":"2024-05-14T11:16:16.297647+0000","last_unstale":"2024-05-14T11:16:17.302531+0000","last_undegraded":"2024-05-14T11:16:17.302531+0000","last_fullsized":"2024-05-14T11:16:17.302531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807122+0000","last_change":"2024-05-14T11:16:12.247217+0000","last_active":"2024-05-14T11:16:17.807122+0000","last_peered":"2024-05-14T11:16:17.807122+0000","last_clean":"2024-05-14T11:16:17.807122+0000","last_became_active":"2024-05-14T11:16:12.246460+0000","last_became_peered":"2024-05-14T11:16:12.246460+0000","last_unstale":"2024-05-14T11:16:17.807122+0000","last_undegraded":"2024-05-14T11:16:17.807122+0000","last_fullsized":"2024-05-14T11:16:17.807122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301791+0000","last_change":"2024-05-14T11:16:11.246553+0000","last_active":"2024-05-14T11:16:17.301791+0000","last_peered":"2024-05-14T11:16:17.301791+0000","last_clean":"2024-05-14T11:16:17.301791+0000","last_became_active":"2024-05-14T11:16:11.245634+0000","last_became_peered":"2024-05-14T11:16:11.245634+0000","last_unstale":"2024-05-14T11:16:17.301791+0000","last_undegraded":"2024-05-14T11:16:17.301791+0000","last_fullsized":"2024-05-14T11:16:17.301791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300036+0000","last_change":"2024-05-14T11:16:16.331377+0000","last_active":"2024-05-14T11:16:17.300036+0000","last_peered":"2024-05-14T11:16:17.300036+0000","last_clean":"2024-05-14T11:16:17.300036+0000","last_became_active":"2024-05-14T11:16:16.330845+0000","last_became_peered":"2024-05-14T11:16:16.330845+0000","last_unstale":"2024-05-14T11:16:17.300036+0000","last_undegraded":"2024-05-14T11:16:17.300036+0000","last_fullsized":"2024-05-14T11:16:17.300036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:08.942678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809208+0000","last_change":"2024-05-14T11:16:12.265091+0000","last_active":"2024-05-14T11:16:17.809208+0000","last_peered":"2024-05-14T11:16:17.809208+0000","last_clean":"2024-05-14T11:16:17.809208+0000","last_became_active":"2024-05-14T11:16:12.264728+0000","last_became_peered":"2024-05-14T11:16:12.264728+0000","last_unstale":"2024-05-14T11:16:17.809208+0000","last_undegraded":"2024-05-14T11:16:17.809208+0000","last_fullsized":"2024-05-14T11:16:17.809208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807713+0000","last_change":"2024-05-14T11:16:11.253941+0000","last_active":"2024-05-14T11:16:17.807713+0000","last_peered":"2024-05-14T11:16:17.807713+0000","last_clean":"2024-05-14T11:16:17.807713+0000","last_became_active":"2024-05-14T11:16:11.253534+0000","last_became_peered":"2024-05-14T11:16:11.253534+0000","last_unstale":"2024-05-14T11:16:17.807713+0000","last_undegraded":"2024-05-14T11:16:17.807713+0000","last_fullsized":"2024-05-14T11:16:17.807713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:37.581488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808083+0000","last_change":"2024-05-14T11:16:16.338462+0000","last_active":"2024-05-14T11:16:17.808083+0000","last_peered":"2024-05-14T11:16:17.808083+0000","last_clean":"2024-05-14T11:16:17.808083+0000","last_became_active":"2024-05-14T11:16:16.337908+0000","last_became_peered":"2024-05-14T11:16:16.337908+0000","last_unstale":"2024-05-14T11:16:17.808083+0000","last_undegraded":"2024-05-14T11:16:17.808083+0000","last_fullsized":"2024-05-14T11:16:17.808083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:28:29.309169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807237+0000","last_change":"2024-05-14T11:16:12.249280+0000","last_active":"2024-05-14T11:16:17.807237+0000","last_peered":"2024-05-14T11:16:17.807237+0000","last_clean":"2024-05-14T11:16:17.807237+0000","last_became_active":"2024-05-14T11:16:12.248977+0000","last_became_peered":"2024-05-14T11:16:12.248977+0000","last_unstale":"2024-05-14T11:16:17.807237+0000","last_undegraded":"2024-05-14T11:16:17.807237+0000","last_fullsized":"2024-05-14T11:16:17.807237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807419+0000","last_change":"2024-05-14T11:16:11.255839+0000","last_active":"2024-05-14T11:16:17.807419+0000","last_peered":"2024-05-14T11:16:17.807419+0000","last_clean":"2024-05-14T11:16:17.807419+0000","last_became_active":"2024-05-14T11:16:11.255487+0000","last_became_peered":"2024-05-14T11:16:11.255487+0000","last_unstale":"2024-05-14T11:16:17.807419+0000","last_undegraded":"2024-05-14T11:16:17.807419+0000","last_fullsized":"2024-05-14T11:16:17.807419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:59.335122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806362+0000","last_change":"2024-05-14T11:16:16.330365+0000","last_active":"2024-05-14T11:16:17.806362+0000","last_peered":"2024-05-14T11:16:17.806362+0000","last_clean":"2024-05-14T11:16:17.806362+0000","last_became_active":"2024-05-14T11:16:16.329833+0000","last_became_peered":"2024-05-14T11:16:16.329833+0000","last_unstale":"2024-05-14T11:16:17.806362+0000","last_undegraded":"2024-05-14T11:16:17.806362+0000","last_fullsized":"2024-05-14T11:16:17.806362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807044+0000","last_change":"2024-05-14T11:16:12.267992+0000","last_active":"2024-05-14T11:16:17.807044+0000","last_peered":"2024-05-14T11:16:17.807044+0000","last_clean":"2024-05-14T11:16:17.807044+0000","last_became_active":"2024-05-14T11:16:12.267183+0000","last_became_peered":"2024-05-14T11:16:12.267183+0000","last_unstale":"2024-05-14T11:16:17.807044+0000","last_undegraded":"2024-05-14T11:16:17.807044+0000","last_fullsized":"2024-05-14T11:16:17.807044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807300+0000","last_change":"2024-05-14T11:16:11.247765+0000","last_active":"2024-05-14T11:16:17.807300+0000","last_peered":"2024-05-14T11:16:17.807300+0000","last_clean":"2024-05-14T11:16:17.807300+0000","last_became_active":"2024-05-14T11:16:11.247503+0000","last_became_peered":"2024-05-14T11:16:11.247503+0000","last_unstale":"2024-05-14T11:16:17.807300+0000","last_undegraded":"2024-05-14T11:16:17.807300+0000","last_fullsized":"2024-05-14T11:16:17.807300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:31.712266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808159+0000","last_change":"2024-05-14T11:16:16.304314+0000","last_active":"2024-05-14T11:16:17.808159+0000","last_peered":"2024-05-14T11:16:17.808159+0000","last_clean":"2024-05-14T11:16:17.808159+0000","last_became_active":"2024-05-14T11:16:16.303370+0000","last_became_peered":"2024-05-14T11:16:16.303370+0000","last_unstale":"2024-05-14T11:16:17.808159+0000","last_undegraded":"2024-05-14T11:16:17.808159+0000","last_fullsized":"2024-05-14T11:16:17.808159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:54:00.581348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301326+0000","last_change":"2024-05-14T11:16:12.258856+0000","last_active":"2024-05-14T11:16:17.301326+0000","last_peered":"2024-05-14T11:16:17.301326+0000","last_clean":"2024-05-14T11:16:17.301326+0000","last_became_active":"2024-05-14T11:16:12.257816+0000","last_became_peered":"2024-05-14T11:16:12.257816+0000","last_unstale":"2024-05-14T11:16:17.301326+0000","last_undegraded":"2024-05-14T11:16:17.301326+0000","last_fullsized":"2024-05-14T11:16:17.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:19.338856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807816+0000","last_change":"2024-05-14T11:16:11.257469+0000","last_active":"2024-05-14T11:16:17.807816+0000","last_peered":"2024-05-14T11:16:17.807816+0000","last_clean":"2024-05-14T11:16:17.807816+0000","last_became_active":"2024-05-14T11:16:11.253425+0000","last_became_peered":"2024-05-14T11:16:11.253425+0000","last_unstale":"2024-05-14T11:16:17.807816+0000","last_undegraded":"2024-05-14T11:16:17.807816+0000","last_fullsized":"2024-05-14T11:16:17.807816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807184+0000","last_change":"2024-05-14T11:16:16.305620+0000","last_active":"2024-05-14T11:16:17.807184+0000","last_peered":"2024-05-14T11:16:17.807184+0000","last_clean":"2024-05-14T11:16:17.807184+0000","last_became_active":"2024-05-14T11:16:16.305156+0000","last_became_peered":"2024-05-14T11:16:16.305156+0000","last_unstale":"2024-05-14T11:16:17.807184+0000","last_undegraded":"2024-05-14T11:16:17.807184+0000","last_fullsized":"2024-05-14T11:16:17.807184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806230+0000","last_change":"2024-05-14T11:16:12.245405+0000","last_active":"2024-05-14T11:16:17.806230+0000","last_peered":"2024-05-14T11:16:17.806230+0000","last_clean":"2024-05-14T11:16:17.806230+0000","last_became_active":"2024-05-14T11:16:12.245076+0000","last_became_peered":"2024-05-14T11:16:12.245076+0000","last_unstale":"2024-05-14T11:16:17.806230+0000","last_undegraded":"2024-05-14T11:16:17.806230+0000","last_fullsized":"2024-05-14T11:16:17.806230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297955+0000","last_change":"2024-05-14T11:16:11.245815+0000","last_active":"2024-05-14T11:16:17.297955+0000","last_peered":"2024-05-14T11:16:17.297955+0000","last_clean":"2024-05-14T11:16:17.297955+0000","last_became_active":"2024-05-14T11:16:11.245487+0000","last_became_peered":"2024-05-14T11:16:11.245487+0000","last_unstale":"2024-05-14T11:16:17.297955+0000","last_undegraded":"2024-05-14T11:16:17.297955+0000","last_fullsized":"2024-05-14T11:16:17.297955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809501+0000","last_change":"2024-05-14T11:16:16.304799+0000","last_active":"2024-05-14T11:16:17.809501+0000","last_peered":"2024-05-14T11:16:17.809501+0000","last_clean":"2024-05-14T11:16:17.809501+0000","last_became_active":"2024-05-14T11:16:16.304223+0000","last_became_peered":"2024-05-14T11:16:16.304223+0000","last_unstale":"2024-05-14T11:16:17.809501+0000","last_undegraded":"2024-05-14T11:16:17.809501+0000","last_fullsized":"2024-05-14T11:16:17.809501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.805980+0000","last_change":"2024-05-14T11:16:12.242987+0000","last_active":"2024-05-14T11:16:17.805980+0000","last_peered":"2024-05-14T11:16:17.805980+0000","last_clean":"2024-05-14T11:16:17.805980+0000","last_became_active":"2024-05-14T11:16:12.242648+0000","last_became_peered":"2024-05-14T11:16:12.242648+0000","last_unstale":"2024-05-14T11:16:17.805980+0000","last_undegraded":"2024-05-14T11:16:17.805980+0000","last_fullsized":"2024-05-14T11:16:17.805980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:37.421472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806812+0000","last_change":"2024-05-14T11:16:11.254596+0000","last_active":"2024-05-14T11:16:17.806812+0000","last_peered":"2024-05-14T11:16:17.806812+0000","last_clean":"2024-05-14T11:16:17.806812+0000","last_became_active":"2024-05-14T11:16:11.254304+0000","last_became_peered":"2024-05-14T11:16:11.254304+0000","last_unstale":"2024-05-14T11:16:17.806812+0000","last_undegraded":"2024-05-14T11:16:17.806812+0000","last_fullsized":"2024-05-14T11:16:17.806812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806903+0000","last_change":"2024-05-14T11:16:16.333086+0000","last_active":"2024-05-14T11:16:17.806903+0000","last_peered":"2024-05-14T11:16:17.806903+0000","last_clean":"2024-05-14T11:16:17.806903+0000","last_became_active":"2024-05-14T11:16:16.332633+0000","last_became_peered":"2024-05-14T11:16:16.332633+0000","last_unstale":"2024-05-14T11:16:17.806903+0000","last_undegraded":"2024-05-14T11:16:17.806903+0000","last_fullsized":"2024-05-14T11:16:17.806903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:46.707261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808690+0000","last_change":"2024-05-14T11:16:12.267597+0000","last_active":"2024-05-14T11:16:17.808690+0000","last_peered":"2024-05-14T11:16:17.808690+0000","last_clean":"2024-05-14T11:16:17.808690+0000","last_became_active":"2024-05-14T11:16:12.267050+0000","last_became_peered":"2024-05-14T11:16:12.267050+0000","last_unstale":"2024-05-14T11:16:17.808690+0000","last_undegraded":"2024-05-14T11:16:17.808690+0000","last_fullsized":"2024-05-14T11:16:17.808690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806566+0000","last_change":"2024-05-14T11:16:11.253533+0000","last_active":"2024-05-14T11:16:17.806566+0000","last_peered":"2024-05-14T11:16:17.806566+0000","last_clean":"2024-05-14T11:16:17.806566+0000","last_became_active":"2024-05-14T11:16:11.253104+0000","last_became_peered":"2024-05-14T11:16:11.253104+0000","last_unstale":"2024-05-14T11:16:17.806566+0000","last_undegraded":"2024-05-14T11:16:17.806566+0000","last_fullsized":"2024-05-14T11:16:17.806566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806962+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:16:17.806962+0000","last_peered":"2024-05-14T11:16:17.806962+0000","last_clean":"2024-05-14T11:16:17.806962+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:16:17.806962+0000","last_undegraded":"2024-05-14T11:16:17.806962+0000","last_fullsized":"2024-05-14T11:16:17.806962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299583+0000","last_change":"2024-05-14T11:16:16.330966+0000","last_active":"2024-05-14T11:16:17.299583+0000","last_peered":"2024-05-14T11:16:17.299583+0000","last_clean":"2024-05-14T11:16:17.299583+0000","last_became_active":"2024-05-14T11:16:16.330413+0000","last_became_peered":"2024-05-14T11:16:16.330413+0000","last_unstale":"2024-05-14T11:16:17.299583+0000","last_undegraded":"2024-05-14T11:16:17.299583+0000","last_fullsized":"2024-05-14T11:16:17.299583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:26.206373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807128+0000","last_change":"2024-05-14T11:16:12.263387+0000","last_active":"2024-05-14T11:16:17.807128+0000","last_peered":"2024-05-14T11:16:17.807128+0000","last_clean":"2024-05-14T11:16:17.807128+0000","last_became_active":"2024-05-14T11:16:12.262944+0000","last_became_peered":"2024-05-14T11:16:12.262944+0000","last_unstale":"2024-05-14T11:16:17.807128+0000","last_undegraded":"2024-05-14T11:16:17.807128+0000","last_fullsized":"2024-05-14T11:16:17.807128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298224+0000","last_change":"2024-05-14T11:16:11.259096+0000","last_active":"2024-05-14T11:16:17.298224+0000","last_peered":"2024-05-14T11:16:17.298224+0000","last_clean":"2024-05-14T11:16:17.298224+0000","last_became_active":"2024-05-14T11:16:11.258840+0000","last_became_peered":"2024-05-14T11:16:11.258840+0000","last_unstale":"2024-05-14T11:16:17.298224+0000","last_undegraded":"2024-05-14T11:16:17.298224+0000","last_fullsized":"2024-05-14T11:16:17.298224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300200+0000","last_change":"2024-05-14T11:16:16.333088+0000","last_active":"2024-05-14T11:16:17.300200+0000","last_peered":"2024-05-14T11:16:17.300200+0000","last_clean":"2024-05-14T11:16:17.300200+0000","last_became_active":"2024-05-14T11:16:16.332585+0000","last_became_peered":"2024-05-14T11:16:16.332585+0000","last_unstale":"2024-05-14T11:16:17.300200+0000","last_undegraded":"2024-05-14T11:16:17.300200+0000","last_fullsized":"2024-05-14T11:16:17.300200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806820+0000","last_change":"2024-05-14T11:16:12.270318+0000","last_active":"2024-05-14T11:16:17.806820+0000","last_peered":"2024-05-14T11:16:17.806820+0000","last_clean":"2024-05-14T11:16:17.806820+0000","last_became_active":"2024-05-14T11:16:12.269365+0000","last_became_peered":"2024-05-14T11:16:12.269365+0000","last_unstale":"2024-05-14T11:16:17.806820+0000","last_undegraded":"2024-05-14T11:16:17.806820+0000","last_fullsized":"2024-05-14T11:16:17.806820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807501+0000","last_change":"2024-05-14T11:16:11.252737+0000","last_active":"2024-05-14T11:16:17.807501+0000","last_peered":"2024-05-14T11:16:17.807501+0000","last_clean":"2024-05-14T11:16:17.807501+0000","last_became_active":"2024-05-14T11:16:11.252365+0000","last_became_peered":"2024-05-14T11:16:11.252365+0000","last_unstale":"2024-05-14T11:16:17.807501+0000","last_undegraded":"2024-05-14T11:16:17.807501+0000","last_fullsized":"2024-05-14T11:16:17.807501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806471+0000","last_change":"2024-05-14T11:16:16.302121+0000","last_active":"2024-05-14T11:16:17.806471+0000","last_peered":"2024-05-14T11:16:17.806471+0000","last_clean":"2024-05-14T11:16:17.806471+0000","last_became_active":"2024-05-14T11:16:16.299059+0000","last_became_peered":"2024-05-14T11:16:16.299059+0000","last_unstale":"2024-05-14T11:16:17.806471+0000","last_undegraded":"2024-05-14T11:16:17.806471+0000","last_fullsized":"2024-05-14T11:16:17.806471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807801+0000","last_change":"2024-05-14T11:16:12.247488+0000","last_active":"2024-05-14T11:16:17.807801+0000","last_peered":"2024-05-14T11:16:17.807801+0000","last_clean":"2024-05-14T11:16:17.807801+0000","last_became_active":"2024-05-14T11:16:12.247235+0000","last_became_peered":"2024-05-14T11:16:12.247235+0000","last_unstale":"2024-05-14T11:16:17.807801+0000","last_undegraded":"2024-05-14T11:16:17.807801+0000","last_fullsized":"2024-05-14T11:16:17.807801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:22.549789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298399+0000","last_change":"2024-05-14T11:16:11.246157+0000","last_active":"2024-05-14T11:16:17.298399+0000","last_peered":"2024-05-14T11:16:17.298399+0000","last_clean":"2024-05-14T11:16:17.298399+0000","last_became_active":"2024-05-14T11:16:11.245408+0000","last_became_peered":"2024-05-14T11:16:11.245408+0000","last_unstale":"2024-05-14T11:16:17.298399+0000","last_undegraded":"2024-05-14T11:16:17.298399+0000","last_fullsized":"2024-05-14T11:16:17.298399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806090+0000","last_change":"2024-05-14T11:16:16.330283+0000","last_active":"2024-05-14T11:16:17.806090+0000","last_peered":"2024-05-14T11:16:17.806090+0000","last_clean":"2024-05-14T11:16:17.806090+0000","last_became_active":"2024-05-14T11:16:16.329844+0000","last_became_peered":"2024-05-14T11:16:16.329844+0000","last_unstale":"2024-05-14T11:16:17.806090+0000","last_undegraded":"2024-05-14T11:16:17.806090+0000","last_fullsized":"2024-05-14T11:16:17.806090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:05.995998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807746+0000","last_change":"2024-05-14T11:16:12.248542+0000","last_active":"2024-05-14T11:16:17.807746+0000","last_peered":"2024-05-14T11:16:17.807746+0000","last_clean":"2024-05-14T11:16:17.807746+0000","last_became_active":"2024-05-14T11:16:12.248214+0000","last_became_peered":"2024-05-14T11:16:12.248214+0000","last_unstale":"2024-05-14T11:16:17.807746+0000","last_undegraded":"2024-05-14T11:16:17.807746+0000","last_fullsized":"2024-05-14T11:16:17.807746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:00.728760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302636+0000","last_change":"2024-05-14T11:16:11.241324+0000","last_active":"2024-05-14T11:16:17.302636+0000","last_peered":"2024-05-14T11:16:17.302636+0000","last_clean":"2024-05-14T11:16:17.302636+0000","last_became_active":"2024-05-14T11:16:11.239970+0000","last_became_peered":"2024-05-14T11:16:11.239970+0000","last_unstale":"2024-05-14T11:16:17.302636+0000","last_undegraded":"2024-05-14T11:16:17.302636+0000","last_fullsized":"2024-05-14T11:16:17.302636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:44.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299463+0000","last_change":"2024-05-14T11:16:16.293327+0000","last_active":"2024-05-14T11:16:17.299463+0000","last_peered":"2024-05-14T11:16:17.299463+0000","last_clean":"2024-05-14T11:16:17.299463+0000","last_became_active":"2024-05-14T11:16:16.292800+0000","last_became_peered":"2024-05-14T11:16:16.292800+0000","last_unstale":"2024-05-14T11:16:17.299463+0000","last_undegraded":"2024-05-14T11:16:17.299463+0000","last_fullsized":"2024-05-14T11:16:17.299463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:39.377103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808057+0000","last_change":"2024-05-14T11:16:12.248759+0000","last_active":"2024-05-14T11:16:17.808057+0000","last_peered":"2024-05-14T11:16:17.808057+0000","last_clean":"2024-05-14T11:16:17.808057+0000","last_became_active":"2024-05-14T11:16:12.248407+0000","last_became_peered":"2024-05-14T11:16:12.248407+0000","last_unstale":"2024-05-14T11:16:17.808057+0000","last_undegraded":"2024-05-14T11:16:17.808057+0000","last_fullsized":"2024-05-14T11:16:17.808057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297834+0000","last_change":"2024-05-14T11:16:11.248167+0000","last_active":"2024-05-14T11:16:17.297834+0000","last_peered":"2024-05-14T11:16:17.297834+0000","last_clean":"2024-05-14T11:16:17.297834+0000","last_became_active":"2024-05-14T11:16:11.245949+0000","last_became_peered":"2024-05-14T11:16:11.245949+0000","last_unstale":"2024-05-14T11:16:17.297834+0000","last_undegraded":"2024-05-14T11:16:17.297834+0000","last_fullsized":"2024-05-14T11:16:17.297834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807832+0000","last_change":"2024-05-14T11:16:16.290089+0000","last_active":"2024-05-14T11:16:17.807832+0000","last_peered":"2024-05-14T11:16:17.807832+0000","last_clean":"2024-05-14T11:16:17.807832+0000","last_became_active":"2024-05-14T11:16:16.287987+0000","last_became_peered":"2024-05-14T11:16:16.287987+0000","last_unstale":"2024-05-14T11:16:17.807832+0000","last_undegraded":"2024-05-14T11:16:17.807832+0000","last_fullsized":"2024-05-14T11:16:17.807832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:31:26.599395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299766+0000","last_change":"2024-05-14T11:16:12.248930+0000","last_active":"2024-05-14T11:16:17.299766+0000","last_peered":"2024-05-14T11:16:17.299766+0000","last_clean":"2024-05-14T11:16:17.299766+0000","last_became_active":"2024-05-14T11:16:12.248159+0000","last_became_peered":"2024-05-14T11:16:12.248159+0000","last_unstale":"2024-05-14T11:16:17.299766+0000","last_undegraded":"2024-05-14T11:16:17.299766+0000","last_fullsized":"2024-05-14T11:16:17.299766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299880+0000","last_change":"2024-05-14T11:16:11.247396+0000","last_active":"2024-05-14T11:16:17.299880+0000","last_peered":"2024-05-14T11:16:17.299880+0000","last_clean":"2024-05-14T11:16:17.299880+0000","last_became_active":"2024-05-14T11:16:11.246855+0000","last_became_peered":"2024-05-14T11:16:11.246855+0000","last_unstale":"2024-05-14T11:16:17.299880+0000","last_undegraded":"2024-05-14T11:16:17.299880+0000","last_fullsized":"2024-05-14T11:16:17.299880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808975+0000","last_change":"2024-05-14T11:16:16.332903+0000","last_active":"2024-05-14T11:16:17.808975+0000","last_peered":"2024-05-14T11:16:17.808975+0000","last_clean":"2024-05-14T11:16:17.808975+0000","last_became_active":"2024-05-14T11:16:16.332436+0000","last_became_peered":"2024-05-14T11:16:16.332436+0000","last_unstale":"2024-05-14T11:16:17.808975+0000","last_undegraded":"2024-05-14T11:16:17.808975+0000","last_fullsized":"2024-05-14T11:16:17.808975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:01.254012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807525+0000","last_change":"2024-05-14T11:16:12.248442+0000","last_active":"2024-05-14T11:16:17.807525+0000","last_peered":"2024-05-14T11:16:17.807525+0000","last_clean":"2024-05-14T11:16:17.807525+0000","last_became_active":"2024-05-14T11:16:12.248082+0000","last_became_peered":"2024-05-14T11:16:12.248082+0000","last_unstale":"2024-05-14T11:16:17.807525+0000","last_undegraded":"2024-05-14T11:16:17.807525+0000","last_fullsized":"2024-05-14T11:16:17.807525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:45.923026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807708+0000","last_change":"2024-05-14T11:16:11.257458+0000","last_active":"2024-05-14T11:16:17.807708+0000","last_peered":"2024-05-14T11:16:17.807708+0000","last_clean":"2024-05-14T11:16:17.807708+0000","last_became_active":"2024-05-14T11:16:11.252978+0000","last_became_peered":"2024-05-14T11:16:11.252978+0000","last_unstale":"2024-05-14T11:16:17.807708+0000","last_undegraded":"2024-05-14T11:16:17.807708+0000","last_fullsized":"2024-05-14T11:16:17.807708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808827+0000","last_change":"2024-05-14T11:16:16.330174+0000","last_active":"2024-05-14T11:16:17.808827+0000","last_peered":"2024-05-14T11:16:17.808827+0000","last_clean":"2024-05-14T11:16:17.808827+0000","last_became_active":"2024-05-14T11:16:16.329696+0000","last_became_peered":"2024-05-14T11:16:16.329696+0000","last_unstale":"2024-05-14T11:16:17.808827+0000","last_undegraded":"2024-05-14T11:16:17.808827+0000","last_fullsized":"2024-05-14T11:16:17.808827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:14.122764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806365+0000","last_change":"2024-05-14T11:16:12.246355+0000","last_active":"2024-05-14T11:16:17.806365+0000","last_peered":"2024-05-14T11:16:17.806365+0000","last_clean":"2024-05-14T11:16:17.806365+0000","last_became_active":"2024-05-14T11:16:12.245693+0000","last_became_peered":"2024-05-14T11:16:12.245693+0000","last_unstale":"2024-05-14T11:16:17.806365+0000","last_undegraded":"2024-05-14T11:16:17.806365+0000","last_fullsized":"2024-05-14T11:16:17.806365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806645+0000","last_change":"2024-05-14T11:16:11.249709+0000","last_active":"2024-05-14T11:16:17.806645+0000","last_peered":"2024-05-14T11:16:17.806645+0000","last_clean":"2024-05-14T11:16:17.806645+0000","last_became_active":"2024-05-14T11:16:11.245550+0000","last_became_peered":"2024-05-14T11:16:11.245550+0000","last_unstale":"2024-05-14T11:16:17.806645+0000","last_undegraded":"2024-05-14T11:16:17.806645+0000","last_fullsized":"2024-05-14T11:16:17.806645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:18:18.498642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806772+0000","last_change":"2024-05-14T11:16:16.330492+0000","last_active":"2024-05-14T11:16:17.806772+0000","last_peered":"2024-05-14T11:16:17.806772+0000","last_clean":"2024-05-14T11:16:17.806772+0000","last_became_active":"2024-05-14T11:16:16.330027+0000","last_became_peered":"2024-05-14T11:16:16.330027+0000","last_unstale":"2024-05-14T11:16:17.806772+0000","last_undegraded":"2024-05-14T11:16:17.806772+0000","last_fullsized":"2024-05-14T11:16:17.806772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807029+0000","last_change":"2024-05-14T11:16:12.263610+0000","last_active":"2024-05-14T11:16:17.807029+0000","last_peered":"2024-05-14T11:16:17.807029+0000","last_clean":"2024-05-14T11:16:17.807029+0000","last_became_active":"2024-05-14T11:16:12.263072+0000","last_became_peered":"2024-05-14T11:16:12.263072+0000","last_unstale":"2024-05-14T11:16:17.807029+0000","last_undegraded":"2024-05-14T11:16:17.807029+0000","last_fullsized":"2024-05-14T11:16:17.807029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:23.206421+0000","last_change":"2024-05-14T11:16:11.250108+0000","last_active":"2024-05-14T11:16:23.206421+0000","last_peered":"2024-05-14T11:16:23.206421+0000","last_clean":"2024-05-14T11:16:23.206421+0000","last_became_active":"2024-05-14T11:16:11.249460+0000","last_became_peered":"2024-05-14T11:16:11.249460+0000","last_unstale":"2024-05-14T11:16:23.206421+0000","last_undegraded":"2024-05-14T11:16:23.206421+0000","last_fullsized":"2024-05-14T11:16:23.206421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:41.248263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809402+0000","last_change":"2024-05-14T11:16:16.305560+0000","last_active":"2024-05-14T11:16:17.809402+0000","last_peered":"2024-05-14T11:16:17.809402+0000","last_clean":"2024-05-14T11:16:17.809402+0000","last_became_active":"2024-05-14T11:16:16.305166+0000","last_became_peered":"2024-05-14T11:16:16.305166+0000","last_unstale":"2024-05-14T11:16:17.809402+0000","last_undegraded":"2024-05-14T11:16:17.809402+0000","last_fullsized":"2024-05-14T11:16:17.809402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:24:04.778365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302423+0000","last_change":"2024-05-14T11:16:12.248795+0000","last_active":"2024-05-14T11:16:17.302423+0000","last_peered":"2024-05-14T11:16:17.302423+0000","last_clean":"2024-05-14T11:16:17.302423+0000","last_became_active":"2024-05-14T11:16:12.248478+0000","last_became_peered":"2024-05-14T11:16:12.248478+0000","last_unstale":"2024-05-14T11:16:17.302423+0000","last_undegraded":"2024-05-14T11:16:17.302423+0000","last_fullsized":"2024-05-14T11:16:17.302423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807383+0000","last_change":"2024-05-14T11:16:11.259967+0000","last_active":"2024-05-14T11:16:17.807383+0000","last_peered":"2024-05-14T11:16:17.807383+0000","last_clean":"2024-05-14T11:16:17.807383+0000","last_became_active":"2024-05-14T11:16:11.259703+0000","last_became_peered":"2024-05-14T11:16:11.259703+0000","last_unstale":"2024-05-14T11:16:17.807383+0000","last_undegraded":"2024-05-14T11:16:17.807383+0000","last_fullsized":"2024-05-14T11:16:17.807383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301098+0000","last_change":"2024-05-14T11:16:16.293106+0000","last_active":"2024-05-14T11:16:17.301098+0000","last_peered":"2024-05-14T11:16:17.301098+0000","last_clean":"2024-05-14T11:16:17.301098+0000","last_became_active":"2024-05-14T11:16:16.292666+0000","last_became_peered":"2024-05-14T11:16:16.292666+0000","last_unstale":"2024-05-14T11:16:17.301098+0000","last_undegraded":"2024-05-14T11:16:17.301098+0000","last_fullsized":"2024-05-14T11:16:17.301098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:37.752207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807505+0000","last_change":"2024-05-14T11:16:12.260898+0000","last_active":"2024-05-14T11:16:17.807505+0000","last_peered":"2024-05-14T11:16:17.807505+0000","last_clean":"2024-05-14T11:16:17.807505+0000","last_became_active":"2024-05-14T11:16:12.260114+0000","last_became_peered":"2024-05-14T11:16:12.260114+0000","last_unstale":"2024-05-14T11:16:17.807505+0000","last_undegraded":"2024-05-14T11:16:17.807505+0000","last_fullsized":"2024-05-14T11:16:17.807505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:06:34.906554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806807+0000","last_change":"2024-05-14T11:16:11.254169+0000","last_active":"2024-05-14T11:16:17.806807+0000","last_peered":"2024-05-14T11:16:17.806807+0000","last_clean":"2024-05-14T11:16:17.806807+0000","last_became_active":"2024-05-14T11:16:11.253658+0000","last_became_peered":"2024-05-14T11:16:11.253658+0000","last_unstale":"2024-05-14T11:16:17.806807+0000","last_undegraded":"2024-05-14T11:16:17.806807+0000","last_fullsized":"2024-05-14T11:16:17.806807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:02.459616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808298+0000","last_change":"2024-05-14T11:16:16.298722+0000","last_active":"2024-05-14T11:16:17.808298+0000","last_peered":"2024-05-14T11:16:17.808298+0000","last_clean":"2024-05-14T11:16:17.808298+0000","last_became_active":"2024-05-14T11:16:16.298312+0000","last_became_peered":"2024-05-14T11:16:16.298312+0000","last_unstale":"2024-05-14T11:16:17.808298+0000","last_undegraded":"2024-05-14T11:16:17.808298+0000","last_fullsized":"2024-05-14T11:16:17.808298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:53:33.877677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806669+0000","last_change":"2024-05-14T11:16:12.269044+0000","last_active":"2024-05-14T11:16:17.806669+0000","last_peered":"2024-05-14T11:16:17.806669+0000","last_clean":"2024-05-14T11:16:17.806669+0000","last_became_active":"2024-05-14T11:16:12.268428+0000","last_became_peered":"2024-05-14T11:16:12.268428+0000","last_unstale":"2024-05-14T11:16:17.806669+0000","last_undegraded":"2024-05-14T11:16:17.806669+0000","last_fullsized":"2024-05-14T11:16:17.806669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:33.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301709+0000","last_change":"2024-05-14T11:16:11.247431+0000","last_active":"2024-05-14T11:16:17.301709+0000","last_peered":"2024-05-14T11:16:17.301709+0000","last_clean":"2024-05-14T11:16:17.301709+0000","last_became_active":"2024-05-14T11:16:11.247003+0000","last_became_peered":"2024-05-14T11:16:11.247003+0000","last_unstale":"2024-05-14T11:16:17.301709+0000","last_undegraded":"2024-05-14T11:16:17.301709+0000","last_fullsized":"2024-05-14T11:16:17.301709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:17.483475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808398+0000","last_change":"2024-05-14T11:16:16.338217+0000","last_active":"2024-05-14T11:16:17.808398+0000","last_peered":"2024-05-14T11:16:17.808398+0000","last_clean":"2024-05-14T11:16:17.808398+0000","last_became_active":"2024-05-14T11:16:16.337645+0000","last_became_peered":"2024-05-14T11:16:16.337645+0000","last_unstale":"2024-05-14T11:16:17.808398+0000","last_undegraded":"2024-05-14T11:16:17.808398+0000","last_fullsized":"2024-05-14T11:16:17.808398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:50:21.563593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806740+0000","last_change":"2024-05-14T11:16:16.337358+0000","last_active":"2024-05-14T11:16:17.806740+0000","last_peered":"2024-05-14T11:16:17.806740+0000","last_clean":"2024-05-14T11:16:17.806740+0000","last_became_active":"2024-05-14T11:16:16.336916+0000","last_became_peered":"2024-05-14T11:16:16.336916+0000","last_unstale":"2024-05-14T11:16:17.806740+0000","last_undegraded":"2024-05-14T11:16:17.806740+0000","last_fullsized":"2024-05-14T11:16:17.806740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:58.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806581+0000","last_change":"2024-05-14T11:16:12.267101+0000","last_active":"2024-05-14T11:16:17.806581+0000","last_peered":"2024-05-14T11:16:17.806581+0000","last_clean":"2024-05-14T11:16:17.806581+0000","last_became_active":"2024-05-14T11:16:12.266805+0000","last_became_peered":"2024-05-14T11:16:12.266805+0000","last_unstale":"2024-05-14T11:16:17.806581+0000","last_undegraded":"2024-05-14T11:16:17.806581+0000","last_fullsized":"2024-05-14T11:16:17.806581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806424+0000","last_change":"2024-05-14T11:16:11.259588+0000","last_active":"2024-05-14T11:16:17.806424+0000","last_peered":"2024-05-14T11:16:17.806424+0000","last_clean":"2024-05-14T11:16:17.806424+0000","last_became_active":"2024-05-14T11:16:11.259246+0000","last_became_peered":"2024-05-14T11:16:11.259246+0000","last_unstale":"2024-05-14T11:16:17.806424+0000","last_undegraded":"2024-05-14T11:16:17.806424+0000","last_fullsized":"2024-05-14T11:16:17.806424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:12:29.493930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301555+0000","last_change":"2024-05-14T11:16:16.299514+0000","last_active":"2024-05-14T11:16:17.301555+0000","last_peered":"2024-05-14T11:16:17.301555+0000","last_clean":"2024-05-14T11:16:17.301555+0000","last_became_active":"2024-05-14T11:16:16.299075+0000","last_became_peered":"2024-05-14T11:16:16.299075+0000","last_unstale":"2024-05-14T11:16:17.301555+0000","last_undegraded":"2024-05-14T11:16:17.301555+0000","last_fullsized":"2024-05-14T11:16:17.301555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:26:49.972631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808226+0000","last_change":"2024-05-14T11:16:12.258195+0000","last_active":"2024-05-14T11:16:17.808226+0000","last_peered":"2024-05-14T11:16:17.808226+0000","last_clean":"2024-05-14T11:16:17.808226+0000","last_became_active":"2024-05-14T11:16:12.257876+0000","last_became_peered":"2024-05-14T11:16:12.257876+0000","last_unstale":"2024-05-14T11:16:17.808226+0000","last_undegraded":"2024-05-14T11:16:17.808226+0000","last_fullsized":"2024-05-14T11:16:17.808226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:52:39.616467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808524+0000","last_change":"2024-05-14T11:16:11.246503+0000","last_active":"2024-05-14T11:16:17.808524+0000","last_peered":"2024-05-14T11:16:17.808524+0000","last_clean":"2024-05-14T11:16:17.808524+0000","last_became_active":"2024-05-14T11:16:11.246056+0000","last_became_peered":"2024-05-14T11:16:11.246056+0000","last_unstale":"2024-05-14T11:16:17.808524+0000","last_undegraded":"2024-05-14T11:16:17.808524+0000","last_fullsized":"2024-05-14T11:16:17.808524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:32.026203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300138+0000","last_change":"2024-05-14T11:16:11.258034+0000","last_active":"2024-05-14T11:16:17.300138+0000","last_peered":"2024-05-14T11:16:17.300138+0000","last_clean":"2024-05-14T11:16:17.300138+0000","last_became_active":"2024-05-14T11:16:11.257706+0000","last_became_peered":"2024-05-14T11:16:11.257706+0000","last_unstale":"2024-05-14T11:16:17.300138+0000","last_undegraded":"2024-05-14T11:16:17.300138+0000","last_fullsized":"2024-05-14T11:16:17.300138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:01:18.331919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806611+0000","last_change":"2024-05-14T11:16:12.248022+0000","last_active":"2024-05-14T11:16:17.806611+0000","last_peered":"2024-05-14T11:16:17.806611+0000","last_clean":"2024-05-14T11:16:17.806611+0000","last_became_active":"2024-05-14T11:16:12.247640+0000","last_became_peered":"2024-05-14T11:16:12.247640+0000","last_unstale":"2024-05-14T11:16:17.806611+0000","last_undegraded":"2024-05-14T11:16:17.806611+0000","last_fullsized":"2024-05-14T11:16:17.806611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298100+0000","last_change":"2024-05-14T11:16:16.296944+0000","last_active":"2024-05-14T11:16:17.298100+0000","last_peered":"2024-05-14T11:16:17.298100+0000","last_clean":"2024-05-14T11:16:17.298100+0000","last_became_active":"2024-05-14T11:16:16.296515+0000","last_became_peered":"2024-05-14T11:16:16.296515+0000","last_unstale":"2024-05-14T11:16:17.298100+0000","last_undegraded":"2024-05-14T11:16:17.298100+0000","last_fullsized":"2024-05-14T11:16:17.298100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:11:41.471078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809032+0000","last_change":"2024-05-14T11:16:11.253359+0000","last_active":"2024-05-14T11:16:17.809032+0000","last_peered":"2024-05-14T11:16:17.809032+0000","last_clean":"2024-05-14T11:16:17.809032+0000","last_became_active":"2024-05-14T11:16:11.248847+0000","last_became_peered":"2024-05-14T11:16:11.248847+0000","last_unstale":"2024-05-14T11:16:17.809032+0000","last_undegraded":"2024-05-14T11:16:17.809032+0000","last_fullsized":"2024-05-14T11:16:17.809032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:10.205173+0000","last_clean_scrub_stamp":"2024-05-14T11:16:10.205173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:56.331087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301381+0000","last_change":"2024-05-14T11:16:12.239027+0000","last_active":"2024-05-14T11:16:17.301381+0000","last_peered":"2024-05-14T11:16:17.301381+0000","last_clean":"2024-05-14T11:16:17.301381+0000","last_became_active":"2024-05-14T11:16:12.238687+0000","last_became_peered":"2024-05-14T11:16:12.238687+0000","last_unstale":"2024-05-14T11:16:17.301381+0000","last_undegraded":"2024-05-14T11:16:17.301381+0000","last_fullsized":"2024-05-14T11:16:17.301381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:38:18.490412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809262+0000","last_change":"2024-05-14T11:16:16.293565+0000","last_active":"2024-05-14T11:16:17.809262+0000","last_peered":"2024-05-14T11:16:17.809262+0000","last_clean":"2024-05-14T11:16:17.809262+0000","last_became_active":"2024-05-14T11:16:16.293094+0000","last_became_peered":"2024-05-14T11:16:16.293094+0000","last_unstale":"2024-05-14T11:16:17.809262+0000","last_undegraded":"2024-05-14T11:16:17.809262+0000","last_fullsized":"2024-05-14T11:16:17.809262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:15.261716+0000","last_clean_scrub_stamp":"2024-05-14T11:16:15.261716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:03:43.070705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.299522+0000","last_change":"2024-05-14T11:16:12.249274+0000","last_active":"2024-05-14T11:16:17.299522+0000","last_peered":"2024-05-14T11:16:17.299522+0000","last_clean":"2024-05-14T11:16:17.299522+0000","last_became_active":"2024-05-14T11:16:12.248560+0000","last_became_peered":"2024-05-14T11:16:12.248560+0000","last_unstale":"2024-05-14T11:16:17.299522+0000","last_undegraded":"2024-05-14T11:16:17.299522+0000","last_fullsized":"2024-05-14T11:16:17.299522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.297943+0000","last_change":"2024-05-14T11:16:12.267227+0000","last_active":"2024-05-14T11:16:17.297943+0000","last_peered":"2024-05-14T11:16:17.297943+0000","last_clean":"2024-05-14T11:16:17.297943+0000","last_became_active":"2024-05-14T11:16:12.266918+0000","last_became_peered":"2024-05-14T11:16:12.266918+0000","last_unstale":"2024-05-14T11:16:17.297943+0000","last_undegraded":"2024-05-14T11:16:17.297943+0000","last_fullsized":"2024-05-14T11:16:17.297943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:38.588425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807593+0000","last_change":"2024-05-14T11:16:12.259446+0000","last_active":"2024-05-14T11:16:17.807593+0000","last_peered":"2024-05-14T11:16:17.807593+0000","last_clean":"2024-05-14T11:16:17.807593+0000","last_became_active":"2024-05-14T11:16:12.259111+0000","last_became_peered":"2024-05-14T11:16:12.259111+0000","last_unstale":"2024-05-14T11:16:17.807593+0000","last_undegraded":"2024-05-14T11:16:17.807593+0000","last_fullsized":"2024-05-14T11:16:17.807593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806487+0000","last_change":"2024-05-14T11:16:12.259006+0000","last_active":"2024-05-14T11:16:17.806487+0000","last_peered":"2024-05-14T11:16:17.806487+0000","last_clean":"2024-05-14T11:16:17.806487+0000","last_became_active":"2024-05-14T11:16:12.258686+0000","last_became_peered":"2024-05-14T11:16:12.258686+0000","last_unstale":"2024-05-14T11:16:17.806487+0000","last_undegraded":"2024-05-14T11:16:17.806487+0000","last_fullsized":"2024-05-14T11:16:17.806487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808880+0000","last_change":"2024-05-14T11:16:12.263813+0000","last_active":"2024-05-14T11:16:17.808880+0000","last_peered":"2024-05-14T11:16:17.808880+0000","last_clean":"2024-05-14T11:16:17.808880+0000","last_became_active":"2024-05-14T11:16:12.263020+0000","last_became_peered":"2024-05-14T11:16:12.263020+0000","last_unstale":"2024-05-14T11:16:17.808880+0000","last_undegraded":"2024-05-14T11:16:17.808880+0000","last_fullsized":"2024-05-14T11:16:17.808880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807160+0000","last_change":"2024-05-14T11:16:12.265745+0000","last_active":"2024-05-14T11:16:17.807160+0000","last_peered":"2024-05-14T11:16:17.807160+0000","last_clean":"2024-05-14T11:16:17.807160+0000","last_became_active":"2024-05-14T11:16:12.262165+0000","last_became_peered":"2024-05-14T11:16:12.262165+0000","last_unstale":"2024-05-14T11:16:17.807160+0000","last_undegraded":"2024-05-14T11:16:17.807160+0000","last_fullsized":"2024-05-14T11:16:17.807160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301482+0000","last_change":"2024-05-14T11:16:12.250079+0000","last_active":"2024-05-14T11:16:17.301482+0000","last_peered":"2024-05-14T11:16:17.301482+0000","last_clean":"2024-05-14T11:16:17.301482+0000","last_became_active":"2024-05-14T11:16:12.246835+0000","last_became_peered":"2024-05-14T11:16:12.246835+0000","last_unstale":"2024-05-14T11:16:17.301482+0000","last_undegraded":"2024-05-14T11:16:17.301482+0000","last_fullsized":"2024-05-14T11:16:17.301482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.809094+0000","last_change":"2024-05-14T11:16:12.264222+0000","last_active":"2024-05-14T11:16:17.809094+0000","last_peered":"2024-05-14T11:16:17.809094+0000","last_clean":"2024-05-14T11:16:17.809094+0000","last_became_active":"2024-05-14T11:16:12.263773+0000","last_became_peered":"2024-05-14T11:16:12.263773+0000","last_unstale":"2024-05-14T11:16:17.809094+0000","last_undegraded":"2024-05-14T11:16:17.809094+0000","last_fullsized":"2024-05-14T11:16:17.809094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:09.716369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806261+0000","last_change":"2024-05-14T11:16:12.260462+0000","last_active":"2024-05-14T11:16:17.806261+0000","last_peered":"2024-05-14T11:16:17.806261+0000","last_clean":"2024-05-14T11:16:17.806261+0000","last_became_active":"2024-05-14T11:16:12.260089+0000","last_became_peered":"2024-05-14T11:16:12.260089+0000","last_unstale":"2024-05-14T11:16:17.806261+0000","last_undegraded":"2024-05-14T11:16:17.806261+0000","last_fullsized":"2024-05-14T11:16:17.806261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:58:04.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808385+0000","last_change":"2024-05-14T11:16:12.257286+0000","last_active":"2024-05-14T11:16:17.808385+0000","last_peered":"2024-05-14T11:16:17.808385+0000","last_clean":"2024-05-14T11:16:17.808385+0000","last_became_active":"2024-05-14T11:16:12.256949+0000","last_became_peered":"2024-05-14T11:16:12.256949+0000","last_unstale":"2024-05-14T11:16:17.808385+0000","last_undegraded":"2024-05-14T11:16:17.808385+0000","last_fullsized":"2024-05-14T11:16:17.808385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807462+0000","last_change":"2024-05-14T11:16:12.266841+0000","last_active":"2024-05-14T11:16:17.807462+0000","last_peered":"2024-05-14T11:16:17.807462+0000","last_clean":"2024-05-14T11:16:17.807462+0000","last_became_active":"2024-05-14T11:16:12.266472+0000","last_became_peered":"2024-05-14T11:16:12.266472+0000","last_unstale":"2024-05-14T11:16:17.807462+0000","last_undegraded":"2024-05-14T11:16:17.807462+0000","last_fullsized":"2024-05-14T11:16:17.807462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806284+0000","last_change":"2024-05-14T11:16:12.245994+0000","last_active":"2024-05-14T11:16:17.806284+0000","last_peered":"2024-05-14T11:16:17.806284+0000","last_clean":"2024-05-14T11:16:17.806284+0000","last_became_active":"2024-05-14T11:16:12.243237+0000","last_became_peered":"2024-05-14T11:16:12.243237+0000","last_unstale":"2024-05-14T11:16:17.806284+0000","last_undegraded":"2024-05-14T11:16:17.806284+0000","last_fullsized":"2024-05-14T11:16:17.806284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300549+0000","last_change":"2024-05-14T11:16:12.240444+0000","last_active":"2024-05-14T11:16:17.300549+0000","last_peered":"2024-05-14T11:16:17.300549+0000","last_clean":"2024-05-14T11:16:17.300549+0000","last_became_active":"2024-05-14T11:16:12.240076+0000","last_became_peered":"2024-05-14T11:16:12.240076+0000","last_unstale":"2024-05-14T11:16:17.300549+0000","last_undegraded":"2024-05-14T11:16:17.300549+0000","last_fullsized":"2024-05-14T11:16:17.300549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:12.433973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807295+0000","last_change":"2024-05-14T11:16:12.243507+0000","last_active":"2024-05-14T11:16:17.807295+0000","last_peered":"2024-05-14T11:16:17.807295+0000","last_clean":"2024-05-14T11:16:17.807295+0000","last_became_active":"2024-05-14T11:16:12.243180+0000","last_became_peered":"2024-05-14T11:16:12.243180+0000","last_unstale":"2024-05-14T11:16:17.807295+0000","last_undegraded":"2024-05-14T11:16:17.807295+0000","last_fullsized":"2024-05-14T11:16:17.807295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.298355+0000","last_change":"2024-05-14T11:16:12.250650+0000","last_active":"2024-05-14T11:16:17.298355+0000","last_peered":"2024-05-14T11:16:17.298355+0000","last_clean":"2024-05-14T11:16:17.298355+0000","last_became_active":"2024-05-14T11:16:12.250259+0000","last_became_peered":"2024-05-14T11:16:12.250259+0000","last_unstale":"2024-05-14T11:16:17.298355+0000","last_undegraded":"2024-05-14T11:16:17.298355+0000","last_fullsized":"2024-05-14T11:16:17.298355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:32.719203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807449+0000","last_change":"2024-05-14T11:16:12.267003+0000","last_active":"2024-05-14T11:16:17.807449+0000","last_peered":"2024-05-14T11:16:17.807449+0000","last_clean":"2024-05-14T11:16:17.807449+0000","last_became_active":"2024-05-14T11:16:12.266655+0000","last_became_peered":"2024-05-14T11:16:12.266655+0000","last_unstale":"2024-05-14T11:16:17.807449+0000","last_undegraded":"2024-05-14T11:16:17.807449+0000","last_fullsized":"2024-05-14T11:16:17.807449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:56.531742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806108+0000","last_change":"2024-05-14T11:16:12.251621+0000","last_active":"2024-05-14T11:16:17.806108+0000","last_peered":"2024-05-14T11:16:17.806108+0000","last_clean":"2024-05-14T11:16:17.806108+0000","last_became_active":"2024-05-14T11:16:12.251284+0000","last_became_peered":"2024-05-14T11:16:12.251284+0000","last_unstale":"2024-05-14T11:16:17.806108+0000","last_undegraded":"2024-05-14T11:16:17.806108+0000","last_fullsized":"2024-05-14T11:16:17.806108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806728+0000","last_change":"2024-05-14T11:16:12.247078+0000","last_active":"2024-05-14T11:16:17.806728+0000","last_peered":"2024-05-14T11:16:17.806728+0000","last_clean":"2024-05-14T11:16:17.806728+0000","last_became_active":"2024-05-14T11:16:12.246785+0000","last_became_peered":"2024-05-14T11:16:12.246785+0000","last_unstale":"2024-05-14T11:16:17.806728+0000","last_undegraded":"2024-05-14T11:16:17.806728+0000","last_fullsized":"2024-05-14T11:16:17.806728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806255+0000","last_change":"2024-05-14T11:16:12.246002+0000","last_active":"2024-05-14T11:16:17.806255+0000","last_peered":"2024-05-14T11:16:17.806255+0000","last_clean":"2024-05-14T11:16:17.806255+0000","last_became_active":"2024-05-14T11:16:12.245378+0000","last_became_peered":"2024-05-14T11:16:12.245378+0000","last_unstale":"2024-05-14T11:16:17.806255+0000","last_undegraded":"2024-05-14T11:16:17.806255+0000","last_fullsized":"2024-05-14T11:16:17.806255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:42.753210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808869+0000","last_change":"2024-05-14T11:16:12.264249+0000","last_active":"2024-05-14T11:16:17.808869+0000","last_peered":"2024-05-14T11:16:17.808869+0000","last_clean":"2024-05-14T11:16:17.808869+0000","last_became_active":"2024-05-14T11:16:12.263604+0000","last_became_peered":"2024-05-14T11:16:12.263604+0000","last_unstale":"2024-05-14T11:16:17.808869+0000","last_undegraded":"2024-05-14T11:16:17.808869+0000","last_fullsized":"2024-05-14T11:16:17.808869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.302334+0000","last_change":"2024-05-14T11:16:12.239836+0000","last_active":"2024-05-14T11:16:17.302334+0000","last_peered":"2024-05-14T11:16:17.302334+0000","last_clean":"2024-05-14T11:16:17.302334+0000","last_became_active":"2024-05-14T11:16:12.236431+0000","last_became_peered":"2024-05-14T11:16:12.236431+0000","last_unstale":"2024-05-14T11:16:17.302334+0000","last_undegraded":"2024-05-14T11:16:17.302334+0000","last_fullsized":"2024-05-14T11:16:17.302334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:03.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.300974+0000","last_change":"2024-05-14T11:16:12.273349+0000","last_active":"2024-05-14T11:16:17.300974+0000","last_peered":"2024-05-14T11:16:17.300974+0000","last_clean":"2024-05-14T11:16:17.300974+0000","last_became_active":"2024-05-14T11:16:12.272507+0000","last_became_peered":"2024-05-14T11:16:12.272507+0000","last_unstale":"2024-05-14T11:16:17.300974+0000","last_undegraded":"2024-05-14T11:16:17.300974+0000","last_fullsized":"2024-05-14T11:16:17.300974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:06.321831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.301473+0000","last_change":"2024-05-14T11:16:12.257302+0000","last_active":"2024-05-14T11:16:17.301473+0000","last_peered":"2024-05-14T11:16:17.301473+0000","last_clean":"2024-05-14T11:16:17.301473+0000","last_became_active":"2024-05-14T11:16:12.256830+0000","last_became_peered":"2024-05-14T11:16:12.256830+0000","last_unstale":"2024-05-14T11:16:17.301473+0000","last_undegraded":"2024-05-14T11:16:17.301473+0000","last_fullsized":"2024-05-14T11:16:17.301473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807763+0000","last_change":"2024-05-14T11:16:12.264613+0000","last_active":"2024-05-14T11:16:17.807763+0000","last_peered":"2024-05-14T11:16:17.807763+0000","last_clean":"2024-05-14T11:16:17.807763+0000","last_became_active":"2024-05-14T11:16:12.264294+0000","last_became_peered":"2024-05-14T11:16:12.264294+0000","last_unstale":"2024-05-14T11:16:17.807763+0000","last_undegraded":"2024-05-14T11:16:17.807763+0000","last_fullsized":"2024-05-14T11:16:17.807763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:19:42.657656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.808085+0000","last_change":"2024-05-14T11:16:12.267492+0000","last_active":"2024-05-14T11:16:17.808085+0000","last_peered":"2024-05-14T11:16:17.808085+0000","last_clean":"2024-05-14T11:16:17.808085+0000","last_became_active":"2024-05-14T11:16:12.267005+0000","last_became_peered":"2024-05-14T11:16:12.267005+0000","last_unstale":"2024-05-14T11:16:17.808085+0000","last_undegraded":"2024-05-14T11:16:17.808085+0000","last_fullsized":"2024-05-14T11:16:17.808085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807982+0000","last_change":"2024-05-14T11:16:12.252241+0000","last_active":"2024-05-14T11:16:17.807982+0000","last_peered":"2024-05-14T11:16:17.807982+0000","last_clean":"2024-05-14T11:16:17.807982+0000","last_became_active":"2024-05-14T11:16:12.251844+0000","last_became_peered":"2024-05-14T11:16:12.251844+0000","last_unstale":"2024-05-14T11:16:17.807982+0000","last_undegraded":"2024-05-14T11:16:17.807982+0000","last_fullsized":"2024-05-14T11:16:17.807982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:37.416383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807948+0000","last_change":"2024-05-14T11:16:12.254254+0000","last_active":"2024-05-14T11:16:17.807948+0000","last_peered":"2024-05-14T11:16:17.807948+0000","last_clean":"2024-05-14T11:16:17.807948+0000","last_became_active":"2024-05-14T11:16:12.253441+0000","last_became_peered":"2024-05-14T11:16:12.253441+0000","last_unstale":"2024-05-14T11:16:17.807948+0000","last_undegraded":"2024-05-14T11:16:17.807948+0000","last_fullsized":"2024-05-14T11:16:17.807948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:35.327498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807243+0000","last_change":"2024-05-14T11:16:12.244922+0000","last_active":"2024-05-14T11:16:17.807243+0000","last_peered":"2024-05-14T11:16:17.807243+0000","last_clean":"2024-05-14T11:16:17.807243+0000","last_became_active":"2024-05-14T11:16:12.244534+0000","last_became_peered":"2024-05-14T11:16:12.244534+0000","last_unstale":"2024-05-14T11:16:17.807243+0000","last_undegraded":"2024-05-14T11:16:17.807243+0000","last_fullsized":"2024-05-14T11:16:17.807243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:22.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806550+0000","last_change":"2024-05-14T11:16:12.270470+0000","last_active":"2024-05-14T11:16:17.806550+0000","last_peered":"2024-05-14T11:16:17.806550+0000","last_clean":"2024-05-14T11:16:17.806550+0000","last_became_active":"2024-05-14T11:16:12.269667+0000","last_became_peered":"2024-05-14T11:16:12.269667+0000","last_unstale":"2024-05-14T11:16:17.806550+0000","last_undegraded":"2024-05-14T11:16:17.806550+0000","last_fullsized":"2024-05-14T11:16:17.806550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:29:09.290352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.806567+0000","last_change":"2024-05-14T11:16:12.254558+0000","last_active":"2024-05-14T11:16:17.806567+0000","last_peered":"2024-05-14T11:16:17.806567+0000","last_clean":"2024-05-14T11:16:17.806567+0000","last_became_active":"2024-05-14T11:16:12.254245+0000","last_became_peered":"2024-05-14T11:16:12.254245+0000","last_unstale":"2024-05-14T11:16:17.806567+0000","last_undegraded":"2024-05-14T11:16:17.806567+0000","last_fullsized":"2024-05-14T11:16:17.806567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:19.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807656+0000","last_change":"2024-05-14T11:16:12.256145+0000","last_active":"2024-05-14T11:16:17.807656+0000","last_peered":"2024-05-14T11:16:17.807656+0000","last_clean":"2024-05-14T11:16:17.807656+0000","last_became_active":"2024-05-14T11:16:12.255806+0000","last_became_peered":"2024-05-14T11:16:12.255806+0000","last_unstale":"2024-05-14T11:16:17.807656+0000","last_undegraded":"2024-05-14T11:16:17.807656+0000","last_fullsized":"2024-05-14T11:16:17.807656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:38:14.175588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-14T11:16:17.807528+0000","last_change":"2024-05-14T11:16:12.252502+0000","last_active":"2024-05-14T11:16:17.807528+0000","last_peered":"2024-05-14T11:16:17.807528+0000","last_clean":"2024-05-14T11:16:17.807528+0000","last_became_active":"2024-05-14T11:16:12.251431+0000","last_became_peered":"2024-05-14T11:16:12.251431+0000","last_unstale":"2024-05-14T11:16:17.807528+0000","last_undegraded":"2024-05-14T11:16:17.807528+0000","last_fullsized":"2024-05-14T11:16:17.807528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:11.212110+0000","last_clean_scrub_stamp":"2024-05-14T11:16:11.212110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:28:54.216494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:16:38.713 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:16:38.713 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T11:16:38.713 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-14T11:16:38.713 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:16:38.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 Processor -- start 2024-05-14T11:16:38.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 -- start start 2024-05-14T11:16:38.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 0x7f12780ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 0x7f12780b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12780b4a10 con 0x7f12780cdf80 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f12780b3060 con 0x7f12780d1c20 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f12780b3300 con 0x7f12780b40c0 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127d800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 0x7f12780b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127ce00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f12780cdf80 0x7f12780d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:53138/0 (socket says 172.21.15.116:53138) 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f127ce00640 1 -- 172.21.15.116:0/3868767436 learned_addr learned my addr 172.21.15.116:0/3868767436 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.896+0000 7f126fe00640 1 --2- 172.21.15.116:0/3868767436 >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 0x7f12780ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f1277e00640 1 -- 172.21.15.116:0/3868767436 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217282464 0 0) 0x7f12780b4a10 con 0x7f12780cdf80 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f127d800640 1 -- 172.21.15.116:0/3868767436 >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 msgr2=0x7f12780ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f127d800640 1 --2- 172.21.15.116:0/3868767436 >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 0x7f12780ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f127d800640 1 -- 172.21.15.116:0/3868767436 >> v1:172.21.15.149:6789/0 conn(0x7f12780cdf80 legacy=0x7f12780d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f127d800640 1 -- 172.21.15.116:0/3868767436 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12780b35e0 con 0x7f12780b40c0 2024-05-14T11:16:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f126fe00640 1 --2- 172.21.15.116:0/3868767436 >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 0x7f12780ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:38.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f127d800640 1 --2- 172.21.15.116:0/3868767436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 0x7f12780b44d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1270009d90 tx=0x7f127001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6135642aebb25fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f1277e00640 1 -- 172.21.15.116:0/3868767436 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12700b2070 con 0x7f12780b40c0 2024-05-14T11:16:38.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f1277e00640 1 -- 172.21.15.116:0/3868767436 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12700a5a00 con 0x7f12780b40c0 2024-05-14T11:16:38.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.900+0000 7f1277e00640 1 -- 172.21.15.116:0/3868767436 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12700a5dd0 con 0x7f12780b40c0 2024-05-14T11:16:38.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- 172.21.15.116:0/3868767436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 msgr2=0x7f12780b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 --2- 172.21.15.116:0/3868767436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 0x7f12780b44d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1270009d90 tx=0x7f127001c940 comp rx=0 tx=0).stop 2024-05-14T11:16:38.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- 172.21.15.116:0/3868767436 shutdown_connections 2024-05-14T11:16:38.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 --2- 172.21.15.116:0/3868767436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780b40c0 0x7f12780b44d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 --2- 172.21.15.116:0/3868767436 >> v2:172.21.15.149:3300/0 conn(0x7f12780d1c20 0x7f12780ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- 172.21.15.116:0/3868767436 >> 172.21.15.116:0/3868767436 conn(0x7f12780d0770 msgr2=0x7f12780d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- 172.21.15.116:0/3868767436 shutdown_connections 2024-05-14T11:16:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- 172.21.15.116:0/3868767436 wait complete. 2024-05-14T11:16:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 Processor -- start 2024-05-14T11:16:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 -- start start 2024-05-14T11:16:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.904+0000 7f127f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12780cdf80 0x7f127815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780ccf10 0x7f127815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f127806ff40 con 0x7f12780b40c0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f127806fdc0 con 0x7f12780cdf80 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f127816cd60 con 0x7f12780ccf10 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58622/0 (socket says 172.21.15.116:58622) 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 -- 172.21.15.116:0/1427591407 learned_addr learned my addr 172.21.15.116:0/1427591407 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f126fe00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12780cdf80 0x7f127815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127d800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780ccf10 0x7f127815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 -- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780ccf10 msgr2=0x7f127815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780ccf10 0x7f127815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 -- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12780cdf80 msgr2=0x7f127815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12780cdf80 0x7f127815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 -- 172.21.15.116:0/1427591407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12780cb290 con 0x7f12780b40c0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127ce00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f126800c460 tx=0x7f126800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268018070 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268007500 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268004a30 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12780cb570 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12780cbb30 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127806a9c0 con 0x7f12780b40c0 2024-05-14T11:16:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.908+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1268004bd0 con 0x7f12780b40c0 2024-05-14T11:16:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.912+0000 7f1275600640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 0x7f1254071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.912+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f126808e870 con 0x7f12780b40c0 2024-05-14T11:16:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.912+0000 7f126fe00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 0x7f1254071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.912+0000 7f126fe00640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 0x7f1254071940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f127815bca0 tx=0x7f1260009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:38.916+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f126805adc0 con 0x7f12780b40c0 2024-05-14T11:16:39.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f12780b9590 con 0x7f12780b40c0 2024-05-14T11:16:39.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f1275600640 1 -- 172.21.15.116:0/1427591407 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f126805f0c0 con 0x7f12780b40c0 2024-05-14T11:16:39.111 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:39.111 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T11:16:39.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 msgr2=0x7f1254071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 0x7f1254071940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f127815bca0 tx=0x7f1260009190 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 msgr2=0x7f12780cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f126800c460 tx=0x7f126800c920 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 shutdown_connections 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f125406f430 0x7f1254071940 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12780cdf80 0x7f127815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12780ccf10 0x7f127815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 --2- 172.21.15.116:0/1427591407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12780b40c0 0x7f12780cc9d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 >> 172.21.15.116:0/1427591407 conn(0x7f12780d0770 msgr2=0x7f12780c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 shutdown_connections 2024-05-14T11:16:39.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.108+0000 7f127f800640 1 -- 172.21.15.116:0/1427591407 wait complete. 2024-05-14T11:16:39.132 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-14T11:16:39.133 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.276+0000 7f1a7ca00640 1 Processor -- start 2024-05-14T11:16:39.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.276+0000 7f1a7ca00640 1 -- start start 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.276+0000 7f1a7ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 0x7f1a78074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a7ca00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7815d340 con 0x7f1a780d1d80 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a7ca00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7815d620 con 0x7f1a78074b70 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a7ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7815d900 con 0x7f1a780af420 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a76800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1a780d1d80 0x7f1a7806edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51756/0 (socket says 172.21.15.116:51756) 2024-05-14T11:16:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a76800640 1 -- 172.21.15.116:0/1815519267 learned_addr learned my addr 172.21.15.116:0/1815519267 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75e00640 1 --2- 172.21.15.116:0/1815519267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 0x7f1a78074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 418813213 0 0) 0x7f1a7815d340 con 0x7f1a780d1d80 2024-05-14T11:16:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a5c003530 con 0x7f1a780d1d80 2024-05-14T11:16:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531535242 0 0) 0x7f1a7815d620 con 0x7f1a78074b70 2024-05-14T11:16:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a7815d340 con 0x7f1a78074b70 2024-05-14T11:16:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2781659828 0 0) 0x7f1a5c003530 con 0x7f1a780d1d80 2024-05-14T11:16:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75e00640 1 -- 172.21.15.116:0/1815519267 >> v1:172.21.15.149:6789/0 conn(0x7f1a78074b70 legacy=0x7f1a780d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75e00640 1 -- 172.21.15.116:0/1815519267 >> v1:172.21.15.116:6789/0 conn(0x7f1a780d1d80 legacy=0x7f1a7806edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.280+0000 7f1a75e00640 1 -- 172.21.15.116:0/1815519267 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7815dbe0 con 0x7f1a780af420 2024-05-14T11:16:39.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a75e00640 1 --2- 172.21.15.116:0/1815519267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 0x7f1a78074630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097f0 tx=0x7f1a6801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6bc5a009482fddd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680b1070 con 0x7f1a780af420 2024-05-14T11:16:39.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a680ac400 con 0x7f1a780af420 2024-05-14T11:16:39.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a75400640 1 -- 172.21.15.116:0/1815519267 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70001a80 con 0x7f1a780af420 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- 172.21.15.116:0/1815519267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 msgr2=0x7f1a78074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/1815519267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 0x7f1a78074630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097f0 tx=0x7f1a6801c400 comp rx=0 tx=0).stop 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- 172.21.15.116:0/1815519267 shutdown_connections 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/1815519267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a780af420 0x7f1a78074630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- 172.21.15.116:0/1815519267 >> 172.21.15.116:0/1815519267 conn(0x7f1a780b4620 msgr2=0x7f1a780b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- 172.21.15.116:0/1815519267 shutdown_connections 2024-05-14T11:16:39.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- 172.21.15.116:0/1815519267 wait complete. 2024-05-14T11:16:39.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 Processor -- start 2024-05-14T11:16:39.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.284+0000 7f1a7ca00640 1 -- start start 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a780af420 0x7f1a7809a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a7813ea90 0x7f1a7809a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7816b430 con 0x7f1a78074b70 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7816b2b0 con 0x7f1a780af420 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7816b5b0 con 0x7f1a7813ea90 2024-05-14T11:16:39.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58630/0 (socket says 172.21.15.116:58630) 2024-05-14T11:16:39.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 -- 172.21.15.116:0/2191472855 learned_addr learned my addr 172.21.15.116:0/2191472855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 -- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a7813ea90 msgr2=0x7f1a7809a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a7813ea90 0x7f1a7809a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 -- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a780af420 msgr2=0x7f1a7809a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a780af420 0x7f1a7809a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 -- 172.21.15.116:0/2191472855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a780b1000 con 0x7f1a78074b70 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a75e00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097f0 tx=0x7f1a6801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680bb020 con 0x7f1a78074b70 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68004220 con 0x7f1a78074b70 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7814ff90 con 0x7f1a78074b70 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680b1040 con 0x7f1a78074b70 2024-05-14T11:16:39.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a780b12e0 con 0x7f1a78074b70 2024-05-14T11:16:39.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.288+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7806a9c0 con 0x7f1a78074b70 2024-05-14T11:16:39.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.292+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1a68003620 con 0x7f1a78074b70 2024-05-14T11:16:39.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.292+0000 7f1a6ea00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 0x7f1a54071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.292+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1a681290b0 con 0x7f1a78074b70 2024-05-14T11:16:39.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.292+0000 7f1a6d400640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 0x7f1a54071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.292+0000 7f1a6d400640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 0x7f1a54071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a7809ae40 tx=0x7f1a600079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.296+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1a68015ff0 con 0x7f1a78074b70 2024-05-14T11:16:39.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a7806fe00 con 0x7f1a78074b70 2024-05-14T11:16:39.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a6ea00640 1 -- 172.21.15.116:0/2191472855 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f1a680f5bc0 con 0x7f1a78074b70 2024-05-14T11:16:39.491 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:39.491 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:39.491 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 msgr2=0x7f1a54071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 0x7f1a54071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a7809ae40 tx=0x7f1a600079e0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 msgr2=0x7f1a78150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a680097f0 tx=0x7f1a6801ac40 comp rx=0 tx=0).stop 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.488+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 shutdown_connections 2024-05-14T11:16:39.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1a5406f400 0x7f1a54071910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a780af420 0x7f1a7809a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1a7813ea90 0x7f1a7809a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 --2- 172.21.15.116:0/2191472855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a78074b70 0x7f1a78150a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 >> 172.21.15.116:0/2191472855 conn(0x7f1a780b4620 msgr2=0x7f1a78073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 shutdown_connections 2024-05-14T11:16:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.492+0000 7f1a7ca00640 1 -- 172.21.15.116:0/2191472855 wait complete. 2024-05-14T11:16:39.513 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:39.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.656+0000 7f4740c00640 1 Processor -- start 2024-05-14T11:16:39.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.656+0000 7f4740c00640 1 -- start start 2024-05-14T11:16:39.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4740c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c0d1670 0x7f473c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4740c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0af380 0x7f473c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4740c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473c0d1fc0 con 0x7f473c156fa0 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4740c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f473c0706f0 con 0x7f473c0d1670 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4740c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f473c1573b0 con 0x7f473c0af380 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c0d1670 0x7f473c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c0d1670 0x7f473c0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58644/0 (socket says 172.21.15.116:58644) 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 -- 172.21.15.116:0/3808290318 learned_addr learned my addr 172.21.15.116:0/3808290318 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f473a800640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0af380 0x7f473c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 -- 172.21.15.116:0/3808290318 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0af380 msgr2=0x7f473c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:39.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0af380 0x7f473c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 -- 172.21.15.116:0/3808290318 >> v1:172.21.15.149:6789/0 conn(0x7f473c156fa0 legacy=0x7f473c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:39.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 -- 172.21.15.116:0/3808290318 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c160510 con 0x7f473c0d1670 2024-05-14T11:16:39.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.660+0000 7f4731400640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0d1670 0x7f473c0d1a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4724009870 tx=0x7f472401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f88a8c96c158f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4739400640 1 -- 172.21.15.116:0/3808290318 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240b0070 con 0x7f473c0d1670 2024-05-14T11:16:39.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4739400640 1 -- 172.21.15.116:0/3808290318 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47240ab600 con 0x7f473c0d1670 2024-05-14T11:16:39.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4739400640 1 -- 172.21.15.116:0/3808290318 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240ab9d0 con 0x7f473c0d1670 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- 172.21.15.116:0/3808290318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0d1670 msgr2=0x7f473c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0d1670 0x7f473c0d1a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4724009870 tx=0x7f472401c380 comp rx=0 tx=0).stop 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- 172.21.15.116:0/3808290318 shutdown_connections 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0af380 0x7f473c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 --2- 172.21.15.116:0/3808290318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0d1670 0x7f473c0d1a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- 172.21.15.116:0/3808290318 >> 172.21.15.116:0/3808290318 conn(0x7f473c0b4620 msgr2=0x7f473c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- 172.21.15.116:0/3808290318 shutdown_connections 2024-05-14T11:16:39.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- 172.21.15.116:0/3808290318 wait complete. 2024-05-14T11:16:39.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 Processor -- start 2024-05-14T11:16:39.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.664+0000 7f4740c00640 1 -- start start 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0d1670 0x7f473c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 0x7f473c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0b0660 con 0x7f473c0af380 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0b04e0 con 0x7f473c156fa0 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f473c169810 con 0x7f473c0d1670 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58660/0 (socket says 172.21.15.116:58660) 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 -- 172.21.15.116:0/3839202533 learned_addr learned my addr 172.21.15.116:0/3839202533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:39.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4731400640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0d1670 0x7f473c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f473a800640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 0x7f473c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 -- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0d1670 msgr2=0x7f473c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0d1670 0x7f473c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 -- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 msgr2=0x7f473c14d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 0x7f473c14d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 -- 172.21.15.116:0/3839202533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c14aeb0 con 0x7f473c0af380 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f473a800640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 0x7f473c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4739e00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b3e0 tx=0x7f472c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c015020 con 0x7f473c0af380 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c011830 con 0x7f473c0af380 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c011c60 con 0x7f473c0af380 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c14b330 con 0x7f473c0af380 2024-05-14T11:16:39.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c14b8f0 con 0x7f473c0af380 2024-05-14T11:16:39.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.668+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c06a9c0 con 0x7f473c0af380 2024-05-14T11:16:39.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.672+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f472c018020 con 0x7f473c0af380 2024-05-14T11:16:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.672+0000 7f4732a00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 0x7f4718071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.672+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f472c08f2f0 con 0x7f473c0af380 2024-05-14T11:16:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.672+0000 7f4731400640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 0x7f4718071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.672+0000 7f4731400640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 0x7f4718071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47240ad000 tx=0x7f4724008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:39.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.676+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f472c01f030 con 0x7f473c0af380 2024-05-14T11:16:39.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.864+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f473c0cd660 con 0x7f473c0af380 2024-05-14T11:16:39.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4732a00640 1 -- 172.21.15.116:0/3839202533 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f472c05be90 con 0x7f473c0af380 2024-05-14T11:16:39.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:39.870 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:39.870 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:16:39.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 msgr2=0x7f4718071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 0x7f4718071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47240ad000 tx=0x7f4724008040 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 msgr2=0x7f473c14a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b3e0 tx=0x7f472c00b8a0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 shutdown_connections 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f400 0x7f4718071910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c156fa0 0x7f473c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0d1670 0x7f473c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 --2- 172.21.15.116:0/3839202533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c0af380 0x7f473c14a970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 >> 172.21.15.116:0/3839202533 conn(0x7f473c0b4620 msgr2=0x7f473c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 shutdown_connections 2024-05-14T11:16:39.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:39.868+0000 7f4740c00640 1 -- 172.21.15.116:0/3839202533 wait complete. 2024-05-14T11:16:39.891 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:40.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 Processor -- start 2024-05-14T11:16:40.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 -- start start 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe5c0b0770 0x7efe5c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c06cc40 con 0x7efe5c06f730 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5c06cf10 con 0x7efe5c072a30 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe61600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efe5c06db20 con 0x7efe5c0b0770 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe5c0b0770 0x7efe5c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe5c0b0770 0x7efe5c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58664/0 (socket says 172.21.15.116:58664) 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 -- 172.21.15.116:0/2190044777 learned_addr learned my addr 172.21.15.116:0/2190044777 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe53e00640 1 --1- 172.21.15.116:0/2190044777 >> v1:172.21.15.116:6789/0 conn(0x7efe5c06f730 0x7efe5c06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51758/0 (socket says 172.21.15.116:51758) 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 -- 172.21.15.116:0/2190044777 >> v1:172.21.15.149:6789/0 conn(0x7efe5c072a30 legacy=0x7efe5c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 -- 172.21.15.116:0/2190044777 >> v1:172.21.15.116:6789/0 conn(0x7efe5c06f730 legacy=0x7efe5c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.036+0000 7efe5aa00640 1 -- 172.21.15.116:0/2190044777 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c06d1f0 con 0x7efe5c0b0770 2024-05-14T11:16:40.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe5aa00640 1 --2- 172.21.15.116:0/2190044777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c0b0770 0x7efe5c06c700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00bb10 tx=0x7efe4c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e66a8f0b24bcfb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe5a000640 1 -- 172.21.15.116:0/2190044777 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0b4070 con 0x7efe5c0b0770 2024-05-14T11:16:40.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe5a000640 1 -- 172.21.15.116:0/2190044777 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c0a7bc0 con 0x7efe5c0b0770 2024-05-14T11:16:40.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe5a000640 1 -- 172.21.15.116:0/2190044777 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0af440 con 0x7efe5c0b0770 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 -- 172.21.15.116:0/2190044777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c0b0770 msgr2=0x7efe5c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 --2- 172.21.15.116:0/2190044777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c0b0770 0x7efe5c06c700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00bb10 tx=0x7efe4c01eb70 comp rx=0 tx=0).stop 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 -- 172.21.15.116:0/2190044777 shutdown_connections 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 --2- 172.21.15.116:0/2190044777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c0b0770 0x7efe5c06c700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 -- 172.21.15.116:0/2190044777 >> 172.21.15.116:0/2190044777 conn(0x7efe5c0b4620 msgr2=0x7efe5c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 -- 172.21.15.116:0/2190044777 shutdown_connections 2024-05-14T11:16:40.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.040+0000 7efe61600640 1 -- 172.21.15.116:0/2190044777 wait complete. 2024-05-14T11:16:40.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 Processor -- start 2024-05-14T11:16:40.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 -- start start 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe5c06f730 0x7efe5c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 0x7efe5c144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c073ae0 con 0x7efe5c072a30 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c073960 con 0x7efe5c06f730 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe61600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c169fb0 con 0x7efe5c0b0770 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58674/0 (socket says 172.21.15.116:58674) 2024-05-14T11:16:40.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 -- 172.21.15.116:0/123260063 learned_addr learned my addr 172.21.15.116:0/123260063 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe5aa00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe5c06f730 0x7efe5c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe5b400640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 0x7efe5c144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 -- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 msgr2=0x7efe5c144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 0x7efe5c144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 -- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe5c06f730 msgr2=0x7efe5c155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe5c06f730 0x7efe5c155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 -- 172.21.15.116:0/123260063 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c154ce0 con 0x7efe5c072a30 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe5b400640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 0x7efe5c144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe53e00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c3c0 tx=0x7efe4400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440074e0 con 0x7efe5c072a30 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe44007680 con 0x7efe5c072a30 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.044+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004a30 con 0x7efe5c072a30 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c154fc0 con 0x7efe5c072a30 2024-05-14T11:16:40.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c155580 con 0x7efe5c072a30 2024-05-14T11:16:40.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c154a50 con 0x7efe5c072a30 2024-05-14T11:16:40.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe4401a020 con 0x7efe5c072a30 2024-05-14T11:16:40.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe52a00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 0x7efe38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efe4408ee90 con 0x7efe5c072a30 2024-05-14T11:16:40.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.048+0000 7efe5aa00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 0x7efe38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.052+0000 7efe5aa00640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 0x7efe38071910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b1000 tx=0x7efe4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.052+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efe4405b9a0 con 0x7efe5c072a30 2024-05-14T11:16:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.212+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe5c06a9c0 con 0x7efe5c072a30 2024-05-14T11:16:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe52a00640 1 -- 172.21.15.116:0/123260063 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7efe44090020 con 0x7efe5c072a30 2024-05-14T11:16:40.217 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:40.218 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:16.676359+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 msgr2=0x7efe38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 0x7efe38071910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b1000 tx=0x7efe4c008040 comp rx=0 tx=0).stop 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 msgr2=0x7efe5c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c3c0 tx=0x7efe4400c880 comp rx=0 tx=0).stop 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 shutdown_connections 2024-05-14T11:16:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe3806f400 0x7efe38071910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe5c0b0770 0x7efe5c144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe5c072a30 0x7efe5c154170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 --2- 172.21.15.116:0/123260063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe5c06f730 0x7efe5c155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 >> 172.21.15.116:0/123260063 conn(0x7efe5c0b4620 msgr2=0x7efe5c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 shutdown_connections 2024-05-14T11:16:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.216+0000 7efe61600640 1 -- 172.21.15.116:0/123260063 wait complete. 2024-05-14T11:16:40.239 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:16:40.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 Processor -- start 2024-05-14T11:16:40.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 -- start start 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5090160080 0x7f50900cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50900d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f50900d1a50 0x7f50900b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f50900d1e60 con 0x7f50900d1a50 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f50900b2f40 con 0x7f5090160080 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5095e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50900b3220 con 0x7f50900ce4c0 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f50900d1a50 0x7f50900b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f50900d1a50 0x7f50900b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58686/0 (socket says 172.21.15.116:58686) 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 -- 172.21.15.116:0/3390167557 learned_addr learned my addr 172.21.15.116:0/3390167557 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5086000640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50900d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508ea00640 1 --2- 172.21.15.116:0/3390167557 >> v2:172.21.15.149:3300/0 conn(0x7f5090160080 0x7f50900cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 -- 172.21.15.116:0/3390167557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 msgr2=0x7f50900d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:40.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50900d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 -- 172.21.15.116:0/3390167557 >> v2:172.21.15.149:3300/0 conn(0x7f5090160080 msgr2=0x7f50900cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:40.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 --2- 172.21.15.116:0/3390167557 >> v2:172.21.15.149:3300/0 conn(0x7f5090160080 0x7f50900cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f508f400640 1 -- 172.21.15.116:0/3390167557 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50900b3500 con 0x7f50900d1a50 2024-05-14T11:16:40.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.388+0000 7f5086000640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50900d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:40.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f508f400640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50900d1a50 0x7f50900b3fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f509006e1c0 tx=0x7f508801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3caf71a952166e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f508e000640 1 -- 172.21.15.116:0/3390167557 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880a5720 con 0x7f50900d1a50 2024-05-14T11:16:40.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f508e000640 1 -- 172.21.15.116:0/3390167557 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50880a58c0 con 0x7f50900d1a50 2024-05-14T11:16:40.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f508e000640 1 -- 172.21.15.116:0/3390167557 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880a5c90 con 0x7f50900d1a50 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 -- 172.21.15.116:0/3390167557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50900d1a50 msgr2=0x7f50900b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50900d1a50 0x7f50900b3fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f509006e1c0 tx=0x7f508801c8a0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 -- 172.21.15.116:0/3390167557 shutdown_connections 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50900d1a50 0x7f50900b3fe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 --2- 172.21.15.116:0/3390167557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50900d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 --2- 172.21.15.116:0/3390167557 >> v2:172.21.15.149:3300/0 conn(0x7f5090160080 0x7f50900cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 -- 172.21.15.116:0/3390167557 >> 172.21.15.116:0/3390167557 conn(0x7f50900d0770 msgr2=0x7f50900d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 -- 172.21.15.116:0/3390167557 shutdown_connections 2024-05-14T11:16:40.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.392+0000 7f5095e00640 1 -- 172.21.15.116:0/3390167557 wait complete. 2024-05-14T11:16:40.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 Processor -- start 2024-05-14T11:16:40.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 -- start start 2024-05-14T11:16:40.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50901588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50900d1a50 0x7f50900cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f509006ff50 con 0x7f5090160080 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f509006fdd0 con 0x7f50900d1a50 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f5095e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50901639e0 con 0x7f50900ce4c0 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.396+0000 7f508f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58700/0 (socket says 172.21.15.116:58700) 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 -- 172.21.15.116:0/1435596557 learned_addr learned my addr 172.21.15.116:0/1435596557 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:40.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508ea00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50901588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5086000640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50900d1a50 0x7f50900cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 -- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 msgr2=0x7f50901588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50901588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 -- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50900d1a50 msgr2=0x7f50900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50900d1a50 0x7f50900cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 -- 172.21.15.116:0/1435596557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5090161600 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f508f400640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50880af040 tx=0x7f5088019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880bc020 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50880b8690 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880b8a80 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50901618e0 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5090161ea0 con 0x7f5090160080 2024-05-14T11:16:40.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509006a9c0 con 0x7f5090160080 2024-05-14T11:16:40.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.400+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50880bf020 con 0x7f5090160080 2024-05-14T11:16:40.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.404+0000 7f5087e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 0x7f506c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:40.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.404+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f508812a870 con 0x7f5090160080 2024-05-14T11:16:40.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.404+0000 7f508ea00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 0x7f506c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:40.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.404+0000 7f508ea00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 0x7f506c071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50900749e0 tx=0x7f5080008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:40.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.408+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f50880165e0 con 0x7f5090160080 2024-05-14T11:16:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.596+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50900b9590 con 0x7f5090160080 2024-05-14T11:16:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5087e00640 1 -- 172.21.15.116:0/1435596557 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f50880f6dd0 con 0x7f5090160080 2024-05-14T11:16:40.602 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:40.602 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:16:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 msgr2=0x7f506c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 0x7f506c071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50900749e0 tx=0x7f5080008040 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 msgr2=0x7f5090071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50880af040 tx=0x7f5088019640 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 shutdown_connections 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f506c06f400 0x7f506c071910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50900d1a50 0x7f50900cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5090160080 0x7f5090071300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 --2- 172.21.15.116:0/1435596557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50900ce4c0 0x7f50901588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 >> 172.21.15.116:0/1435596557 conn(0x7f50900d0770 msgr2=0x7f50900ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 shutdown_connections 2024-05-14T11:16:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:40.600+0000 7f5095e00640 1 -- 172.21.15.116:0/1435596557 wait complete. 2024-05-14T11:16:40.689 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:16:40.692 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:16:41.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:16:11.615741+0000', 'modified': '2024-05-14T11:16:31.822599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/2939305126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2939305126}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2939305126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4284, 'qdb_cluster': [4284]} 2024-05-14T11:16:41.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:16:41.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:16:41.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 Processor -- start 2024-05-14T11:16:41.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 -- start start 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4ec00b52f0 0x7f4ec0073e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec0074770 con 0x7f4ec0074360 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00b6390 con 0x7f4ec00b7d00 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec6000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec0070a30 con 0x7f4ec00b52f0 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.172+0000 7f4ec4c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4ec00b52f0 0x7f4ec0073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4eb7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4ec0074360 0x7f4ec00b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51772/0 (socket says 172.21.15.116:51772) 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4eb7e00640 1 -- 172.21.15.116:0/4068643982 learned_addr learned my addr 172.21.15.116:0/4068643982 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ec4c00640 1 -- 172.21.15.116:0/4068643982 >> v1:172.21.15.149:6789/0 conn(0x7f4ec00b7d00 legacy=0x7f4ec01618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ec4c00640 1 -- 172.21.15.116:0/4068643982 >> v1:172.21.15.116:6789/0 conn(0x7f4ec0074360 legacy=0x7f4ec00b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ec4c00640 1 -- 172.21.15.116:0/4068643982 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec0147570 con 0x7f4ec00b52f0 2024-05-14T11:16:41.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/4068643982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec0073e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00097c0 tx=0x7f4eb001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6a02947ffa9280 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ebfe00640 1 -- 172.21.15.116:0/4068643982 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00b0070 con 0x7f4ec00b52f0 2024-05-14T11:16:41.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ebfe00640 1 -- 172.21.15.116:0/4068643982 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb00ab600 con 0x7f4ec00b52f0 2024-05-14T11:16:41.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.176+0000 7f4ebfe00640 1 -- 172.21.15.116:0/4068643982 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00ab9d0 con 0x7f4ec00b52f0 2024-05-14T11:16:41.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- 172.21.15.116:0/4068643982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 msgr2=0x7f4ec0073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 --2- 172.21.15.116:0/4068643982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec0073e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00097c0 tx=0x7f4eb001c330 comp rx=0 tx=0).stop 2024-05-14T11:16:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- 172.21.15.116:0/4068643982 shutdown_connections 2024-05-14T11:16:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 --2- 172.21.15.116:0/4068643982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec0073e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- 172.21.15.116:0/4068643982 >> 172.21.15.116:0/4068643982 conn(0x7f4ec00d28e0 msgr2=0x7f4ec00d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- 172.21.15.116:0/4068643982 shutdown_connections 2024-05-14T11:16:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- 172.21.15.116:0/4068643982 wait complete. 2024-05-14T11:16:41.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 Processor -- start 2024-05-14T11:16:41.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.180+0000 7f4ec6000640 1 -- start start 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ec0074360 0x7f4ec00d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4ec00b7d00 0x7f4ec01e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00706c0 con 0x7f4ec00b52f0 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00703e0 con 0x7f4ec0074360 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec00b5700 con 0x7f4ec00b7d00 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58724/0 (socket says 172.21.15.116:58724) 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 -- 172.21.15.116:0/1774482522 learned_addr learned my addr 172.21.15.116:0/1774482522 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ec0074360 0x7f4ec00d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec5600640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4ec00b7d00 0x7f4ec01e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 -- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4ec00b7d00 msgr2=0x7f4ec01e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4ec00b7d00 0x7f4ec01e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 -- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ec0074360 msgr2=0x7f4ec00d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ec0074360 0x7f4ec00d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 -- 172.21.15.116:0/1774482522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec00ae870 con 0x7f4ec00b52f0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800faa0 tx=0x7f4ea800ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ebd600640 1 -- 172.21.15.116:0/1774482522 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea801b070 con 0x7f4ec00b52f0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ebd600640 1 -- 172.21.15.116:0/1774482522 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8007d00 con 0x7f4ec00b52f0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec00d0470 con 0x7f4ec00b52f0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ebd600640 1 -- 172.21.15.116:0/1774482522 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800a310 con 0x7f4ec00b52f0 2024-05-14T11:16:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.184+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec00d0a30 con 0x7f4ec00b52f0 2024-05-14T11:16:41.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ebd600640 1 -- 172.21.15.116:0/1774482522 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ea801e070 con 0x7f4ec00b52f0 2024-05-14T11:16:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ebd600640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 0x7f4e9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ebd600640 1 -- 172.21.15.116:0/1774482522 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4ea8091e40 con 0x7f4ec00b52f0 2024-05-14T11:16:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 0x7f4e9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ececbc000 1 Processor -- start 2024-05-14T11:16:41.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ececbc000 1 -- start start 2024-05-14T11:16:41.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.188+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 0x7f4e9c071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00ad000 tx=0x7f4eb00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b4ef7c0 0x55862ba29320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba29860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55862b8c6620 con 0x55862b8c16b0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55862b7e1740 con 0x55862b4ef7c0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55862b82f090 con 0x55862b8d7f80 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58738/0 (socket says 172.21.15.116:58738) 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 -- 172.21.15.116:0/238394565 learned_addr learned my addr 172.21.15.116:0/238394565 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b4ef7c0 0x55862ba29320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba29860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 -- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 msgr2=0x55862ba29860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba29860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 -- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b4ef7c0 msgr2=0x55862ba29320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b4ef7c0 0x55862ba29320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 -- 172.21.15.116:0/238394565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862b9c30a0 con 0x55862b8c16b0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ec5600640 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b6bd0 tx=0x7f4eb8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67015bfef879c69d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4eb5200640 1 -- 172.21.15.116:0/238394565 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800fc40 con 0x55862b8c16b0 2024-05-14T11:16:41.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4eb5200640 1 -- 172.21.15.116:0/238394565 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb8018070 con 0x55862b8c16b0 2024-05-14T11:16:41.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4eb5200640 1 -- 172.21.15.116:0/238394565 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8003980 con 0x55862b8c16b0 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- 172.21.15.116:0/238394565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 msgr2=0x55862ba29da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b6bd0 tx=0x7f4eb8001640 comp rx=0 tx=0).stop 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- 172.21.15.116:0/238394565 shutdown_connections 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8c16b0 0x55862ba29da0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba29860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 --2- 172.21.15.116:0/238394565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b4ef7c0 0x55862ba29320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- 172.21.15.116:0/238394565 >> 172.21.15.116:0/238394565 conn(0x55862b9ae340 msgr2=0x55862b9ddd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- 172.21.15.116:0/238394565 shutdown_connections 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- 172.21.15.116:0/238394565 wait complete. 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 -- start start 2024-05-14T11:16:41.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.192+0000 7f4ececbc000 1 Processor -- start 2024-05-14T11:16:41.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 0x55862ba2f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55862b82b050 con 0x55862b8d7f80 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55862b7e7ec0 con 0x55862b8c16b0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55862b7cec90 con 0x55862b4ef7c0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 10 client.0 _ll_register_callbacks cb 0x55862b9cae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58740/0 (socket says 172.21.15.116:58740) 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 -- 172.21.15.116:0/635487438 learned_addr learned my addr 172.21.15.116:0/635487438 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 0x55862ba2f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 -- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 msgr2=0x55862ba2f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 0x55862ba2f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 -- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 msgr2=0x55862ba2fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba2fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 -- 172.21.15.116:0/635487438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55862b8d96f0 con 0x55862b8d7f80 2024-05-14T11:16:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 0x55862ba2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec5600640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55862b9bfd00 tx=0x7f4eb8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fff86b601d1d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80052d0 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4eb8005470 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb8018070 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4eb802e050 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 10 client.4659 fetch_fsmap learned FSMap version 9 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55862b8d56a0 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb8018070 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4eb8031020 con 0x55862b8d7f80 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 10 client.4659 fetch_fsmap finished waiting for FSMap version 9 2024-05-14T11:16:41.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 10 client.4659 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:16:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 20 client.4659 populate_metadata read hostname 'smithi116' 2024-05-14T11:16:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4d47d0 0x55862ba0e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 4 client.4659 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:16:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 --> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55862b7c7be0 con 0x55862b4d47d0 2024-05-14T11:16:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.196+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4d47d0 0x55862ba0e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4d47d0 0x55862ba0e460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98ba2dbf6b662b2 server_cookie=b91099defd8c627b in_seq=0 out_seq=0 2024-05-14T11:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ead600640 10 client.4659 ms_handle_connect on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ead600640 1 -- 172.21.15.116:0/635487438 <== mds.0 v2:172.21.15.116:6834/2939305126 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55862b7c7be0 con 0x55862b4d47d0 2024-05-14T11:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ead600640 10 client.4659 handle_command_reply: tid=0 2024-05-14T11:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 client.4659 shutdown 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 msgr2=0x55862ba0d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55862b9bfd00 tx=0x7f4eb8001260 comp rx=0 tx=0).stop 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 shutdown_connections 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4d47d0 0x55862ba0e460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba2fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ead600640 0 client.4659 ms_handle_reset on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b8d7f80 0x55862ba0d720 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 --2- 172.21.15.116:0/635487438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b4ef7c0 0x55862ba2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 >> 172.21.15.116:0/635487438 conn(0x55862b9ae340 msgr2=0x55862b9de490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 shutdown_connections 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 1 -- 172.21.15.116:0/635487438 wait complete. 2024-05-14T11:16:41.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.200+0000 7f4ececbc000 20 client.4659 trim_cache size 0 max 16384 2024-05-14T11:16:41.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.216+0000 7f4ececbc000 1 Processor -- start 2024-05-14T11:16:41.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.216+0000 7f4ececbc000 1 -- start start 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba97c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0202540 con 0x55862b4ef7c0 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0202b90 con 0x55862b8c16b0 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55862b7c47d0 con 0x55862b8d7f80 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58748/0 (socket says 172.21.15.116:58748) 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 -- 172.21.15.116:0/14616567 learned_addr learned my addr 172.21.15.116:0/14616567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec5600640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba97c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 -- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 msgr2=0x55862ba98150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba98150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 -- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 msgr2=0x55862ba97c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba97c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 -- 172.21.15.116:0/14616567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862b808af0 con 0x55862b4ef7c0 2024-05-14T11:16:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba97c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4eb001c370 tx=0x7f4eb0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43848f6659b3f156 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4eafe00640 1 -- 172.21.15.116:0/14616567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00b0040 con 0x55862b4ef7c0 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4eafe00640 1 -- 172.21.15.116:0/14616567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb00ce070 con 0x55862b4ef7c0 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4eafe00640 1 -- 172.21.15.116:0/14616567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00c9430 con 0x55862b4ef7c0 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- 172.21.15.116:0/14616567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 msgr2=0x55862ba976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4eb001c370 tx=0x7f4eb0009190 comp rx=0 tx=0).stop 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- 172.21.15.116:0/14616567 shutdown_connections 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8c16b0 0x55862ba97c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8d7f80 0x55862ba98150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 --2- 172.21.15.116:0/14616567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba976d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- 172.21.15.116:0/14616567 >> 172.21.15.116:0/14616567 conn(0x55862b9ae340 msgr2=0x55862ba35e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- 172.21.15.116:0/14616567 shutdown_connections 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.220+0000 7f4ececbc000 1 -- 172.21.15.116:0/14616567 wait complete. 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- start start 2024-05-14T11:16:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 Processor -- start 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8c16b0 0x55862ba2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8d7f80 0x55862ba58480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55862b7ec920 con 0x55862b4ef7c0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55862b82b050 con 0x55862b8d7f80 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55862b7e1910 con 0x55862b8c16b0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 10 client.0 _ll_register_callbacks cb 0x7f4eb0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58764/0 (socket says 172.21.15.116:58764) 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 -- 172.21.15.116:0/2280230294 learned_addr learned my addr 172.21.15.116:0/2280230294 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8c16b0 0x55862ba2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec5600640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8d7f80 0x55862ba58480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 -- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8c16b0 msgr2=0x55862ba2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8c16b0 0x55862ba2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 -- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8d7f80 msgr2=0x55862ba58480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8d7f80 0x55862ba58480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 -- 172.21.15.116:0/2280230294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55862b807e40 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ec4c00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00ae040 tx=0x7f4eb001ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce7363bcb96d10e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00d7060 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4eb00cba20 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb00ce070 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f4eb00b0040 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 10 client.4665 fetch_fsmap learned FSMap version 9 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55862b8af4e0 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb00ce070 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f4eb00cbc20 con 0x55862b4ef7c0 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 10 client.4665 fetch_fsmap finished waiting for FSMap version 9 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 10 client.4665 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:16:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 20 client.4665 populate_metadata read hostname 'smithi116' 2024-05-14T11:16:41.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4c2830 0x55862baa38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 4 client.4665 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:16:41.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 --> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] -- command(tid 0: {"prefix": "status"}) -- 0x55862b83cca0 con 0x55862b4c2830 2024-05-14T11:16:41.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.224+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4c2830 0x55862baa38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4eb7e00640 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4c2830 0x55862baa38f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f29c66d42d661b server_cookie=f391f8731a587638 in_seq=0 out_seq=0 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ead600640 10 client.4665 ms_handle_connect on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ead600640 1 -- 172.21.15.116:0/2280230294 <== mds.0 v2:172.21.15.116:6834/2939305126 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55862b83cca0 con 0x55862b4c2830 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ead600640 10 client.4665 handle_command_reply: tid=0 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 client.4665 shutdown 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 msgr2=0x55862ba2ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00ae040 tx=0x7f4eb001ab70 comp rx=0 tx=0).stop 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 shutdown_connections 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:6834/2939305126,v1:172.21.15.116:6836/2939305126] conn(0x55862b4c2830 0x55862baa38f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55862b8d7f80 0x55862ba58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55862b8c16b0 0x55862ba2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ead600640 0 client.4665 ms_handle_reset on v2:172.21.15.116:6834/2939305126 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 --2- 172.21.15.116:0/2280230294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55862b4ef7c0 0x55862ba2ddd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 >> 172.21.15.116:0/2280230294 conn(0x55862b9ae340 msgr2=0x55862ba0c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 shutdown_connections 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 1 -- 172.21.15.116:0/2280230294 wait complete. 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ececbc000 20 client.4665 trim_cache size 0 max 16384 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4284, 2024-05-14T11:16:41.233 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 29.008889315000001, 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 9, 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 24, 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 24, 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 34.558245739999997 2024-05-14T11:16:41.234 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 msgr2=0x7f4e9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 0x7f4e9c071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00ad000 tx=0x7f4eb00a6000 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 msgr2=0x7f4ec015d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800faa0 tx=0x7f4ea800ff60 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 shutdown_connections 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e9c06f400 0x7f4e9c071910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4ec00b7d00 0x7f4ec01e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec00b52f0 0x7f4ec015d400 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 --2- 172.21.15.116:0/1774482522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ec0074360 0x7f4ec00d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 >> 172.21.15.116:0/1774482522 conn(0x7f4ec00d28e0 msgr2=0x7f4ec015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 shutdown_connections 2024-05-14T11:16:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.228+0000 7f4ec6000640 1 -- 172.21.15.116:0/1774482522 wait complete. 2024-05-14T11:16:41.250 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T11:16:41.260 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T11:16:41.260 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T11:16:41.261 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T11:16:41.261 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-14T11:16:41.293 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-14T11:16:41.293 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-14T11:16:41.294 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:41.294 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:41.294 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:41.294 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-14T11:16:41.294 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T11:16:41.295 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:41.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.295 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-05-14T11:16:41.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.305 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-05-14T11:16:41.375 INFO:teuthology.orchestra.run.smithi149.stderr:Cannot find device "ceph-brx" 2024-05-14T11:16:41.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:16:41.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.378 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:16:41.386 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-14T11:16:41.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T11:16:41.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-05-14T11:16:41.612 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-14T11:16:41.612 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:16:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.752+0000 7fae05e00640 1 Processor -- start 2024-05-14T11:16:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.752+0000 7fae05e00640 1 -- start start 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fae05e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae000b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fae05e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae000b2400 0x7fae000d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fae05e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000d3d30 con 0x7fae000d3920 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fae05e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fae000b4890 con 0x7fae000b2400 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fae05e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fae000d52c0 con 0x7fae000d23b0 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae000b2400 0x7fae000d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae000b2400 0x7fae000d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58780/0 (socket says 172.21.15.116:58780) 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 -- 172.21.15.116:0/3539978651 learned_addr learned my addr 172.21.15.116:0/3539978651 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadff400640 1 --1- 172.21.15.116:0/3539978651 >> v1:172.21.15.116:6789/0 conn(0x7fae000d3920 0x7fae000d5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51786/0 (socket says 172.21.15.116:51786) 2024-05-14T11:16:41.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadfea00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae000b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 -- 172.21.15.116:0/3539978651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 msgr2=0x7fae000b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae000b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 -- 172.21.15.116:0/3539978651 >> v1:172.21.15.116:6789/0 conn(0x7fae000d3920 legacy=0x7fae000d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadf7e00640 1 -- 172.21.15.116:0/3539978651 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000b80e0 con 0x7fae000b2400 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadfea00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae000b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:41.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.756+0000 7fadfe000640 1 -- 172.21.15.116:0/3539978651 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276950477 0 0) 0x7fadf80014a0 con 0x7fae000d3920 2024-05-14T11:16:41.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fadf7e00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae000d33e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fade8009870 tx=0x7fade801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f4e88a3fca71c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fadfe000640 1 -- 172.21.15.116:0/3539978651 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80b2070 con 0x7fae000b2400 2024-05-14T11:16:41.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fadfe000640 1 -- 172.21.15.116:0/3539978651 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade80a9070 con 0x7fae000b2400 2024-05-14T11:16:41.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fadfe000640 1 -- 172.21.15.116:0/3539978651 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80ad880 con 0x7fae000b2400 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 -- 172.21.15.116:0/3539978651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 msgr2=0x7fae000d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae000d33e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fade8009870 tx=0x7fade801c560 comp rx=0 tx=0).stop 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 -- 172.21.15.116:0/3539978651 shutdown_connections 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae000d33e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 --2- 172.21.15.116:0/3539978651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae000b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 -- 172.21.15.116:0/3539978651 >> 172.21.15.116:0/3539978651 conn(0x7fae0009ea30 msgr2=0x7fae0008cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 -- 172.21.15.116:0/3539978651 shutdown_connections 2024-05-14T11:16:41.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.760+0000 7fae05e00640 1 -- 172.21.15.116:0/3539978651 wait complete. 2024-05-14T11:16:41.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 Processor -- start 2024-05-14T11:16:41.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- start start 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae0015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae000d3920 0x7fae0015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d6270 con 0x7fae000b2400 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d27c0 con 0x7fae000d3920 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fae000b2810 con 0x7fae000d23b0 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58794/0 (socket says 172.21.15.116:58794) 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 -- 172.21.15.116:0/2282150057 learned_addr learned my addr 172.21.15.116:0/2282150057 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 -- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 msgr2=0x7fae0015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae0015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 -- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae000d3920 msgr2=0x7fae0015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae000d3920 0x7fae0015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 -- 172.21.15.116:0/2282150057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0006fce0 con 0x7fae000b2400 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadfea00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d7d0 tx=0x7fadf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0018020 con 0x7fae000b2400 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf00096a0 con 0x7fae000b2400 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0009a90 con 0x7fae000b2400 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae000a1670 con 0x7fae000b2400 2024-05-14T11:16:41.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.764+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae000a1c30 con 0x7fae000b2400 2024-05-14T11:16:41.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0006a9c0 con 0x7fae000b2400 2024-05-14T11:16:41.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fadf001b020 con 0x7fae000b2400 2024-05-14T11:16:41.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fadf7400640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:41.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fadf008fa50 con 0x7fae000b2400 2024-05-14T11:16:41.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fadf7e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:41.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.768+0000 7fadf7e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade80af000 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:41.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.772+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fadf0022020 con 0x7fae000b2400 2024-05-14T11:16:41.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae000d4390 con 0x7fae000b2400 2024-05-14T11:16:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fadf7400640 1 -- 172.21.15.116:0/2282150057 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fadf005c680 con 0x7fae000b2400 2024-05-14T11:16:41.971 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 msgr2=0x7faddc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fade80af000 tx=0x7fade8008040 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 msgr2=0x7fae0015fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d7d0 tx=0x7fadf000dc90 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 shutdown_connections 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faddc06f430 0x7faddc071940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae000d3920 0x7fae0015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae000d23b0 0x7fae0015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 --2- 172.21.15.116:0/2282150057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000b2400 0x7fae0015fe90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 >> 172.21.15.116:0/2282150057 conn(0x7fae0009ea30 msgr2=0x7fae0015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 shutdown_connections 2024-05-14T11:16:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:41.968+0000 7fae05e00640 1 -- 172.21.15.116:0/2282150057 wait complete. 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:16:41.992 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:16:41.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:16:41.993 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:16:41.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:41.993 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:42.003 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:16:42.004 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:42.004 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T11:16:42.005 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:42.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:42.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:42.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:42.159 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:16:42.222 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:16:42.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:42.224 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:42.280 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:16:42.281 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:42.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:42.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:42.419 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:16:42.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:42.428 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-05-14T11:16:42.481 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:42.481 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:42.481 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:42.481 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-14T11:16:42.481 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:42.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:42.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:42.606 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-14T11:16:42.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-14T11:16:42.607 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:42.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:42.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:42.745 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:42.745 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:42.745 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:16:42.745 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T11:16:42.745 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:42.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:42.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:42.871 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:16:42.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:42.871 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:42.880 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:16:42.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:42.880 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:42.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:42.993 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:43.046 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:43.047 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:43.048 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:43.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:43.049 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:43.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:16:43.100 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:16:43.143 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:43.163 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:43.218 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:43.220 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:43.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:43.221 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:43.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:16:43.244+0000 7f54daaa8580 -1 init, newargv = 0x7f54cc01b2f0 newargc=15 2024-05-14T11:16:43.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[35350]: starting ceph client 2024-05-14T11:16:43.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[35350]: starting fuse 2024-05-14T11:16:44.359 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:44.383 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:44.435 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:44.436 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:44.437 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:44.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:44.438 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:44.489 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:16:44.490 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:44.490 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:16:44.491 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:16:44.619 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.35350.asok 2024-05-14T11:16:44.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:44.632 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35350.asok 2024-05-14T11:16:44.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:44.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35350.asok status 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:16:44.809 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "35350", 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4696, 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4696 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:44.810 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1561851645 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1561851645 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4696 192.168.0.1:0/1561851645", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/1561851645", 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:16:44.811 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 9, 2024-05-14T11:16:44.812 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2024-05-14T11:16:44.812 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:16:44.812 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:16:44.812 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:16:44.812 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:16:44.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:16:44.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 Processor -- start 2024-05-14T11:16:44.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 -- start start 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 0x7f276c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c0d1e60 con 0x7f276c160080 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c0b2f40 con 0x7f276c0b3fe0 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2774200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f276c0b3220 con 0x7f276c0d1510 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2771800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f276c160080 0x7f276c0cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51800/0 (socket says 172.21.15.116:51800) 2024-05-14T11:16:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2771800640 1 -- 172.21.15.116:0/2484247790 learned_addr learned my addr 172.21.15.116:0/2484247790 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2770e00640 1 --2- 172.21.15.116:0/2484247790 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 0x7f276c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381904144 0 0) 0x7f2760001120 con 0x7f276c160080 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2754003530 con 0x7f276c160080 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238555345 0 0) 0x7f276c0b2f40 con 0x7f276c0b3fe0 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2760001120 con 0x7f276c0b3fe0 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2770e00640 1 -- 172.21.15.116:0/2484247790 >> v1:172.21.15.149:6789/0 conn(0x7f276c0b3fe0 legacy=0x7f276c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.968+0000 7f2770e00640 1 -- 172.21.15.116:0/2484247790 >> v1:172.21.15.116:6789/0 conn(0x7f276c160080 legacy=0x7f276c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f2770e00640 1 -- 172.21.15.116:0/2484247790 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c0b3500 con 0x7f276c0d1510 2024-05-14T11:16:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f2770e00640 1 --2- 172.21.15.116:0/2484247790 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 0x7f276c0d1920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f275801c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=692d6a4fa4891d9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580b1070 con 0x7f276c0d1510 2024-05-14T11:16:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27580ac5f0 con 0x7f276c0d1510 2024-05-14T11:16:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f276be00640 1 -- 172.21.15.116:0/2484247790 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580ac9a0 con 0x7f276c0d1510 2024-05-14T11:16:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f2774200640 1 -- 172.21.15.116:0/2484247790 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 msgr2=0x7f276c0d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.972+0000 7f2774200640 1 --2- 172.21.15.116:0/2484247790 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 0x7f276c0d1920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f275801c390 comp rx=0 tx=0).stop 2024-05-14T11:16:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- 172.21.15.116:0/2484247790 shutdown_connections 2024-05-14T11:16:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 --2- 172.21.15.116:0/2484247790 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c0d1510 0x7f276c0d1920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- 172.21.15.116:0/2484247790 >> 172.21.15.116:0/2484247790 conn(0x7f276c0d0770 msgr2=0x7f276c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- 172.21.15.116:0/2484247790 shutdown_connections 2024-05-14T11:16:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- 172.21.15.116:0/2484247790 wait complete. 2024-05-14T11:16:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 Processor -- start 2024-05-14T11:16:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- start start 2024-05-14T11:16:44.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f276c0b3fe0 0x7f276c150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:44.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c160080 0x7f276c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f276c06ff50 con 0x7f276c0d1510 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f276c06fdd0 con 0x7f276c0b3fe0 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.976+0000 7f2774200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f276c16dd50 con 0x7f276c160080 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58798/0 (socket says 172.21.15.116:58798) 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 -- 172.21.15.116:0/3094887753 learned_addr learned my addr 172.21.15.116:0/3094887753 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2771800640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f276c0b3fe0 0x7f276c150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2772200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c160080 0x7f276c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 -- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c160080 msgr2=0x7f276c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c160080 0x7f276c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 -- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f276c0b3fe0 msgr2=0x7f276c150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f276c0b3fe0 0x7f276c150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:44.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 -- 172.21.15.116:0/3094887753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c0b2c40 con 0x7f276c0d1510 2024-05-14T11:16:44.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2770e00640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f2758019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:44.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580b9020 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27580b5410 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c150ad0 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580b1040 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c151850 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c06a9c0 con 0x7f276c0d1510 2024-05-14T11:16:44.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.980+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f27580bc020 con 0x7f276c0d1510 2024-05-14T11:16:44.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.984+0000 7f2769600640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 0x7f274c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:44.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.984+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2758128e40 con 0x7f276c0d1510 2024-05-14T11:16:44.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.984+0000 7f2771800640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 0x7f274c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:44.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.984+0000 7f2771800640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 0x7f274c071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2760007ab0 tx=0x7f2760009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:44.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:44.988+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f27580c3070 con 0x7f276c0d1510 2024-05-14T11:16:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.180+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f276c06e600 con 0x7f276c0d1510 2024-05-14T11:16:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.180+0000 7f2769600640 1 -- 172.21.15.116:0/3094887753 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f27580a9070 con 0x7f276c0d1510 2024-05-14T11:16:45.184 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T11:16:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 msgr2=0x7f274c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 0x7f274c071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2760007ab0 tx=0x7f2760009040 comp rx=0 tx=0).stop 2024-05-14T11:16:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 msgr2=0x7f276c0c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f2758019130 comp rx=0 tx=0).stop 2024-05-14T11:16:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 shutdown_connections 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f274c06f400 0x7f274c071910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f276c160080 0x7f276c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f276c0d1510 0x7f276c0c9410 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 --2- 172.21.15.116:0/3094887753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f276c0b3fe0 0x7f276c150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 >> 172.21.15.116:0/3094887753 conn(0x7f276c0d0770 msgr2=0x7f276c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 shutdown_connections 2024-05-14T11:16:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45.184+0000 7f2774200640 1 -- 172.21.15.116:0/3094887753 wait complete. 2024-05-14T11:16:45.205 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:16:45.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:16:45.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-14T11:16:45.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.206 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:16:45.216 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:45.217 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.218 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:16:45.278 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:16:45.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.280 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:45.332 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:45.333 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:45.333 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:16:45.333 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:45.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:45.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.467 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:16:45.476 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:16:45.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.476 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-05-14T11:16:45.528 INFO:teuthology.orchestra.run.smithi116.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-14T11:16:45.529 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:45.529 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:45.529 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:16:45.529 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-14T11:16:45.529 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:45.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:45.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:45.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.655 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-14T11:16:45.675 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether d2:d2:ca:5d:94:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::d0d2:caff:fe5d:94fe/64 scope link 2024-05-14T11:16:45.676 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:45.677 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.744 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:45.745 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether d2:d2:ca:5d:94:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-14T11:16:45.745 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-14T11:16:45.745 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.745 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::d0d2:caff:fe5d:94fe/64 scope link 2024-05-14T11:16:45.745 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:45.746 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-14T11:16:45.746 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:45.746 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-14T11:16:45.747 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:45.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:45.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:45.923 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:45.923 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:16:45.923 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2024-05-14T11:16:45.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.1 master ceph-brx 2024-05-14T11:16:45.924 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:16:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:46.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:46.050 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-14T11:16:46.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:46.050 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-14T11:16:46.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:46.059 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:46.112 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:46.172 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:16:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:16:46.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:46.229 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:46.280 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:16:46.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-05-14T11:16:46.280 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-14T11:16:46.323 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:16:46.343 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:46.394 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:16:46.394 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:16:46.394 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:16:46.394 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:16:46.394 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:16:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:16:46.396 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:46.397 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:16:46.397 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:16:46.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:46.398 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:16:46.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:2024-05-14T11:16:46.424+0000 7fca92247580 -1 init, newargv = 0x7fca7801b4e0 newargc=15 2024-05-14T11:16:46.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:ceph-fuse[35495]: starting ceph client 2024-05-14T11:16:46.436 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi116.stderr:ceph-fuse[35495]: starting fuse 2024-05-14T11:16:46.438 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:16:46.438 INFO:teuthology.orchestra.run.smithi116.stdout:45 2024-05-14T11:16:46.438 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:46.439 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.2")) 2024-05-14T11:16:46.440 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:16:46.560 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.2.35495.asok 2024-05-14T11:16:46.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:46.571 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35495.asok 2024-05-14T11:16:46.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:46.571 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35495.asok status 2024-05-14T11:16:46.748 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:16:46.748 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:16:46.748 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:16:46.748 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:16:46.748 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "2", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "35495", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4692, 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4692 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:46.749 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.2:0", 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1001769267 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.2:0", 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1001769267 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4692 192.168.0.2:0/1001769267", 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.2:0/1001769267", 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 9, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:16:46.750 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:16:46.751 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:16:46.751 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:16:46.765 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:16:46.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.908+0000 7ff025800640 1 Processor -- start 2024-05-14T11:16:46.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.908+0000 7ff025800640 1 -- start start 2024-05-14T11:16:46.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff025800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 0x7ff0200aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff025800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff0200d1670 0x7ff0200d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff025800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02006ff20 con 0x7ff020161600 2024-05-14T11:16:46.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff025800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff02006d710 con 0x7ff0200d1670 2024-05-14T11:16:46.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff025800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff0200ae100 con 0x7ff02015db30 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff020161600 0x7ff020161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51812/0 (socket says 172.21.15.116:51812) 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01f400640 1 -- 172.21.15.116:0/3550970172 learned_addr learned my addr 172.21.15.116:0/3550970172 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff016000640 1 --2- 172.21.15.116:0/3550970172 >> v2:172.21.15.149:3300/0 conn(0x7ff0200d1670 0x7ff0200d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01ea00640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 0x7ff0200aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01e000640 1 -- 172.21.15.116:0/3550970172 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1498540720 0 0) 0x7ff02006ff20 con 0x7ff020161600 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01e000640 1 -- 172.21.15.116:0/3550970172 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff004003530 con 0x7ff020161600 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff016000640 1 -- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 msgr2=0x7ff0200aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff016000640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 0x7ff0200aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff016000640 1 -- 172.21.15.116:0/3550970172 >> v1:172.21.15.116:6789/0 conn(0x7ff020161600 legacy=0x7ff020161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff016000640 1 -- 172.21.15.116:0/3550970172 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0200d1fc0 con 0x7ff0200d1670 2024-05-14T11:16:46.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.912+0000 7ff01ea00640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 0x7ff0200aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:46.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff016000640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0200d1670 0x7ff0200d1a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0080097c0 tx=0x7ff00801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b2ecaf8ad796b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:46.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff01e000640 1 -- 172.21.15.116:0/3550970172 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080b0070 con 0x7ff0200d1670 2024-05-14T11:16:46.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff01e000640 1 -- 172.21.15.116:0/3550970172 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0080ab600 con 0x7ff0200d1670 2024-05-14T11:16:46.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff01e000640 1 -- 172.21.15.116:0/3550970172 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080ab9d0 con 0x7ff0200d1670 2024-05-14T11:16:46.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0200d1670 msgr2=0x7ff0200d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:46.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0200d1670 0x7ff0200d1a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0080097c0 tx=0x7ff00801c410 comp rx=0 tx=0).stop 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- 172.21.15.116:0/3550970172 shutdown_connections 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0200d1670 0x7ff0200d1a80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 --2- 172.21.15.116:0/3550970172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff02015db30 0x7ff0200aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- 172.21.15.116:0/3550970172 >> 172.21.15.116:0/3550970172 conn(0x7ff0200b4620 msgr2=0x7ff0200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- 172.21.15.116:0/3550970172 shutdown_connections 2024-05-14T11:16:46.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- 172.21.15.116:0/3550970172 wait complete. 2024-05-14T11:16:46.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 Processor -- start 2024-05-14T11:16:46.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.916+0000 7ff025800640 1 -- start start 2024-05-14T11:16:46.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff02015db30 0x7ff02014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 0x7ff020149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff02015df40 con 0x7ff0200d1670 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0200aefe0 con 0x7ff02015db30 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff02016af80 con 0x7ff020161600 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58800/0 (socket says 172.21.15.116:58800) 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 -- 172.21.15.116:0/1352432162 learned_addr learned my addr 172.21.15.116:0/1352432162 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff016000640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff02015db30 0x7ff02014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01f400640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 0x7ff020149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 -- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 msgr2=0x7ff020149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 0x7ff020149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 -- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff02015db30 msgr2=0x7ff02014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff02015db30 0x7ff02014b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:46.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 -- 172.21.15.116:0/1352432162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0201607a0 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01f400640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 0x7ff020149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff01ea00640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e9b0 tx=0x7ff01000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010017070 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01000ca00 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01000cdd0 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02014e7b0 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02014dc80 con 0x7ff0200d1670 2024-05-14T11:16:46.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.920+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02006a9c0 con 0x7ff0200d1670 2024-05-14T11:16:46.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.924+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff01001a070 con 0x7ff0200d1670 2024-05-14T11:16:46.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.924+0000 7ff017400640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 0x7feffc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:46.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.924+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff01008f4d0 con 0x7ff0200d1670 2024-05-14T11:16:46.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.924+0000 7ff016000640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 0x7feffc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:46.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.924+0000 7ff016000640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 0x7feffc071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:46.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:46.928+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff01005c070 con 0x7ff0200d1670 2024-05-14T11:16:47.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0200aea30 con 0x7ff0200d1670 2024-05-14T11:16:47.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff017400640 1 -- 172.21.15.116:0/1352432162 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff01005b970 con 0x7ff0200d1670 2024-05-14T11:16:47.126 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 msgr2=0x7feffc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 0x7feffc071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).stop 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 msgr2=0x7ff020160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e9b0 tx=0x7ff01000ee70 comp rx=0 tx=0).stop 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 shutdown_connections 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feffc06f430 0x7feffc071940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff02015db30 0x7ff02014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff020161600 0x7ff020149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:47.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 --2- 172.21.15.116:0/1352432162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0200d1670 0x7ff020160260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:47.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 >> 172.21.15.116:0/1352432162 conn(0x7ff0200b4620 msgr2=0x7ff0200b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:47.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 shutdown_connections 2024-05-14T11:16:47.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:47.124+0000 7ff025800640 1 -- 172.21.15.116:0/1352432162 wait complete. 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:47.147 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:16:47.148 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:16:47.148 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:16:47.148 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-14T11:16:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:47.148 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link 2024-05-14T11:16:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:47.160 INFO:teuthology.orchestra.run.smithi149.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:16:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 02:42:40:78:73:f9 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:16:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:47.160 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add name ceph-brx type bridge 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr flush dev ceph-brx 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set ceph-brx up 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T11:16:47.161 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:47.254 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:47.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:47.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:47.324 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:16:47.342 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-05-14T11:16:47.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:47.345 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-05-14T11:16:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-05-14T11:16:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:16:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:16:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:16:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:47.422 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:16:47.423 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:47.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:47.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:47.555 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-05-14T11:16:47.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:47.564 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-05-14T11:16:47.614 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:47.614 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:47.614 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:47.614 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-14T11:16:47.614 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:47.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:47.733 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:47.740 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-14T11:16:47.740 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:47.786 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:47.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:47.901 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:47.901 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:47.901 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.0 up 2024-05-14T11:16:47.901 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T11:16:47.901 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:47.950 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:47.981 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:47.988 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-14T11:16:47.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:47.988 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:47.998 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-14T11:16:47.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:47.998 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:48.054 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:48.122 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:48.171 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:48.172 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:48.173 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:48.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:48.174 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:48.226 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:16:48.226 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-14T11:16:48.272 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:48.293 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:48.343 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:48.343 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:48.344 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:48.345 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:48.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:48.346 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:48.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:2024-05-14T11:16:48.379+0000 7f5d4d68e580 -1 init, newargv = 0x7f5d34005ec0 newargc=15 2024-05-14T11:16:48.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[33289]: starting ceph client 2024-05-14T11:16:48.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[33289]: starting fuse 2024-05-14T11:16:49.491 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:49.512 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:49.563 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:49.563 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:49.565 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:49.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:49.566 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:49.618 INFO:teuthology.orchestra.run.smithi149.stdout:44 2024-05-14T11:16:49.618 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> import glob 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> import re 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> import os 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> import subprocess 2024-05-14T11:16:49.618 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> def _find_admin_socket(client_name): 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> files = glob.glob(asok_path) 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> # Given a non-glob path, it better be there 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> if "*" not in asok_path: 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> assert(len(files) == 1) 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> return files[0] 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> for f in files: 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> contents = proc_f.read() 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> if mountpoint in contents: 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> return f 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:16:49.619 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:49.620 DEBUG:teuthology.orchestra.run.smithi149:> print(_find_admin_socket("client.1")) 2024-05-14T11:16:49.620 DEBUG:teuthology.orchestra.run.smithi149:> ' 2024-05-14T11:16:49.743 INFO:teuthology.orchestra.run.smithi149.stdout:/var/run/ceph/ceph-client.1.33289.asok 2024-05-14T11:16:49.746 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:49.754 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33289.asok 2024-05-14T11:16:49.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:49.754 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33289.asok status 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "metadata": { 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "entity_id": "1", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "hostname": "smithi149", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "pid": "33289", 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: "root": "/" 2024-05-14T11:16:49.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_count": 0, 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_pinned_count": 0, 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "id": 4720, 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "inst": { 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "name": { 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "client", 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "num": 4720 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-05-14T11:16:49.937 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 496606771 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 496606771 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "inst_str": "client.4720 192.168.0.1:0/496606771", 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "addr_str": "192.168.0.1:0/496606771", 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "inode_count": 1, 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_epoch": 9, 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch": 24, 2024-05-14T11:16:49.938 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:16:49.939 INFO:teuthology.orchestra.run.smithi149.stdout: "blocklisted": false, 2024-05-14T11:16:49.939 INFO:teuthology.orchestra.run.smithi149.stdout: "fs_name": "cephfs" 2024-05-14T11:16:49.939 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-14T11:16:49.952 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:16:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 Processor -- start 2024-05-14T11:16:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 -- start start 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300d1c20 0x7f9d300ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d300b40c0 0x7f9d300b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d300b4a10 con 0x7f9d300cdf80 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9d300b3060 con 0x7f9d300b40c0 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d36e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300b3300 con 0x7f9d300d1c20 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d2fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d300cdf80 0x7f9d300d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58964/0 (socket says 172.21.15.116:58964) 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d2fe00640 1 -- 172.21.15.116:0/296947706 learned_addr learned my addr 172.21.15.116:0/296947706 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d34e00640 1 --2- 172.21.15.116:0/296947706 >> v2:172.21.15.116:3300/0 conn(0x7f9d300b40c0 0x7f9d300b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d2f400640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300d1c20 0x7f9d300ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d34e00640 1 -- 172.21.15.116:0/296947706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300d1c20 msgr2=0x7f9d300ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d34e00640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300d1c20 0x7f9d300ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d34e00640 1 -- 172.21.15.116:0/296947706 >> v1:172.21.15.116:6789/0 conn(0x7f9d300cdf80 legacy=0x7f9d300d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d34e00640 1 -- 172.21.15.116:0/296947706 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300b35e0 con 0x7f9d300b40c0 2024-05-14T11:16:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.108+0000 7f9d2ea00640 1 -- 172.21.15.116:0/296947706 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823038285 0 0) 0x7f9d300b4a10 con 0x7f9d300cdf80 2024-05-14T11:16:50.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d34e00640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300b40c0 0x7f9d300b44d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d280097c0 tx=0x7f9d2801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d230781b90bbfb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:50.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d2ea00640 1 -- 172.21.15.116:0/296947706 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a7070 con 0x7f9d300b40c0 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d2ea00640 1 -- 172.21.15.116:0/296947706 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d280b0070 con 0x7f9d300b40c0 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d2ea00640 1 -- 172.21.15.116:0/296947706 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280ab890 con 0x7f9d300b40c0 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 -- 172.21.15.116:0/296947706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300b40c0 msgr2=0x7f9d300b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300b40c0 0x7f9d300b44d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d280097c0 tx=0x7f9d2801c470 comp rx=0 tx=0).stop 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 -- 172.21.15.116:0/296947706 shutdown_connections 2024-05-14T11:16:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300b40c0 0x7f9d300b44d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 --2- 172.21.15.116:0/296947706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300d1c20 0x7f9d300ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 -- 172.21.15.116:0/296947706 >> 172.21.15.116:0/296947706 conn(0x7f9d300d0770 msgr2=0x7f9d300d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 -- 172.21.15.116:0/296947706 shutdown_connections 2024-05-14T11:16:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.112+0000 7f9d36e00640 1 -- 172.21.15.116:0/296947706 wait complete. 2024-05-14T11:16:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 Processor -- start 2024-05-14T11:16:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 -- start start 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 0x7f9d301574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 0x7f9d3015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3006ff40 con 0x7f9d300cdf80 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3006fdc0 con 0x7f9d3015d970 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d36e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3016cd60 con 0x7f9d300b40c0 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44554/0 (socket says 172.21.15.116:44554) 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d2f400640 1 -- 172.21.15.116:0/524709320 learned_addr learned my addr 172.21.15.116:0/524709320 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d2fe00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 0x7f9d301574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.116+0000 7f9d34e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 0x7f9d3015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 -- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 msgr2=0x7f9d301574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 0x7f9d301574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 -- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 msgr2=0x7f9d3015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 0x7f9d3015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 -- 172.21.15.116:0/524709320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30154da0 con 0x7f9d300cdf80 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2fe00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 0x7f9d301574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d34e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 0x7f9d3015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2f400640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00d650 tx=0x7f9d1c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00b660 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1c019070 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3015e2c0 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00ba20 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d30157ed0 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3006a9c0 con 0x7f9d300cdf80 2024-05-14T11:16:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d1c00bbc0 con 0x7f9d300cdf80 2024-05-14T11:16:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 0x7f9d10071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9d1c08f9e0 con 0x7f9d300cdf80 2024-05-14T11:16:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.120+0000 7f9d2fe00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 0x7f9d10071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.124+0000 7f9d2fe00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 0x7f9d10071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:50.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.128+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9d3015e2c0 con 0x7f9d300cdf80 2024-05-14T11:16:50.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9d300b9590 con 0x7f9d300cdf80 2024-05-14T11:16:50.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d27e00640 1 -- 172.21.15.116:0/524709320 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f9d1c05c580 con 0x7f9d300cdf80 2024-05-14T11:16:50.323 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 msgr2=0x7f9d10071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 0x7f9d10071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20005b80 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 msgr2=0x7f9d30154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00d650 tx=0x7f9d1c00db10 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 shutdown_connections 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9d1006f430 0x7f9d10071940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d3015d970 0x7f9d3015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300cdf80 0x7f9d30154860 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 --2- 172.21.15.116:0/524709320 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d300b40c0 0x7f9d301574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 >> 172.21.15.116:0/524709320 conn(0x7f9d300d0770 msgr2=0x7f9d3006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 shutdown_connections 2024-05-14T11:16:50.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:50.320+0000 7f9d36e00640 1 -- 172.21.15.116:0/524709320 wait complete. 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:16:50.344 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-14T11:16:50.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.344 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-05-14T11:16:50.353 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:50.353 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:50.353 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:50.353 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.353 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link 2024-05-14T11:16:50.354 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 02:42:40:78:73:f9 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:16:50.355 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.356 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:16:50.416 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-05-14T11:16:50.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.418 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-05-14T11:16:50.477 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-05-14T11:16:50.477 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:16:50.477 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:16:50.477 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:16:50.478 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:16:50.478 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:50.478 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:50.478 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:16:50.478 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:16:50.478 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:16:50.479 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:50.563 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:50.604 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:50.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.611 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-05-14T11:16:50.620 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-14T11:16:50.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.621 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-05-14T11:16:50.669 INFO:teuthology.orchestra.run.smithi149.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-14T11:16:50.670 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:50.670 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:50.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:50.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-14T11:16:50.670 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:50.755 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:50.785 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:50.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.792 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether e2:84:75:11:4f:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::e084:75ff:fe11:4f70/64 scope link 2024-05-14T11:16:50.813 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:50.815 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-14T11:16:50.877 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:16:50.877 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:16:50.877 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:16:50.877 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether e2:84:75:11:4f:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::e084:75ff:fe11:4f70/64 scope link 2024-05-14T11:16:50.878 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:16:50.879 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-14T11:16:50.879 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-14T11:16:50.880 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:50.968 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:51.052 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:51.059 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:16:51.059 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-05-14T11:16:51.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.1 up 2024-05-14T11:16:51.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.1 master ceph-brx 2024-05-14T11:16:51.059 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-05-14T11:16:51.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:16:51.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:51.142 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-14T11:16:51.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:51.142 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:51.198 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-14T11:16:51.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:16:51.198 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:51.250 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:51.309 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:51.363 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:51.363 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:51.364 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:51.366 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:51.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:51.366 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:51.422 INFO:teuthology.orchestra.run.smithi149.stdout:44 2024-05-14T11:16:51.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-05-14T11:16:51.422 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-14T11:16:51.473 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:51.492 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:51.543 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:51.543 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:51.543 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:51.543 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:51.543 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:51.544 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:51.545 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:51.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:51.546 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:51.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:2024-05-14T11:16:51.575+0000 7f6a2a4cc580 -1 init, newargv = 0x7f6a1c01b2f0 newargc=15 2024-05-14T11:16:51.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:ceph-fuse[33408]: starting ceph client 2024-05-14T11:16:51.585 INFO:teuthology.orchestra.run.smithi149.stdout:44 2024-05-14T11:16:51.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:ceph-fuse[33408]: starting fuse 2024-05-14T11:16:52.687 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-05-14T11:16:52.708 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-05-14T11:16:52.759 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/run/lock 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/fs/nfsd 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:16:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:/run/rpc_pipefs 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:16:52.761 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:52.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:52.762 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-05-14T11:16:52.813 INFO:teuthology.orchestra.run.smithi149.stdout:44 2024-05-14T11:16:52.814 INFO:teuthology.orchestra.run.smithi149.stdout:45 2024-05-14T11:16:52.814 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> import glob 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> import re 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> import os 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> import subprocess 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> def _find_admin_socket(client_name): 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> files = glob.glob(asok_path) 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-14T11:16:52.814 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> # Given a non-glob path, it better be there 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> if "*" not in asok_path: 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> assert(len(files) == 1) 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> return files[0] 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> for f in files: 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> contents = proc_f.read() 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> if mountpoint in contents: 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> return f 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> 2024-05-14T11:16:52.815 DEBUG:teuthology.orchestra.run.smithi149:> print(_find_admin_socket("client.3")) 2024-05-14T11:16:52.816 DEBUG:teuthology.orchestra.run.smithi149:> ' 2024-05-14T11:16:52.933 INFO:teuthology.orchestra.run.smithi149.stdout:/var/run/ceph/ceph-client.3.33408.asok 2024-05-14T11:16:52.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-14T11:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:16:52.945 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33408.asok 2024-05-14T11:16:52.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:52.946 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33408.asok status 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "metadata": { 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "entity_id": "3", 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "hostname": "smithi149", 2024-05-14T11:16:53.122 INFO:teuthology.orchestra.run.smithi149.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "pid": "33408", 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "root": "/" 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_count": 0, 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_pinned_count": 0, 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "id": 4732, 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "inst": { 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "name": { 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "client", 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "num": 4732 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:53.123 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.2:0", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 2071737367 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.2:0", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 2071737367 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "inst_str": "client.4732 192.168.0.2:0/2071737367", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "addr_str": "192.168.0.2:0/2071737367", 2024-05-14T11:16:53.124 INFO:teuthology.orchestra.run.smithi149.stdout: "inode_count": 1, 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_epoch": 9, 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch": 24, 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout: "blocklisted": false, 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout: "fs_name": "cephfs" 2024-05-14T11:16:53.125 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-14T11:16:53.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.138 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:16:53.138 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:53.148 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:16:53.149 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:53.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:53.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.214 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:16:53.215 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:53.266 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:16:53.266 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:53.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.266 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:53.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.329 DEBUG:teuthology.orchestra.run.smithi149:> stat --file-system '--printf=%T 2024-05-14T11:16:53.330 DEBUG:teuthology.orchestra.run.smithi149:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:53.339 INFO:teuthology.orchestra.run.smithi149.stdout:fuseblk 2024-05-14T11:16:53.339 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:53.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.339 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:53.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.409 DEBUG:teuthology.orchestra.run.smithi149:> stat --file-system '--printf=%T 2024-05-14T11:16:53.410 DEBUG:teuthology.orchestra.run.smithi149:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:53.463 INFO:teuthology.orchestra.run.smithi149.stdout:fuseblk 2024-05-14T11:16:53.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:53.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:53.463 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-14T11:16:53.525 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-14T11:16:53.564 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:16:53.564 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:16:53.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 Processor -- start 2024-05-14T11:16:53.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 -- start start 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd6e00d1670 0x7fd6e00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e006ff20 con 0x7fd6e015db30 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e00d1fc0 con 0x7fd6e006c700 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e7e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e00ae100 con 0x7fd6e00d1670 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e4a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd6e00d1670 0x7fd6e00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e5400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd6e015db30 0x7fd6e00aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58966/0 (socket says 172.21.15.116:58966) 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.716+0000 7fd6e5400640 1 -- 172.21.15.116:0/421631621 learned_addr learned my addr 172.21.15.116:0/421631621 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6e4a00640 1 -- 172.21.15.116:0/421631621 >> v1:172.21.15.149:6789/0 conn(0x7fd6e006c700 legacy=0x7fd6e006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6e4a00640 1 -- 172.21.15.116:0/421631621 >> v1:172.21.15.116:6789/0 conn(0x7fd6e015db30 legacy=0x7fd6e00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6e4a00640 1 -- 172.21.15.116:0/421631621 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e0070470 con 0x7fd6e00d1670 2024-05-14T11:16:53.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6e4a00640 1 --2- 172.21.15.116:0/421631621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e00d1a80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae280 tx=0x7fd6cc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a8065b3beb70f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:53.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6dfe00640 1 -- 172.21.15.116:0/421631621 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0b0070 con 0x7fd6e00d1670 2024-05-14T11:16:53.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6dfe00640 1 -- 172.21.15.116:0/421631621 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc0ab600 con 0x7fd6e00d1670 2024-05-14T11:16:53.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.720+0000 7fd6dfe00640 1 -- 172.21.15.116:0/421631621 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0ab9d0 con 0x7fd6e00d1670 2024-05-14T11:16:53.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- 172.21.15.116:0/421631621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 msgr2=0x7fd6e00d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/421631621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e00d1a80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae280 tx=0x7fd6cc01c380 comp rx=0 tx=0).stop 2024-05-14T11:16:53.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- 172.21.15.116:0/421631621 shutdown_connections 2024-05-14T11:16:53.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/421631621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e00d1a80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- 172.21.15.116:0/421631621 >> 172.21.15.116:0/421631621 conn(0x7fd6e00b4620 msgr2=0x7fd6e00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:53.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- 172.21.15.116:0/421631621 shutdown_connections 2024-05-14T11:16:53.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- 172.21.15.116:0/421631621 wait complete. 2024-05-14T11:16:53.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 Processor -- start 2024-05-14T11:16:53.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.724+0000 7fd6e7e00640 1 -- start start 2024-05-14T11:16:53.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6e006c700 0x7fd6e014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:53.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e015db30 0x7fd6e0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e016bf90 con 0x7fd6e00d1670 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e015df40 con 0x7fd6e015db30 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e016c110 con 0x7fd6e006c700 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44578/0 (socket says 172.21.15.116:44578) 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 -- 172.21.15.116:0/595424025 learned_addr learned my addr 172.21.15.116:0/595424025 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e5400640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6e006c700 0x7fd6e014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e5e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e015db30 0x7fd6e0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 -- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6e006c700 msgr2=0x7fd6e014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6e006c700 0x7fd6e014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 -- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e015db30 msgr2=0x7fd6e0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e015db30 0x7fd6e0159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 -- 172.21.15.116:0/595424025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01556f0 con 0x7fd6e00d1670 2024-05-14T11:16:53.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e4a00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0ae040 tx=0x7fd6cc019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0b8020 con 0x7fd6e00d1670 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc0b4630 con 0x7fd6e00d1670 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01559d0 con 0x7fd6e00d1670 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0b0040 con 0x7fd6e00d1670 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e0155f90 con 0x7fd6e00d1670 2024-05-14T11:16:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e006a9c0 con 0x7fd6e00d1670 2024-05-14T11:16:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.728+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6cc0bb020 con 0x7fd6e00d1670 2024-05-14T11:16:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.732+0000 7fd6dd600640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 0x7fd6c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.732+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fd6cc129300 con 0x7fd6e00d1670 2024-05-14T11:16:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.732+0000 7fd6e5400640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 0x7fd6c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.732+0000 7fd6e5400640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 0x7fd6c0071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000f70 tx=0x7fd6d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:53.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.736+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd6cc0c2030 con 0x7fd6e00d1670 2024-05-14T11:16:53.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.892+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6e00b3bf0 con 0x7fd6e00d1670 2024-05-14T11:16:53.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6dd600640 1 -- 172.21.15.116:0/595424025 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18969 (secure 0 0 0) 0x7fd6cc016090 con 0x7fd6e00d1670 2024-05-14T11:16:53.897 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:16:53.897 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:53.384022+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 msgr2=0x7fd6c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 0x7fd6c0071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000f70 tx=0x7fd6d4009040 comp rx=0 tx=0).stop 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 msgr2=0x7fd6e01594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0ae040 tx=0x7fd6cc019120 comp rx=0 tx=0).stop 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 shutdown_connections 2024-05-14T11:16:53.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6c006f400 0x7fd6c0071910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e015db30 0x7fd6e0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00d1670 0x7fd6e01594c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 --2- 172.21.15.116:0/595424025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6e006c700 0x7fd6e014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 >> 172.21.15.116:0/595424025 conn(0x7fd6e00b4620 msgr2=0x7fd6e00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 shutdown_connections 2024-05-14T11:16:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:53.896+0000 7fd6e7e00640 1 -- 172.21.15.116:0/595424025 wait complete. 2024-05-14T11:16:53.919 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T11:16:09.523112+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T11:16:10.584877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-14T11:16:14.274286+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-14T11:16:53.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:16:54.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.068+0000 7ff3a9c00640 1 Processor -- start 2024-05-14T11:16:54.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.068+0000 7ff3a9c00640 1 -- start start 2024-05-14T11:16:54.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a9c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3a40cd1c0 0x7ff3a40cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40d1730 0x7ff3a40d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a9c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a406f0a0 con 0x7ff3a4160670 2024-05-14T11:16:54.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a9c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a40cbb60 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a9c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a40d2080 con 0x7ff3a40d1730 2024-05-14T11:16:54.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3a40cd1c0 0x7ff3a40cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff3a4160670 0x7ff3a4160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58978/0 (socket says 172.21.15.116:58978) 2024-05-14T11:16:54.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a3400640 1 -- 172.21.15.116:0/1841988274 learned_addr learned my addr 172.21.15.116:0/1841988274 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:54.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff39a000640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40d1730 0x7ff3a40d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2000640 1 -- 172.21.15.116:0/1841988274 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170547160 0 0) 0x7ff3a406f0a0 con 0x7ff3a4160670 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2000640 1 -- 172.21.15.116:0/1841988274 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff388003530 con 0x7ff3a4160670 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2a00640 1 -- 172.21.15.116:0/1841988274 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40d1730 msgr2=0x7ff3a40d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2a00640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40d1730 0x7ff3a40d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2a00640 1 -- 172.21.15.116:0/1841988274 >> v1:172.21.15.116:6789/0 conn(0x7ff3a4160670 legacy=0x7ff3a4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.072+0000 7ff3a2a00640 1 -- 172.21.15.116:0/1841988274 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a406f420 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a2a00640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a40cd1c0 0x7ff3a40cd5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3940097f0 tx=0x7ff39401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53303e51bbd755a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a2000640 1 -- 172.21.15.116:0/1841988274 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3940ab640 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a2000640 1 -- 172.21.15.116:0/1841988274 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3940b0070 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a2000640 1 -- 172.21.15.116:0/1841988274 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3940aba00 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1841988274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a40cd1c0 msgr2=0x7ff3a40cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a40cd1c0 0x7ff3a40cd5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3940097f0 tx=0x7ff39401c440 comp rx=0 tx=0).stop 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1841988274 shutdown_connections 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40d1730 0x7ff3a40d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1841988274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a40cd1c0 0x7ff3a40cd5d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1841988274 >> 172.21.15.116:0/1841988274 conn(0x7ff3a40b4720 msgr2=0x7ff3a40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1841988274 shutdown_connections 2024-05-14T11:16:54.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1841988274 wait complete. 2024-05-14T11:16:54.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 Processor -- start 2024-05-14T11:16:54.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.076+0000 7ff3a9c00640 1 -- start start 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40cd1c0 0x7ff3a415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3a40d1730 0x7ff3a4155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a415cf10 con 0x7ff3a4160670 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a406ffa0 con 0x7ff3a40d1730 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a406d710 con 0x7ff3a40cd1c0 2024-05-14T11:16:54.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44596/0 (socket says 172.21.15.116:44596) 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 -- 172.21.15.116:0/1453349579 learned_addr learned my addr 172.21.15.116:0/1453349579 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a2a00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40cd1c0 0x7ff3a415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff39a000640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3a40d1730 0x7ff3a4155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 -- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40cd1c0 msgr2=0x7ff3a415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40cd1c0 0x7ff3a415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 -- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3a40d1730 msgr2=0x7ff3a4155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3a40d1730 0x7ff3a4155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 -- 172.21.15.116:0/1453349579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a41559c0 con 0x7ff3a4160670 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a3400640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00ca70 tx=0x7ff39c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00ab70 con 0x7ff3a4160670 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff39c00ad10 con 0x7ff3a4160670 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a4158890 con 0x7ff3a4160670 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c012620 con 0x7ff3a4160670 2024-05-14T11:16:54.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a415a8b0 con 0x7ff3a4160670 2024-05-14T11:16:54.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.080+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a406a9c0 con 0x7ff3a4160670 2024-05-14T11:16:54.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.084+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff39c0127c0 con 0x7ff3a4160670 2024-05-14T11:16:54.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.084+0000 7ff39b400640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 0x7ff380071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.084+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7ff39c08fbd0 con 0x7ff3a4160670 2024-05-14T11:16:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.084+0000 7ff3a2a00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 0x7ff380071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.084+0000 7ff3a2a00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 0x7ff380071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3940af000 tx=0x7ff3940a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.088+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff39c05bfc0 con 0x7ff3a4160670 2024-05-14T11:16:54.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3a406eb80 con 0x7ff3a4160670 2024-05-14T11:16:54.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff39b400640 1 -- 172.21.15.116:0/1453349579 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff39c0602c0 con 0x7ff3a4160670 2024-05-14T11:16:54.250 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 msgr2=0x7ff380071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 0x7ff380071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3940af000 tx=0x7ff3940a7000 comp rx=0 tx=0).stop 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 msgr2=0x7ff3a40c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00ca70 tx=0x7ff39c00cf30 comp rx=0 tx=0).stop 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 shutdown_connections 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff38006f400 0x7ff380071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3a40d1730 0x7ff3a4155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3a4160670 0x7ff3a40c6800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 --2- 172.21.15.116:0/1453349579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff3a40cd1c0 0x7ff3a415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 >> 172.21.15.116:0/1453349579 conn(0x7ff3a40b4720 msgr2=0x7ff3a4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 shutdown_connections 2024-05-14T11:16:54.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.248+0000 7ff3a9c00640 1 -- 172.21.15.116:0/1453349579 wait complete. 2024-05-14T11:16:54.271 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T11:16:54.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 Processor -- start 2024-05-14T11:16:54.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 -- start start 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f54f80cfa70 0x7f54f806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f80cbb20 con 0x7f54f815cab0 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f80cbdc0 con 0x7f54f80cd6d0 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.420+0000 7f54f7200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f54f80cc9d0 con 0x7f54f80cfa70 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54ed400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f54f815cab0 0x7f54f80cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58994/0 (socket says 172.21.15.116:58994) 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54ed400640 1 -- 172.21.15.116:0/1331465233 learned_addr learned my addr 172.21.15.116:0/1331465233 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:54.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f6800640 1 --2- 172.21.15.116:0/1331465233 >> v2:172.21.15.149:3300/0 conn(0x7f54f80cfa70 0x7f54f806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728179703 0 0) 0x7f54f80cbb20 con 0x7f54f815cab0 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54dc003530 con 0x7f54f815cab0 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254030500 0 0) 0x7f54f80cbdc0 con 0x7f54f80cd6d0 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54f80cbb20 con 0x7f54f80cd6d0 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f6800640 1 -- 172.21.15.116:0/1331465233 >> v1:172.21.15.149:6789/0 conn(0x7f54f80cd6d0 legacy=0x7f54f80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f6800640 1 -- 172.21.15.116:0/1331465233 >> v1:172.21.15.116:6789/0 conn(0x7f54f815cab0 legacy=0x7f54f80cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f6800640 1 -- 172.21.15.116:0/1331465233 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f80cc0a0 con 0x7f54f80cfa70 2024-05-14T11:16:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.424+0000 7f54f6800640 1 --2- 172.21.15.116:0/1331465233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cfa70 0x7f54f806eb00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097c0 tx=0x7f54f001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=415384b39f0d7ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b0070 con 0x7f54f80cfa70 2024-05-14T11:16:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f00ab600 con 0x7f54f80cfa70 2024-05-14T11:16:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f5400640 1 -- 172.21.15.116:0/1331465233 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00ab9d0 con 0x7f54f80cfa70 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- 172.21.15.116:0/1331465233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cfa70 msgr2=0x7f54f806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 --2- 172.21.15.116:0/1331465233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cfa70 0x7f54f806eb00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097c0 tx=0x7f54f001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- 172.21.15.116:0/1331465233 shutdown_connections 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 --2- 172.21.15.116:0/1331465233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cfa70 0x7f54f806eb00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- 172.21.15.116:0/1331465233 >> 172.21.15.116:0/1331465233 conn(0x7f54f80d12f0 msgr2=0x7f54f80d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- 172.21.15.116:0/1331465233 shutdown_connections 2024-05-14T11:16:54.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- 172.21.15.116:0/1331465233 wait complete. 2024-05-14T11:16:54.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 Processor -- start 2024-05-14T11:16:54.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.428+0000 7f54f7200640 1 -- start start 2024-05-14T11:16:54.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cd6d0 0x7f54f809c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54f80c8680 0x7f54f80739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f54f806f450 con 0x7f54f815cab0 2024-05-14T11:16:54.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54f806de40 con 0x7f54f80cd6d0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f54f806f5d0 con 0x7f54f80c8680 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44604/0 (socket says 172.21.15.116:44604) 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 -- 172.21.15.116:0/1782896189 learned_addr learned my addr 172.21.15.116:0/1782896189 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f5e00640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cd6d0 0x7f54f809c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f6800640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54f80c8680 0x7f54f80739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 -- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54f80c8680 msgr2=0x7f54f80739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54f80c8680 0x7f54f80739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 -- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cd6d0 msgr2=0x7f54f809c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cd6d0 0x7f54f809c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 -- 172.21.15.116:0/1782896189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f80c4f10 con 0x7f54f815cab0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54ed400640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54e000cb50 tx=0x7f54e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e000ad40 con 0x7f54f815cab0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0012400 con 0x7f54f815cab0 2024-05-14T11:16:54.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f80c51f0 con 0x7f54f815cab0 2024-05-14T11:16:54.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e00127d0 con 0x7f54f815cab0 2024-05-14T11:16:54.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f80c57b0 con 0x7f54f815cab0 2024-05-14T11:16:54.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.432+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f806a9c0 con 0x7f54f815cab0 2024-05-14T11:16:54.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.436+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54e0012970 con 0x7f54f815cab0 2024-05-14T11:16:54.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.436+0000 7f54eea00640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 0x7f54d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.436+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f54e008fcd0 con 0x7f54f815cab0 2024-05-14T11:16:54.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.436+0000 7f54f5e00640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 0x7f54d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.436+0000 7f54f5e00640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 0x7f54d4071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c10 tx=0x7f54e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.440+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f54e005c0c0 con 0x7f54f815cab0 2024-05-14T11:16:54.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f54f80afc40 con 0x7f54f815cab0 2024-05-14T11:16:54.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54eea00640 1 -- 172.21.15.116:0/1782896189 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f54e00603c0 con 0x7f54f815cab0 2024-05-14T11:16:54.603 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 msgr2=0x7f54d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 0x7f54d4071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c10 tx=0x7f54e8009040 comp rx=0 tx=0).stop 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 msgr2=0x7f54f80cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54e000cb50 tx=0x7f54e000a430 comp rx=0 tx=0).stop 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 shutdown_connections 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f54d406f400 0x7f54d4071910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54f80c8680 0x7f54f80739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54f815cab0 0x7f54f80cae50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 --2- 172.21.15.116:0/1782896189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f80cd6d0 0x7f54f809c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 >> 172.21.15.116:0/1782896189 conn(0x7f54f80d12f0 msgr2=0x7f54f815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 shutdown_connections 2024-05-14T11:16:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.600+0000 7f54f7200640 1 -- 172.21.15.116:0/1782896189 wait complete. 2024-05-14T11:16:54.624 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T11:16:54.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 Processor -- start 2024-05-14T11:16:54.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 -- start start 2024-05-14T11:16:54.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c15c1d0 0x7f473c0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0cdd40 0x7f473c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f473c15fc50 0x7f473c160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f473c06d790 con 0x7f473c15c1d0 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f473c0ce690 con 0x7f473c15fc50 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4743c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f473c06cc10 con 0x7f473c0cdd40 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c15c1d0 0x7f473c0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f473c15c1d0 0x7f473c0d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44610/0 (socket says 172.21.15.116:44610) 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 -- 172.21.15.116:0/2120778796 learned_addr learned my addr 172.21.15.116:0/2120778796 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4733e00640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0cdd40 0x7f473c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741c00640 1 --2- 172.21.15.116:0/2120778796 >> v2:172.21.15.149:3300/0 conn(0x7f473c15fc50 0x7f473c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 -- 172.21.15.116:0/2120778796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0cdd40 msgr2=0x7f473c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0cdd40 0x7f473c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 -- 172.21.15.116:0/2120778796 >> v2:172.21.15.149:3300/0 conn(0x7f473c15fc50 msgr2=0x7f473c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 --2- 172.21.15.116:0/2120778796 >> v2:172.21.15.149:3300/0 conn(0x7f473c15fc50 0x7f473c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.772+0000 7f4741200640 1 -- 172.21.15.116:0/2120778796 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c06cef0 con 0x7f473c15c1d0 2024-05-14T11:16:54.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4741200640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15c1d0 0x7f473c0d0c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f473c06cb20 tx=0x7f472c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fe6553d765026e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f473be00640 1 -- 172.21.15.116:0/2120778796 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0b4070 con 0x7f473c15c1d0 2024-05-14T11:16:54.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f473be00640 1 -- 172.21.15.116:0/2120778796 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c0a7970 con 0x7f473c15c1d0 2024-05-14T11:16:54.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f473be00640 1 -- 172.21.15.116:0/2120778796 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0a7d40 con 0x7f473c15c1d0 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 -- 172.21.15.116:0/2120778796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15c1d0 msgr2=0x7f473c0d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15c1d0 0x7f473c0d0c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f473c06cb20 tx=0x7f472c01eb60 comp rx=0 tx=0).stop 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 -- 172.21.15.116:0/2120778796 shutdown_connections 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 --2- 172.21.15.116:0/2120778796 >> v2:172.21.15.149:3300/0 conn(0x7f473c15fc50 0x7f473c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c0cdd40 0x7f473c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 --2- 172.21.15.116:0/2120778796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15c1d0 0x7f473c0d0c50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 -- 172.21.15.116:0/2120778796 >> 172.21.15.116:0/2120778796 conn(0x7f473c0b4720 msgr2=0x7f473c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 -- 172.21.15.116:0/2120778796 shutdown_connections 2024-05-14T11:16:54.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.776+0000 7f4743c00640 1 -- 172.21.15.116:0/2120778796 wait complete. 2024-05-14T11:16:54.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 Processor -- start 2024-05-14T11:16:54.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 -- start start 2024-05-14T11:16:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c0cdd40 0x7f473c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c15c1d0 0x7f473c099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0b0b70 con 0x7f473c15fc50 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f473c0b09f0 con 0x7f473c0cdd40 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4743c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f473c0cbd60 con 0x7f473c15c1d0 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44622/0 (socket says 172.21.15.116:44622) 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 -- 172.21.15.116:0/1932566550 learned_addr learned my addr 172.21.15.116:0/1932566550 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741200640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c0cdd40 0x7f473c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4733e00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c15c1d0 0x7f473c099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 -- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c15c1d0 msgr2=0x7f473c099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c15c1d0 0x7f473c099ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 -- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c0cdd40 msgr2=0x7f473c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c0cdd40 0x7f473c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.780+0000 7f4741c00640 1 -- 172.21.15.116:0/1932566550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c159950 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4741c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4734007d80 tx=0x7f473400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734014420 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4734019070 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c156ba0 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734014800 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1568c0 con 0x7f473c15fc50 2024-05-14T11:16:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c156340 con 0x7f473c15fc50 2024-05-14T11:16:54.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47340149a0 con 0x7f473c15fc50 2024-05-14T11:16:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 0x7f4718071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.784+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f473408fd80 con 0x7f473c15fc50 2024-05-14T11:16:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.788+0000 7f4741200640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 0x7f4718071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.788+0000 7f4741200640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 0x7f4718071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b1000 tx=0x7f472c0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.792+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f473c156340 con 0x7f473c15fc50 2024-05-14T11:16:54.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f473c069de0 con 0x7f473c15fc50 2024-05-14T11:16:54.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4739600640 1 -- 172.21.15.116:0/1932566550 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f473405c2a0 con 0x7f473c15fc50 2024-05-14T11:16:54.950 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 msgr2=0x7f4718071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 0x7f4718071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b1000 tx=0x7f472c0a9070 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 msgr2=0x7f473c073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4734007d80 tx=0x7f473400e4b0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 shutdown_connections 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f471806f430 0x7f4718071940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f473c15fc50 0x7f473c073840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f473c15c1d0 0x7f473c099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 --2- 172.21.15.116:0/1932566550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473c0cdd40 0x7f473c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 >> 172.21.15.116:0/1932566550 conn(0x7f473c0b4720 msgr2=0x7f473c0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 shutdown_connections 2024-05-14T11:16:54.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:54.948+0000 7f4743c00640 1 -- 172.21.15.116:0/1932566550 wait complete. 2024-05-14T11:16:54.971 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-14T11:16:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.116+0000 7fd7c2200640 1 Processor -- start 2024-05-14T11:16:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.116+0000 7fd7c2200640 1 -- start start 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7c2200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd7bc0d1e40 0x7fd7bc06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7c2200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7bc0cde70 0x7fd7bc1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7c2200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc06f860 con 0x7fd7bc06f450 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7c2200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7bc06db80 con 0x7fd7bc0cde70 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7c2200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd7bc06fae0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7bc0cde70 0x7fd7bc1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7bc0cde70 0x7fd7bc1675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44624/0 (socket says 172.21.15.116:44624) 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 -- 172.21.15.116:0/3750147959 learned_addr learned my addr 172.21.15.116:0/3750147959 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7b2a00640 1 --2- 172.21.15.116:0/3750147959 >> v2:172.21.15.149:3300/0 conn(0x7fd7bc0d1e40 0x7fd7bc06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 -- 172.21.15.116:0/3750147959 >> v2:172.21.15.149:3300/0 conn(0x7fd7bc0d1e40 msgr2=0x7fd7bc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 --2- 172.21.15.116:0/3750147959 >> v2:172.21.15.149:3300/0 conn(0x7fd7bc0d1e40 0x7fd7bc06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 -- 172.21.15.116:0/3750147959 >> v1:172.21.15.149:6789/0 conn(0x7fd7bc06f450 legacy=0x7fd7bc0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.120+0000 7fd7bbe00640 1 -- 172.21.15.116:0/3750147959 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc06fdc0 con 0x7fd7bc0cde70 2024-05-14T11:16:55.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7bbe00640 1 --2- 172.21.15.116:0/3750147959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc1675f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0d2430 tx=0x7fd7b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9795090714eaad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7baa00640 1 -- 172.21.15.116:0/3750147959 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40ab610 con 0x7fd7bc0cde70 2024-05-14T11:16:55.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7baa00640 1 -- 172.21.15.116:0/3750147959 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b40b0070 con 0x7fd7bc0cde70 2024-05-14T11:16:55.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7baa00640 1 -- 172.21.15.116:0/3750147959 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40ab9d0 con 0x7fd7bc0cde70 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 -- 172.21.15.116:0/3750147959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0cde70 msgr2=0x7fd7bc1675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 --2- 172.21.15.116:0/3750147959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc1675f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0d2430 tx=0x7fd7b401c520 comp rx=0 tx=0).stop 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 -- 172.21.15.116:0/3750147959 shutdown_connections 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 --2- 172.21.15.116:0/3750147959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc1675f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 --2- 172.21.15.116:0/3750147959 >> v2:172.21.15.149:3300/0 conn(0x7fd7bc0d1e40 0x7fd7bc06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 -- 172.21.15.116:0/3750147959 >> 172.21.15.116:0/3750147959 conn(0x7fd7bc0b4720 msgr2=0x7fd7bc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 -- 172.21.15.116:0/3750147959 shutdown_connections 2024-05-14T11:16:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 -- 172.21.15.116:0/3750147959 wait complete. 2024-05-14T11:16:55.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.124+0000 7fd7c2200640 1 Processor -- start 2024-05-14T11:16:55.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- start start 2024-05-14T11:16:55.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7bc06f450 0x7fd7bc156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc16a4d0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc16a350 con 0x7fd7bc0cde70 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc16a650 con 0x7fd7bc06f450 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44638/0 (socket says 172.21.15.116:44638) 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 -- 172.21.15.116:0/1229273964 learned_addr learned my addr 172.21.15.116:0/1229273964 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bb400640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7bc06f450 0x7fd7bc156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7b2a00640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 -- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7bc06f450 msgr2=0x7fd7bc156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7bc06f450 0x7fd7bc156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 -- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc0cde70 msgr2=0x7fd7bc0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc0c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 -- 172.21.15.116:0/1229273964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc155570 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7bbe00640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7b401a150 tx=0x7fd7b401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40be5f0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b40be790 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc155850 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40beb80 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc155e10 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.128+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc06a9c0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.132+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7b40b43f0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.132+0000 7fd7b3e00640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 0x7fd798071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.132+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fd7b412a230 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.132+0000 7fd7bb400640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 0x7fd798071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.132+0000 7fd7bb400640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 0x7fd798071940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0c83b0 tx=0x7fd7ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.136+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd7b40c4060 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd7bc069de0 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7b3e00640 1 -- 172.21.15.116:0/1229273964 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd7b40a8050 con 0x7fd7bc0d1e40 2024-05-14T11:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 msgr2=0x7fd798071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 0x7fd798071940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0c83b0 tx=0x7fd7ac009040 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 msgr2=0x7fd7bc0ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7b401a150 tx=0x7fd7b401ad60 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 shutdown_connections 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd79806f430 0x7fd798071940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc0cde70 0x7fd7bc0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7bc0d1e40 0x7fd7bc0ca780 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 --2- 172.21.15.116:0/1229273964 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7bc06f450 0x7fd7bc156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 >> 172.21.15.116:0/1229273964 conn(0x7fd7bc0b4720 msgr2=0x7fd7bc0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 shutdown_connections 2024-05-14T11:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.296+0000 7fd7c2200640 1 -- 172.21.15.116:0/1229273964 wait complete. 2024-05-14T11:16:55.320 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-14T11:16:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.464+0000 7fd583e00640 1 Processor -- start 2024-05-14T11:16:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.464+0000 7fd583e00640 1 -- start start 2024-05-14T11:16:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd583e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd57c0702e0 0x7fd57c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd583e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c06ee40 con 0x7fd57c0d1bb0 2024-05-14T11:16:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd583e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c0d1fc0 con 0x7fd57c15eb40 2024-05-14T11:16:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd583e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0aef20 con 0x7fd57c0702e0 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd580a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd57c0d1bb0 0x7fd57c0ae100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59008/0 (socket says 172.21.15.116:59008) 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd580a00640 1 -- 172.21.15.116:0/3572906925 learned_addr learned my addr 172.21.15.116:0/3572906925 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd581400640 1 --2- 172.21.15.116:0/3572906925 >> v2:172.21.15.116:3300/0 conn(0x7fd57c0702e0 0x7fd57c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd57be00640 1 -- 172.21.15.116:0/3572906925 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104908050 0 0) 0x7fd57c06ee40 con 0x7fd57c0d1bb0 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd581400640 1 -- 172.21.15.116:0/3572906925 >> v1:172.21.15.149:6789/0 conn(0x7fd57c15eb40 legacy=0x7fd57c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd581400640 1 -- 172.21.15.116:0/3572906925 >> v1:172.21.15.116:6789/0 conn(0x7fd57c0d1bb0 legacy=0x7fd57c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.468+0000 7fd581400640 1 -- 172.21.15.116:0/3572906925 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c06fe70 con 0x7fd57c0702e0 2024-05-14T11:16:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd581400640 1 --2- 172.21.15.116:0/3572906925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c0702e0 0x7fd57c0d1670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd570009d90 tx=0x7fd57001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad056599c6c543 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd57be00640 1 -- 172.21.15.116:0/3572906925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700b2070 con 0x7fd57c0702e0 2024-05-14T11:16:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd57be00640 1 -- 172.21.15.116:0/3572906925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5700a5a00 con 0x7fd57c0702e0 2024-05-14T11:16:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd57be00640 1 -- 172.21.15.116:0/3572906925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700a5dd0 con 0x7fd57c0702e0 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 -- 172.21.15.116:0/3572906925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c0702e0 msgr2=0x7fd57c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 --2- 172.21.15.116:0/3572906925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c0702e0 0x7fd57c0d1670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd570009d90 tx=0x7fd57001c940 comp rx=0 tx=0).stop 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 -- 172.21.15.116:0/3572906925 shutdown_connections 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 --2- 172.21.15.116:0/3572906925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c0702e0 0x7fd57c0d1670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 -- 172.21.15.116:0/3572906925 >> 172.21.15.116:0/3572906925 conn(0x7fd57c0b4620 msgr2=0x7fd57c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 -- 172.21.15.116:0/3572906925 shutdown_connections 2024-05-14T11:16:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.472+0000 7fd583e00640 1 -- 172.21.15.116:0/3572906925 wait complete. 2024-05-14T11:16:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 Processor -- start 2024-05-14T11:16:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- start start 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd57c0d1bb0 0x7fd57c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 0x7fd57c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c15ef50 con 0x7fd57c15eb40 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c06f950 con 0x7fd57c0702e0 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c15f0d0 con 0x7fd57c0d1bb0 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 0x7fd57c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36842/0 (socket says 172.21.15.116:36842) 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581400640 1 -- 172.21.15.116:0/4087543120 learned_addr learned my addr 172.21.15.116:0/4087543120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd580a00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd57c0d1bb0 0x7fd57c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 -- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd57c0d1bb0 msgr2=0x7fd57c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd57c0d1bb0 0x7fd57c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 -- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 msgr2=0x7fd57c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 -- 172.21.15.116:0/4087543120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c156720 con 0x7fd57c15eb40 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581400640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:55.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd581e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 0x7fd57c0d0150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c3f0 tx=0x7fd57400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574017070 con 0x7fd57c15eb40 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5740052d0 con 0x7fd57c15eb40 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574003820 con 0x7fd57c15eb40 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c0ce2b0 con 0x7fd57c15eb40 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.476+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c158240 con 0x7fd57c15eb40 2024-05-14T11:16:55.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c158490 con 0x7fd57c15eb40 2024-05-14T11:16:55.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd57401a070 con 0x7fd57c15eb40 2024-05-14T11:16:55.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd579600640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 0x7fd55c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:55.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fd57408e860 con 0x7fd57c15eb40 2024-05-14T11:16:55.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd581400640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 0x7fd55c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:55.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.480+0000 7fd581400640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 0x7fd55c071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd57001a570 tx=0x7fd5700a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.484+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd57c0d1fc0 con 0x7fd57c15eb40 2024-05-14T11:16:55.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:55.696+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fd57c0ce040 con 0x7fd57c15eb40 2024-05-14T11:16:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd579600640 1 -- 172.21.15.116:0/4087543120 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v65) ==== 145+0+0 (secure 0 0 0) 0x7fd574005480 con 0x7fd57c15eb40 2024-05-14T11:16:56.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 msgr2=0x7fd55c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 0x7fd55c071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd57001a570 tx=0x7fd5700a7070 comp rx=0 tx=0).stop 2024-05-14T11:16:56.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 msgr2=0x7fd57c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 0x7fd57c0d0150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c3f0 tx=0x7fd57400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 shutdown_connections 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd55c06f400 0x7fd55c071910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd57c15eb40 0x7fd57c0d0150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd57c0d1bb0 0x7fd57c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 --2- 172.21.15.116:0/4087543120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd57c0702e0 0x7fd57c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 >> 172.21.15.116:0/4087543120 conn(0x7fd57c0b4620 msgr2=0x7fd57c14aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 shutdown_connections 2024-05-14T11:16:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.404+0000 7fd583e00640 1 -- 172.21.15.116:0/4087543120 wait complete. 2024-05-14T11:16:56.428 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:16:56.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 Processor -- start 2024-05-14T11:16:56.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 -- start start 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f651815daf0 0x7f65180b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f6518161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65180cea10 con 0x7f65180b3c90 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f65180b2f80 con 0x7f651815daf0 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.576+0000 7f651e600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f65180b6c30 con 0x7f6518161580 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f650ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f65180b3c90 0x7f65180b40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59022/0 (socket says 172.21.15.116:59022) 2024-05-14T11:16:56.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f650ea00640 1 -- 172.21.15.116:0/187559927 learned_addr learned my addr 172.21.15.116:0/187559927 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:56.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517400640 1 --2- 172.21.15.116:0/187559927 >> v2:172.21.15.149:3300/0 conn(0x7f651815daf0 0x7f65180b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f6518161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883572024 0 0) 0x7f65180cea10 con 0x7f65180b3c90 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc003530 con 0x7f65180b3c90 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 192247701 0 0) 0x7f64fc003530 con 0x7f65180b3c90 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 msgr2=0x7f6518161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f6518161990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 >> v2:172.21.15.149:3300/0 conn(0x7f651815daf0 msgr2=0x7f65180b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 --2- 172.21.15.116:0/187559927 >> v2:172.21.15.149:3300/0 conn(0x7f651815daf0 0x7f65180b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65180b31d0 con 0x7f65180b3c90 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f6500001700 con 0x7f65180b3c90 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517400640 1 --2- 172.21.15.116:0/187559927 >> v2:172.21.15.149:3300/0 conn(0x7f651815daf0 0x7f65180b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:16:56.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 >> v1:172.21.15.116:6789/0 conn(0x7f65180b3c90 legacy=0x7f65180b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc003a10 0x7f64fc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f64fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64fc004d50 0x7f64fc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f65180b2f80 con 0x7f64fc004d50 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65180b6c30 con 0x7f64fc003a10 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc00ba80 con 0x7f6518161580 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64fc004d50 0x7f64fc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517400640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc003a10 0x7f64fc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f650ea00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f64fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 -- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 msgr2=0x7f64fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6518161580 0x7f64fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 -- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc003a10 msgr2=0x7f64fc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:56.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc003a10 0x7f64fc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.580+0000 7f6517e00640 1 -- 172.21.15.116:0/187559927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc003530 con 0x7f64fc004d50 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f6517e00640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64fc004d50 0x7f64fc005160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6510009f90 tx=0x7f651001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5f6f1e5f736a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65100b2070 con 0x7f64fc004d50 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65100a5c70 con 0x7f64fc004d50 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f6516a00640 1 -- 172.21.15.116:0/187559927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510009c10 con 0x7f64fc004d50 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 -- 172.21.15.116:0/187559927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64fc004d50 msgr2=0x7f64fc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 --2- 172.21.15.116:0/187559927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64fc004d50 0x7f64fc005160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6510009f90 tx=0x7f651001cc60 comp rx=0 tx=0).stop 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f6517e00640 1 -- 172.21.15.116:0/187559927 reap_dead start 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 -- 172.21.15.116:0/187559927 shutdown_connections 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 -- 172.21.15.116:0/187559927 >> 172.21.15.116:0/187559927 conn(0x7f65180b5710 msgr2=0x7f65180d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 -- 172.21.15.116:0/187559927 shutdown_connections 2024-05-14T11:16:56.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 -- 172.21.15.116:0/187559927 wait complete. 2024-05-14T11:16:56.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.584+0000 7f651e600640 1 Processor -- start 2024-05-14T11:16:56.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- start start 2024-05-14T11:16:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64fc004d50 0x7f65180b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f651815bd70 0x7f65181f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f65180b9340 con 0x7f65180ae880 2024-05-14T11:16:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65180b3790 con 0x7f651815bd70 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f65180b94c0 con 0x7f64fc004d50 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44674/0 (socket says 172.21.15.116:44674) 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 -- 172.21.15.116:0/2681869337 learned_addr learned my addr 172.21.15.116:0/2681869337 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f6517400640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64fc004d50 0x7f65180b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f6517e00640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f651815bd70 0x7f65181f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 -- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64fc004d50 msgr2=0x7f65180b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64fc004d50 0x7f65180b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 -- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f651815bd70 msgr2=0x7f65181f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f651815bd70 0x7f65181f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 -- 172.21.15.116:0/2681869337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6518155290 con 0x7f65180ae880 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650ea00640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f650000e850 tx=0x7f650000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000c8d0 con 0x7f65180ae880 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6500018070 con 0x7f65180ae880 2024-05-14T11:16:56.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000cc90 con 0x7f65180ae880 2024-05-14T11:16:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65180af060 con 0x7f65180ae880 2024-05-14T11:16:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65180af620 con 0x7f65180ae880 2024-05-14T11:16:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.588+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651806a9c0 con 0x7f65180ae880 2024-05-14T11:16:56.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.592+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6500013420 con 0x7f65180ae880 2024-05-14T11:16:56.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.592+0000 7f650fe00640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 0x7f64f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:56.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.592+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f650008f1b0 con 0x7f65180ae880 2024-05-14T11:16:56.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.592+0000 7f6517400640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 0x7f64f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:56.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.592+0000 7f6517400640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 0x7f64f4071940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65080093f0 tx=0x7f6508009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:56.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.596+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f650005b5a0 con 0x7f65180ae880 2024-05-14T11:16:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.760+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f65180aedf0 con 0x7f65180ae880 2024-05-14T11:16:56.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.764+0000 7f650fe00640 1 -- 172.21.15.116:0/2681869337 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f650005f8a0 con 0x7f65180ae880 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 msgr2=0x7f64f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 0x7f64f4071940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65080093f0 tx=0x7f6508009190 comp rx=0 tx=0).stop 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 msgr2=0x7f6518154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f650000e850 tx=0x7f650000ed10 comp rx=0 tx=0).stop 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 shutdown_connections 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64f406f430 0x7f64f4071940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f651815bd70 0x7f65181f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65180ae880 0x7f6518154d50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 --2- 172.21.15.116:0/2681869337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64fc004d50 0x7f65180b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:56.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 >> 172.21.15.116:0/2681869337 conn(0x7f65180b5710 msgr2=0x7f651815dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:56.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 shutdown_connections 2024-05-14T11:16:56.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:56.768+0000 7f651e600640 1 -- 172.21.15.116:0/2681869337 wait complete. 2024-05-14T11:16:56.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:16:56.789 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:16:56.879 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:16:56.973 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:16:56.973 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:56.974 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:56.982 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:56.982 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:57.032 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.216+0000 7fb7b9400640 1 Processor -- start 2024-05-14T11:16:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.216+0000 7fb7b9400640 1 -- start start 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b9400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7b40d1670 0x7fb7b40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b9400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b406ff20 con 0x7fb7b415db30 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b9400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b40d1fc0 con 0x7fb7b406c700 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b9400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40ae100 con 0x7fb7b40d1670 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b2800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb7b415db30 0x7fb7b40aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59024/0 (socket says 172.21.15.116:59024) 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7a9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7b40d1670 0x7fb7b40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7b2800640 1 -- 172.21.15.116:0/3592290172 learned_addr learned my addr 172.21.15.116:0/3592290172 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:57.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7a9e00640 1 -- 172.21.15.116:0/3592290172 >> v1:172.21.15.149:6789/0 conn(0x7fb7b406c700 legacy=0x7fb7b406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7a9e00640 1 -- 172.21.15.116:0/3592290172 >> v1:172.21.15.116:6789/0 conn(0x7fb7b415db30 legacy=0x7fb7b40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.220+0000 7fb7a9e00640 1 -- 172.21.15.116:0/3592290172 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b4070470 con 0x7fb7b40d1670 2024-05-14T11:16:57.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7a9e00640 1 --2- 172.21.15.116:0/3592290172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b40d1670 0x7fb7b40d1a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40ae280 tx=0x7fb79c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b87044e0c3c4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b1e00640 1 -- 172.21.15.116:0/3592290172 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0b0070 con 0x7fb7b40d1670 2024-05-14T11:16:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b1e00640 1 -- 172.21.15.116:0/3592290172 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c0ab600 con 0x7fb7b40d1670 2024-05-14T11:16:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b1e00640 1 -- 172.21.15.116:0/3592290172 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0ab9d0 con 0x7fb7b40d1670 2024-05-14T11:16:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 -- 172.21.15.116:0/3592290172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b40d1670 msgr2=0x7fb7b40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 --2- 172.21.15.116:0/3592290172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b40d1670 0x7fb7b40d1a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40ae280 tx=0x7fb79c01c380 comp rx=0 tx=0).stop 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 -- 172.21.15.116:0/3592290172 shutdown_connections 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 --2- 172.21.15.116:0/3592290172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b40d1670 0x7fb7b40d1a80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 -- 172.21.15.116:0/3592290172 >> 172.21.15.116:0/3592290172 conn(0x7fb7b40b4620 msgr2=0x7fb7b40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 -- 172.21.15.116:0/3592290172 shutdown_connections 2024-05-14T11:16:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 -- 172.21.15.116:0/3592290172 wait complete. 2024-05-14T11:16:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.224+0000 7fb7b9400640 1 Processor -- start 2024-05-14T11:16:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- start start 2024-05-14T11:16:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 0x7fb7b4149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 0x7fb7b414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b416bf90 con 0x7fb7b406c700 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b415df40 con 0x7fb7b40d1670 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b416c110 con 0x7fb7b415db30 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44688/0 (socket says 172.21.15.116:44688) 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 -- 172.21.15.116:0/1594348071 learned_addr learned my addr 172.21.15.116:0/1594348071 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 -- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 msgr2=0x7fb7b414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7a9e00640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 0x7fb7b4149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b3200640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 0x7fb7b414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 0x7fb7b414a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 -- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 msgr2=0x7fb7b4149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 0x7fb7b4149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 -- 172.21.15.116:0/1594348071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b41451b0 con 0x7fb7b406c700 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7a9e00640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 0x7fb7b4149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b3200640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 0x7fb7b414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b2800640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c500 tx=0x7fb7a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4017070 con 0x7fb7b406c700 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a4005cf0 con 0x7fb7b406c700 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40072f0 con 0x7fb7b406c700 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b4159760 con 0x7fb7b406c700 2024-05-14T11:16:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b4147420 con 0x7fb7b406c700 2024-05-14T11:16:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.228+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b40998b0 con 0x7fb7b406c700 2024-05-14T11:16:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.232+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7a401a070 con 0x7fb7b406c700 2024-05-14T11:16:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.232+0000 7fb7ab400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 0x7fb790071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.232+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fb7a408e9d0 con 0x7fb7b406c700 2024-05-14T11:16:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.232+0000 7fb7a9e00640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 0x7fb790071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.232+0000 7fb7a9e00640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 0x7fb790071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0ad000 tx=0x7fb79c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.236+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb7a405ae50 con 0x7fb7b406c700 2024-05-14T11:16:57.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.408+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb7b406a9c0 con 0x7fb7b406c700 2024-05-14T11:16:57.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.408+0000 7fb7ab400640 1 -- 172.21.15.116:0/1594348071 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fb7a405f150 con 0x7fb7b406c700 2024-05-14T11:16:57.412 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 msgr2=0x7fb790071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 0x7fb790071910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0ad000 tx=0x7fb79c008040 comp rx=0 tx=0).stop 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 msgr2=0x7fb7b4098540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c500 tx=0x7fb7a400c9c0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 shutdown_connections 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb79006f400 0x7fb790071910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b40d1670 0x7fb7b4149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7b415db30 0x7fb7b414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 --2- 172.21.15.116:0/1594348071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b406c700 0x7fb7b4098540 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 >> 172.21.15.116:0/1594348071 conn(0x7fb7b40b4620 msgr2=0x7fb7b40b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 shutdown_connections 2024-05-14T11:16:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.412+0000 7fb7b9400640 1 -- 172.21.15.116:0/1594348071 wait complete. 2024-05-14T11:16:57.434 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:57.435 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:57.443 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:57.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.628+0000 7fc916800640 1 Processor -- start 2024-05-14T11:16:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.628+0000 7fc916800640 1 -- start start 2024-05-14T11:16:57.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc916800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc91015db30 0x7fc9100aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc916800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc91006c700 0x7fc91006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc916800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc91006ff20 con 0x7fc9100d1670 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc916800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc9100ae100 con 0x7fc91006c700 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc916800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc9100ae870 con 0x7fc91015db30 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc91006c700 0x7fc91006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc91006c700 0x7fc91006cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41936/0 (socket says 172.21.15.116:41936) 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 -- 172.21.15.116:0/1613200900 learned_addr learned my addr 172.21.15.116:0/1613200900 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90f400640 1 --2- 172.21.15.116:0/1613200900 >> v2:172.21.15.149:3300/0 conn(0x7fc91015db30 0x7fc9100aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 -- 172.21.15.116:0/1613200900 >> v2:172.21.15.149:3300/0 conn(0x7fc91015db30 msgr2=0x7fc9100aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 --2- 172.21.15.116:0/1613200900 >> v2:172.21.15.149:3300/0 conn(0x7fc91015db30 0x7fc9100aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 -- 172.21.15.116:0/1613200900 >> v1:172.21.15.149:6789/0 conn(0x7fc9100d1670 legacy=0x7fc9100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 -- 172.21.15.116:0/1613200900 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc910070470 con 0x7fc91006c700 2024-05-14T11:16:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.632+0000 7fc90fe00640 1 --2- 172.21.15.116:0/1613200900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91006cb10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc910071f20 tx=0x7fc90801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8817947ae7ce33b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc90e000640 1 -- 172.21.15.116:0/1613200900 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a56d0 con 0x7fc91006c700 2024-05-14T11:16:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc90e000640 1 -- 172.21.15.116:0/1613200900 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9080a5870 con 0x7fc91006c700 2024-05-14T11:16:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc90e000640 1 -- 172.21.15.116:0/1613200900 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a5c40 con 0x7fc91006c700 2024-05-14T11:16:57.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- 172.21.15.116:0/1613200900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 msgr2=0x7fc91006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 --2- 172.21.15.116:0/1613200900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91006cb10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc910071f20 tx=0x7fc90801c920 comp rx=0 tx=0).stop 2024-05-14T11:16:57.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- 172.21.15.116:0/1613200900 shutdown_connections 2024-05-14T11:16:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 --2- 172.21.15.116:0/1613200900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91006cb10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 --2- 172.21.15.116:0/1613200900 >> v2:172.21.15.149:3300/0 conn(0x7fc91015db30 0x7fc9100aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- 172.21.15.116:0/1613200900 >> 172.21.15.116:0/1613200900 conn(0x7fc9100b4620 msgr2=0x7fc9100b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- 172.21.15.116:0/1613200900 shutdown_connections 2024-05-14T11:16:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- 172.21.15.116:0/1613200900 wait complete. 2024-05-14T11:16:57.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 Processor -- start 2024-05-14T11:16:57.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.636+0000 7fc916800640 1 -- start start 2024-05-14T11:16:57.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d1670 0x7fc9101613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc91015db30 0x7fc9101618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc91016a770 con 0x7fc91006c700 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc91015df40 con 0x7fc91015db30 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc91016a8f0 con 0x7fc9100d1670 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41946/0 (socket says 172.21.15.116:41946) 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 -- 172.21.15.116:0/2692438461 learned_addr learned my addr 172.21.15.116:0/2692438461 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90fe00640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc91015db30 0x7fc9101618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90ea00640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d1670 0x7fc9101613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 -- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d1670 msgr2=0x7fc9101613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d1670 0x7fc9101613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 -- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc91015db30 msgr2=0x7fc9101618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc91015db30 0x7fc9101618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 -- 172.21.15.116:0/2692438461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91014e0f0 con 0x7fc91006c700 2024-05-14T11:16:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc90f400640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e870 tx=0x7fc90000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900017070 con 0x7fc91006c700 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90000c8e0 con 0x7fc91006c700 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc910167c50 con 0x7fc91006c700 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000ccb0 con 0x7fc91006c700 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91014e3b0 con 0x7fc91006c700 2024-05-14T11:16:57.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.640+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91006a9c0 con 0x7fc91006c700 2024-05-14T11:16:57.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.644+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc90001a070 con 0x7fc91006c700 2024-05-14T11:16:57.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.644+0000 7fc907400640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 0x7fc8ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.644+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fc90008f3a0 con 0x7fc91006c700 2024-05-14T11:16:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.644+0000 7fc90ea00640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 0x7fc8ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.644+0000 7fc90ea00640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 0x7fc8ec071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc91015cb40 tx=0x7fc8f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:57.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.648+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc90005b950 con 0x7fc91006c700 2024-05-14T11:16:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9100aebd0 con 0x7fc91006c700 2024-05-14T11:16:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc907400640 1 -- 172.21.15.116:0/2692438461 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc90005fc50 con 0x7fc91006c700 2024-05-14T11:16:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 msgr2=0x7fc8ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 0x7fc8ec071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc91015cb40 tx=0x7fc8f8009040 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 msgr2=0x7fc91009a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e870 tx=0x7fc90000ed30 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 shutdown_connections 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f430 0x7fc8ec071940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc91015db30 0x7fc9101618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d1670 0x7fc9101613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 --2- 172.21.15.116:0/2692438461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc91006c700 0x7fc91009a780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 >> 172.21.15.116:0/2692438461 conn(0x7fc9100b4620 msgr2=0x7fc910159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 shutdown_connections 2024-05-14T11:16:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:57.820+0000 7fc916800640 1 -- 172.21.15.116:0/2692438461 wait complete. 2024-05-14T11:16:57.843 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:16:57.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:57.843 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:16:57.863 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:16:57.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:16:57.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:57.863 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:16:57.917 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:16:57.917 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:57.926 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:16:57.926 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:57.978 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.120+0000 7f8ec2c00640 1 Processor -- start 2024-05-14T11:16:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.120+0000 7f8ec2c00640 1 -- start start 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec2c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ebc15db30 0x7f8ebc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec2c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc06ff20 con 0x7f8ebc06c700 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec2c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc070470 con 0x7f8ebc0d1670 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec2c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ebc0ae100 con 0x7f8ebc15db30 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ebbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ebc15db30 0x7f8ebc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec0c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8ebc06c700 0x7f8ebc06cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60246/0 (socket says 172.21.15.116:60246) 2024-05-14T11:16:58.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ec0c00640 1 -- 172.21.15.116:0/2606994975 learned_addr learned my addr 172.21.15.116:0/2606994975 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:58.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2606994975 >> v1:172.21.15.149:6789/0 conn(0x7f8ebc0d1670 legacy=0x7f8ebc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2606994975 >> v1:172.21.15.116:6789/0 conn(0x7f8ebc06c700 legacy=0x7f8ebc06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.124+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2606994975 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc070750 con 0x7f8ebc15db30 2024-05-14T11:16:58.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/2606994975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc15db30 0x7f8ebc0aeaa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0ae280 tx=0x7f8ea401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2692703e1854d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2606994975 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40b0070 con 0x7f8ebc15db30 2024-05-14T11:16:58.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2606994975 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea40ab600 con 0x7f8ebc15db30 2024-05-14T11:16:58.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2606994975 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40ab9d0 con 0x7f8ebc15db30 2024-05-14T11:16:58.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 -- 172.21.15.116:0/2606994975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc15db30 msgr2=0x7f8ebc0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/2606994975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc15db30 0x7f8ebc0aeaa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0ae280 tx=0x7f8ea401c410 comp rx=0 tx=0).stop 2024-05-14T11:16:58.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 -- 172.21.15.116:0/2606994975 shutdown_connections 2024-05-14T11:16:58.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/2606994975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc15db30 0x7f8ebc0aeaa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 -- 172.21.15.116:0/2606994975 >> 172.21.15.116:0/2606994975 conn(0x7f8ebc0b4620 msgr2=0x7f8ebc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:58.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 -- 172.21.15.116:0/2606994975 shutdown_connections 2024-05-14T11:16:58.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.128+0000 7f8ec2c00640 1 -- 172.21.15.116:0/2606994975 wait complete. 2024-05-14T11:16:58.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 Processor -- start 2024-05-14T11:16:58.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- start start 2024-05-14T11:16:58.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 0x7f8ebc0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ebc15db30 0x7f8ebc155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc16bf90 con 0x7f8ebc06c700 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc15df40 con 0x7f8ebc0d1670 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc16c110 con 0x7f8ebc15db30 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41968/0 (socket says 172.21.15.116:41968) 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3675692632 learned_addr learned my addr 172.21.15.116:0/3675692632 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebb400640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 0x7f8ebc0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec0c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ebc15db30 0x7f8ebc155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ebc15db30 msgr2=0x7f8ebc155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ebc15db30 0x7f8ebc155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 msgr2=0x7f8ebc0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 0x7f8ebc0d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3675692632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc1672e0 con 0x7f8ebc06c700 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebb400640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 0x7f8ebc0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ea401a040 tx=0x7f8ea401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40b0050 con 0x7f8ebc06c700 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea40bc070 con 0x7f8ebc06c700 2024-05-14T11:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc1675c0 con 0x7f8ebc06c700 2024-05-14T11:16:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40b5a70 con 0x7f8ebc06c700 2024-05-14T11:16:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.132+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc0cd7e0 con 0x7f8ebc06c700 2024-05-14T11:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc0cda10 con 0x7f8ebc06c700 2024-05-14T11:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ea40ab480 con 0x7f8ebc06c700 2024-05-14T11:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8eb3e00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 0x7f8e98071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f8ea4129ff0 con 0x7f8ebc06c700 2024-05-14T11:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8ebb400640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 0x7f8e98071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.136+0000 7f8ebb400640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 0x7f8e98071940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc15d8d0 tx=0x7f8eac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.140+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8ebc0cda10 con 0x7f8ebc06c700 2024-05-14T11:16:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.312+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ebc06a9c0 con 0x7f8ebc06c700 2024-05-14T11:16:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.312+0000 7f8eb3e00640 1 -- 172.21.15.116:0/3675692632 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f8ea4016090 con 0x7f8ebc06c700 2024-05-14T11:16:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 msgr2=0x7f8e98071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 0x7f8e98071940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc15d8d0 tx=0x7f8eac009040 comp rx=0 tx=0).stop 2024-05-14T11:16:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 msgr2=0x7f8ebc153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ea401a040 tx=0x7f8ea401ac50 comp rx=0 tx=0).stop 2024-05-14T11:16:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 shutdown_connections 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8e9806f430 0x7f8e98071940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ebc0d1670 0x7f8ebc0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ebc15db30 0x7f8ebc155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 --2- 172.21.15.116:0/3675692632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ebc06c700 0x7f8ebc153510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 >> 172.21.15.116:0/3675692632 conn(0x7f8ebc0b4620 msgr2=0x7f8ebc15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 shutdown_connections 2024-05-14T11:16:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.316+0000 7f8ec2c00640 1 -- 172.21.15.116:0/3675692632 wait complete. 2024-05-14T11:16:58.338 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:16:58.338 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:58.347 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 Processor -- start 2024-05-14T11:16:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 -- start start 2024-05-14T11:16:58.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2b00b0770 0x7ff2b006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff2b0072a30 0x7ff2b0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b006cc40 con 0x7ff2b006f730 2024-05-14T11:16:58.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b006db20 con 0x7ff2b00b0770 2024-05-14T11:16:58.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.496+0000 7ff2b7e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b006cf10 con 0x7ff2b0072a30 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b4a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff2b006f730 0x7ff2b006fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60252/0 (socket says 172.21.15.116:60252) 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b4a00640 1 -- 172.21.15.116:0/108053504 learned_addr learned my addr 172.21.15.116:0/108053504 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2b00b0770 0x7ff2b006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5e00640 1 --2- 172.21.15.116:0/108053504 >> v2:172.21.15.149:3300/0 conn(0x7ff2b0072a30 0x7ff2b0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2afe00640 1 -- 172.21.15.116:0/108053504 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100278694 0 0) 0x7ff2b006cc40 con 0x7ff2b006f730 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2afe00640 1 -- 172.21.15.116:0/108053504 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff298003530 con 0x7ff2b006f730 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 -- 172.21.15.116:0/108053504 >> v2:172.21.15.149:3300/0 conn(0x7ff2b0072a30 msgr2=0x7ff2b0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 --2- 172.21.15.116:0/108053504 >> v2:172.21.15.149:3300/0 conn(0x7ff2b0072a30 0x7ff2b0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 -- 172.21.15.116:0/108053504 >> v1:172.21.15.116:6789/0 conn(0x7ff2b006f730 legacy=0x7ff2b006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 -- 172.21.15.116:0/108053504 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b006d1f0 con 0x7ff2b00b0770 2024-05-14T11:16:58.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2b5400640 1 --2- 172.21.15.116:0/108053504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b00b0770 0x7ff2b006c700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00729c0 tx=0x7ff2a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4826a50b21a1a6c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2afe00640 1 -- 172.21.15.116:0/108053504 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40b0070 con 0x7ff2b00b0770 2024-05-14T11:16:58.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2afe00640 1 -- 172.21.15.116:0/108053504 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a40ab600 con 0x7ff2b00b0770 2024-05-14T11:16:58.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.500+0000 7ff2afe00640 1 -- 172.21.15.116:0/108053504 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40ab9b0 con 0x7ff2b00b0770 2024-05-14T11:16:58.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- 172.21.15.116:0/108053504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b00b0770 msgr2=0x7ff2b006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/108053504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b00b0770 0x7ff2b006c700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00729c0 tx=0x7ff2a401c3c0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- 172.21.15.116:0/108053504 shutdown_connections 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/108053504 >> v2:172.21.15.149:3300/0 conn(0x7ff2b0072a30 0x7ff2b0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/108053504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b00b0770 0x7ff2b006c700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- 172.21.15.116:0/108053504 >> 172.21.15.116:0/108053504 conn(0x7ff2b00b4620 msgr2=0x7ff2b00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- 172.21.15.116:0/108053504 shutdown_connections 2024-05-14T11:16:58.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- 172.21.15.116:0/108053504 wait complete. 2024-05-14T11:16:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 Processor -- start 2024-05-14T11:16:58.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.504+0000 7ff2b7e00640 1 -- start start 2024-05-14T11:16:58.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 0x7ff2b015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2b00b0770 0x7ff2b01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0073500 con 0x7ff2b006f730 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0073380 con 0x7ff2b0072a30 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b0073680 con 0x7ff2b00b0770 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41980/0 (socket says 172.21.15.116:41980) 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 -- 172.21.15.116:0/414489401 learned_addr learned my addr 172.21.15.116:0/414489401 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 -- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2b00b0770 msgr2=0x7ff2b01539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2b00b0770 0x7ff2b01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 -- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 msgr2=0x7ff2b015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b4a00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 0x7ff2b015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 0x7ff2b015ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 -- 172.21.15.116:0/414489401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b015da90 con 0x7ff2b006f730 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b4a00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 0x7ff2b015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b5400640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4019ff0 tx=0x7ff2a401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40b0040 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a4004030 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4006e20 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b015dd70 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b015e330 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b006a9c0 con 0x7ff2b006f730 2024-05-14T11:16:58.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.508+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2a4003620 con 0x7ff2b006f730 2024-05-14T11:16:58.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.512+0000 7ff2ad600640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 0x7ff290071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:58.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.512+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7ff2a41296a0 con 0x7ff2b006f730 2024-05-14T11:16:58.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.512+0000 7ff2b4a00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 0x7ff290071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:58.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.512+0000 7ff2b4a00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 0x7ff290071940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00ccab0 tx=0x7ff29c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:58.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.516+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2a4016070 con 0x7ff2b006f730 2024-05-14T11:16:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2b006cbc0 con 0x7ff2b006f730 2024-05-14T11:16:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2ad600640 1 -- 172.21.15.116:0/414489401 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7ff2a40f6170 con 0x7ff2b006f730 2024-05-14T11:16:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 msgr2=0x7ff290071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 0x7ff290071940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00ccab0 tx=0x7ff29c009040 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 msgr2=0x7ff2b014ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4019ff0 tx=0x7ff2a401ac00 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 shutdown_connections 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff29006f430 0x7ff290071940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2b0072a30 0x7ff2b015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2b00b0770 0x7ff2b01539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 --2- 172.21.15.116:0/414489401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2b006f730 0x7ff2b014ea60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.684+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 >> 172.21.15.116:0/414489401 conn(0x7ff2b00b4620 msgr2=0x7ff2b0155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.688+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 shutdown_connections 2024-05-14T11:16:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:58.688+0000 7ff2b7e00640 1 -- 172.21.15.116:0/414489401 wait complete. 2024-05-14T11:16:58.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi149.front.sepia.ceph.com... 2024-05-14T11:16:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:58.709 DEBUG:teuthology.orchestra.run.smithi149:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-14T11:16:58.744 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:16:58.744 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:16:58.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:58.744 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-14T11:16:58.847 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:58.847 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:58.855 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:58.856 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:58.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 Processor -- start 2024-05-14T11:16:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 -- start start 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 0x7fddd406de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fddd4071550 0x7fddd406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd406cc10 con 0x7fddd40c5210 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fddd40b7390 con 0x7fddd4071550 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fdddaa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fddd415eb40 con 0x7fddd406c700 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fddd4071550 0x7fddd406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fddd4071550 0x7fddd406e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41988/0 (socket says 172.21.15.116:41988) 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 -- 172.21.15.116:0/987938825 learned_addr learned my addr 172.21.15.116:0/987938825 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd3400640 1 --2- 172.21.15.116:0/987938825 >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 0x7fddd406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 -- 172.21.15.116:0/987938825 >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 msgr2=0x7fddd406de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 --2- 172.21.15.116:0/987938825 >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 0x7fddd406de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 -- 172.21.15.116:0/987938825 >> v1:172.21.15.149:6789/0 conn(0x7fddd40c5210 legacy=0x7fddd4070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd8a00640 1 -- 172.21.15.116:0/987938825 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd415ee20 con 0x7fddd4071550 2024-05-14T11:16:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.100+0000 7fddd3400640 1 --2- 172.21.15.116:0/987938825 >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 0x7fddd406de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:16:59.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fddd8a00640 1 --2- 172.21.15.116:0/987938825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd4071550 0x7fddd406e340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cf180 tx=0x7fddcc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b05aa56b60b698 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fddd2a00640 1 -- 172.21.15.116:0/987938825 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0ac020 con 0x7fddd4071550 2024-05-14T11:16:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fddd2a00640 1 -- 172.21.15.116:0/987938825 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc0a6410 con 0x7fddd4071550 2024-05-14T11:16:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fddd2a00640 1 -- 172.21.15.116:0/987938825 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0a67e0 con 0x7fddd4071550 2024-05-14T11:16:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 -- 172.21.15.116:0/987938825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd4071550 msgr2=0x7fddd406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 --2- 172.21.15.116:0/987938825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd4071550 0x7fddd406e340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fddd40cf180 tx=0x7fddcc01c780 comp rx=0 tx=0).stop 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 -- 172.21.15.116:0/987938825 shutdown_connections 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 --2- 172.21.15.116:0/987938825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd4071550 0x7fddd406e340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 --2- 172.21.15.116:0/987938825 >> v2:172.21.15.149:3300/0 conn(0x7fddd406c700 0x7fddd406de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 -- 172.21.15.116:0/987938825 >> 172.21.15.116:0/987938825 conn(0x7fddd40ce570 msgr2=0x7fddd40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 -- 172.21.15.116:0/987938825 shutdown_connections 2024-05-14T11:16:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.104+0000 7fdddaa00640 1 -- 172.21.15.116:0/987938825 wait complete. 2024-05-14T11:16:59.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 Processor -- start 2024-05-14T11:16:59.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- start start 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddd406c700 0x7fddd4152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 0x7fddd41510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fddd40cd470 con 0x7fddd40c5210 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fddd40cd2f0 con 0x7fddd406c700 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fddd40cd5f0 con 0x7fddd4071550 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42004/0 (socket says 172.21.15.116:42004) 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 -- 172.21.15.116:0/710540228 learned_addr learned my addr 172.21.15.116:0/710540228 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd3e00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddd406c700 0x7fddd4152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd3400640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 0x7fddd41510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 -- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 msgr2=0x7fddd41510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 0x7fddd41510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 -- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddd406c700 msgr2=0x7fddd4152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddd406c700 0x7fddd4152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 -- 172.21.15.116:0/710540228 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd4160d20 con 0x7fddd40c5210 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd3400640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 0x7fddd41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddd8a00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0ab040 tx=0x7fddcc019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0b7070 con 0x7fddd40c5210 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc0b1420 con 0x7fddd40c5210 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0b1810 con 0x7fddd40c5210 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd4161000 con 0x7fddd40c5210 2024-05-14T11:16:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.108+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd40cfa40 con 0x7fddd40c5210 2024-05-14T11:16:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd406a9c0 con 0x7fddd40c5210 2024-05-14T11:16:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fddcc0af020 con 0x7fddd40c5210 2024-05-14T11:16:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fddcbe00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fddcc129950 con 0x7fddd40c5210 2024-05-14T11:16:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fddd3e00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.112+0000 7fddd3e00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 0x7fddb4071910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.116+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fddd406cc10 con 0x7fddd40c5210 2024-05-14T11:16:59.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.288+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fddd4154400 con 0x7fddd40c5210 2024-05-14T11:16:59.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.288+0000 7fddcbe00640 1 -- 172.21.15.116:0/710540228 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fddd4154400 con 0x7fddd40c5210 2024-05-14T11:16:59.292 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 msgr2=0x7fddb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 0x7fddb4071910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4009060 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 msgr2=0x7fddd40d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0ab040 tx=0x7fddcc019520 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 shutdown_connections 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd40c5210 0x7fddd40d0240 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fddd4071550 0x7fddd41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 --2- 172.21.15.116:0/710540228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddd406c700 0x7fddd4152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 >> 172.21.15.116:0/710540228 conn(0x7fddd40ce570 msgr2=0x7fddd40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 shutdown_connections 2024-05-14T11:16:59.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.292+0000 7fdddaa00640 1 -- 172.21.15.116:0/710540228 wait complete. 2024-05-14T11:16:59.314 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:16:59.314 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:59.323 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:16:59.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.508+0000 7f56fc000640 1 Processor -- start 2024-05-14T11:16:59.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.508+0000 7f56fc000640 1 -- start start 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fc000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f40b0770 0x7f56f406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fc000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56f4072a30 0x7f56f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fc000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f406cc40 con 0x7f56f406f730 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fc000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f56f406db20 con 0x7f56f4072a30 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fc000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56f406cf10 con 0x7f56f40b0770 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fa000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56f4072a30 0x7f56f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56f8c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f56f406f730 0x7f56f406fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60258/0 (socket says 172.21.15.116:60258) 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56f8c00640 1 -- 172.21.15.116:0/3946683897 learned_addr learned my addr 172.21.15.116:0/3946683897 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56f9600640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f40b0770 0x7f56f406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fa000640 1 -- 172.21.15.116:0/3946683897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f40b0770 msgr2=0x7f56f406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:16:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fa000640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f40b0770 0x7f56f406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fa000640 1 -- 172.21.15.116:0/3946683897 >> v1:172.21.15.116:6789/0 conn(0x7f56f406f730 legacy=0x7f56f406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56fa000640 1 -- 172.21.15.116:0/3946683897 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f406d1f0 con 0x7f56f4072a30 2024-05-14T11:16:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.512+0000 7f56f3e00640 1 -- 172.21.15.116:0/3946683897 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614112841 0 0) 0x7f56f406cc40 con 0x7f56f406f730 2024-05-14T11:16:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fa000640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f4072a30 0x7f56f4072e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009d90 tx=0x7f56ec01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903a18b3eb719f84 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56f3e00640 1 -- 172.21.15.116:0/3946683897 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0b2070 con 0x7f56f4072a30 2024-05-14T11:16:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56f3e00640 1 -- 172.21.15.116:0/3946683897 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec0a5990 con 0x7f56f4072a30 2024-05-14T11:16:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56f3e00640 1 -- 172.21.15.116:0/3946683897 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0a5d60 con 0x7f56f4072a30 2024-05-14T11:16:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 -- 172.21.15.116:0/3946683897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f4072a30 msgr2=0x7f56f4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f4072a30 0x7f56f4072e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009d90 tx=0x7f56ec01c8d0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 -- 172.21.15.116:0/3946683897 shutdown_connections 2024-05-14T11:16:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f4072a30 0x7f56f4072e40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 --2- 172.21.15.116:0/3946683897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f40b0770 0x7f56f406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 -- 172.21.15.116:0/3946683897 >> 172.21.15.116:0/3946683897 conn(0x7f56f40b4620 msgr2=0x7f56f40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 -- 172.21.15.116:0/3946683897 shutdown_connections 2024-05-14T11:16:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.516+0000 7f56fc000640 1 -- 172.21.15.116:0/3946683897 wait complete. 2024-05-14T11:16:59.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 Processor -- start 2024-05-14T11:16:59.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- start start 2024-05-14T11:16:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f406f730 0x7f56f414ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f4072a30 0x7f56f4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d17f0 con 0x7f56f40b0770 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40d1670 con 0x7f56f406f730 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56f40d1970 con 0x7f56f4072a30 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42022/0 (socket says 172.21.15.116:42022) 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 -- 172.21.15.116:0/1228818162 learned_addr learned my addr 172.21.15.116:0/1228818162 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56f9600640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f406f730 0x7f56f414ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56f8c00640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f4072a30 0x7f56f4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 -- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f4072a30 msgr2=0x7f56f4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f4072a30 0x7f56f4142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 -- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f406f730 msgr2=0x7f56f414ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f406f730 0x7f56f414ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 -- 172.21.15.116:0/1228818162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f415b650 con 0x7f56f40b0770 2024-05-14T11:16:59.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fa000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009d90 tx=0x7f56ec019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0bd020 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec0b3040 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f415b930 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0b2050 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.520+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f415bef0 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f4157c00 con 0x7f56f40b0770 2024-05-14T11:16:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56ec0ad710 con 0x7f56f40b0770 2024-05-14T11:16:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56f1600640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 0x7f56d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:16:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f56ec12a220 con 0x7f56f40b0770 2024-05-14T11:16:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56f9600640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 0x7f56d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:16:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.524+0000 7f56f9600640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 0x7f56d4071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56f40d1240 tx=0x7f56e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:16:59.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.528+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f56ec0c7020 con 0x7f56f40b0770 2024-05-14T11:16:59.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.696+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f56f406a9c0 con 0x7f56f40b0770 2024-05-14T11:16:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.696+0000 7f56f1600640 1 -- 172.21.15.116:0/1228818162 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f56ec016580 con 0x7f56f40b0770 2024-05-14T11:16:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 msgr2=0x7f56d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 0x7f56d4071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56f40d1240 tx=0x7f56e8008040 comp rx=0 tx=0).stop 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 msgr2=0x7f56f4153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009d90 tx=0x7f56ec019670 comp rx=0 tx=0).stop 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 shutdown_connections 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f56d406f400 0x7f56d4071910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f40b0770 0x7f56f4153d70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56f4072a30 0x7f56f4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 --2- 172.21.15.116:0/1228818162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f406f730 0x7f56f414ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:16:59.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 >> 172.21.15.116:0/1228818162 conn(0x7f56f40b4620 msgr2=0x7f56f4155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:16:59.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 shutdown_connections 2024-05-14T11:16:59.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:16:59.700+0000 7f56fc000640 1 -- 172.21.15.116:0/1228818162 wait complete. 2024-05-14T11:16:59.723 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:16:59.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:59.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-14T11:16:59.743 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:16:59.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:16:59.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:16:59.743 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-14T11:16:59.797 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:16:59.797 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:59.806 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:16:59.806 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:16:59.858 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:17:00.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 Processor -- start 2024-05-14T11:17:00.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 -- start start 2024-05-14T11:17:00.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 0x7f48d40d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f48d40ce9d0 0x7f48d40b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d40d1f90 con 0x7f48d40b9020 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f48d40d1d40 con 0x7f48d40ce9d0 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40d2270 con 0x7f48d41621b0 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f48d40b9020 0x7f48d40ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60264/0 (socket says 172.21.15.116:60264) 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2000640 1 -- 172.21.15.116:0/335000828 learned_addr learned my addr 172.21.15.116:0/335000828 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2a00640 1 --2- 172.21.15.116:0/335000828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 0x7f48d40d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d3400640 1 --2- 172.21.15.116:0/335000828 >> v2:172.21.15.149:3300/0 conn(0x7f48d40ce9d0 0x7f48d40b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d1600640 1 -- 172.21.15.116:0/335000828 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029864168 0 0) 0x7f48d40d1f90 con 0x7f48d40b9020 2024-05-14T11:17:00.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d1600640 1 -- 172.21.15.116:0/335000828 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48b8003530 con 0x7f48d40b9020 2024-05-14T11:17:00.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2a00640 1 -- 172.21.15.116:0/335000828 >> v2:172.21.15.149:3300/0 conn(0x7f48d40ce9d0 msgr2=0x7f48d40b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:00.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2a00640 1 --2- 172.21.15.116:0/335000828 >> v2:172.21.15.149:3300/0 conn(0x7f48d40ce9d0 0x7f48d40b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2a00640 1 -- 172.21.15.116:0/335000828 >> v1:172.21.15.116:6789/0 conn(0x7f48d40b9020 legacy=0x7f48d40ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.004+0000 7f48d2a00640 1 -- 172.21.15.116:0/335000828 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d40d2550 con 0x7f48d41621b0 2024-05-14T11:17:00.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d2a00640 1 --2- 172.21.15.116:0/335000828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 0x7f48d40d2d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48c40097c0 tx=0x7f48c401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ee92c4ac12fd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d1600640 1 -- 172.21.15.116:0/335000828 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b0070 con 0x7f48d41621b0 2024-05-14T11:17:00.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d1600640 1 -- 172.21.15.116:0/335000828 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40ab600 con 0x7f48d41621b0 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d1600640 1 -- 172.21.15.116:0/335000828 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40ab9d0 con 0x7f48d41621b0 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 -- 172.21.15.116:0/335000828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 msgr2=0x7f48d40d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 --2- 172.21.15.116:0/335000828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 0x7f48d40d2d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48c40097c0 tx=0x7f48c401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 -- 172.21.15.116:0/335000828 shutdown_connections 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 --2- 172.21.15.116:0/335000828 >> v2:172.21.15.149:3300/0 conn(0x7f48d40ce9d0 0x7f48d40b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 --2- 172.21.15.116:0/335000828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d41621b0 0x7f48d40d2d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 -- 172.21.15.116:0/335000828 >> 172.21.15.116:0/335000828 conn(0x7f48d40d0770 msgr2=0x7f48d40d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:00.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 -- 172.21.15.116:0/335000828 shutdown_connections 2024-05-14T11:17:00.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.008+0000 7f48d9600640 1 -- 172.21.15.116:0/335000828 wait complete. 2024-05-14T11:17:00.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 Processor -- start 2024-05-14T11:17:00.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 -- start start 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 0x7f48d4200200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d40cafe0 0x7f48d4070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48d415e600 con 0x7f48d41621b0 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48d415e480 con 0x7f48d40cafe0 2024-05-14T11:17:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48d415e780 con 0x7f48d40b9020 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42028/0 (socket says 172.21.15.116:42028) 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d2000640 1 -- 172.21.15.116:0/3862284346 learned_addr learned my addr 172.21.15.116:0/3862284346 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.012+0000 7f48d2a00640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 0x7f48d4200200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d3400640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d40cafe0 0x7f48d4070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 -- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d40cafe0 msgr2=0x7f48d4070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d40cafe0 0x7f48d4070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 -- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 msgr2=0x7f48d40caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 -- 172.21.15.116:0/3862284346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d40cb810 con 0x7f48d40b9020 2024-05-14T11:17:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2000640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d2a00640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 0x7f48d4200200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ab000 tx=0x7f48c4019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b0040 con 0x7f48d40b9020 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40ba070 con 0x7f48d40b9020 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d40cbaf0 con 0x7f48d40b9020 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4006e20 con 0x7f48d40b9020 2024-05-14T11:17:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d40cc0b0 con 0x7f48d40b9020 2024-05-14T11:17:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d406a9c0 con 0x7f48d40b9020 2024-05-14T11:17:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.016+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f48c40b4650 con 0x7f48d40b9020 2024-05-14T11:17:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.020+0000 7f48caa00640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 0x7f48b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.020+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f48c4129ca0 con 0x7f48d40b9020 2024-05-14T11:17:00.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.020+0000 7f48d2000640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 0x7f48b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.020+0000 7f48d2000640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 0x7f48b4071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f48c0007fd0 tx=0x7f48c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.024+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f48c4016090 con 0x7f48d40b9020 2024-05-14T11:17:00.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.192+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48d4161a90 con 0x7f48d40b9020 2024-05-14T11:17:00.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.192+0000 7f48caa00640 1 -- 172.21.15.116:0/3862284346 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f48c40f6190 con 0x7f48d40b9020 2024-05-14T11:17:00.196 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:17:00.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 msgr2=0x7f48b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 0x7f48b4071910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f48c0007fd0 tx=0x7f48c0009040 comp rx=0 tx=0).stop 2024-05-14T11:17:00.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 msgr2=0x7f48d4200200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 0x7f48d4200200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ab000 tx=0x7f48c4019240 comp rx=0 tx=0).stop 2024-05-14T11:17:00.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 shutdown_connections 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f48b406f400 0x7f48b4071910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d40cafe0 0x7f48d4070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48d41621b0 0x7f48d40caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 --2- 172.21.15.116:0/3862284346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48d40b9020 0x7f48d4200200 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 >> 172.21.15.116:0/3862284346 conn(0x7f48d40d0770 msgr2=0x7f48d40d35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 shutdown_connections 2024-05-14T11:17:00.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.196+0000 7f48d9600640 1 -- 172.21.15.116:0/3862284346 wait complete. 2024-05-14T11:17:00.226 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:17:00.227 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:00.238 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:17:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.388+0000 7fa431e00640 1 Processor -- start 2024-05-14T11:17:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.388+0000 7fa431e00640 1 -- start start 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa431e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa42c0ce9d0 0x7fa42c0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa431e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0d3160 con 0x7fa42c1621b0 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa431e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0d1f90 con 0x7fa42c0d2d50 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa431e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa42c0d1d40 con 0x7fa42c0ce9d0 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42aa00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa42c1621b0 0x7fa42c0b9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60272/0 (socket says 172.21.15.116:60272) 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42aa00640 1 -- 172.21.15.116:0/1573754055 learned_addr learned my addr 172.21.15.116:0/1573754055 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42b400640 1 --2- 172.21.15.116:0/1573754055 >> v2:172.21.15.149:3300/0 conn(0x7fa42c0ce9d0 0x7fa42c0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184631313 0 0) 0x7fa42c0d3160 con 0x7fa42c1621b0 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa410003530 con 0x7fa42c1621b0 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142766690 0 0) 0x7fa42c0d1f90 con 0x7fa42c0d2d50 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa42c0d3160 con 0x7fa42c0d2d50 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42b400640 1 -- 172.21.15.116:0/1573754055 >> v1:172.21.15.149:6789/0 conn(0x7fa42c0d2d50 legacy=0x7fa42c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42b400640 1 -- 172.21.15.116:0/1573754055 >> v1:172.21.15.116:6789/0 conn(0x7fa42c1621b0 legacy=0x7fa42c0b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.392+0000 7fa42b400640 1 -- 172.21.15.116:0/1573754055 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0d2270 con 0x7fa42c0ce9d0 2024-05-14T11:17:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa42b400640 1 --2- 172.21.15.116:0/1573754055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c0ce9d0 0x7fa42c0b2f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa424009d90 tx=0x7fa42401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd1f739c434d91e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b2070 con 0x7fa42c0ce9d0 2024-05-14T11:17:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4240a5830 con 0x7fa42c0ce9d0 2024-05-14T11:17:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa42a000640 1 -- 172.21.15.116:0/1573754055 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240a5be0 con 0x7fa42c0ce9d0 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 -- 172.21.15.116:0/1573754055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c0ce9d0 msgr2=0x7fa42c0b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 --2- 172.21.15.116:0/1573754055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c0ce9d0 0x7fa42c0b2f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa424009d90 tx=0x7fa42401c970 comp rx=0 tx=0).stop 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 -- 172.21.15.116:0/1573754055 shutdown_connections 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 --2- 172.21.15.116:0/1573754055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c0ce9d0 0x7fa42c0b2f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 -- 172.21.15.116:0/1573754055 >> 172.21.15.116:0/1573754055 conn(0x7fa42c0d0770 msgr2=0x7fa42c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 -- 172.21.15.116:0/1573754055 shutdown_connections 2024-05-14T11:17:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.396+0000 7fa431e00640 1 -- 172.21.15.116:0/1573754055 wait complete. 2024-05-14T11:17:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 Processor -- start 2024-05-14T11:17:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- start start 2024-05-14T11:17:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c1621b0 0x7fa42c168da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa42c06e0a0 0x7fa42c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c15e600 con 0x7fa42c0d2d50 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c15e480 con 0x7fa42c1621b0 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c15e780 con 0x7fa42c06e0a0 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42032/0 (socket says 172.21.15.116:42032) 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 -- 172.21.15.116:0/2637027931 learned_addr learned my addr 172.21.15.116:0/2637027931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa423e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c1621b0 0x7fa42c168da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42b400640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa42c06e0a0 0x7fa42c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 -- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa42c06e0a0 msgr2=0x7fa42c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa42c06e0a0 0x7fa42c06e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 -- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c1621b0 msgr2=0x7fa42c168da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c1621b0 0x7fa42c168da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 -- 172.21.15.116:0/2637027931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c1692e0 con 0x7fa42c0d2d50 2024-05-14T11:17:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa42aa00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c400 tx=0x7fa41c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c017070 con 0x7fa42c0d2d50 2024-05-14T11:17:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c005cf0 con 0x7fa42c0d2d50 2024-05-14T11:17:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c159c00 con 0x7fa42c0d2d50 2024-05-14T11:17:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c007310 con 0x7fa42c0d2d50 2024-05-14T11:17:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.400+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c167aa0 con 0x7fa42c0d2d50 2024-05-14T11:17:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.404+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c06a9c0 con 0x7fa42c0d2d50 2024-05-14T11:17:00.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.404+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa41c01a070 con 0x7fa42c0d2d50 2024-05-14T11:17:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.408+0000 7fa423400640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 0x7fa408071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.408+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fa41c08e930 con 0x7fa42c0d2d50 2024-05-14T11:17:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.408+0000 7fa423e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 0x7fa408071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.408+0000 7fa423e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 0x7fa408071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c10 tx=0x7fa414009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.412+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa41c05ae50 con 0x7fa42c0d2d50 2024-05-14T11:17:00.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.584+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa42c15f5c0 con 0x7fa42c0d2d50 2024-05-14T11:17:00.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.584+0000 7fa423400640 1 -- 172.21.15.116:0/2637027931 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa41c05f150 con 0x7fa42c0d2d50 2024-05-14T11:17:00.588 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:17:00.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 msgr2=0x7fa408071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 0x7fa408071940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c10 tx=0x7fa414009040 comp rx=0 tx=0).stop 2024-05-14T11:17:00.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 msgr2=0x7fa42c168860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c400 tx=0x7fa41c00c8c0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 shutdown_connections 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa40806f430 0x7fa408071940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa42c1621b0 0x7fa42c168da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa42c06e0a0 0x7fa42c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 --2- 172.21.15.116:0/2637027931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa42c0d2d50 0x7fa42c168860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 >> 172.21.15.116:0/2637027931 conn(0x7fa42c0d0770 msgr2=0x7fa42c15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 shutdown_connections 2024-05-14T11:17:00.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.588+0000 7fa431e00640 1 -- 172.21.15.116:0/2637027931 wait complete. 2024-05-14T11:17:00.609 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi149.front.sepia.ceph.com... 2024-05-14T11:17:00.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:00.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-14T11:17:00.630 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:00.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:17:00.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:00.630 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-14T11:17:00.683 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:17:00.683 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:00.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 Processor -- start 2024-05-14T11:17:00.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 -- start start 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 0x7f0a2006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a200d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0a2006e9e0 0x7f0a2006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0a201573b0 con 0x7f0a2006e9e0 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0a20161120 con 0x7f0a20156fa0 2024-05-14T11:17:00.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a25a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200d1fc0 con 0x7f0a200d1670 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0a2006e9e0 0x7f0a2006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 0x7f0a2006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a16000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a200d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 0x7f0a2006ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:47754/0 (socket says 172.21.15.116:47754) 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0a2006e9e0 0x7f0a2006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42044/0 (socket says 172.21.15.116:42044) 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1ea00640 1 -- 172.21.15.116:0/3993612175 learned_addr learned my addr 172.21.15.116:0/3993612175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 -- 172.21.15.116:0/3993612175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 msgr2=0x7f0a200d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 --2- 172.21.15.116:0/3993612175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a200d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 -- 172.21.15.116:0/3993612175 >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 msgr2=0x7f0a2006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:00.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 --2- 172.21.15.116:0/3993612175 >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 0x7f0a2006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.828+0000 7f0a1f400640 1 -- 172.21.15.116:0/3993612175 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a20160510 con 0x7f0a2006e9e0 2024-05-14T11:17:00.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a1f400640 1 --2- 172.21.15.116:0/3993612175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2006edf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a200d2170 tx=0x7f0a1801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8d2e710b95fbcc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a1e000640 1 -- 172.21.15.116:0/3993612175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180b2070 con 0x7f0a2006e9e0 2024-05-14T11:17:00.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a1e000640 1 -- 172.21.15.116:0/3993612175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a180a5980 con 0x7f0a2006e9e0 2024-05-14T11:17:00.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a1e000640 1 -- 172.21.15.116:0/3993612175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180a5d50 con 0x7f0a2006e9e0 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 -- 172.21.15.116:0/3993612175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 msgr2=0x7f0a2006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 --2- 172.21.15.116:0/3993612175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2006edf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a200d2170 tx=0x7f0a1801c900 comp rx=0 tx=0).stop 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 -- 172.21.15.116:0/3993612175 shutdown_connections 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 --2- 172.21.15.116:0/3993612175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2006edf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 --2- 172.21.15.116:0/3993612175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a200d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 --2- 172.21.15.116:0/3993612175 >> v2:172.21.15.149:3300/0 conn(0x7f0a20156fa0 0x7f0a2006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 -- 172.21.15.116:0/3993612175 >> 172.21.15.116:0/3993612175 conn(0x7f0a200b4620 msgr2=0x7f0a200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 -- 172.21.15.116:0/3993612175 shutdown_connections 2024-05-14T11:17:00.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.832+0000 7f0a25a00640 1 -- 172.21.15.116:0/3993612175 wait complete. 2024-05-14T11:17:00.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 Processor -- start 2024-05-14T11:17:00.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 -- start start 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a2014bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a20156fa0 0x7f0a2014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a20070520 con 0x7f0a2006e9e0 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2006f740 con 0x7f0a20156fa0 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200706a0 con 0x7f0a200d1670 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42048/0 (socket says 172.21.15.116:42048) 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 -- 172.21.15.116:0/4130996538 learned_addr learned my addr 172.21.15.116:0/4130996538 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a16000640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a2014bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1f400640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a20156fa0 0x7f0a2014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 -- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 msgr2=0x7f0a2014bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a2014bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 -- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a20156fa0 msgr2=0x7f0a2014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a20156fa0 0x7f0a2014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 -- 172.21.15.116:0/4130996538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a200cf160 con 0x7f0a2006e9e0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a1ea00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e8a0 tx=0x7f0a1000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10017070 con 0x7f0a2006e9e0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1000c940 con 0x7f0a2006e9e0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a200ce3a0 con 0x7f0a2006e9e0 2024-05-14T11:17:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.836+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cd10 con 0x7f0a2006e9e0 2024-05-14T11:17:00.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a200ce960 con 0x7f0a2006e9e0 2024-05-14T11:17:00.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2006a9c0 con 0x7f0a2006e9e0 2024-05-14T11:17:00.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a1001a070 con 0x7f0a2006e9e0 2024-05-14T11:17:00.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a17400640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 0x7f09fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:00.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f0a1008f5d0 con 0x7f0a2006e9e0 2024-05-14T11:17:00.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a16000640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 0x7f09fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:00.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.840+0000 7f0a16000640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 0x7f09fc071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a20069bf0 tx=0x7f0a08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:00.844+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0a1005c010 con 0x7f0a2006e9e0 2024-05-14T11:17:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.036+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a200b1d40 con 0x7f0a2006e9e0 2024-05-14T11:17:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.036+0000 7f0a17400640 1 -- 172.21.15.116:0/4130996538 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0a1005b910 con 0x7f0a2006e9e0 2024-05-14T11:17:01.041 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:01.041 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:17:01.041 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 msgr2=0x7f09fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 0x7f09fc071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a20069bf0 tx=0x7f0a08008040 comp rx=0 tx=0).stop 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 msgr2=0x7f0a2015bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e8a0 tx=0x7f0a1000ed60 comp rx=0 tx=0).stop 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 shutdown_connections 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f09fc06f400 0x7f09fc071910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a20156fa0 0x7f0a2014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a200d1670 0x7f0a2014bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 --2- 172.21.15.116:0/4130996538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a2006e9e0 0x7f0a2015bb20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 >> 172.21.15.116:0/4130996538 conn(0x7f0a200b4620 msgr2=0x7f0a20150800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 shutdown_connections 2024-05-14T11:17:01.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.040+0000 7f0a25a00640 1 -- 172.21.15.116:0/4130996538 wait complete. 2024-05-14T11:17:01.063 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:01.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.204+0000 7f603ec00640 1 Processor -- start 2024-05-14T11:17:01.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.204+0000 7f603ec00640 1 -- start start 2024-05-14T11:17:01.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603ec00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6038164f80 0x7f60381673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603ec00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603806cf10 con 0x7f60380b0770 2024-05-14T11:17:01.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603ec00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603806d1f0 con 0x7f603806f730 2024-05-14T11:17:01.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603ec00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f603806db20 con 0x7f6038164f80 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f6037e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f60380b0770 0x7f603806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60276/0 (socket says 172.21.15.116:60276) 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f6037e00640 1 -- 172.21.15.116:0/2391100314 learned_addr learned my addr 172.21.15.116:0/2391100314 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603cc00640 1 --2- 172.21.15.116:0/2391100314 >> v2:172.21.15.149:3300/0 conn(0x7f6038164f80 0x7f60381673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603cc00640 1 -- 172.21.15.116:0/2391100314 >> v1:172.21.15.149:6789/0 conn(0x7f603806f730 legacy=0x7f603806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603cc00640 1 -- 172.21.15.116:0/2391100314 >> v1:172.21.15.116:6789/0 conn(0x7f60380b0770 legacy=0x7f603806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f6037400640 1 -- 172.21.15.116:0/2391100314 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293186897 0 0) 0x7f603806cf10 con 0x7f60380b0770 2024-05-14T11:17:01.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.208+0000 7f603cc00640 1 -- 172.21.15.116:0/2391100314 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603806d4d0 con 0x7f6038164f80 2024-05-14T11:17:01.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603cc00640 1 --2- 172.21.15.116:0/2391100314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381673a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6030009d90 tx=0x7f603001c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e0b4f81d9300e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f6037400640 1 -- 172.21.15.116:0/2391100314 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300b2070 con 0x7f6038164f80 2024-05-14T11:17:01.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f6037400640 1 -- 172.21.15.116:0/2391100314 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60300a5890 con 0x7f6038164f80 2024-05-14T11:17:01.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f6037400640 1 -- 172.21.15.116:0/2391100314 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300a5c60 con 0x7f6038164f80 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 -- 172.21.15.116:0/2391100314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 msgr2=0x7f60381673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 --2- 172.21.15.116:0/2391100314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381673a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6030009d90 tx=0x7f603001c8b0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 -- 172.21.15.116:0/2391100314 shutdown_connections 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 --2- 172.21.15.116:0/2391100314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381673a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 -- 172.21.15.116:0/2391100314 >> 172.21.15.116:0/2391100314 conn(0x7f60380b4620 msgr2=0x7f60380b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 -- 172.21.15.116:0/2391100314 shutdown_connections 2024-05-14T11:17:01.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.212+0000 7f603ec00640 1 -- 172.21.15.116:0/2391100314 wait complete. 2024-05-14T11:17:01.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 Processor -- start 2024-05-14T11:17:01.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- start start 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f603806f730 0x7f6038165390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f603816ab80 con 0x7f60380b0770 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f603816aa00 con 0x7f6038164f80 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f603816ad00 con 0x7f603806f730 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42060/0 (socket says 172.21.15.116:42060) 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 -- 172.21.15.116:0/2347953949 learned_addr learned my addr 172.21.15.116:0/2347953949 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:01.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603cc00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f6037e00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f603806f730 0x7f6038165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 -- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f603806f730 msgr2=0x7f6038165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f603806f730 0x7f6038165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 -- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 msgr2=0x7f60381670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 -- 172.21.15.116:0/2347953949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6038165b50 con 0x7f60380b0770 2024-05-14T11:17:01.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f602f400640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f602000c420 tx=0x7f602000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020017070 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60200052d0 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6038098630 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200054c0 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.216+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6038098bf0 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.220+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603806a9c0 con 0x7f60380b0770 2024-05-14T11:17:01.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.220+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f602001a070 con 0x7f60380b0770 2024-05-14T11:17:01.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.220+0000 7f6034c00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.220+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f602008ea60 con 0x7f60380b0770 2024-05-14T11:17:01.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.220+0000 7f6037e00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.224+0000 7f6037e00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6028007c40 tx=0x7f6028007b90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.224+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6038165b50 con 0x7f60380b0770 2024-05-14T11:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.416+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60381658e0 con 0x7f60380b0770 2024-05-14T11:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.416+0000 7f6034c00640 1 -- 172.21.15.116:0/2347953949 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f602005aee0 con 0x7f60380b0770 2024-05-14T11:17:01.420 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:01.421 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:17:01.421 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 msgr2=0x7f6014071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6028007c40 tx=0x7f6028007b90 comp rx=0 tx=0).stop 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 msgr2=0x7f603814b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f602000c420 tx=0x7f602000c8e0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 shutdown_connections 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038164f80 0x7f60381670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380b0770 0x7f603814b680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 --2- 172.21.15.116:0/2347953949 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f603806f730 0x7f6038165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 >> 172.21.15.116:0/2347953949 conn(0x7f60380b4620 msgr2=0x7f60380af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 shutdown_connections 2024-05-14T11:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.420+0000 7f603ec00640 1 -- 172.21.15.116:0/2347953949 wait complete. 2024-05-14T11:17:01.443 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:01.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 Processor -- start 2024-05-14T11:17:01.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 -- start start 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcc7806fdb0 0x7fcc780701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 0x7fcc78074060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc780b52f0 con 0x7fcc7806cf90 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcc78070700 con 0x7fcc7806fdb0 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc7d200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcc780745a0 con 0x7fcc78073c50 2024-05-14T11:17:01.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcc7806fdb0 0x7fcc780701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcc7806fdb0 0x7fcc780701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42066/0 (socket says 172.21.15.116:42066) 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 -- 172.21.15.116:0/2016720695 learned_addr learned my addr 172.21.15.116:0/2016720695 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc76a00640 1 --2- 172.21.15.116:0/2016720695 >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 0x7fcc78074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 -- 172.21.15.116:0/2016720695 >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 msgr2=0x7fcc78074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 --2- 172.21.15.116:0/2016720695 >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 0x7fcc78074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 -- 172.21.15.116:0/2016720695 >> v1:172.21.15.149:6789/0 conn(0x7fcc7806cf90 legacy=0x7fcc7806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc77400640 1 -- 172.21.15.116:0/2016720695 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7815dff0 con 0x7fcc7806fdb0 2024-05-14T11:17:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.588+0000 7fcc76a00640 1 --2- 172.21.15.116:0/2016720695 >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 0x7fcc78074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:01.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc77400640 1 --2- 172.21.15.116:0/2016720695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7806fdb0 0x7fcc780701c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc780b85b0 tx=0x7fcc6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c11fbd4b118887 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc76000640 1 -- 172.21.15.116:0/2016720695 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680b2070 con 0x7fcc7806fdb0 2024-05-14T11:17:01.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc76000640 1 -- 172.21.15.116:0/2016720695 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc680ad680 con 0x7fcc7806fdb0 2024-05-14T11:17:01.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc76000640 1 -- 172.21.15.116:0/2016720695 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680ada50 con 0x7fcc7806fdb0 2024-05-14T11:17:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 -- 172.21.15.116:0/2016720695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7806fdb0 msgr2=0x7fcc780701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 --2- 172.21.15.116:0/2016720695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7806fdb0 0x7fcc780701c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc780b85b0 tx=0x7fcc6801c490 comp rx=0 tx=0).stop 2024-05-14T11:17:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 -- 172.21.15.116:0/2016720695 shutdown_connections 2024-05-14T11:17:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 --2- 172.21.15.116:0/2016720695 >> v2:172.21.15.149:3300/0 conn(0x7fcc78073c50 0x7fcc78074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 --2- 172.21.15.116:0/2016720695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc7806fdb0 0x7fcc780701c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 -- 172.21.15.116:0/2016720695 >> 172.21.15.116:0/2016720695 conn(0x7fcc780d26e0 msgr2=0x7fcc780d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 -- 172.21.15.116:0/2016720695 shutdown_connections 2024-05-14T11:17:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.592+0000 7fcc7d200640 1 -- 172.21.15.116:0/2016720695 wait complete. 2024-05-14T11:17:01.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 Processor -- start 2024-05-14T11:17:01.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 -- start start 2024-05-14T11:17:01.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc7806cf90 0x7fcc780c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7806fdb0 0x7fcc78143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7806e270 con 0x7fcc78073c50 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78070950 con 0x7fcc7806fdb0 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc7d200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7816b080 con 0x7fcc7806cf90 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42082/0 (socket says 172.21.15.116:42082) 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 -- 172.21.15.116:0/3669616196 learned_addr learned my addr 172.21.15.116:0/3669616196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc76a00640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7806fdb0 0x7fcc78143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77400640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc7806cf90 0x7fcc780c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 -- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc7806cf90 msgr2=0x7fcc780c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc7806cf90 0x7fcc780c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 -- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7806fdb0 msgr2=0x7fcc78143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7806fdb0 0x7fcc78143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.596+0000 7fcc77e00640 1 -- 172.21.15.116:0/3669616196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc780c7d80 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc77e00640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c740 tx=0x7fcc7000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70003b80 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7000ce50 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc78143db0 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000a850 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7815cd60 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7815c7f0 con 0x7fcc78073c50 2024-05-14T11:17:01.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcc7000a9f0 con 0x7fcc78073c50 2024-05-14T11:17:01.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 0x7fcc58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fcc7008f440 con 0x7fcc78073c50 2024-05-14T11:17:01.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.600+0000 7fcc77400640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 0x7fcc58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.604+0000 7fcc77400640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 0x7fcc58071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc680af000 tx=0x7fcc68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.608+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcc780b52f0 con 0x7fcc78073c50 2024-05-14T11:17:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.764+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc78050880 con 0x7fcc78073c50 2024-05-14T11:17:01.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.764+0000 7fcc6f400640 1 -- 172.21.15.116:0/3669616196 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18969 (secure 0 0 0) 0x7fcc7005b8c0 con 0x7fcc78073c50 2024-05-14T11:17:01.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:01.769 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:53.384022+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 msgr2=0x7fcc58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 0x7fcc58071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc680af000 tx=0x7fcc68008040 comp rx=0 tx=0).stop 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 msgr2=0x7fcc780cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000c740 tx=0x7fcc7000cc00 comp rx=0 tx=0).stop 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 shutdown_connections 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcc5806f400 0x7fcc58071910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc7806fdb0 0x7fcc78143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc78073c50 0x7fcc780cf110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 --2- 172.21.15.116:0/3669616196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcc7806cf90 0x7fcc780c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 >> 172.21.15.116:0/3669616196 conn(0x7fcc780d26e0 msgr2=0x7fcc7806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 shutdown_connections 2024-05-14T11:17:01.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.768+0000 7fcc7d200640 1 -- 172.21.15.116:0/3669616196 wait complete. 2024-05-14T11:17:01.793 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:17:01.793 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:01.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.936+0000 7f6d29200640 1 Processor -- start 2024-05-14T11:17:01.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.936+0000 7f6d29200640 1 -- start start 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d29200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d240af830 0x7f6d240d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d29200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6d240d1bb0 0x7f6d2406edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d29200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d240d1fc0 con 0x7f6d240af420 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d29200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2415d340 con 0x7f6d240af830 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d29200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2415d620 con 0x7f6d240d1bb0 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d240af830 0x7f6d240d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d240af830 0x7f6d240d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42086/0 (socket says 172.21.15.116:42086) 2024-05-14T11:17:01.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 -- 172.21.15.116:0/2247557983 learned_addr learned my addr 172.21.15.116:0/2247557983 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:01.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d22800640 1 --2- 172.21.15.116:0/2247557983 >> v2:172.21.15.149:3300/0 conn(0x7f6d240d1bb0 0x7f6d2406edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 -- 172.21.15.116:0/2247557983 >> v2:172.21.15.149:3300/0 conn(0x7f6d240d1bb0 msgr2=0x7f6d2406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:01.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 --2- 172.21.15.116:0/2247557983 >> v2:172.21.15.149:3300/0 conn(0x7f6d240d1bb0 0x7f6d2406edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 -- 172.21.15.116:0/2247557983 >> v1:172.21.15.149:6789/0 conn(0x7f6d240af420 legacy=0x7f6d24074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.940+0000 7f6d21400640 1 -- 172.21.15.116:0/2247557983 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2415d900 con 0x7f6d240af830 2024-05-14T11:17:01.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d21400640 1 --2- 172.21.15.116:0/2247557983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d240d1670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d24160420 tx=0x7f6d1001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b315dbd9f3fbda8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d20a00640 1 -- 172.21.15.116:0/2247557983 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100ab430 con 0x7f6d240af830 2024-05-14T11:17:01.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d20a00640 1 -- 172.21.15.116:0/2247557983 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d100b0070 con 0x7f6d240af830 2024-05-14T11:17:01.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d20a00640 1 -- 172.21.15.116:0/2247557983 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100ab7f0 con 0x7f6d240af830 2024-05-14T11:17:01.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 -- 172.21.15.116:0/2247557983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 msgr2=0x7f6d240d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 --2- 172.21.15.116:0/2247557983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d240d1670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d24160420 tx=0x7f6d1001c4c0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 -- 172.21.15.116:0/2247557983 shutdown_connections 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 --2- 172.21.15.116:0/2247557983 >> v2:172.21.15.149:3300/0 conn(0x7f6d240d1bb0 0x7f6d2406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 --2- 172.21.15.116:0/2247557983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d240d1670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 -- 172.21.15.116:0/2247557983 >> 172.21.15.116:0/2247557983 conn(0x7f6d240b4620 msgr2=0x7f6d240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 -- 172.21.15.116:0/2247557983 shutdown_connections 2024-05-14T11:17:01.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 -- 172.21.15.116:0/2247557983 wait complete. 2024-05-14T11:17:01.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.944+0000 7f6d29200640 1 Processor -- start 2024-05-14T11:17:01.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- start start 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d240af420 0x7f6d2409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d240d1bb0 0x7f6d2409bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24169880 con 0x7f6d240af830 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d24169a00 con 0x7f6d240af420 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6d24169b80 con 0x7f6d240d1bb0 2024-05-14T11:17:01.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42098/0 (socket says 172.21.15.116:42098) 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 -- 172.21.15.116:0/1304589363 learned_addr learned my addr 172.21.15.116:0/1304589363 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21e00640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d240af420 0x7f6d2409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d22800640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d240d1bb0 0x7f6d2409bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 -- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d240d1bb0 msgr2=0x7f6d2409bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d240d1bb0 0x7f6d2409bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 -- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d240af420 msgr2=0x7f6d2409eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d240af420 0x7f6d2409eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 -- 172.21.15.116:0/1304589363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240b3480 con 0x7f6d240af830 2024-05-14T11:17:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d21400640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d1001ab70 tx=0x7f6d1001ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100be5f0 con 0x7f6d240af830 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d100be790 con 0x7f6d240af830 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d240b3760 con 0x7f6d240af830 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100beb40 con 0x7f6d240af830 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d241f7930 con 0x7f6d240af830 2024-05-14T11:17:01.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.948+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2406a9c0 con 0x7f6d240af830 2024-05-14T11:17:01.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.952+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d100b43f0 con 0x7f6d240af830 2024-05-14T11:17:01.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.952+0000 7f6d1a000640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 0x7f6d04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:01.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.952+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f6d100a8070 con 0x7f6d240af830 2024-05-14T11:17:01.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.952+0000 7f6d21e00640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 0x7f6d04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:01.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.952+0000 7f6d21e00640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 0x7f6d04071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d14007fd0 tx=0x7f6d14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:01.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:01.956+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6d10016110 con 0x7f6d240af830 2024-05-14T11:17:02.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d240b3930 con 0x7f6d240af830 2024-05-14T11:17:02.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d1a000640 1 -- 172.21.15.116:0/1304589363 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f6d100f6020 con 0x7f6d240af830 2024-05-14T11:17:02.151 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:02.151 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:17:02.151 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 msgr2=0x7f6d04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 0x7f6d04071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d14007fd0 tx=0x7f6d14009040 comp rx=0 tx=0).stop 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 msgr2=0x7f6d241f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d1001ab70 tx=0x7f6d1001ad00 comp rx=0 tx=0).stop 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 shutdown_connections 2024-05-14T11:17:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d0406f400 0x7f6d04071910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d240d1bb0 0x7f6d2409bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d240af830 0x7f6d241f70f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 --2- 172.21.15.116:0/1304589363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d240af420 0x7f6d2409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 >> 172.21.15.116:0/1304589363 conn(0x7f6d240b4620 msgr2=0x7f6d2415ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 shutdown_connections 2024-05-14T11:17:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.148+0000 7f6d29200640 1 -- 172.21.15.116:0/1304589363 wait complete. 2024-05-14T11:17:02.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:02.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 Processor -- start 2024-05-14T11:17:02.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 -- start start 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f69d40d2d50 0x7f69d40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69d40ce800 0x7f69d40b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d40cec10 con 0x7f69d41621b0 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f69d40d1d40 con 0x7f69d40ce800 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d9e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f69d40d3160 con 0x7f69d40d2d50 2024-05-14T11:17:02.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d2a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f69d41621b0 0x7f69d40b9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60280/0 (socket says 172.21.15.116:60280) 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d2a00640 1 -- 172.21.15.116:0/3431009925 learned_addr learned my addr 172.21.15.116:0/3431009925 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.316+0000 7f69d3400640 1 --2- 172.21.15.116:0/3431009925 >> v2:172.21.15.116:3300/0 conn(0x7f69d40ce800 0x7f69d40b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69ca000640 1 --2- 172.21.15.116:0/3431009925 >> v2:172.21.15.149:3300/0 conn(0x7f69d40d2d50 0x7f69d40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d2000640 1 -- 172.21.15.116:0/3431009925 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498012966 0 0) 0x7f69c4001120 con 0x7f69d41621b0 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d2000640 1 -- 172.21.15.116:0/3431009925 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69b8003530 con 0x7f69d41621b0 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d3400640 1 -- 172.21.15.116:0/3431009925 >> v2:172.21.15.149:3300/0 conn(0x7f69d40d2d50 msgr2=0x7f69d40ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d3400640 1 --2- 172.21.15.116:0/3431009925 >> v2:172.21.15.149:3300/0 conn(0x7f69d40d2d50 0x7f69d40ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d3400640 1 -- 172.21.15.116:0/3431009925 >> v1:172.21.15.116:6789/0 conn(0x7f69d41621b0 legacy=0x7f69d40b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d3400640 1 -- 172.21.15.116:0/3431009925 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d40d1f90 con 0x7f69d40ce800 2024-05-14T11:17:02.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d3400640 1 --2- 172.21.15.116:0/3431009925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d40ce800 0x7f69d40b2f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69d40d1ec0 tx=0x7f69cc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78341dadea4261d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d2000640 1 -- 172.21.15.116:0/3431009925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0b2070 con 0x7f69d40ce800 2024-05-14T11:17:02.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d2000640 1 -- 172.21.15.116:0/3431009925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc0a57f0 con 0x7f69d40ce800 2024-05-14T11:17:02.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.320+0000 7f69d2000640 1 -- 172.21.15.116:0/3431009925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0a5bc0 con 0x7f69d40ce800 2024-05-14T11:17:02.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- 172.21.15.116:0/3431009925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d40ce800 msgr2=0x7f69d40b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3431009925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d40ce800 0x7f69d40b2f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69d40d1ec0 tx=0x7f69cc01c930 comp rx=0 tx=0).stop 2024-05-14T11:17:02.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- 172.21.15.116:0/3431009925 shutdown_connections 2024-05-14T11:17:02.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3431009925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d40ce800 0x7f69d40b2f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3431009925 >> v2:172.21.15.149:3300/0 conn(0x7f69d40d2d50 0x7f69d40ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- 172.21.15.116:0/3431009925 >> 172.21.15.116:0/3431009925 conn(0x7f69d40d0770 msgr2=0x7f69d40d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:02.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- 172.21.15.116:0/3431009925 shutdown_connections 2024-05-14T11:17:02.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- 172.21.15.116:0/3431009925 wait complete. 2024-05-14T11:17:02.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 Processor -- start 2024-05-14T11:17:02.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.324+0000 7f69d9e00640 1 -- start start 2024-05-14T11:17:02.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 0x7f69d4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69d40d2d50 0x7f69d4159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69d415e3f0 con 0x7f69d41621b0 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69d415e270 con 0x7f69d40ce800 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69d415e570 con 0x7f69d40d2d50 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42110/0 (socket says 172.21.15.116:42110) 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 -- 172.21.15.116:0/3317313638 learned_addr learned my addr 172.21.15.116:0/3317313638 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d2a00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 0x7f69d4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69ca000640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69d40d2d50 0x7f69d4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 -- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69d40d2d50 msgr2=0x7f69d4159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69d40d2d50 0x7f69d4159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 -- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 msgr2=0x7f69d4150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 0x7f69d4150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 -- 172.21.15.116:0/3317313638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d4159a50 con 0x7f69d41621b0 2024-05-14T11:17:02.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d2a00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 0x7f69d4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d3400640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69cc01ce10 tx=0x7f69cc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0b2050 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc0b3040 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0ad650 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d4159d30 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d4151280 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d41514e0 con 0x7f69d41621b0 2024-05-14T11:17:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.328+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69cc0c0020 con 0x7f69d41621b0 2024-05-14T11:17:02.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.332+0000 7f69cbe00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 0x7f69b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.332+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f69cc129e10 con 0x7f69d41621b0 2024-05-14T11:17:02.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.332+0000 7f69d2a00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 0x7f69b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.332+0000 7f69d2a00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 0x7f69b0071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69c4005890 tx=0x7f69c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.336+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f69cc12d030 con 0x7f69d41621b0 2024-05-14T11:17:02.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69d406a9c0 con 0x7f69d41621b0 2024-05-14T11:17:02.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69cbe00640 1 -- 172.21.15.116:0/3317313638 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f69cc016520 con 0x7f69d41621b0 2024-05-14T11:17:02.535 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:02.535 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1384955686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1384955686},{"type":"v1","addr":"172.21.15.116:6837","nonce":1384955686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/856824707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":856824707},{"type":"v1","addr":"172.21.15.149:6834","nonce":856824707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/506974787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":506974787},{"type":"v1","addr":"172.21.15.149:6835","nonce":506974787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:16:11.615741+0000","modified":"2024-05-14T11:16:31.822599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2939305126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2939305126},{"type":"v1","addr":"172.21.15.116:6836","nonce":2939305126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-05-14T11:17:02.535 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 msgr2=0x7f69b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 0x7f69b0071940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69c4005890 tx=0x7f69c4009040 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 msgr2=0x7f69d4156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69cc01ce10 tx=0x7f69cc0196d0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 shutdown_connections 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69b006f430 0x7f69b0071940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41621b0 0x7f69d4156520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69d40d2d50 0x7f69d4159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 --2- 172.21.15.116:0/3317313638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69d40ce800 0x7f69d4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 >> 172.21.15.116:0/3317313638 conn(0x7f69d40d0770 msgr2=0x7f69d40704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:02.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 shutdown_connections 2024-05-14T11:17:02.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.532+0000 7f69d9e00640 1 -- 172.21.15.116:0/3317313638 wait complete. 2024-05-14T11:17:02.555 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:02.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 Processor -- start 2024-05-14T11:17:02.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 -- start start 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f25c40cfc70 0x7f25c4167920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c40d1340 con 0x7f25c4160d40 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c40d1620 con 0x7f25c40b68e0 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25ca600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f25c40b1f90 con 0x7f25c40cfc70 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f25c4160d40 0x7f25c40cf560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60282/0 (socket says 172.21.15.116:60282) 2024-05-14T11:17:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c3400640 1 -- 172.21.15.116:0/2380174375 learned_addr learned my addr 172.21.15.116:0/2380174375 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1284300850 0 0) 0x7f25c40d1340 con 0x7f25c4160d40 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a8003530 con 0x7f25c4160d40 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25baa00640 1 --2- 172.21.15.116:0/2380174375 >> v2:172.21.15.149:3300/0 conn(0x7f25c40cfc70 0x7f25c4167920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612708342 0 0) 0x7f25a8003530 con 0x7f25c4160d40 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 >> v2:172.21.15.149:3300/0 conn(0x7f25c40cfc70 msgr2=0x7f25c4167920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 --2- 172.21.15.116:0/2380174375 >> v2:172.21.15.149:3300/0 conn(0x7f25c40cfc70 0x7f25c4167920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 >> v1:172.21.15.149:6789/0 conn(0x7f25c40b68e0 legacy=0x7f25c4169e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c40d1900 con 0x7f25c4160d40 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f25b4001700 con 0x7f25c4160d40 2024-05-14T11:17:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 >> v1:172.21.15.116:6789/0 conn(0x7f25c4160d40 legacy=0x7f25c40cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8003a60 0x7f25a8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 0x7f25a8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25a8005260 0x7f25a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f25a800c1c0 con 0x7f25a8003a60 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25a800c370 con 0x7f25a8004810 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.700+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f25a800c570 con 0x7f25a8005260 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25c3400640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8003a60 0x7f25a8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 0x7f25a8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25c3e00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25a8005260 0x7f25a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 -- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25a8005260 msgr2=0x7f25a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25a8005260 0x7f25a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 -- 172.21.15.116:0/2380174375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8003a60 msgr2=0x7f25a8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8003a60 0x7f25a8003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 -- 172.21.15.116:0/2380174375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a8003530 con 0x7f25a8004810 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25c3400640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8003a60 0x7f25a8003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:02.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25c3e00640 1 -- 172.21.15.116:0/2380174375 reap_dead start 2024-05-14T11:17:02.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.704+0000 7f25baa00640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 0x7f25a8004c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009ae0 tx=0x7f25ac01ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e77086857f1543 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0b2070 con 0x7f25a8004810 2024-05-14T11:17:02.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ac0a5520 con 0x7f25a8004810 2024-05-14T11:17:02.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25c2a00640 1 -- 172.21.15.116:0/2380174375 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0096b0 con 0x7f25a8004810 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 msgr2=0x7f25a8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 0x7f25a8004c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009ae0 tx=0x7f25ac01ca70 comp rx=0 tx=0).stop 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- 172.21.15.116:0/2380174375 shutdown_connections 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 --2- 172.21.15.116:0/2380174375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25a8004810 0x7f25a8004c20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- 172.21.15.116:0/2380174375 >> 172.21.15.116:0/2380174375 conn(0x7f25c4071370 msgr2=0x7f25c4071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- 172.21.15.116:0/2380174375 shutdown_connections 2024-05-14T11:17:02.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- 172.21.15.116:0/2380174375 wait complete. 2024-05-14T11:17:02.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 Processor -- start 2024-05-14T11:17:02.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.708+0000 7f25ca600640 1 -- start start 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25c4151090 0x7f25c41514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25c414dec0 0x7f25c414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f25c40b5a50 con 0x7f25a8004810 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25c40b58d0 con 0x7f25c4151090 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f25c40b5bd0 con 0x7f25c414dec0 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42130/0 (socket says 172.21.15.116:42130) 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 -- 172.21.15.116:0/2653046137 learned_addr learned my addr 172.21.15.116:0/2653046137 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:02.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25baa00640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25c4151090 0x7f25c41514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3e00640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25c414dec0 0x7f25c414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 -- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25c414dec0 msgr2=0x7f25c414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25c414dec0 0x7f25c414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 -- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25c4151090 msgr2=0x7f25c41514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25c4151090 0x7f25c41514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 -- 172.21.15.116:0/2653046137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c415aca0 con 0x7f25a8004810 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25c3400640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25b40072f0 tx=0x7f25b40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b401a070 con 0x7f25a8004810 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b400fa20 con 0x7f25a8004810 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4007d70 con 0x7f25a8004810 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c415af10 con 0x7f25a8004810 2024-05-14T11:17:02.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c415b4d0 con 0x7f25a8004810 2024-05-14T11:17:02.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.712+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c406a9c0 con 0x7f25a8004810 2024-05-14T11:17:02.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.716+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25b401d360 con 0x7f25a8004810 2024-05-14T11:17:02.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.716+0000 7f25bbe00640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 0x7f25a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:02.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.716+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f25b408efc0 con 0x7f25a8004810 2024-05-14T11:17:02.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.716+0000 7f25baa00640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 0x7f25a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:02.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.716+0000 7f25baa00640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 0x7f25a0071940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0af000 tx=0x7f25ac0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:02.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.720+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f25b405b430 con 0x7f25a8004810 2024-05-14T11:17:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.880+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25c40707d0 con 0x7f25a8004810 2024-05-14T11:17:02.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.880+0000 7f25bbe00640 1 -- 172.21.15.116:0/2653046137 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18969 (secure 0 0 0) 0x7f25b405f730 con 0x7f25a8004810 2024-05-14T11:17:02.884 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:02.884 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:16:53.384022+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:16:09.523112+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T11:16:10.584877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T11:16:14.274286+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 msgr2=0x7f25a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 0x7f25a0071940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0af000 tx=0x7f25ac0046d0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 msgr2=0x7f25c40aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25b40072f0 tx=0x7f25b40077b0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 shutdown_connections 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f25a006f430 0x7f25a0071940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25c4151090 0x7f25c41514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25c414dec0 0x7f25c414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 --2- 172.21.15.116:0/2653046137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25a8004810 0x7f25c40aadf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 >> 172.21.15.116:0/2653046137 conn(0x7f25c4071370 msgr2=0x7f25c40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 shutdown_connections 2024-05-14T11:17:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:02.884+0000 7f25ca600640 1 -- 172.21.15.116:0/2653046137 wait complete. 2024-05-14T11:17:02.906 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:17:03.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.048+0000 7fb60f400640 1 Processor -- start 2024-05-14T11:17:03.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.048+0000 7fb60f400640 1 -- start start 2024-05-14T11:17:03.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6080b0770 0x7fb60806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60f400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb608072a30 0x7fb608072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60f400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb60806cc40 con 0x7fb60806f730 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60f400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb60806db20 con 0x7fb6080b0770 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60f400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb60806cf10 con 0x7fb608072a30 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6080b0770 0x7fb60806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6080b0770 0x7fb60806c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42134/0 (socket says 172.21.15.116:42134) 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 -- 172.21.15.116:0/3772760836 learned_addr learned my addr 172.21.15.116:0/3772760836 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60d400640 1 --2- 172.21.15.116:0/3772760836 >> v2:172.21.15.149:3300/0 conn(0x7fb608072a30 0x7fb608072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 -- 172.21.15.116:0/3772760836 >> v2:172.21.15.149:3300/0 conn(0x7fb608072a30 msgr2=0x7fb608072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 --2- 172.21.15.116:0/3772760836 >> v2:172.21.15.149:3300/0 conn(0x7fb608072a30 0x7fb608072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 -- 172.21.15.116:0/3772760836 >> v1:172.21.15.149:6789/0 conn(0x7fb60806f730 legacy=0x7fb60806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:03.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.052+0000 7fb60ca00640 1 -- 172.21.15.116:0/3772760836 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60806d1f0 con 0x7fb6080b0770 2024-05-14T11:17:03.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60ca00640 1 --2- 172.21.15.116:0/3772760836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6080b0770 0x7fb60806c700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6080729c0 tx=0x7fb5f801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ced37932026c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb607e00640 1 -- 172.21.15.116:0/3772760836 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b1070 con 0x7fb6080b0770 2024-05-14T11:17:03.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb607e00640 1 -- 172.21.15.116:0/3772760836 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80a8050 con 0x7fb6080b0770 2024-05-14T11:17:03.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb607e00640 1 -- 172.21.15.116:0/3772760836 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80ac860 con 0x7fb6080b0770 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 -- 172.21.15.116:0/3772760836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6080b0770 msgr2=0x7fb60806c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 --2- 172.21.15.116:0/3772760836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6080b0770 0x7fb60806c700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6080729c0 tx=0x7fb5f801c460 comp rx=0 tx=0).stop 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 -- 172.21.15.116:0/3772760836 shutdown_connections 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 --2- 172.21.15.116:0/3772760836 >> v2:172.21.15.149:3300/0 conn(0x7fb608072a30 0x7fb608072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 --2- 172.21.15.116:0/3772760836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6080b0770 0x7fb60806c700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 -- 172.21.15.116:0/3772760836 >> 172.21.15.116:0/3772760836 conn(0x7fb6080b4620 msgr2=0x7fb6080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 -- 172.21.15.116:0/3772760836 shutdown_connections 2024-05-14T11:17:03.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.056+0000 7fb60f400640 1 -- 172.21.15.116:0/3772760836 wait complete. 2024-05-14T11:17:03.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 Processor -- start 2024-05-14T11:17:03.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- start start 2024-05-14T11:17:03.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb608072a30 0x7fb6080cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6080b0770 0x7fb6080cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb608073500 con 0x7fb60806f730 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb608073380 con 0x7fb6080b0770 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb608073680 con 0x7fb608072a30 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42150/0 (socket says 172.21.15.116:42150) 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 -- 172.21.15.116:0/12327279 learned_addr learned my addr 172.21.15.116:0/12327279 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60d400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6080b0770 0x7fb6080cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb5ffe00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb608072a30 0x7fb6080cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 -- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb608072a30 msgr2=0x7fb6080cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb608072a30 0x7fb6080cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 -- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6080b0770 msgr2=0x7fb6080cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6080b0770 0x7fb6080cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 -- 172.21.15.116:0/12327279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080cb030 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60ca00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8019070 tx=0x7fb5f801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80bc070 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80b7650 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6080cb310 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b1040 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6080cb8d0 con 0x7fb60806f730 2024-05-14T11:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.060+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60806a9c0 con 0x7fb60806f730 2024-05-14T11:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.064+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5f80bf070 con 0x7fb60806f730 2024-05-14T11:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.064+0000 7fb605600640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 0x7fb5e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.064+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7fb5f81290c0 con 0x7fb60806f730 2024-05-14T11:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.064+0000 7fb5ffe00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 0x7fb5e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.064+0000 7fb5ffe00640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 0x7fb5e4071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6081fc800 tx=0x7fb5f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.068+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb5f8016110 con 0x7fb60806f730 2024-05-14T11:17:03.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.260+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb60806f510 con 0x7fb60806f730 2024-05-14T11:17:03.264 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:17:03.260+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:17:03.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.264+0000 7fb605600640 1 -- 172.21.15.116:0/12327279 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fb5f80f5b90 con 0x7fb60806f730 2024-05-14T11:17:03.266 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.264+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 msgr2=0x7fb5e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.264+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 0x7fb5e4071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6081fc800 tx=0x7fb5f0009040 comp rx=0 tx=0).stop 2024-05-14T11:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.264+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 msgr2=0x7fb60809aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.264+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8019070 tx=0x7fb5f801aca0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 shutdown_connections 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5e406f400 0x7fb5e4071910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6080b0770 0x7fb6080cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb608072a30 0x7fb6080cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 --2- 172.21.15.116:0/12327279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb60806f730 0x7fb60809aa40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 >> 172.21.15.116:0/12327279 conn(0x7fb6080b4620 msgr2=0x7fb6080b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 shutdown_connections 2024-05-14T11:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.268+0000 7fb60f400640 1 -- 172.21.15.116:0/12327279 wait complete. 2024-05-14T11:17:03.281 INFO:tasks.ceph.mds.c.smithi116.stderr: -3579> 2024-05-14T11:16:18.197+0000 7f4603200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:17:03.299 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:17:03.408 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:17:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.448+0000 7ff1bfc00640 1 Processor -- start 2024-05-14T11:17:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.448+0000 7ff1bfc00640 1 -- start start 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bfc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1b80c70e0 0x7ff1b815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bfc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 0x7ff1b8163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bfc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b80c74f0 con 0x7ff1b81642d0 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bfc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b80cee00 con 0x7ff1b80c70e0 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bfc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b80cfe50 con 0x7ff1b8163980 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1b80c70e0 0x7ff1b815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1b80c70e0 0x7ff1b815fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42168/0 (socket says 172.21.15.116:42168) 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 -- 172.21.15.116:0/2909523885 learned_addr learned my addr 172.21.15.116:0/2909523885 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1afe00640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 0x7ff1b8163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 -- 172.21.15.116:0/2909523885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 msgr2=0x7ff1b8163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:03.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 0x7ff1b8163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 -- 172.21.15.116:0/2909523885 >> v1:172.21.15.149:6789/0 conn(0x7ff1b81642d0 legacy=0x7ff1b806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1bd200640 1 -- 172.21.15.116:0/2909523885 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b806e260 con 0x7ff1b80c70e0 2024-05-14T11:17:03.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.452+0000 7ff1afe00640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 0x7ff1b8163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bd200640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b80c70e0 0x7ff1b815fbc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80098a0 tx=0x7ff1a801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2fe40f278e46bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1b7e00640 1 -- 172.21.15.116:0/2909523885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80b5070 con 0x7ff1b80c70e0 2024-05-14T11:17:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1b7e00640 1 -- 172.21.15.116:0/2909523885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80a8a30 con 0x7ff1b80c70e0 2024-05-14T11:17:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1b7e00640 1 -- 172.21.15.116:0/2909523885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80a8e00 con 0x7ff1b80c70e0 2024-05-14T11:17:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 -- 172.21.15.116:0/2909523885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b80c70e0 msgr2=0x7ff1b815fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b80c70e0 0x7ff1b815fbc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80098a0 tx=0x7ff1a801ed30 comp rx=0 tx=0).stop 2024-05-14T11:17:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 -- 172.21.15.116:0/2909523885 shutdown_connections 2024-05-14T11:17:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b8163980 0x7ff1b8163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/2909523885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b80c70e0 0x7ff1b815fbc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 -- 172.21.15.116:0/2909523885 >> 172.21.15.116:0/2909523885 conn(0x7ff1b80d0670 msgr2=0x7ff1b80d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 -- 172.21.15.116:0/2909523885 shutdown_connections 2024-05-14T11:17:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.456+0000 7ff1bfc00640 1 -- 172.21.15.116:0/2909523885 wait complete. 2024-05-14T11:17:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 Processor -- start 2024-05-14T11:17:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 -- start start 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b80c70e0 0x7ff1b81ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1b81642d0 0x7ff1b81f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8164870 con 0x7ff1b8163980 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b806eac0 con 0x7ff1b81642d0 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bfc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b8160100 con 0x7ff1b80c70e0 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42182/0 (socket says 172.21.15.116:42182) 2024-05-14T11:17:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 -- 172.21.15.116:0/3301001304 learned_addr learned my addr 172.21.15.116:0/3301001304 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bd200640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b80c70e0 0x7ff1b81ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1bdc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1b81642d0 0x7ff1b81f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 -- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b80c70e0 msgr2=0x7ff1b81ea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b80c70e0 0x7ff1b81ea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 -- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1b81642d0 msgr2=0x7ff1b81f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1b81642d0 0x7ff1b81f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 -- 172.21.15.116:0/3301001304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b81611f0 con 0x7ff1b8163980 2024-05-14T11:17:03.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1afe00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000d800 tx=0x7ff1a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0009640 con 0x7ff1b8163980 2024-05-14T11:17:03.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a0019070 con 0x7ff1b8163980 2024-05-14T11:17:03.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.460+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0009a20 con 0x7ff1b8163980 2024-05-14T11:17:03.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.464+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81614d0 con 0x7ff1b8163980 2024-05-14T11:17:03.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.464+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b8161a90 con 0x7ff1b8163980 2024-05-14T11:17:03.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.464+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b806a9c0 con 0x7ff1b8163980 2024-05-14T11:17:03.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.464+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff1a0009bc0 con 0x7ff1b8163980 2024-05-14T11:17:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.468+0000 7ff1b5600640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 0x7ff194071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.468+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6829+0+0 (secure 0 0 0) 0x7ff1a0090630 con 0x7ff1b8163980 2024-05-14T11:17:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.468+0000 7ff1bd200640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 0x7ff194071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:03.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.468+0000 7ff1bd200640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 0x7ff194071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b2000 tx=0x7ff1a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.472+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff1a005ca70 con 0x7ff1b8163980 2024-05-14T11:17:03.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:03.660+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff1b8160f10 con 0x7ff1b8163980 2024-05-14T11:17:04.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.264+0000 7ff1b5600640 1 -- 172.21.15.116:0/3301001304 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff1a0060d70 con 0x7ff1b8163980 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 msgr2=0x7ff194071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 0x7ff194071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b2000 tx=0x7ff1a8009040 comp rx=0 tx=0).stop 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 msgr2=0x7ff1b814aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000d800 tx=0x7ff1a000dcc0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 shutdown_connections 2024-05-14T11:17:04.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff19406f400 0x7ff194071910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1b81642d0 0x7ff1b81f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1b8163980 0x7ff1b814aa00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 --2- 172.21.15.116:0/3301001304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1b80c70e0 0x7ff1b81ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 >> 172.21.15.116:0/3301001304 conn(0x7ff1b80d0670 msgr2=0x7ff1b80c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 shutdown_connections 2024-05-14T11:17:04.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.268+0000 7ff1bfc00640 1 -- 172.21.15.116:0/3301001304 wait complete. 2024-05-14T11:17:04.294 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 Processor -- start 2024-05-14T11:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 -- start start 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f76f406ef80 0x7f76f40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f415d360 con 0x7f76f4070640 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f76f415dc90 con 0x7f76f406ef80 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76fa800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f76f415d640 con 0x7f76f4070a50 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76f3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f76f4070640 0x7f76f406e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60310/0 (socket says 172.21.15.116:60310) 2024-05-14T11:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76f3400640 1 -- 172.21.15.116:0/1774938979 learned_addr learned my addr 172.21.15.116:0/1774938979 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1774938979 >> v2:172.21.15.149:3300/0 conn(0x7f76f406ef80 0x7f76f40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76f3e00640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76f2a00640 1 -- 172.21.15.116:0/1774938979 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587472314 0 0) 0x7f76f415d360 con 0x7f76f4070640 2024-05-14T11:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76f2a00640 1 -- 172.21.15.116:0/1774938979 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d8003530 con 0x7f76f4070640 2024-05-14T11:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76ebe00640 1 -- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 msgr2=0x7f76f40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76ebe00640 1 -- 172.21.15.116:0/1774938979 >> v1:172.21.15.116:6789/0 conn(0x7f76f4070640 legacy=0x7f76f406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.452+0000 7f76ebe00640 1 -- 172.21.15.116:0/1774938979 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f415d920 con 0x7f76f406ef80 2024-05-14T11:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f406ef80 0x7f76f40cf280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b44278561ee735d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76f2a00640 1 -- 172.21.15.116:0/1774938979 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0aa620 con 0x7f76f406ef80 2024-05-14T11:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76f2a00640 1 -- 172.21.15.116:0/1774938979 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc0af070 con 0x7f76f406ef80 2024-05-14T11:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76f2a00640 1 -- 172.21.15.116:0/1774938979 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0aa9e0 con 0x7f76f406ef80 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 -- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f406ef80 msgr2=0x7f76f40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f406ef80 0x7f76f40cf280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 -- 172.21.15.116:0/1774938979 shutdown_connections 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 --2- 172.21.15.116:0/1774938979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f406ef80 0x7f76f40cf280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 -- 172.21.15.116:0/1774938979 >> 172.21.15.116:0/1774938979 conn(0x7f76f40cde80 msgr2=0x7f76f40ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 -- 172.21.15.116:0/1774938979 shutdown_connections 2024-05-14T11:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.456+0000 7f76fa800640 1 -- 172.21.15.116:0/1774938979 wait complete. 2024-05-14T11:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 Processor -- start 2024-05-14T11:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 -- start start 2024-05-14T11:17:04.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f40ab4d0 0x7f76f40aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41699a0 con 0x7f76f4070640 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76f40cda10 con 0x7f76f40ab4d0 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76fa800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f76f4169b20 con 0x7f76f4070a50 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42198/0 (socket says 172.21.15.116:42198) 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 -- 172.21.15.116:0/1797448867 learned_addr learned my addr 172.21.15.116:0/1797448867 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3e00640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f40ab4d0 0x7f76f40aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 -- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 msgr2=0x7f76f415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 -- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f40ab4d0 msgr2=0x7f76f40aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f40ab4d0 0x7f76f40aa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76f3400640 1 -- 172.21.15.116:0/1797448867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f40ac880 con 0x7f76f4070640 2024-05-14T11:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.460+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76f3400640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c430 tx=0x7f76e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4017070 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e4005cf0 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40072f0 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f40acb60 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f40ae800 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f40ad080 con 0x7f76f4070640 2024-05-14T11:17:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76e401a070 con 0x7f76f4070640 2024-05-14T11:17:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 0x7f76d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6829+0+0 (secure 0 0 0) 0x7f76e408e8d0 con 0x7f76f4070640 2024-05-14T11:17:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.464+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 0x7f76d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.468+0000 7f76ebe00640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 0x7f76d0071940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ae000 tx=0x7f76dc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.468+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f76e405aef0 con 0x7f76f4070640 2024-05-14T11:17:04.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:04.628+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76f40ac5c0 con 0x7f76f4070640 2024-05-14T11:17:05.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.444+0000 7f76eb400640 1 -- 172.21.15.116:0/1797448867 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f76e405f1f0 con 0x7f76f4070640 2024-05-14T11:17:05.448 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:17:05.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 msgr2=0x7f76d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 0x7f76d0071940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ae000 tx=0x7f76dc0a7000 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 msgr2=0x7f76f415a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c430 tx=0x7f76e400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 shutdown_connections 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76d006f430 0x7f76d0071940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76f40ab4d0 0x7f76f40aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76f4070a50 0x7f76f415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 --2- 172.21.15.116:0/1797448867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76f4070640 0x7f76f415a2d0 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c430 tx=0x7f76e400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 >> 172.21.15.116:0/1797448867 conn(0x7f76f40cde80 msgr2=0x7f76f4158160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:05.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.448+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 shutdown_connections 2024-05-14T11:17:05.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.452+0000 7f76fa800640 1 -- 172.21.15.116:0/1797448867 wait complete. 2024-05-14T11:17:05.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:17:05.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 Processor -- start 2024-05-14T11:17:05.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 -- start start 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f93fc06fdc0 0x7f93fc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93fc0d04c0 con 0x7f93fc0cd620 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93fc0d07a0 con 0x7f93fc0cda30 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f9402c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f93fc0d0270 con 0x7f93fc06fdc0 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f93fbe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f93fc0cd620 0x7f93fc06f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60324/0 (socket says 172.21.15.116:60324) 2024-05-14T11:17:05.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.628+0000 7f93fbe00640 1 -- 172.21.15.116:0/315693426 learned_addr learned my addr 172.21.15.116:0/315693426 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93fb400640 1 --2- 172.21.15.116:0/315693426 >> v2:172.21.15.149:3300/0 conn(0x7f93fc06fdc0 0x7f93fc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843380388 0 0) 0x7f93fc0d04c0 con 0x7f93fc0cd620 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93e0003530 con 0x7f93fc0cd620 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3483060024 0 0) 0x7f93e0003530 con 0x7f93fc0cd620 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 >> v2:172.21.15.149:3300/0 conn(0x7f93fc06fdc0 msgr2=0x7f93fc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 --2- 172.21.15.116:0/315693426 >> v2:172.21.15.149:3300/0 conn(0x7f93fc06fdc0 0x7f93fc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 >> v1:172.21.15.149:6789/0 conn(0x7f93fc0cda30 legacy=0x7f93fc06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc0b08c0 con 0x7f93fc0cd620 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f93ec001700 con 0x7f93fc0cd620 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 >> v1:172.21.15.116:6789/0 conn(0x7f93fc0cd620 legacy=0x7f93fc06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93e0003a60 0x7f93e0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93e0004810 0x7f93e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc0d0270 con 0x7f93e0005260 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93e000c1f0 con 0x7f93e0003a60 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f93e000c3f0 con 0x7f93e0004810 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93fbe00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93e0003a60 0x7f93e0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f93fb400640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93e0004810 0x7f93e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 -- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93e0004810 msgr2=0x7f93e0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93e0004810 0x7f93e0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 -- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93e0003a60 msgr2=0x7f93e0003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:05.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93e0003a60 0x7f93e0003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.632+0000 7f9400c00640 1 -- 172.21.15.116:0/315693426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e0003530 con 0x7f93e0005260 2024-05-14T11:17:05.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9400c00640 1 -- 172.21.15.116:0/315693426 reap_dead start 2024-05-14T11:17:05.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9400c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93e000b9d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001e90 tx=0x7f93f401c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42293ffc3bd8df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:05.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4001ae0 con 0x7f93e0005260 2024-05-14T11:17:05.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f4003b80 con 0x7f93e0005260 2024-05-14T11:17:05.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f93faa00640 1 -- 172.21.15.116:0/315693426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40050d0 con 0x7f93e0005260 2024-05-14T11:17:05.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 msgr2=0x7f93e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93e000b9d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001e90 tx=0x7f93f401c770 comp rx=0 tx=0).stop 2024-05-14T11:17:05.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- 172.21.15.116:0/315693426 shutdown_connections 2024-05-14T11:17:05.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 --2- 172.21.15.116:0/315693426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93e000b9d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- 172.21.15.116:0/315693426 >> 172.21.15.116:0/315693426 conn(0x7f93fc0cde80 msgr2=0x7f93fc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:05.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- 172.21.15.116:0/315693426 shutdown_connections 2024-05-14T11:17:05.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- 172.21.15.116:0/315693426 wait complete. 2024-05-14T11:17:05.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 Processor -- start 2024-05-14T11:17:05.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.636+0000 7f9402c00640 1 -- start start 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 0x7f93fc147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93fc0aed80 0x7f93fc0af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc0cda30 con 0x7f93e0005260 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc0b0c30 con 0x7f93fc144520 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f93fc0cdbb0 con 0x7f93fc0aed80 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42220/0 (socket says 172.21.15.116:42220) 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 -- 172.21.15.116:0/2768341296 learned_addr learned my addr 172.21.15.116:0/2768341296 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fb400640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 0x7f93fc147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9400c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93fc0aed80 0x7f93fc0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 -- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93fc0aed80 msgr2=0x7f93fc0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93fc0aed80 0x7f93fc0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 -- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 msgr2=0x7f93fc147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 0x7f93fc147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 -- 172.21.15.116:0/2768341296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc0b0320 con 0x7f93e0005260 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fb400640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 0x7f93fc147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93fbe00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007340 tx=0x7f93ec007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec01a070 con 0x7f93e0005260 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ec0016b0 con 0x7f93e0005260 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00fc00 con 0x7f93e0005260 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc0af6d0 con 0x7f93e0005260 2024-05-14T11:17:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.640+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc144fa0 con 0x7f93e0005260 2024-05-14T11:17:05.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc06aa30 con 0x7f93e0005260 2024-05-14T11:17:05.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93ec01d360 con 0x7f93e0005260 2024-05-14T11:17:05.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f93f3e00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 0x7f93dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:05.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6391+0+0 (secure 0 0 0) 0x7f93ec08ed20 con 0x7f93e0005260 2024-05-14T11:17:05.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f93fb400640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 0x7f93dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:05.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.644+0000 7f93fb400640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 0x7f93dc071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0afa20 tx=0x7f93e8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:05.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.648+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f93fc0af6d0 con 0x7f93e0005260 2024-05-14T11:17:05.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:05.808+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93fc0cfb40 con 0x7f93e0005260 2024-05-14T11:17:06.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.452+0000 7f93f3e00640 1 -- 172.21.15.116:0/2768341296 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f93ec05b390 con 0x7f93e0005260 2024-05-14T11:17:06.454 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.452+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 msgr2=0x7f93dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.452+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 0x7f93dc071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0afa20 tx=0x7f93e8009190 comp rx=0 tx=0).stop 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.452+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 msgr2=0x7f93fc1fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.452+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f93ec007340 tx=0x7f93ec007740 comp rx=0 tx=0).stop 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 shutdown_connections 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f93dc06f430 0x7f93dc071940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93fc144520 0x7f93fc147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93fc0aed80 0x7f93fc0af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 --2- 172.21.15.116:0/2768341296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93e0005260 0x7f93fc1fa1e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 >> 172.21.15.116:0/2768341296 conn(0x7f93fc0cde80 msgr2=0x7f93fc06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:06.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 shutdown_connections 2024-05-14T11:17:06.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.456+0000 7f9402c00640 1 -- 172.21.15.116:0/2768341296 wait complete. 2024-05-14T11:17:06.484 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-14T11:17:06.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.632+0000 7fec43600640 1 Processor -- start 2024-05-14T11:17:06.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 -- start start 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fec3c070640 0x7fec3c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fec3c070a50 0x7fec3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c1421f0 0x7fec3c142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c15dc90 con 0x7fec3c070640 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fec3c15d360 con 0x7fec3c070a50 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec43600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c15d640 con 0x7fec3c1421f0 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fec3c070640 0x7fec3c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fec3c070640 0x7fec3c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42224/0 (socket says 172.21.15.116:42224) 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 -- 172.21.15.116:0/4068845829 learned_addr learned my addr 172.21.15.116:0/4068845829 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec41600640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c1421f0 0x7fec3c142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec33e00640 1 --2- 172.21.15.116:0/4068845829 >> v2:172.21.15.149:3300/0 conn(0x7fec3c070a50 0x7fec3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 -- 172.21.15.116:0/4068845829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c1421f0 msgr2=0x7fec3c142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c1421f0 0x7fec3c142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 -- 172.21.15.116:0/4068845829 >> v2:172.21.15.149:3300/0 conn(0x7fec3c070a50 msgr2=0x7fec3c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 --2- 172.21.15.116:0/4068845829 >> v2:172.21.15.149:3300/0 conn(0x7fec3c070a50 0x7fec3c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.636+0000 7fec40c00640 1 -- 172.21.15.116:0/4068845829 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c15d920 con 0x7fec3c070640 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec40c00640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c070640 0x7fec3c06e870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec3c15d850 tx=0x7fec2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b614641314ceab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec3be00640 1 -- 172.21.15.116:0/4068845829 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0aa660 con 0x7fec3c070640 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec3be00640 1 -- 172.21.15.116:0/4068845829 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c0af070 con 0x7fec3c070640 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec3be00640 1 -- 172.21.15.116:0/4068845829 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0aaa20 con 0x7fec3c070640 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 -- 172.21.15.116:0/4068845829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c070640 msgr2=0x7fec3c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c070640 0x7fec3c06e870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec3c15d850 tx=0x7fec2c01c520 comp rx=0 tx=0).stop 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 -- 172.21.15.116:0/4068845829 shutdown_connections 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c1421f0 0x7fec3c142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 --2- 172.21.15.116:0/4068845829 >> v2:172.21.15.149:3300/0 conn(0x7fec3c070a50 0x7fec3c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 --2- 172.21.15.116:0/4068845829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c070640 0x7fec3c06e870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 -- 172.21.15.116:0/4068845829 >> 172.21.15.116:0/4068845829 conn(0x7fec3c0cde80 msgr2=0x7fec3c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:06.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 -- 172.21.15.116:0/4068845829 shutdown_connections 2024-05-14T11:17:06.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.640+0000 7fec43600640 1 -- 172.21.15.116:0/4068845829 wait complete. 2024-05-14T11:17:06.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 Processor -- start 2024-05-14T11:17:06.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 -- start start 2024-05-14T11:17:06.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec3c070640 0x7fec3c145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c070a50 0x7fec3c140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0cd980 con 0x7fec3c1421f0 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c0cd800 con 0x7fec3c070640 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c0cfa90 con 0x7fec3c070a50 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42226/0 (socket says 172.21.15.116:42226) 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 -- 172.21.15.116:0/679992340 learned_addr learned my addr 172.21.15.116:0/679992340 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec40c00640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec3c070640 0x7fec3c145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec33e00640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c070a50 0x7fec3c140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 -- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c070a50 msgr2=0x7fec3c140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c070a50 0x7fec3c140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 -- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec3c070640 msgr2=0x7fec3c145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec3c070640 0x7fec3c145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 -- 172.21.15.116:0/679992340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c06cf90 con 0x7fec3c1421f0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec41600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec34009cb0 tx=0x7fec3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34018020 con 0x7fec3c1421f0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34014480 con 0x7fec3c1421f0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34014870 con 0x7fec3c1421f0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.644+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c06d270 con 0x7fec3c1421f0 2024-05-14T11:17:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c06d830 con 0x7fec3c1421f0 2024-05-14T11:17:06.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c06aa30 con 0x7fec3c1421f0 2024-05-14T11:17:06.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec3401b020 con 0x7fec3c1421f0 2024-05-14T11:17:06.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec39600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 0x7fec18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:06.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fec3408fe50 con 0x7fec3c1421f0 2024-05-14T11:17:06.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec40c00640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 0x7fec18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:06.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.648+0000 7fec40c00640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 0x7fec18071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ae000 tx=0x7fec2c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:06.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.652+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fec34022030 con 0x7fec3c1421f0 2024-05-14T11:17:06.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:06.820+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec3c06cd30 con 0x7fec3c1421f0 2024-05-14T11:17:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.460+0000 7fec39600640 1 -- 172.21.15.116:0/679992340 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fec34010040 con 0x7fec3c1421f0 2024-05-14T11:17:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data_ec' removed 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 msgr2=0x7fec18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 0x7fec18071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ae000 tx=0x7fec2c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 msgr2=0x7fec3c15bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec34009cb0 tx=0x7fec3400c490 comp rx=0 tx=0).stop 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 shutdown_connections 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec1806f430 0x7fec18071940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec3c1421f0 0x7fec3c15bae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec3c070a50 0x7fec3c140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 --2- 172.21.15.116:0/679992340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec3c070640 0x7fec3c145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 >> 172.21.15.116:0/679992340 conn(0x7fec3c0cde80 msgr2=0x7fec3c13ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 shutdown_connections 2024-05-14T11:17:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.464+0000 7fec43600640 1 -- 172.21.15.116:0/679992340 wait complete. 2024-05-14T11:17:07.492 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:17:07.492 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:17:07.492 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:17:07.493 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:07.493 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:17:07.493 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:17:07.493 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:17:07.493 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:07.493 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:17:07.493 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:17:07.493 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:17:07.494 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:07.494 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:17:07.494 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:17:07.494 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:17:07.494 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:07.506 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:17:07.507 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:17:07.508 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:17:07.508 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:17:07.509 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:17:07.509 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:17:07.515 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:17:07.515 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:17:07.557 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:17:07.557 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:07.559 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:17:07.559 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:07.562 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:17:07.563 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:07.565 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:17:07.565 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:07.565 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:17:07.661 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:17:07.664 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:17:07.666 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:17:07.668 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:17:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.720+0000 7fa6e4800640 1 Processor -- start 2024-05-14T11:17:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.720+0000 7fa6e4800640 1 -- start start 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e4800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 0x7fa6dc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e4800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa6dc0af380 0x7fa6dc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e4800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc0d1fc0 con 0x7fa6dc156fa0 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e4800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc0706f0 con 0x7fa6dc0af380 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e4800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc1573b0 con 0x7fa6dc0d1670 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa6dc156fa0 0x7fa6dc06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40390/0 (socket says 172.21.15.116:40390) 2024-05-14T11:17:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1e00640 1 -- 172.21.15.116:0/609102337 learned_addr learned my addr 172.21.15.116:0/609102337 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1400640 1 --2- 172.21.15.116:0/609102337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 0x7fa6dc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e2800640 1 --2- 172.21.15.116:0/609102337 >> v2:172.21.15.149:3300/0 conn(0x7fa6dc0af380 0x7fa6dc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e0a00640 1 -- 172.21.15.116:0/609102337 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056752359 0 0) 0x7fa6d0001120 con 0x7fa6dc156fa0 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e0a00640 1 -- 172.21.15.116:0/609102337 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6c4003530 con 0x7fa6dc156fa0 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1400640 1 -- 172.21.15.116:0/609102337 >> v2:172.21.15.149:3300/0 conn(0x7fa6dc0af380 msgr2=0x7fa6dc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1400640 1 --2- 172.21.15.116:0/609102337 >> v2:172.21.15.149:3300/0 conn(0x7fa6dc0af380 0x7fa6dc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1400640 1 -- 172.21.15.116:0/609102337 >> v1:172.21.15.116:6789/0 conn(0x7fa6dc156fa0 legacy=0x7fa6dc06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.724+0000 7fa6e1400640 1 -- 172.21.15.116:0/609102337 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc160510 con 0x7fa6dc0d1670 2024-05-14T11:17:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e1400640 1 --2- 172.21.15.116:0/609102337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 0x7fa6dc0d1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0097c0 tx=0x7fa6cc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f1057ed77eae9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e0a00640 1 -- 172.21.15.116:0/609102337 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0b2070 con 0x7fa6dc0d1670 2024-05-14T11:17:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e0a00640 1 -- 172.21.15.116:0/609102337 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0ad640 con 0x7fa6dc0d1670 2024-05-14T11:17:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e0a00640 1 -- 172.21.15.116:0/609102337 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0ada10 con 0x7fa6dc0d1670 2024-05-14T11:17:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- 172.21.15.116:0/609102337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 msgr2=0x7fa6dc0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 --2- 172.21.15.116:0/609102337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 0x7fa6dc0d1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0097c0 tx=0x7fa6cc01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- 172.21.15.116:0/609102337 shutdown_connections 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 --2- 172.21.15.116:0/609102337 >> v2:172.21.15.149:3300/0 conn(0x7fa6dc0af380 0x7fa6dc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 --2- 172.21.15.116:0/609102337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc0d1670 0x7fa6dc0d1a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- 172.21.15.116:0/609102337 >> 172.21.15.116:0/609102337 conn(0x7fa6dc0b4620 msgr2=0x7fa6dc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- 172.21.15.116:0/609102337 shutdown_connections 2024-05-14T11:17:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- 172.21.15.116:0/609102337 wait complete. 2024-05-14T11:17:07.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 Processor -- start 2024-05-14T11:17:07.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.728+0000 7fa6e4800640 1 -- start start 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc0d1670 0x7fa6dc16b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc156fa0 0x7fa6dc152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b0660 con 0x7fa6dc0af380 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b04e0 con 0x7fa6dc0d1670 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc06c700 con 0x7fa6dc156fa0 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43990/0 (socket says 172.21.15.116:43990) 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 -- 172.21.15.116:0/44854418 learned_addr learned my addr 172.21.15.116:0/44854418 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e2800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc156fa0 0x7fa6dc152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1400640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc0d1670 0x7fa6dc16b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 -- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc156fa0 msgr2=0x7fa6dc152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc156fa0 0x7fa6dc152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 -- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc0d1670 msgr2=0x7fa6dc16b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc0d1670 0x7fa6dc16b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 -- 172.21.15.116:0/44854418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc165d40 con 0x7fa6dc0af380 2024-05-14T11:17:07.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e1e00640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c760 tx=0x7fa6d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0017070 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d0005ac0 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc1fcdd0 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0001d20 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc15c4e0 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.732+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc155680 con 0x7fa6dc0af380 2024-05-14T11:17:07.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.736+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa6d001a070 con 0x7fa6dc0af380 2024-05-14T11:17:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.736+0000 7fa6d6000640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 0x7fa6c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.736+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7fa6d008e370 con 0x7fa6dc0af380 2024-05-14T11:17:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.736+0000 7fa6e1400640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 0x7fa6c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:07.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.736+0000 7fa6e1400640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 0x7fa6c0071940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0af000 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.740+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa6dc0d1fc0 con 0x7fa6dc0af380 2024-05-14T11:17:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:07.908+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa6dc06a9c0 con 0x7fa6dc0af380 2024-05-14T11:17:08.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.472+0000 7fa6d6000640 1 -- 172.21.15.116:0/44854418 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7fa6d00633b0 con 0x7fa6dc0af380 2024-05-14T11:17:08.477 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:17:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 msgr2=0x7fa6c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 0x7fa6c0071940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0af000 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2024-05-14T11:17:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 msgr2=0x7fa6dc1fc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c760 tx=0x7fa6d000cc20 comp rx=0 tx=0).stop 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 shutdown_connections 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa6c006f430 0x7fa6c0071940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc0d1670 0x7fa6dc16b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6dc156fa0 0x7fa6dc152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 --2- 172.21.15.116:0/44854418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6dc0af380 0x7fa6dc1fc890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 >> 172.21.15.116:0/44854418 conn(0x7fa6dc0b4620 msgr2=0x7fa6dc165f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 shutdown_connections 2024-05-14T11:17:08.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.476+0000 7fa6e4800640 1 -- 172.21.15.116:0/44854418 wait complete. 2024-05-14T11:17:08.502 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:17:08.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.644+0000 7f2990800640 1 Processor -- start 2024-05-14T11:17:08.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.644+0000 7f2990800640 1 -- start start 2024-05-14T11:17:08.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f2990800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f298806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f2990800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29880d1670 0x7f29880d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f2990800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29880d1fc0 con 0x7f298806e9e0 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f2990800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f29881573b0 con 0x7f29880d1670 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f2990800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2988161120 con 0x7f2988156fa0 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29880d1670 0x7f29880d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298e800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f298806e9e0 0x7f298806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40394/0 (socket says 172.21.15.116:40394) 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298e800640 1 -- 172.21.15.116:0/1537731255 learned_addr learned my addr 172.21.15.116:0/1537731255 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298de00640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f298806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298d400640 1 -- 172.21.15.116:0/1537731255 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 msgr2=0x7f298806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298d400640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f298806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298d400640 1 -- 172.21.15.116:0/1537731255 >> v1:172.21.15.116:6789/0 conn(0x7f298806e9e0 legacy=0x7f298806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298d400640 1 -- 172.21.15.116:0/1537731255 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2988160510 con 0x7f29880d1670 2024-05-14T11:17:08.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298ca00640 1 -- 172.21.15.116:0/1537731255 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141338850 0 0) 0x7f29840014a0 con 0x7f298806e9e0 2024-05-14T11:17:08.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.648+0000 7f298de00640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f298806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:08.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f298d400640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f29880d1a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29780097c0 tx=0x7f297801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b518e0be143f39d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:08.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f298ca00640 1 -- 172.21.15.116:0/1537731255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780ab610 con 0x7f29880d1670 2024-05-14T11:17:08.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f298ca00640 1 -- 172.21.15.116:0/1537731255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29780b0070 con 0x7f29880d1670 2024-05-14T11:17:08.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f298ca00640 1 -- 172.21.15.116:0/1537731255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780ab9d0 con 0x7f29880d1670 2024-05-14T11:17:08.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 -- 172.21.15.116:0/1537731255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 msgr2=0x7f29880d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f29880d1a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29780097c0 tx=0x7f297801c410 comp rx=0 tx=0).stop 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 -- 172.21.15.116:0/1537731255 shutdown_connections 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f29880d1a80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 --2- 172.21.15.116:0/1537731255 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f298806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 -- 172.21.15.116:0/1537731255 >> 172.21.15.116:0/1537731255 conn(0x7f29880b4620 msgr2=0x7f29880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 -- 172.21.15.116:0/1537731255 shutdown_connections 2024-05-14T11:17:08.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 -- 172.21.15.116:0/1537731255 wait complete. 2024-05-14T11:17:08.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.652+0000 7f2990800640 1 Processor -- start 2024-05-14T11:17:08.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- start start 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f298806e9e0 0x7f2988158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f29881557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2988070520 con 0x7f29880d1670 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29880b07a0 con 0x7f298806e9e0 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f29880706a0 con 0x7f2988156fa0 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44012/0 (socket says 172.21.15.116:44012) 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 -- 172.21.15.116:0/2350228749 learned_addr learned my addr 172.21.15.116:0/2350228749 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298e800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f29881557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298de00640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f298806e9e0 0x7f2988158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 -- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 msgr2=0x7f29881557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f29881557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 -- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f298806e9e0 msgr2=0x7f2988158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f298806e9e0 0x7f2988158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 -- 172.21.15.116:0/2350228749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2988069a50 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f298d400640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2978019080 tx=0x7f297801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780bf610 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29780bf7d0 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780bfbc0 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298815a040 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.656+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29881fa560 con 0x7f29880d1670 2024-05-14T11:17:08.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298806a9c0 con 0x7f29880d1670 2024-05-14T11:17:08.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f29780bd020 con 0x7f29880d1670 2024-05-14T11:17:08.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f2982000640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 0x7f296c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:08.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2978129390 con 0x7f29880d1670 2024-05-14T11:17:08.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f298de00640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 0x7f296c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:08.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.660+0000 7f298de00640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 0x7f296c071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29880b0d60 tx=0x7f297c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:08.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.664+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f29780c6070 con 0x7f29880d1670 2024-05-14T11:17:08.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.884+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f298806ef90 con 0x7f29880d1670 2024-05-14T11:17:08.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.884+0000 7f2982000640 1 -- 172.21.15.116:0/2350228749 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f298815a040 con 0x7f29880d1670 2024-05-14T11:17:08.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:08.888 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:17:08.889 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.890 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.891 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.892 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.893 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.894 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.895 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.896 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:08.897 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.898 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.899 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mds", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0NmrBpiFBAA22cvwKe75HMXJ2FqA89kOA==", 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mds" 2024-05-14T11:17:08.900 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mgr", 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0Nm7CNiFBAA/PCbcOud53RibsQ8qNRdRg==", 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-osd", 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0NmYSxiFBAAhT1d5sRJAG8dpTn6pfCZng==", 2024-05-14T11:17:08.901 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-osd" 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd", 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0NmmzRiFBAAm0AZazQhMopZebTxy2Dsgg==", 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0Nm9DxiFBAArFrzaejXDua0l9iSytMghg==", 2024-05-14T11:17:08.902 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rgw", 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDjR0NmikViFBAAKLRtx81/xIHIvtc/6USoxg==", 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:17:08.903 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:17:08.905 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:17:08.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 msgr2=0x7f296c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 0x7f296c071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29880b0d60 tx=0x7f297c008040 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 msgr2=0x7f298814f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2978019080 tx=0x7f297801acb0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 shutdown_connections 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f296c06f400 0x7f296c071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2988156fa0 0x7f29881557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29880d1670 0x7f298814f900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 --2- 172.21.15.116:0/2350228749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f298806e9e0 0x7f2988158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 >> 172.21.15.116:0/2350228749 conn(0x7f29880b4620 msgr2=0x7f2988166be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 shutdown_connections 2024-05-14T11:17:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:08.888+0000 7f2990800640 1 -- 172.21.15.116:0/2350228749 wait complete. 2024-05-14T11:17:08.910 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-14T11:17:09.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 Processor -- start 2024-05-14T11:17:09.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 -- start start 2024-05-14T11:17:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c44161e80 0x7f0c440d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c440d25b0 0x7f0c440d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c440d18b0 con 0x7f0c440b8fc0 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0c440d1be0 con 0x7f0c440d25b0 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c4a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c440712c0 con 0x7f0c44161e80 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c440d25b0 0x7f0c440d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c440d25b0 0x7f0c440d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44028/0 (socket says 172.21.15.116:44028) 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 -- 172.21.15.116:0/3853124642 learned_addr learned my addr 172.21.15.116:0/3853124642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c3aa00640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c44161e80 0x7f0c440d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 -- 172.21.15.116:0/3853124642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c44161e80 msgr2=0x7f0c440d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c44161e80 0x7f0c440d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 -- 172.21.15.116:0/3853124642 >> v1:172.21.15.149:6789/0 conn(0x7f0c440b8fc0 legacy=0x7f0c44161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.056+0000 7f0c43e00640 1 -- 172.21.15.116:0/3853124642 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c44071510 con 0x7f0c440d25b0 2024-05-14T11:17:09.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c43e00640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440d29c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa98f98397e926b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c42a00640 1 -- 172.21.15.116:0/3853124642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0a57f0 con 0x7f0c440d25b0 2024-05-14T11:17:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c42a00640 1 -- 172.21.15.116:0/3853124642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c0a5990 con 0x7f0c440d25b0 2024-05-14T11:17:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c42a00640 1 -- 172.21.15.116:0/3853124642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0a5d60 con 0x7f0c440d25b0 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 -- 172.21.15.116:0/3853124642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 msgr2=0x7f0c440d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440d29c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c01ca30 comp rx=0 tx=0).stop 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 -- 172.21.15.116:0/3853124642 shutdown_connections 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440d29c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 --2- 172.21.15.116:0/3853124642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c44161e80 0x7f0c440d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 -- 172.21.15.116:0/3853124642 >> 172.21.15.116:0/3853124642 conn(0x7f0c440d3740 msgr2=0x7f0c440d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 -- 172.21.15.116:0/3853124642 shutdown_connections 2024-05-14T11:17:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.060+0000 7f0c4a400640 1 -- 172.21.15.116:0/3853124642 wait complete. 2024-05-14T11:17:09.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 Processor -- start 2024-05-14T11:17:09.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 -- start start 2024-05-14T11:17:09.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c440b8fc0 0x7f0c440c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c4414e6a0 0x7f0c440b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441625b0 con 0x7f0c440d25b0 2024-05-14T11:17:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44162430 con 0x7f0c440b8fc0 2024-05-14T11:17:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.064+0000 7f0c4a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c440b93d0 con 0x7f0c4414e6a0 2024-05-14T11:17:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44032/0 (socket says 172.21.15.116:44032) 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 -- 172.21.15.116:0/2763370802 learned_addr learned my addr 172.21.15.116:0/2763370802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c43400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c440b8fc0 0x7f0c440c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c43e00640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c4414e6a0 0x7f0c440b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 -- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c4414e6a0 msgr2=0x7f0c440b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c4414e6a0 0x7f0c440b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 -- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c440b8fc0 msgr2=0x7f0c440c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c440b8fc0 0x7f0c440c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 -- 172.21.15.116:0/2763370802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c440c8c00 con 0x7f0c440d25b0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3aa00640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d5f0 tx=0x7f0c2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0125f0 con 0x7f0c440d25b0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c019070 con 0x7f0c440d25b0 2024-05-14T11:17:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0129d0 con 0x7f0c440d25b0 2024-05-14T11:17:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c440c8ee0 con 0x7f0c440d25b0 2024-05-14T11:17:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c440c94a0 con 0x7f0c440d25b0 2024-05-14T11:17:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4406a9c0 con 0x7f0c440d25b0 2024-05-14T11:17:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.068+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c2c012b70 con 0x7f0c440d25b0 2024-05-14T11:17:09.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.072+0000 7f0c3be00640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.072+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0c2c08f600 con 0x7f0c440d25b0 2024-05-14T11:17:09.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.072+0000 7f0c43400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.072+0000 7f0c43400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 0x7f0c20071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.076+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0c2c05c610 con 0x7f0c440d25b0 2024-05-14T11:17:09.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.288+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0c440c8990 con 0x7f0c440d25b0 2024-05-14T11:17:09.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.288+0000 7f0c3be00640 1 -- 172.21.15.116:0/2763370802 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f0c2c05bf10 con 0x7f0c440d25b0 2024-05-14T11:17:09.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 msgr2=0x7f0c20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 0x7f0c20071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c34009040 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 msgr2=0x7f0c440b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d5f0 tx=0x7f0c2c00dab0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 shutdown_connections 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c4414e6a0 0x7f0c440b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c440d25b0 0x7f0c440b8070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 --2- 172.21.15.116:0/2763370802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c440b8fc0 0x7f0c440c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 >> 172.21.15.116:0/2763370802 conn(0x7f0c440d3740 msgr2=0x7f0c4406e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 shutdown_connections 2024-05-14T11:17:09.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.292+0000 7f0c4a400640 1 -- 172.21.15.116:0/2763370802 wait complete. 2024-05-14T11:17:09.314 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-14T11:17:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.456+0000 7f599ca00640 1 Processor -- start 2024-05-14T11:17:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.456+0000 7f599ca00640 1 -- start start 2024-05-14T11:17:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f599ca00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f59980b47d0 0x7f59980ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f599ca00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59980b3060 con 0x7f59980d1c20 2024-05-14T11:17:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f599ca00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59980b3300 con 0x7f59980cdf80 2024-05-14T11:17:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f599ca00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f59980b35e0 con 0x7f59980b47d0 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5995400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f59980d1c20 0x7f59980b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40404/0 (socket says 172.21.15.116:40404) 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5995400640 1 -- 172.21.15.116:0/3946615205 learned_addr learned my addr 172.21.15.116:0/3946615205 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5996800640 1 --2- 172.21.15.116:0/3946615205 >> v2:172.21.15.149:3300/0 conn(0x7f59980b47d0 0x7f59980ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758444559 0 0) 0x7f59980b3060 con 0x7f59980d1c20 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f597c003530 con 0x7f59980d1c20 2024-05-14T11:17:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5996800640 1 -- 172.21.15.116:0/3946615205 >> v1:172.21.15.149:6789/0 conn(0x7f59980cdf80 legacy=0x7f59980d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5996800640 1 -- 172.21.15.116:0/3946615205 >> v1:172.21.15.116:6789/0 conn(0x7f59980d1c20 legacy=0x7f59980b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2054070900 0 0) 0x7f597c003530 con 0x7f59980d1c20 2024-05-14T11:17:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.460+0000 7f5996800640 1 -- 172.21.15.116:0/3946615205 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59980b38c0 con 0x7f59980b47d0 2024-05-14T11:17:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f5996800640 1 --2- 172.21.15.116:0/3946615205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980b47d0 0x7f59980ce390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f59900097c0 tx=0x7f599001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8640f7950d7b2553 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900b0070 con 0x7f59980b47d0 2024-05-14T11:17:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59900ab400 con 0x7f59980b47d0 2024-05-14T11:17:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f5994a00640 1 -- 172.21.15.116:0/3946615205 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984001700 con 0x7f59980b47d0 2024-05-14T11:17:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 -- 172.21.15.116:0/3946615205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980b47d0 msgr2=0x7f59980ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 --2- 172.21.15.116:0/3946615205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980b47d0 0x7f59980ce390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f59900097c0 tx=0x7f599001c390 comp rx=0 tx=0).stop 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 -- 172.21.15.116:0/3946615205 shutdown_connections 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 --2- 172.21.15.116:0/3946615205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980b47d0 0x7f59980ce390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 -- 172.21.15.116:0/3946615205 >> 172.21.15.116:0/3946615205 conn(0x7f59980d0770 msgr2=0x7f59980d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 -- 172.21.15.116:0/3946615205 shutdown_connections 2024-05-14T11:17:09.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.464+0000 7f599ca00640 1 -- 172.21.15.116:0/3946615205 wait complete. 2024-05-14T11:17:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 Processor -- start 2024-05-14T11:17:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 -- start start 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f59980b47d0 0x7f599814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 0x7f5998157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f599816dd70 con 0x7f59980c9de0 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f599806ffd0 con 0x7f59980cdf80 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f599816def0 con 0x7f59980b47d0 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44040/0 (socket says 172.21.15.116:44040) 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 -- 172.21.15.116:0/616383955 learned_addr learned my addr 172.21.15.116:0/616383955 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5995400640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 0x7f5998157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5995e00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f59980b47d0 0x7f599814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 -- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f59980b47d0 msgr2=0x7f599814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f59980b47d0 0x7f599814bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 -- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 msgr2=0x7f5998157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 0x7f5998157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 -- 172.21.15.116:0/616383955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599814c350 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5995400640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 0x7f5998157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f5996800640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59900ae040 tx=0x7f5990019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900b0040 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59900bb070 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990003ae0 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.468+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599814c140 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5998156a20 con 0x7f59980c9de0 2024-05-14T11:17:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599814c520 con 0x7f59980c9de0 2024-05-14T11:17:09.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59900b3070 con 0x7f59980c9de0 2024-05-14T11:17:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f598e000640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 0x7f5978071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f5990128de0 con 0x7f59980c9de0 2024-05-14T11:17:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.472+0000 7f5995e00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 0x7f5978071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.476+0000 7f5995e00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 0x7f5978071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59880012f0 tx=0x7f5988009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.480+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5990015fb0 con 0x7f59980c9de0 2024-05-14T11:17:09.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.692+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f599806a9c0 con 0x7f59980c9de0 2024-05-14T11:17:09.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.696+0000 7f598e000640 1 -- 172.21.15.116:0/616383955 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f59900f5e00 con 0x7f59980c9de0 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 msgr2=0x7f5978071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 0x7f5978071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f59880012f0 tx=0x7f5988009040 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 msgr2=0x7f5998157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f59900ae040 tx=0x7f5990019130 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 shutdown_connections 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f597806f430 0x7f5978071940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59980cdf80 0x7f5998157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59980c9de0 0x7f5998157760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 --2- 172.21.15.116:0/616383955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f59980b47d0 0x7f599814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 >> 172.21.15.116:0/616383955 conn(0x7f59980d0770 msgr2=0x7f599815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:09.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 shutdown_connections 2024-05-14T11:17:09.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.700+0000 7f599ca00640 1 -- 172.21.15.116:0/616383955 wait complete. 2024-05-14T11:17:09.722 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-14T11:17:09.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.864+0000 7f8a28e00640 1 Processor -- start 2024-05-14T11:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.864+0000 7f8a28e00640 1 -- start start 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a28e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a24156fa0 0x7f8a2406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a28e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8a240af380 0x7f8a240af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a28e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a24160510 con 0x7f8a240d1670 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a28e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8a241573b0 con 0x7f8a240af380 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a28e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a24161120 con 0x7f8a24156fa0 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a22800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8a240af380 0x7f8a240af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a19400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8a240d1670 0x7f8a240d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40408/0 (socket says 172.21.15.116:40408) 2024-05-14T11:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a19400640 1 -- 172.21.15.116:0/1414330398 learned_addr learned my addr 172.21.15.116:0/1414330398 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a21e00640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a24156fa0 0x7f8a2406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a22800640 1 -- 172.21.15.116:0/1414330398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a24156fa0 msgr2=0x7f8a2406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a22800640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a24156fa0 0x7f8a2406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a22800640 1 -- 172.21.15.116:0/1414330398 >> v1:172.21.15.116:6789/0 conn(0x7f8a240d1670 legacy=0x7f8a240d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a21400640 1 -- 172.21.15.116:0/1414330398 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045823605 0 0) 0x7f8a24160510 con 0x7f8a240d1670 2024-05-14T11:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.868+0000 7f8a22800640 1 -- 172.21.15.116:0/1414330398 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a241607f0 con 0x7f8a240af380 2024-05-14T11:17:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a22800640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a240af380 0x7f8a240af790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009d90 tx=0x7f8a1c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61737ec445429cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a21400640 1 -- 172.21.15.116:0/1414330398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0b2070 con 0x7f8a240af380 2024-05-14T11:17:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a21400640 1 -- 172.21.15.116:0/1414330398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0a5a00 con 0x7f8a240af380 2024-05-14T11:17:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a21400640 1 -- 172.21.15.116:0/1414330398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a5dd0 con 0x7f8a240af380 2024-05-14T11:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 -- 172.21.15.116:0/1414330398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a240af380 msgr2=0x7f8a240af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a240af380 0x7f8a240af790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009d90 tx=0x7f8a1c01c940 comp rx=0 tx=0).stop 2024-05-14T11:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 -- 172.21.15.116:0/1414330398 shutdown_connections 2024-05-14T11:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a240af380 0x7f8a240af790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 --2- 172.21.15.116:0/1414330398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a24156fa0 0x7f8a2406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 -- 172.21.15.116:0/1414330398 >> 172.21.15.116:0/1414330398 conn(0x7f8a240b4620 msgr2=0x7f8a240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 -- 172.21.15.116:0/1414330398 shutdown_connections 2024-05-14T11:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.872+0000 7f8a28e00640 1 -- 172.21.15.116:0/1414330398 wait complete. 2024-05-14T11:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 Processor -- start 2024-05-14T11:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- start start 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 0x7f8a2414aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 0x7f8a241f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b0660 con 0x7f8a24156fa0 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b04e0 con 0x7f8a240af380 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a24169810 con 0x7f8a240d1670 2024-05-14T11:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44062/0 (socket says 172.21.15.116:44062) 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 -- 172.21.15.116:0/854154211 learned_addr learned my addr 172.21.15.116:0/854154211 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a21e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 0x7f8a2414aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a19400640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 0x7f8a241f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 -- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 msgr2=0x7f8a241f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 0x7f8a241f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 -- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 msgr2=0x7f8a2414aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 0x7f8a2414aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 -- 172.21.15.116:0/854154211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a241530e0 con 0x7f8a24156fa0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a21e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 0x7f8a2414aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a19400640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 0x7f8a241f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a22800640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0b0040 tx=0x7f8a1c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0b2050 con 0x7f8a24156fa0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0b3040 con 0x7f8a24156fa0 2024-05-14T11:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241f9460 con 0x7f8a24156fa0 2024-05-14T11:17:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0ad930 con 0x7f8a24156fa0 2024-05-14T11:17:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.876+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241fda30 con 0x7f8a24156fa0 2024-05-14T11:17:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2406a9c0 con 0x7f8a24156fa0 2024-05-14T11:17:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8a1c0c0020 con 0x7f8a24156fa0 2024-05-14T11:17:09.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a1aa00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 0x7f8a00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:09.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f8a1c129b50 con 0x7f8a24156fa0 2024-05-14T11:17:09.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a21e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 0x7f8a00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:09.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.880+0000 7f8a21e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 0x7f8a00071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b0d60 tx=0x7f8a14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:09.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:09.884+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8a1c0c7020 con 0x7f8a24156fa0 2024-05-14T11:17:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.096+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f8a2415f550 con 0x7f8a24156fa0 2024-05-14T11:17:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.100+0000 7f8a1aa00640 1 -- 172.21.15.116:0/854154211 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f8a241530e0 con 0x7f8a24156fa0 2024-05-14T11:17:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 msgr2=0x7f8a00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 0x7f8a00071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b0d60 tx=0x7f8a14008040 comp rx=0 tx=0).stop 2024-05-14T11:17:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 msgr2=0x7f8a2415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0b0040 tx=0x7f8a1c0196e0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 shutdown_connections 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8a0006f400 0x7f8a00071910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a24156fa0 0x7f8a2415b950 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a240d1670 0x7f8a241f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 --2- 172.21.15.116:0/854154211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a240af380 0x7f8a2414aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 >> 172.21.15.116:0/854154211 conn(0x7f8a240b4620 msgr2=0x7f8a241503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 shutdown_connections 2024-05-14T11:17:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.104+0000 7f8a28e00640 1 -- 172.21.15.116:0/854154211 wait complete. 2024-05-14T11:17:10.127 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-14T11:17:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 Processor -- start 2024-05-14T11:17:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 -- start start 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 0x7fe8180d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8180b5660 0x7fe8180b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8180b1f10 con 0x7fe818163860 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180cf6d0 con 0x7fe8180b5660 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe81d000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180cf9b0 con 0x7fe8180d0bf0 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8180b5660 0x7fe8180b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8180b5660 0x7fe8180b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44074/0 (socket says 172.21.15.116:44074) 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 -- 172.21.15.116:0/1907329573 learned_addr learned my addr 172.21.15.116:0/1907329573 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe815e00640 1 --2- 172.21.15.116:0/1907329573 >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 0x7fe8180d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 -- 172.21.15.116:0/1907329573 >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 msgr2=0x7fe8180d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 --2- 172.21.15.116:0/1907329573 >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 0x7fe8180d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 -- 172.21.15.116:0/1907329573 >> v1:172.21.15.149:6789/0 conn(0x7fe818163860 legacy=0x7fe818163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe816800640 1 -- 172.21.15.116:0/1907329573 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8180cfc90 con 0x7fe8180b5660 2024-05-14T11:17:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.272+0000 7fe815e00640 1 --2- 172.21.15.116:0/1907329573 >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 0x7fe8180d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe816800640 1 --2- 172.21.15.116:0/1907329573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe8180b5a70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8180b72f0 tx=0x7fe81001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904f807be5bcf76a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe815400640 1 -- 172.21.15.116:0/1907329573 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a5620 con 0x7fe8180b5660 2024-05-14T11:17:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe815400640 1 -- 172.21.15.116:0/1907329573 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8100a57c0 con 0x7fe8180b5660 2024-05-14T11:17:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe815400640 1 -- 172.21.15.116:0/1907329573 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a5b90 con 0x7fe8180b5660 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 -- 172.21.15.116:0/1907329573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 msgr2=0x7fe8180b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 --2- 172.21.15.116:0/1907329573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe8180b5a70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe8180b72f0 tx=0x7fe81001c960 comp rx=0 tx=0).stop 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 -- 172.21.15.116:0/1907329573 shutdown_connections 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 --2- 172.21.15.116:0/1907329573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe8180b5a70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 --2- 172.21.15.116:0/1907329573 >> v2:172.21.15.149:3300/0 conn(0x7fe8180d0bf0 0x7fe8180d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 -- 172.21.15.116:0/1907329573 >> 172.21.15.116:0/1907329573 conn(0x7fe818071370 msgr2=0x7fe818071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 -- 172.21.15.116:0/1907329573 shutdown_connections 2024-05-14T11:17:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.276+0000 7fe81d000640 1 -- 172.21.15.116:0/1907329573 wait complete. 2024-05-14T11:17:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 Processor -- start 2024-05-14T11:17:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 -- start start 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8180d0bf0 0x7fe818154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe818163860 0x7fe81815a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe81816cd40 con 0x7fe8180b5660 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe81816cbc0 con 0x7fe818163860 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe81d000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe81816cec0 con 0x7fe8180d0bf0 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44088/0 (socket says 172.21.15.116:44088) 2024-05-14T11:17:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 -- 172.21.15.116:0/966869694 learned_addr learned my addr 172.21.15.116:0/966869694 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe816800640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe818163860 0x7fe81815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe80fe00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8180d0bf0 0x7fe818154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 -- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8180d0bf0 msgr2=0x7fe818154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8180d0bf0 0x7fe818154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 -- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe818163860 msgr2=0x7fe81815a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe818163860 0x7fe81815a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 -- 172.21.15.116:0/966869694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81806f530 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.280+0000 7fe815e00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe80800d830 tx=0x7fe80800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800b6c0 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe808019070 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800baa0 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81806e780 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81806ef70 con 0x7fe8180b5660 2024-05-14T11:17:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81806a9c0 con 0x7fe8180b5660 2024-05-14T11:17:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe80801e460 con 0x7fe8180b5660 2024-05-14T11:17:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 0x7fe7f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe80808f760 con 0x7fe8180b5660 2024-05-14T11:17:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.284+0000 7fe80fe00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 0x7fe7f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.288+0000 7fe80fe00640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 0x7fe7f4071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe818164330 tx=0x7fe800009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.288+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe81806f530 con 0x7fe8180b5660 2024-05-14T11:17:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.504+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fe818164360 con 0x7fe8180b5660 2024-05-14T11:17:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.508+0000 7fe80e000640 1 -- 172.21.15.116:0/966869694 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fe80805c800 con 0x7fe8180b5660 2024-05-14T11:17:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 msgr2=0x7fe7f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 0x7fe7f4071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe818164330 tx=0x7fe800009040 comp rx=0 tx=0).stop 2024-05-14T11:17:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 msgr2=0x7fe81806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe80800d830 tx=0x7fe80800dcf0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 shutdown_connections 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7f406f430 0x7fe7f4071940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe818163860 0x7fe81815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8180d0bf0 0x7fe818154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 --2- 172.21.15.116:0/966869694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8180b5660 0x7fe81806e240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 >> 172.21.15.116:0/966869694 conn(0x7fe818071370 msgr2=0x7fe818169f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 shutdown_connections 2024-05-14T11:17:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.512+0000 7fe81d000640 1 -- 172.21.15.116:0/966869694 wait complete. 2024-05-14T11:17:10.534 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-14T11:17:10.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.676+0000 7ff845c00640 1 Processor -- start 2024-05-14T11:17:10.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.676+0000 7ff845c00640 1 -- start start 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff845c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff840156fa0 0x7ff84006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff845c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 0x7ff8400af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff845c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff840160510 con 0x7ff8400d1670 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff845c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff8401573b0 con 0x7ff840156fa0 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff845c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff840161120 con 0x7ff8400af380 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff836000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff8400d1670 0x7ff8400d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40416/0 (socket says 172.21.15.116:40416) 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff836000640 1 -- 172.21.15.116:0/3738888502 learned_addr learned my addr 172.21.15.116:0/3738888502 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83f400640 1 --2- 172.21.15.116:0/3738888502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 0x7ff8400af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83ea00640 1 --2- 172.21.15.116:0/3738888502 >> v2:172.21.15.149:3300/0 conn(0x7ff840156fa0 0x7ff84006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83e000640 1 -- 172.21.15.116:0/3738888502 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190389472 0 0) 0x7ff840160510 con 0x7ff8400d1670 2024-05-14T11:17:10.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83e000640 1 -- 172.21.15.116:0/3738888502 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff824003530 con 0x7ff8400d1670 2024-05-14T11:17:10.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83f400640 1 -- 172.21.15.116:0/3738888502 >> v2:172.21.15.149:3300/0 conn(0x7ff840156fa0 msgr2=0x7ff84006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:10.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83f400640 1 --2- 172.21.15.116:0/3738888502 >> v2:172.21.15.149:3300/0 conn(0x7ff840156fa0 0x7ff84006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83f400640 1 -- 172.21.15.116:0/3738888502 >> v1:172.21.15.116:6789/0 conn(0x7ff8400d1670 legacy=0x7ff8400d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.680+0000 7ff83f400640 1 -- 172.21.15.116:0/3738888502 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401607f0 con 0x7ff8400af380 2024-05-14T11:17:10.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff83f400640 1 --2- 172.21.15.116:0/3738888502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 0x7ff8400af790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8380097c0 tx=0x7ff83801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c1ae42b42c2fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff83e000640 1 -- 172.21.15.116:0/3738888502 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380a7070 con 0x7ff8400af380 2024-05-14T11:17:10.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff83e000640 1 -- 172.21.15.116:0/3738888502 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8380b0070 con 0x7ff8400af380 2024-05-14T11:17:10.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff83e000640 1 -- 172.21.15.116:0/3738888502 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380ab890 con 0x7ff8400af380 2024-05-14T11:17:10.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 -- 172.21.15.116:0/3738888502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 msgr2=0x7ff8400af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 --2- 172.21.15.116:0/3738888502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 0x7ff8400af790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8380097c0 tx=0x7ff83801c470 comp rx=0 tx=0).stop 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 -- 172.21.15.116:0/3738888502 shutdown_connections 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 --2- 172.21.15.116:0/3738888502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8400af380 0x7ff8400af790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 --2- 172.21.15.116:0/3738888502 >> v2:172.21.15.149:3300/0 conn(0x7ff840156fa0 0x7ff84006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 -- 172.21.15.116:0/3738888502 >> 172.21.15.116:0/3738888502 conn(0x7ff8400b4620 msgr2=0x7ff8400b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 -- 172.21.15.116:0/3738888502 shutdown_connections 2024-05-14T11:17:10.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.684+0000 7ff845c00640 1 -- 172.21.15.116:0/3738888502 wait complete. 2024-05-14T11:17:10.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 Processor -- start 2024-05-14T11:17:10.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 -- start start 2024-05-14T11:17:10.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 0x7ff840144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 0x7ff8401f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400b0660 con 0x7ff8400d1670 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400b04e0 con 0x7ff8400af380 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff840169810 con 0x7ff840156fa0 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44092/0 (socket says 172.21.15.116:44092) 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 -- 172.21.15.116:0/4114101679 learned_addr learned my addr 172.21.15.116:0/4114101679 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff83ea00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 0x7ff840144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff83f400640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 0x7ff8401f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 -- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 msgr2=0x7ff8401f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 0x7ff8401f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 -- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 msgr2=0x7ff840144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 0x7ff840144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 -- 172.21.15.116:0/4114101679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84015f1a0 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff83ea00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 0x7ff840144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff83f400640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 0x7ff8401f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff836000640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c430 tx=0x7ff82800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff828005cf0 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff840144a50 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.688+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280072d0 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff840155470 con 0x7ff8400d1670 2024-05-14T11:17:10.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84006a9c0 con 0x7ff8400d1670 2024-05-14T11:17:10.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff82801a070 con 0x7ff8400d1670 2024-05-14T11:17:10.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff837400640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:10.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7ff82808e4b0 con 0x7ff8400d1670 2024-05-14T11:17:10.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff83ea00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:10.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.692+0000 7ff83ea00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 0x7ff81c071940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8400b0d60 tx=0x7ff83000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:10.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.696+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff82805adf0 con 0x7ff8400d1670 2024-05-14T11:17:10.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.912+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff840075dc0 con 0x7ff8400d1670 2024-05-14T11:17:10.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff837400640 1 -- 172.21.15.116:0/4114101679 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7ff82805f0f0 con 0x7ff8400d1670 2024-05-14T11:17:10.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 msgr2=0x7ff81c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 0x7ff81c071940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8400b0d60 tx=0x7ff83000b040 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 msgr2=0x7ff840156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c430 tx=0x7ff82800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 shutdown_connections 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff840156fa0 0x7ff8401f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8400d1670 0x7ff840156230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 --2- 172.21.15.116:0/4114101679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8400af380 0x7ff840144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 >> 172.21.15.116:0/4114101679 conn(0x7ff8400b4620 msgr2=0x7ff8400b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 shutdown_connections 2024-05-14T11:17:10.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:10.916+0000 7ff845c00640 1 -- 172.21.15.116:0/4114101679 wait complete. 2024-05-14T11:17:10.930 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:17:10.933 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:17:10.941 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-14T11:17:11.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.168+0000 7f899c200640 1 Processor -- start 2024-05-14T11:17:11.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.168+0000 7f899c200640 1 -- start start 2024-05-14T11:17:11.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899c200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f899406f730 0x7f899406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899c200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8994070080 con 0x7f89940b0770 2024-05-14T11:17:11.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899c200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899406cf10 con 0x7f8994072a30 2024-05-14T11:17:11.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899c200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f899406db20 con 0x7f899406f730 2024-05-14T11:17:11.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8999800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f89940b0770 0x7f899406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40428/0 (socket says 172.21.15.116:40428) 2024-05-14T11:17:11.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8999800640 1 -- 172.21.15.116:0/1676806195 learned_addr learned my addr 172.21.15.116:0/1676806195 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073203400 0 0) 0x7f8988001120 con 0x7f89940b0770 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f897c003530 con 0x7f89940b0770 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8998e00640 1 --2- 172.21.15.116:0/1676806195 >> v2:172.21.15.149:3300/0 conn(0x7f899406f730 0x7f899406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 583026608 0 0) 0x7f897c003530 con 0x7f89940b0770 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 >> v2:172.21.15.149:3300/0 conn(0x7f899406f730 msgr2=0x7f899406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 --2- 172.21.15.116:0/1676806195 >> v2:172.21.15.149:3300/0 conn(0x7f899406f730 0x7f899406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 >> v1:172.21.15.149:6789/0 conn(0x7f8994072a30 legacy=0x7f8994072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899406d1f0 con 0x7f89940b0770 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f89880018f0 con 0x7f89940b0770 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 >> v1:172.21.15.116:6789/0 conn(0x7f89940b0770 legacy=0x7f899406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f897c003a60 0x7f897c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f897c004810 0x7f897c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f897c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f899406db20 con 0x7f897c005360 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f897c00c290 con 0x7f897c003a60 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f897c00c490 con 0x7f897c004810 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f897c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8999800640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f897c003a60 0x7f897c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f8998e00640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f897c004810 0x7f897c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 -- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f897c004810 msgr2=0x7f897c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f897c004810 0x7f897c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 -- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f897c003a60 msgr2=0x7f897c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f897c003a60 0x7f897c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.172+0000 7f899a200640 1 -- 172.21.15.116:0/1676806195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c003530 con 0x7f897c005360 2024-05-14T11:17:11.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899a200640 1 -- 172.21.15.116:0/1676806195 reap_dead start 2024-05-14T11:17:11.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899a200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f897c00b9d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f899406ee20 tx=0x7f898c01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ddbd560bff77b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c001ae0 con 0x7f897c005360 2024-05-14T11:17:11.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898c003b80 con 0x7f897c005360 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f8993e00640 1 -- 172.21.15.116:0/1676806195 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0050d0 con 0x7f897c005360 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 -- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 msgr2=0x7f897c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f897c00b9d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f899406ee20 tx=0x7f898c01c750 comp rx=0 tx=0).stop 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 -- 172.21.15.116:0/1676806195 shutdown_connections 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 --2- 172.21.15.116:0/1676806195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f897c00b9d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 -- 172.21.15.116:0/1676806195 >> 172.21.15.116:0/1676806195 conn(0x7f89940b4620 msgr2=0x7f89940b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:11.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 -- 172.21.15.116:0/1676806195 shutdown_connections 2024-05-14T11:17:11.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.176+0000 7f899c200640 1 -- 172.21.15.116:0/1676806195 wait complete. 2024-05-14T11:17:11.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 Processor -- start 2024-05-14T11:17:11.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 -- start start 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8994095e80 0x7f89940b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89940b1920 0x7f8994098570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f899406dfb0 con 0x7f897c005360 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f899406d7b0 con 0x7f89940b1920 2024-05-14T11:17:11.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899c200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f899406fb40 con 0x7f8994095e80 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44106/0 (socket says 172.21.15.116:44106) 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 -- 172.21.15.116:0/877748567 learned_addr learned my addr 172.21.15.116:0/877748567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8998e00640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8994095e80 0x7f89940b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f899a200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89940b1920 0x7f8994098570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 -- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8994095e80 msgr2=0x7f89940b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8994095e80 0x7f89940b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 -- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89940b1920 msgr2=0x7f8994098570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89940b1920 0x7f8994098570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 -- 172.21.15.116:0/877748567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940d0ad0 con 0x7f897c005360 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.180+0000 7f8999800640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8988001420 tx=0x7f8988012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898801c070 con 0x7f897c005360 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8988015600 con 0x7f897c005360 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880159f0 con 0x7f897c005360 2024-05-14T11:17:11.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89940d0030 con 0x7f897c005360 2024-05-14T11:17:11.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899409ad50 con 0x7f897c005360 2024-05-14T11:17:11.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899406a9c0 con 0x7f897c005360 2024-05-14T11:17:11.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f898801f070 con 0x7f897c005360 2024-05-14T11:17:11.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 0x7f8974071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f89880915c0 con 0x7f897c005360 2024-05-14T11:17:11.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.184+0000 7f8998e00640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 0x7f8974071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.188+0000 7f8998e00640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 0x7f8974071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89941605f0 tx=0x7f8980009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.188+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8994070080 con 0x7f897c005360 2024-05-14T11:17:11.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.400+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f899406ed20 con 0x7f897c005360 2024-05-14T11:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.404+0000 7f8991600640 1 -- 172.21.15.116:0/877748567 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f898805df80 con 0x7f897c005360 2024-05-14T11:17:11.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 msgr2=0x7f8974071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 0x7f8974071910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89941605f0 tx=0x7f8980009190 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 msgr2=0x7f8994095940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8988001420 tx=0x7f8988012630 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 shutdown_connections 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f897406f400 0x7f8974071910 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89940b1920 0x7f8994098570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8994095e80 0x7f89940b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 --2- 172.21.15.116:0/877748567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f897c005360 0x7f8994095940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 >> 172.21.15.116:0/877748567 conn(0x7f89940b4620 msgr2=0x7f89940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 shutdown_connections 2024-05-14T11:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.408+0000 7f899c200640 1 -- 172.21.15.116:0/877748567 wait complete. 2024-05-14T11:17:11.429 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:17:11.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 Processor -- start 2024-05-14T11:17:11.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 -- start start 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe3740ce5c0 0x7fe37415fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe3740ce9d0 0x7fe3741600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37415c5a0 0x7fe37415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe3740d0230 con 0x7fe37415c5a0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe3740d0510 con 0x7fe3740ce9d0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe379800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe3740d07f0 con 0x7fe3740ce5c0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37415c5a0 0x7fe37415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37415c5a0 0x7fe37415c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44112/0 (socket says 172.21.15.116:44112) 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 -- 172.21.15.116:0/156310577 learned_addr learned my addr 172.21.15.116:0/156310577 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe373e00640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe3740ce5c0 0x7fe37415fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe36be00640 1 --2- 172.21.15.116:0/156310577 >> v2:172.21.15.149:3300/0 conn(0x7fe3740ce9d0 0x7fe3741600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 -- 172.21.15.116:0/156310577 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe3740ce5c0 msgr2=0x7fe37415fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:11.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe3740ce5c0 0x7fe37415fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 -- 172.21.15.116:0/156310577 >> v2:172.21.15.149:3300/0 conn(0x7fe3740ce9d0 msgr2=0x7fe3741600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:11.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 --2- 172.21.15.116:0/156310577 >> v2:172.21.15.149:3300/0 conn(0x7fe3740ce9d0 0x7fe3741600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.576+0000 7fe378e00640 1 -- 172.21.15.116:0/156310577 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3740d0ad0 con 0x7fe37415c5a0 2024-05-14T11:17:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe378e00640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37415c5a0 0x7fe37415c9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe37406e1f0 tx=0x7fe36c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81426f515c2cf08b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe373400640 1 -- 172.21.15.116:0/156310577 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0a56e0 con 0x7fe37415c5a0 2024-05-14T11:17:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe373400640 1 -- 172.21.15.116:0/156310577 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0a5880 con 0x7fe37415c5a0 2024-05-14T11:17:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe373400640 1 -- 172.21.15.116:0/156310577 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0a5c50 con 0x7fe37415c5a0 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 -- 172.21.15.116:0/156310577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37415c5a0 msgr2=0x7fe37415c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37415c5a0 0x7fe37415c9b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe37406e1f0 tx=0x7fe36c01c9f0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 -- 172.21.15.116:0/156310577 shutdown_connections 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37415c5a0 0x7fe37415c9b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 --2- 172.21.15.116:0/156310577 >> v2:172.21.15.149:3300/0 conn(0x7fe3740ce9d0 0x7fe3741600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 --2- 172.21.15.116:0/156310577 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe3740ce5c0 0x7fe37415fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 -- 172.21.15.116:0/156310577 >> 172.21.15.116:0/156310577 conn(0x7fe3740735f0 msgr2=0x7fe374073a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 -- 172.21.15.116:0/156310577 shutdown_connections 2024-05-14T11:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.580+0000 7fe379800640 1 -- 172.21.15.116:0/156310577 wait complete. 2024-05-14T11:17:11.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 Processor -- start 2024-05-14T11:17:11.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 -- start start 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 0x7fe3740b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 0x7fe3740b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe37415d480 con 0x7fe3740ce5c0 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe37415d300 con 0x7fe3740ce9d0 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe379800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe3740d15b0 con 0x7fe37415c5a0 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44120/0 (socket says 172.21.15.116:44120) 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 -- 172.21.15.116:0/731268286 learned_addr learned my addr 172.21.15.116:0/731268286 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe36be00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 0x7fe3740b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe378e00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 0x7fe3740b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 -- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 msgr2=0x7fe3740b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 0x7fe3740b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 -- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 msgr2=0x7fe3740b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 0x7fe3740b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 -- 172.21.15.116:0/731268286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe374143bf0 con 0x7fe3740ce5c0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe378e00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 0x7fe3740b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe36be00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 0x7fe3740b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe373e00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e870 tx=0x7fe36400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.584+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364017070 con 0x7fe3740ce5c0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36400ee60 con 0x7fe3740ce5c0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364004220 con 0x7fe3740ce5c0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe374159640 con 0x7fe3740ce5c0 2024-05-14T11:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37413b500 con 0x7fe3740ce5c0 2024-05-14T11:17:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe374050880 con 0x7fe3740ce5c0 2024-05-14T11:17:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe36400c940 con 0x7fe3740ce5c0 2024-05-14T11:17:11.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe370c00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 0x7fe350071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.588+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe36408f230 con 0x7fe3740ce5c0 2024-05-14T11:17:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.592+0000 7fe36be00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 0x7fe350071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.592+0000 7fe36be00640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 0x7fe350071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3741fc870 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.596+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe374143bf0 con 0x7fe3740ce5c0 2024-05-14T11:17:11.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.752+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe374145760 con 0x7fe3740ce5c0 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.752+0000 7fe370c00640 1 -- 172.21.15.116:0/731268286 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7fe36405c2d0 con 0x7fe3740ce5c0 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:11.757 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:17:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 msgr2=0x7fe350071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 0x7fe350071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3741fc870 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-05-14T11:17:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 msgr2=0x7fe37413afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe36400e870 tx=0x7fe36400ed30 comp rx=0 tx=0).stop 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 shutdown_connections 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe35006f400 0x7fe350071910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3740ce9d0 0x7fe3740b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe37415c5a0 0x7fe3740b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 --2- 172.21.15.116:0/731268286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3740ce5c0 0x7fe37413afc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 >> 172.21.15.116:0/731268286 conn(0x7fe3740735f0 msgr2=0x7fe37415fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 shutdown_connections 2024-05-14T11:17:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.756+0000 7fe379800640 1 -- 172.21.15.116:0/731268286 wait complete. 2024-05-14T11:17:11.778 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:17:11.778 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:17:11.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.924+0000 7f17efe00640 1 Processor -- start 2024-05-14T11:17:11.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.924+0000 7f17efe00640 1 -- start start 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17efe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f17e8070640 0x7f17e806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17efe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e8143010 con 0x7f17e8070a50 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17efe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e815d360 con 0x7f17e80d0290 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17efe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f17e815dc90 con 0x7f17e8070640 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17eca00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f17e8070a50 0x7f17e81421f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40442/0 (socket says 172.21.15.116:40442) 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17eca00640 1 -- 172.21.15.116:0/1255715870 learned_addr learned my addr 172.21.15.116:0/1255715870 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17ed400640 1 --2- 172.21.15.116:0/1255715870 >> v2:172.21.15.149:3300/0 conn(0x7f17e8070640 0x7f17e806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61629588 0 0) 0x7f17e8143010 con 0x7f17e8070a50 2024-05-14T11:17:11.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17d0003530 con 0x7f17e8070a50 2024-05-14T11:17:11.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1785433312 0 0) 0x7f17d0003530 con 0x7f17e8070a50 2024-05-14T11:17:11.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17ed400640 1 -- 172.21.15.116:0/1255715870 >> v1:172.21.15.149:6789/0 conn(0x7f17e80d0290 legacy=0x7f17e8142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17ed400640 1 -- 172.21.15.116:0/1255715870 >> v1:172.21.15.116:6789/0 conn(0x7f17e8070a50 legacy=0x7f17e81421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.928+0000 7f17ed400640 1 -- 172.21.15.116:0/1255715870 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e815d640 con 0x7f17e8070640 2024-05-14T11:17:11.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17ed400640 1 --2- 172.21.15.116:0/1255715870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e8070640 0x7f17e806e870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17e815de10 tx=0x7f17dc01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07d974548b90cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0b0070 con 0x7f17e8070640 2024-05-14T11:17:11.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17dc0a4e50 con 0x7f17e8070640 2024-05-14T11:17:11.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17e7e00640 1 -- 172.21.15.116:0/1255715870 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4001700 con 0x7f17e8070640 2024-05-14T11:17:11.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 -- 172.21.15.116:0/1255715870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e8070640 msgr2=0x7f17e806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 --2- 172.21.15.116:0/1255715870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e8070640 0x7f17e806e870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17e815de10 tx=0x7f17dc01c330 comp rx=0 tx=0).stop 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 -- 172.21.15.116:0/1255715870 shutdown_connections 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 --2- 172.21.15.116:0/1255715870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e8070640 0x7f17e806e870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 -- 172.21.15.116:0/1255715870 >> 172.21.15.116:0/1255715870 conn(0x7f17e80cde80 msgr2=0x7f17e80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 -- 172.21.15.116:0/1255715870 shutdown_connections 2024-05-14T11:17:11.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.932+0000 7f17efe00640 1 -- 172.21.15.116:0/1255715870 wait complete. 2024-05-14T11:17:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 Processor -- start 2024-05-14T11:17:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 -- start start 2024-05-14T11:17:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e8070640 0x7f17e815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e80d0290 0x7f17e81fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17e80cd980 con 0x7f17e8070a50 2024-05-14T11:17:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17e80cd800 con 0x7f17e80d0290 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f17e816b5d0 con 0x7f17e8070640 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44128/0 (socket says 172.21.15.116:44128) 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 -- 172.21.15.116:0/2679048571 learned_addr learned my addr 172.21.15.116:0/2679048571 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17ed400640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e8070640 0x7f17e815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17ede00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e80d0290 0x7f17e81fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 -- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e8070640 msgr2=0x7f17e815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e8070640 0x7f17e815b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 -- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e80d0290 msgr2=0x7f17e81fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e80d0290 0x7f17e81fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 -- 172.21.15.116:0/2679048571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e813b650 con 0x7f17e8070a50 2024-05-14T11:17:11.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17eca00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c700 tx=0x7f17d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4005ae0 con 0x7f17e8070a50 2024-05-14T11:17:11.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d4005ca0 con 0x7f17e8070a50 2024-05-14T11:17:11.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.936+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e813b930 con 0x7f17e8070a50 2024-05-14T11:17:11.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40072b0 con 0x7f17e8070a50 2024-05-14T11:17:11.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e8158a60 con 0x7f17e8070a50 2024-05-14T11:17:11.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e806aa30 con 0x7f17e8070a50 2024-05-14T11:17:11.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17d4007450 con 0x7f17e8070a50 2024-05-14T11:17:11.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17e5600640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 0x7f17c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:11.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f17d408ed30 con 0x7f17e8070a50 2024-05-14T11:17:11.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17ed400640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 0x7f17c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:11.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.940+0000 7f17ed400640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 0x7f17c8071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0ad000 tx=0x7f17dc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:11.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:11.944+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f17d405b790 con 0x7f17e8070a50 2024-05-14T11:17:12.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.104+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f17e80b03a0 con 0x7f17e8070a50 2024-05-14T11:17:12.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.496+0000 7f17e5600640 1 -- 172.21.15.116:0/2679048571 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f17d405fa90 con 0x7f17e8070a50 2024-05-14T11:17:12.498 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.496+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 msgr2=0x7f17c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.496+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 0x7f17c8071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0ad000 tx=0x7f17dc0a6000 comp rx=0 tx=0).stop 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.496+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 msgr2=0x7f17e813a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.496+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c700 tx=0x7f17d400cbc0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 shutdown_connections 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c806f400 0x7f17c8071910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e80d0290 0x7f17e81fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e8070a50 0x7f17e813a980 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 --2- 172.21.15.116:0/2679048571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e8070640 0x7f17e815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 >> 172.21.15.116:0/2679048571 conn(0x7f17e80cde80 msgr2=0x7f17e80b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:12.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 shutdown_connections 2024-05-14T11:17:12.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.500+0000 7f17efe00640 1 -- 172.21.15.116:0/2679048571 wait complete. 2024-05-14T11:17:12.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:17:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 Processor -- start 2024-05-14T11:17:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 -- start start 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d406f0a0 con 0x7fa0d40d1730 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d40cbb60 con 0x7fa0d406c700 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.676+0000 7fa0d9600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d406f420 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44144/0 (socket says 172.21.15.116:44144) 2024-05-14T11:17:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 -- 172.21.15.116:0/967000797 learned_addr learned my addr 172.21.15.116:0/967000797 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:12.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 -- 172.21.15.116:0/967000797 >> v1:172.21.15.149:6789/0 conn(0x7fa0d406c700 legacy=0x7fa0d406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:12.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 -- 172.21.15.116:0/967000797 >> v1:172.21.15.116:6789/0 conn(0x7fa0d40d1730 legacy=0x7fa0d40d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 -- 172.21.15.116:0/967000797 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d406f700 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d2a00640 1 --2- 172.21.15.116:0/967000797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009870 tx=0x7fa0c401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac65198b114e4134 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d1600640 1 -- 172.21.15.116:0/967000797 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40b2070 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d1600640 1 -- 172.21.15.116:0/967000797 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c40ad680 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.680+0000 7fa0d1600640 1 -- 172.21.15.116:0/967000797 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40ada50 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- 172.21.15.116:0/967000797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 msgr2=0x7fa0d40cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 --2- 172.21.15.116:0/967000797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009870 tx=0x7fa0c401c380 comp rx=0 tx=0).stop 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- 172.21.15.116:0/967000797 shutdown_connections 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 --2- 172.21.15.116:0/967000797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d40cd5d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- 172.21.15.116:0/967000797 >> 172.21.15.116:0/967000797 conn(0x7fa0d40b4720 msgr2=0x7fa0d40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- 172.21.15.116:0/967000797 shutdown_connections 2024-05-14T11:17:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- 172.21.15.116:0/967000797 wait complete. 2024-05-14T11:17:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 Processor -- start 2024-05-14T11:17:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.684+0000 7fa0d9600640 1 -- start start 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 0x7fa0d409a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0d40d1730 0x7fa0d4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d416a9c0 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d415cf10 con 0x7fa0d40d1730 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d416ab40 con 0x7fa0d406c700 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44152/0 (socket says 172.21.15.116:44152) 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 -- 172.21.15.116:0/529154981 learned_addr learned my addr 172.21.15.116:0/529154981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2a00640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 0x7fa0d409a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d3400640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0d40d1730 0x7fa0d4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 -- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 msgr2=0x7fa0d409a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 0x7fa0d409a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 -- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0d40d1730 msgr2=0x7fa0d4157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0d40d1730 0x7fa0d4157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 -- 172.21.15.116:0/529154981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d409aaa0 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2a00640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 0x7fa0d409a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d2000640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c3f0 tx=0x7fa0c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0017070 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c00052d0 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d409ae70 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003820 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d40c77e0 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.688+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d40c7270 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.692+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa0c001a070 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.692+0000 7fa0caa00640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 0x7fa0b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:12.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.692+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7fa0c008e630 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.692+0000 7fa0d2a00640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 0x7fa0b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:12.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.692+0000 7fa0d2a00640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 0x7fa0b4071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40af000 tx=0x7fa0c40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:12.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.696+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa0c0005480 con 0x7fa0d40cd1c0 2024-05-14T11:17:12.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:12.856+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa0d406a9c0 con 0x7fa0d40cd1c0 2024-05-14T11:17:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.496+0000 7fa0caa00640 1 -- 172.21.15.116:0/529154981 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7fa0c005b4d0 con 0x7fa0d40cd1c0 2024-05-14T11:17:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 msgr2=0x7fa0b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 0x7fa0b4071940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40af000 tx=0x7fa0c40a7000 comp rx=0 tx=0).stop 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 msgr2=0x7fa0d4156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c3f0 tx=0x7fa0c000c8b0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 shutdown_connections 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0b406f430 0x7fa0b4071940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0d40d1730 0x7fa0d4157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0d40cd1c0 0x7fa0d4156be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 --2- 172.21.15.116:0/529154981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0d406c700 0x7fa0d409a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 >> 172.21.15.116:0/529154981 conn(0x7fa0d40b4720 msgr2=0x7fa0d415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:13.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 shutdown_connections 2024-05-14T11:17:13.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.500+0000 7fa0d9600640 1 -- 172.21.15.116:0/529154981 wait complete. 2024-05-14T11:17:13.545 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:17:13.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 Processor -- start 2024-05-14T11:17:13.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 -- start start 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb14006f450 0x7fb14006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb14006f860 con 0x7fb1400d1730 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb14006fae0 con 0x7fb1400cde70 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb146200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb14006db80 con 0x7fb14006f450 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb136a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb1400d1730 0x7fb1400d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40462/0 (socket says 172.21.15.116:40462) 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb136a00640 1 -- 172.21.15.116:0/1261736072 learned_addr learned my addr 172.21.15.116:0/1261736072 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13f400640 1 --2- 172.21.15.116:0/1261736072 >> v2:172.21.15.149:3300/0 conn(0x7fb14006f450 0x7fb14006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420492052 0 0) 0x7fb128001120 con 0x7fb1400d1730 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb124003530 con 0x7fb1400d1730 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774040003 0 0) 0x7fb14006fae0 con 0x7fb1400cde70 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb128001120 con 0x7fb1400cde70 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13f400640 1 -- 172.21.15.116:0/1261736072 >> v1:172.21.15.149:6789/0 conn(0x7fb1400cde70 legacy=0x7fb1401675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13f400640 1 -- 172.21.15.116:0/1261736072 >> v1:172.21.15.116:6789/0 conn(0x7fb1400d1730 legacy=0x7fb1400d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.692+0000 7fb13f400640 1 -- 172.21.15.116:0/1261736072 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14006fdc0 con 0x7fb14006f450 2024-05-14T11:17:13.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb13f400640 1 --2- 172.21.15.116:0/1261736072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb14006d640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb130009d90 tx=0x7fb13001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bcca1e1fffe31f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:13.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300b2070 con 0x7fb14006f450 2024-05-14T11:17:13.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1300a5920 con 0x7fb14006f450 2024-05-14T11:17:13.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb13ea00640 1 -- 172.21.15.116:0/1261736072 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300a5cd0 con 0x7fb14006f450 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 -- 172.21.15.116:0/1261736072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 msgr2=0x7fb14006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 --2- 172.21.15.116:0/1261736072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb14006d640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb130009d90 tx=0x7fb13001c970 comp rx=0 tx=0).stop 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 -- 172.21.15.116:0/1261736072 shutdown_connections 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 --2- 172.21.15.116:0/1261736072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb14006d640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 -- 172.21.15.116:0/1261736072 >> 172.21.15.116:0/1261736072 conn(0x7fb1400b4720 msgr2=0x7fb1400b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 -- 172.21.15.116:0/1261736072 shutdown_connections 2024-05-14T11:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.696+0000 7fb146200640 1 -- 172.21.15.116:0/1261736072 wait complete. 2024-05-14T11:17:13.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 Processor -- start 2024-05-14T11:17:13.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 -- start start 2024-05-14T11:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb1400c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1400cde70 0x7fb1401605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb14016b6b0 con 0x7fb1400d1730 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb14016b530 con 0x7fb14006f450 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.700+0000 7fb146200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb14016b830 con 0x7fb1400cde70 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44158/0 (socket says 172.21.15.116:44158) 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 -- 172.21.15.116:0/926023473 learned_addr learned my addr 172.21.15.116:0/926023473 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb136a00640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1400cde70 0x7fb1401605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13f400640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb1400c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 -- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1400cde70 msgr2=0x7fb1401605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1400cde70 0x7fb1401605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 -- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 msgr2=0x7fb1400c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb1400c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 -- 172.21.15.116:0/926023473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1400b3bf0 con 0x7fb1400d1730 2024-05-14T11:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb13fe00640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3c0 tx=0x7fb138001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138017070 con 0x7fb1400d1730 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1380052f0 con 0x7fb1400d1730 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1400cb4b0 con 0x7fb1400d1730 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138003840 con 0x7fb1400d1730 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1400c90f0 con 0x7fb1400d1730 2024-05-14T11:17:13.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1400c8e50 con 0x7fb1400d1730 2024-05-14T11:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.704+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb13801a070 con 0x7fb1400d1730 2024-05-14T11:17:13.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.708+0000 7fb137e00640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 0x7fb11c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.708+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fb13808e9a0 con 0x7fb1400d1730 2024-05-14T11:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.708+0000 7fb13f400640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 0x7fb11c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.708+0000 7fb13f400640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 0x7fb11c071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1300af000 tx=0x7fb1300a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:13.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.712+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb1400c8e50 con 0x7fb1400d1730 2024-05-14T11:17:13.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:13.904+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb14006a9c0 con 0x7fb1400d1730 2024-05-14T11:17:14.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.524+0000 7fb137e00640 1 -- 172.21.15.116:0/926023473 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb138021020 con 0x7fb1400d1730 2024-05-14T11:17:14.530 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 5 and data pool 6 2024-05-14T11:17:14.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 msgr2=0x7fb11c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 0x7fb11c071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1300af000 tx=0x7fb1300a7070 comp rx=0 tx=0).stop 2024-05-14T11:17:14.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 msgr2=0x7fb140157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3c0 tx=0x7fb138001e80 comp rx=0 tx=0).stop 2024-05-14T11:17:14.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 shutdown_connections 2024-05-14T11:17:14.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb11c06f400 0x7fb11c071910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1400d1730 0x7fb140157e50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1400cde70 0x7fb1401605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 --2- 172.21.15.116:0/926023473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14006f450 0x7fb1400c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.532+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 >> 172.21.15.116:0/926023473 conn(0x7fb1400b4720 msgr2=0x7fb1400c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:14.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.540+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 shutdown_connections 2024-05-14T11:17:14.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.540+0000 7fb146200640 1 -- 172.21.15.116:0/926023473 wait complete. 2024-05-14T11:17:14.576 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.720+0000 7fcb8d800640 1 Processor -- start 2024-05-14T11:17:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.720+0000 7fcb8d800640 1 -- start start 2024-05-14T11:17:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb8d800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb880d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb8d800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcb8806c700 0x7fcb8806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb8d800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8806ff20 con 0x7fcb8815db30 2024-05-14T11:17:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb8d800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcb880ae100 con 0x7fcb8806c700 2024-05-14T11:17:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb8d800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcb880ae870 con 0x7fcb880d1670 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcb8806c700 0x7fcb8806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb86a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcb8815db30 0x7fcb880aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40464/0 (socket says 172.21.15.116:40464) 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb86a00640 1 -- 172.21.15.116:0/1248165568 learned_addr learned my addr 172.21.15.116:0/1248165568 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcb8806c700 0x7fcb8806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44166/0 (socket says 172.21.15.116:44166) 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb7e000640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb880d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 -- 172.21.15.116:0/1248165568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 msgr2=0x7fcb880d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb880d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 -- 172.21.15.116:0/1248165568 >> v1:172.21.15.116:6789/0 conn(0x7fcb8815db30 legacy=0x7fcb880aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.724+0000 7fcb87400640 1 -- 172.21.15.116:0/1248165568 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb880d1fc0 con 0x7fcb8806c700 2024-05-14T11:17:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb87400640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8806c700 0x7fcb8806cb10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009d90 tx=0x7fcb8001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b63e9320b386963 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb86000640 1 -- 172.21.15.116:0/1248165568 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800b2070 con 0x7fcb8806c700 2024-05-14T11:17:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb86000640 1 -- 172.21.15.116:0/1248165568 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb800a57c0 con 0x7fcb8806c700 2024-05-14T11:17:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb86000640 1 -- 172.21.15.116:0/1248165568 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800a5b90 con 0x7fcb8806c700 2024-05-14T11:17:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- 172.21.15.116:0/1248165568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8806c700 msgr2=0x7fcb8806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8806c700 0x7fcb8806cb10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009d90 tx=0x7fcb8001c810 comp rx=0 tx=0).stop 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- 172.21.15.116:0/1248165568 shutdown_connections 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8806c700 0x7fcb8806cb10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 --2- 172.21.15.116:0/1248165568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb880d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- 172.21.15.116:0/1248165568 >> 172.21.15.116:0/1248165568 conn(0x7fcb880b4620 msgr2=0x7fcb880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- 172.21.15.116:0/1248165568 shutdown_connections 2024-05-14T11:17:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- 172.21.15.116:0/1248165568 wait complete. 2024-05-14T11:17:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 Processor -- start 2024-05-14T11:17:14.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.728+0000 7fcb8d800640 1 -- start start 2024-05-14T11:17:14.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 0x7fcb880d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb8809a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8816a770 con 0x7fcb8815db30 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8815df40 con 0x7fcb8806c700 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8816a8f0 con 0x7fcb880d1670 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44182/0 (socket says 172.21.15.116:44182) 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 -- 172.21.15.116:0/775110080 learned_addr learned my addr 172.21.15.116:0/775110080 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb7e000640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb8809a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb86a00640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 0x7fcb880d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 -- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 msgr2=0x7fcb8809a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb8809a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 -- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 msgr2=0x7fcb880d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 0x7fcb880d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 -- 172.21.15.116:0/775110080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb88149640 con 0x7fcb8815db30 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb86a00640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 0x7fcb880d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb87400640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb800b0040 tx=0x7fcb800195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800b2050 con 0x7fcb8815db30 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb800be070 con 0x7fcb8815db30 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800ad850 con 0x7fcb8815db30 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb88149920 con 0x7fcb8815db30 2024-05-14T11:17:14.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb88149ee0 con 0x7fcb8815db30 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.732+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8806a9c0 con 0x7fcb8815db30 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.736+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb800ad9f0 con 0x7fcb8815db30 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.736+0000 7fcb7f400640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 0x7fcb64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.736+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fcb8012a3c0 con 0x7fcb8815db30 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.736+0000 7fcb86a00640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 0x7fcb64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:14.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.736+0000 7fcb86a00640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 0x7fcb64071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000f70 tx=0x7fcb78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.740+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcb880d1fc0 con 0x7fcb8815db30 2024-05-14T11:17:14.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.900+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb880d0bd0 con 0x7fcb8815db30 2024-05-14T11:17:14.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.900+0000 7fcb7f400640 1 -- 172.21.15.116:0/775110080 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17003 (secure 0 0 0) 0x7fcb80016520 con 0x7fcb8815db30 2024-05-14T11:17:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:14.903 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:13.909735+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:12.118135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:12.868729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:14.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 msgr2=0x7fcb64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 0x7fcb64071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000f70 tx=0x7fcb78009040 comp rx=0 tx=0).stop 2024-05-14T11:17:14.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 msgr2=0x7fcb880cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb800b0040 tx=0x7fcb800195b0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 shutdown_connections 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcb6406f400 0x7fcb64071910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8815db30 0x7fcb880cef90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcb880d1670 0x7fcb8809a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 --2- 172.21.15.116:0/775110080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb8806c700 0x7fcb880d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 >> 172.21.15.116:0/775110080 conn(0x7fcb880b4620 msgr2=0x7fcb88160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 shutdown_connections 2024-05-14T11:17:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:14.904+0000 7fcb8d800640 1 -- 172.21.15.116:0/775110080 wait complete. 2024-05-14T11:17:14.925 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:15.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.068+0000 7f905fe00640 1 Processor -- start 2024-05-14T11:17:15.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.068+0000 7f905fe00640 1 -- start start 2024-05-14T11:17:15.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 0x7f905806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9058070080 con 0x7f90580b0770 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905fe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905806cf10 con 0x7f9058072a30 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f905806db20 con 0x7f905806f730 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905d400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f90580b0770 0x7f905806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40466/0 (socket says 172.21.15.116:40466) 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905d400640 1 -- 172.21.15.116:0/1000390031 learned_addr learned my addr 172.21.15.116:0/1000390031 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905ca00640 1 --2- 172.21.15.116:0/1000390031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 0x7f905806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787380573 0 0) 0x7f904c001120 con 0x7f90580b0770 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9040003530 con 0x7f90580b0770 2024-05-14T11:17:15.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3328228301 0 0) 0x7f9040003530 con 0x7f90580b0770 2024-05-14T11:17:15.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905ca00640 1 -- 172.21.15.116:0/1000390031 >> v1:172.21.15.149:6789/0 conn(0x7f9058072a30 legacy=0x7f9058072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905ca00640 1 -- 172.21.15.116:0/1000390031 >> v1:172.21.15.116:6789/0 conn(0x7f90580b0770 legacy=0x7f905806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.072+0000 7f905ca00640 1 -- 172.21.15.116:0/1000390031 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905806d1f0 con 0x7f905806f730 2024-05-14T11:17:15.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905ca00640 1 --2- 172.21.15.116:0/1000390031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 0x7f905806fb40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90440097f0 tx=0x7f904401c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d40376751213ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440b0070 con 0x7f905806f730 2024-05-14T11:17:15.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90440ab400 con 0x7f905806f730 2024-05-14T11:17:15.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f9057e00640 1 -- 172.21.15.116:0/1000390031 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0018d0 con 0x7f905806f730 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- 172.21.15.116:0/1000390031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 msgr2=0x7f905806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 --2- 172.21.15.116:0/1000390031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 0x7f905806fb40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90440097f0 tx=0x7f904401c340 comp rx=0 tx=0).stop 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- 172.21.15.116:0/1000390031 shutdown_connections 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 --2- 172.21.15.116:0/1000390031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f905806f730 0x7f905806fb40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- 172.21.15.116:0/1000390031 >> 172.21.15.116:0/1000390031 conn(0x7f90580b4620 msgr2=0x7f90580b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- 172.21.15.116:0/1000390031 shutdown_connections 2024-05-14T11:17:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- 172.21.15.116:0/1000390031 wait complete. 2024-05-14T11:17:15.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 Processor -- start 2024-05-14T11:17:15.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.076+0000 7f905fe00640 1 -- start start 2024-05-14T11:17:15.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905806f730 0x7f90580ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90580b0770 0x7f90581f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9058073ae0 con 0x7f9058072a30 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9058073960 con 0x7f905806f730 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f905816b0d0 con 0x7f90580b0770 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44190/0 (socket says 172.21.15.116:44190) 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 -- 172.21.15.116:0/1919864092 learned_addr learned my addr 172.21.15.116:0/1919864092 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905d400640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905806f730 0x7f90580ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905de00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90580b0770 0x7f90581f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 -- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90580b0770 msgr2=0x7f90581f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90580b0770 0x7f90581f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 -- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905806f730 msgr2=0x7f90580ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905806f730 0x7f90580ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 -- 172.21.15.116:0/1919864092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9058149c80 con 0x7f9058072a30 2024-05-14T11:17:15.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905ca00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90440097f0 tx=0x7f90440190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440ba070 con 0x7f9058072a30 2024-05-14T11:17:15.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90440038a0 con 0x7f9058072a30 2024-05-14T11:17:15.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440b0040 con 0x7f9058072a30 2024-05-14T11:17:15.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90580cf1a0 con 0x7f9058072a30 2024-05-14T11:17:15.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90580cf760 con 0x7f9058072a30 2024-05-14T11:17:15.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.080+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905806a9c0 con 0x7f9058072a30 2024-05-14T11:17:15.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.084+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9044003c50 con 0x7f9058072a30 2024-05-14T11:17:15.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.084+0000 7f9055600640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 0x7f9038071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.084+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f90441284f0 con 0x7f9058072a30 2024-05-14T11:17:15.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.084+0000 7f905d400640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 0x7f9038071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.084+0000 7f905d400640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 0x7f9038071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f904c0015a0 tx=0x7f904c001140 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.088+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9044015ff0 con 0x7f9058072a30 2024-05-14T11:17:15.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.244+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90580cab30 con 0x7f9058072a30 2024-05-14T11:17:15.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f9055600640 1 -- 172.21.15.116:0/1919864092 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17003 (secure 0 0 0) 0x7f90440f4b60 con 0x7f9058072a30 2024-05-14T11:17:15.250 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:15.250 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:13.909735+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:12.118135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:12.868729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:15.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 msgr2=0x7f9038071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 0x7f9038071940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f904c0015a0 tx=0x7f904c001140 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 msgr2=0x7f9058149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90440097f0 tx=0x7f90440190e0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 shutdown_connections 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f903806f430 0x7f9038071940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90580b0770 0x7f90581f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9058072a30 0x7f9058149740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 --2- 172.21.15.116:0/1919864092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f905806f730 0x7f90580ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 >> 172.21.15.116:0/1919864092 conn(0x7f90580b4620 msgr2=0x7f905806e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 shutdown_connections 2024-05-14T11:17:15.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.248+0000 7f905fe00640 1 -- 172.21.15.116:0/1919864092 wait complete. 2024-05-14T11:17:15.271 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:17:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.416+0000 7f9121c00640 1 Processor -- start 2024-05-14T11:17:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.416+0000 7f9121c00640 1 -- start start 2024-05-14T11:17:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f911c0d1730 0x7f911c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911c0ce630 con 0x7f911c0ce220 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f911c0d1080 con 0x7f911c0d1730 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d2a20 con 0x7f911c0709c0 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9113400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f911c0ce220 0x7f911c0702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40476/0 (socket says 172.21.15.116:40476) 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9113400640 1 -- 172.21.15.116:0/446504758 learned_addr learned my addr 172.21.15.116:0/446504758 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121200640 1 --2- 172.21.15.116:0/446504758 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f911be00640 1 --2- 172.21.15.116:0/446504758 >> v2:172.21.15.149:3300/0 conn(0x7f911c0d1730 0x7f911c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f911b400640 1 -- 172.21.15.116:0/446504758 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023993179 0 0) 0x7f9104001120 con 0x7f911c0ce220 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f911b400640 1 -- 172.21.15.116:0/446504758 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9100003530 con 0x7f911c0ce220 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121200640 1 -- 172.21.15.116:0/446504758 >> v2:172.21.15.149:3300/0 conn(0x7f911c0d1730 msgr2=0x7f911c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121200640 1 --2- 172.21.15.116:0/446504758 >> v2:172.21.15.149:3300/0 conn(0x7f911c0d1730 0x7f911c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121200640 1 -- 172.21.15.116:0/446504758 >> v1:172.21.15.116:6789/0 conn(0x7f911c0ce220 legacy=0x7f911c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.420+0000 7f9121200640 1 -- 172.21.15.116:0/446504758 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0d1b40 con 0x7f911c0709c0 2024-05-14T11:17:15.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121200640 1 --2- 172.21.15.116:0/446504758 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c070dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f911401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8544b2502d1ccf5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f911b400640 1 -- 172.21.15.116:0/446504758 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140b2070 con 0x7f911c0709c0 2024-05-14T11:17:15.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f911b400640 1 -- 172.21.15.116:0/446504758 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91140ad640 con 0x7f911c0709c0 2024-05-14T11:17:15.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f911b400640 1 -- 172.21.15.116:0/446504758 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140ada10 con 0x7f911c0709c0 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 -- 172.21.15.116:0/446504758 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 msgr2=0x7f911c070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 --2- 172.21.15.116:0/446504758 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c070dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f911401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 -- 172.21.15.116:0/446504758 shutdown_connections 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 --2- 172.21.15.116:0/446504758 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c070dd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 --2- 172.21.15.116:0/446504758 >> v2:172.21.15.149:3300/0 conn(0x7f911c0d1730 0x7f911c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 -- 172.21.15.116:0/446504758 >> 172.21.15.116:0/446504758 conn(0x7f911c0b5670 msgr2=0x7f911c0d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 -- 172.21.15.116:0/446504758 shutdown_connections 2024-05-14T11:17:15.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.424+0000 7f9121c00640 1 -- 172.21.15.116:0/446504758 wait complete. 2024-05-14T11:17:15.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 Processor -- start 2024-05-14T11:17:15.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 -- start start 2024-05-14T11:17:15.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f911c0ce220 0x7f911c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d2eb0 con 0x7f911c0d1730 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d26b0 con 0x7f911c0ce220 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d3030 con 0x7f911c0709c0 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44200/0 (socket says 172.21.15.116:44200) 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 -- 172.21.15.116:0/2043826407 learned_addr learned my addr 172.21.15.116:0/2043826407 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9113400640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f911c0ce220 0x7f911c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f911be00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 -- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 msgr2=0x7f911c0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 -- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f911c0ce220 msgr2=0x7f911c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f911c0ce220 0x7f911c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.428+0000 7f9121200640 1 -- 172.21.15.116:0/2043826407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0b6530 con 0x7f911c0d1730 2024-05-14T11:17:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9121200640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f9114019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140bc070 con 0x7f911c0d1730 2024-05-14T11:17:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91140b65f0 con 0x7f911c0d1730 2024-05-14T11:17:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c0c9b70 con 0x7f911c0d1730 2024-05-14T11:17:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140b2040 con 0x7f911c0d1730 2024-05-14T11:17:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c0b6ad0 con 0x7f911c0d1730 2024-05-14T11:17:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c06a9c0 con 0x7f911c0d1730 2024-05-14T11:17:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91140bf070 con 0x7f911c0d1730 2024-05-14T11:17:15.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.432+0000 7f9118c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 0x7f90f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:15.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.436+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f9114128800 con 0x7f911c0d1730 2024-05-14T11:17:15.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.436+0000 7f911be00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 0x7f90f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:15.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.436+0000 7f911be00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 0x7f90f8071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f910c011340 tx=0x7f910c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.440+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9114016090 con 0x7f911c0d1730 2024-05-14T11:17:15.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:15.636+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f911c0b92b0 con 0x7f911c0d1730 2024-05-14T11:17:16.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9118c00640 1 -- 172.21.15.116:0/2043826407 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f91140f4e70 con 0x7f911c0d1730 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 msgr2=0x7f90f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 0x7f90f8071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f910c011340 tx=0x7f910c00b040 comp rx=0 tx=0).stop 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 msgr2=0x7f911c0c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f9114019240 comp rx=0 tx=0).stop 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 shutdown_connections 2024-05-14T11:17:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90f806f430 0x7f90f8071940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f911c0ce220 0x7f911c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f911c0d1730 0x7f911c0c9330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 --2- 172.21.15.116:0/2043826407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f911c0709c0 0x7f911c0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 >> 172.21.15.116:0/2043826407 conn(0x7f911c0b5670 msgr2=0x7f911c15f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 shutdown_connections 2024-05-14T11:17:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.540+0000 7f9121c00640 1 -- 172.21.15.116:0/2043826407 wait complete. 2024-05-14T11:17:16.568 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:17:16.568 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:17:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 Processor -- start 2024-05-14T11:17:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 -- start start 2024-05-14T11:17:16.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dfc0c74f0 con 0x7f1dfc1642d0 2024-05-14T11:17:16.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc0cee00 con 0x7f1dfc0c70e0 2024-05-14T11:17:16.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1e02000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0cfe50 con 0x7f1dfc163980 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1df2a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.720+0000 7f1dfb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35546/0 (socket says 172.21.15.116:35546) 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1dfb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:42220/0 (socket says 172.21.15.116:42220) 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 -- 172.21.15.116:0/835263345 learned_addr learned my addr 172.21.15.116:0/835263345 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1dfbe00640 1 --1- 172.21.15.116:0/835263345 >> v1:172.21.15.149:6789/0 conn(0x7f1dfc1642d0 0x7f1dfc06efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:56918/0 (socket says 172.21.15.116:56918) 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 -- 172.21.15.116:0/835263345 >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 msgr2=0x7f1dfc15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 --2- 172.21.15.116:0/835263345 >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 -- 172.21.15.116:0/835263345 >> v1:172.21.15.149:6789/0 conn(0x7f1dfc1642d0 legacy=0x7f1dfc06efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 -- 172.21.15.116:0/835263345 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc06e260 con 0x7f1dfc163980 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1dfaa00640 1 -- 172.21.15.116:0/835263345 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004538297 0 0) 0x7f1df40014a0 con 0x7f1dfc1642d0 2024-05-14T11:17:16.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1dfb400640 1 --2- 172.21.15.116:0/835263345 >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:16.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.724+0000 7f1df2a00640 1 --2- 172.21.15.116:0/835263345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009870 tx=0x7f1de401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dda5a8b1d745192 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:16.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1dfaa00640 1 -- 172.21.15.116:0/835263345 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40b5070 con 0x7f1dfc163980 2024-05-14T11:17:16.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1dfaa00640 1 -- 172.21.15.116:0/835263345 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de40a8a50 con 0x7f1dfc163980 2024-05-14T11:17:16.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1dfaa00640 1 -- 172.21.15.116:0/835263345 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40a8e20 con 0x7f1dfc163980 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- 172.21.15.116:0/835263345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 msgr2=0x7f1dfc163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 --2- 172.21.15.116:0/835263345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009870 tx=0x7f1de401ed50 comp rx=0 tx=0).stop 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- 172.21.15.116:0/835263345 shutdown_connections 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 --2- 172.21.15.116:0/835263345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc163980 0x7f1dfc163d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 --2- 172.21.15.116:0/835263345 >> v2:172.21.15.149:3300/0 conn(0x7f1dfc0c70e0 0x7f1dfc15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- 172.21.15.116:0/835263345 >> 172.21.15.116:0/835263345 conn(0x7f1dfc0d0670 msgr2=0x7f1dfc0d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:16.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- 172.21.15.116:0/835263345 shutdown_connections 2024-05-14T11:17:16.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- 172.21.15.116:0/835263345 wait complete. 2024-05-14T11:17:16.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 Processor -- start 2024-05-14T11:17:16.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.728+0000 7f1e02000640 1 -- start start 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc0c70e0 0x7f1dfc0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 0x7f1dfc1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc164870 con 0x7f1dfc1642d0 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc06eac0 con 0x7f1dfc163980 2024-05-14T11:17:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc160100 con 0x7f1dfc0c70e0 2024-05-14T11:17:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44216/0 (socket says 172.21.15.116:44216) 2024-05-14T11:17:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 -- 172.21.15.116:0/2921378729 learned_addr learned my addr 172.21.15.116:0/2921378729 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1df2a00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 0x7f1dfc1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfb400640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc0c70e0 0x7f1dfc0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 -- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc0c70e0 msgr2=0x7f1dfc0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc0c70e0 0x7f1dfc0b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 -- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 msgr2=0x7f1dfc1f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 0x7f1dfc1f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 -- 172.21.15.116:0/2921378729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc158f90 con 0x7f1dfc1642d0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1df2a00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 0x7f1dfc1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1dfbe00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c6b0 tx=0x7f1df400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4018070 con 0x7f1dfc1642d0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df4004db0 con 0x7f1dfc1642d0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc159270 con 0x7f1dfc1642d0 2024-05-14T11:17:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400a880 con 0x7f1dfc1642d0 2024-05-14T11:17:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1f6b30 con 0x7f1dfc1642d0 2024-05-14T11:17:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.732+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc06a9c0 con 0x7f1dfc1642d0 2024-05-14T11:17:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.736+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1df401b070 con 0x7f1dfc1642d0 2024-05-14T11:17:16.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.736+0000 7f1df3e00640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:16.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.736+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1df408ea50 con 0x7f1dfc1642d0 2024-05-14T11:17:16.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.736+0000 7f1dfb400640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:16.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.736+0000 7f1dfb400640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 0x7f1dd8071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009580 tx=0x7f1dec009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:16.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.740+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1dfc0c74f0 con 0x7f1dfc1642d0 2024-05-14T11:17:16.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:16.932+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1dfc0cd960 con 0x7f1dfc1642d0 2024-05-14T11:17:17.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.544+0000 7f1df3e00640 1 -- 172.21.15.116:0/2921378729 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f1df405b0c0 con 0x7f1dfc1642d0 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 msgr2=0x7f1dd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 0x7f1dd8071940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009580 tx=0x7f1dec009190 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 msgr2=0x7f1dfc161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c6b0 tx=0x7f1df400cb70 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 shutdown_connections 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1dd806f430 0x7f1dd8071940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dfc163980 0x7f1dfc1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc1642d0 0x7f1dfc161630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 --2- 172.21.15.116:0/2921378729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dfc0c70e0 0x7f1dfc0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 >> 172.21.15.116:0/2921378729 conn(0x7f1dfc0d0670 msgr2=0x7f1dfc0c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:17.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 shutdown_connections 2024-05-14T11:17:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.548+0000 7f1e02000640 1 -- 172.21.15.116:0/2921378729 wait complete. 2024-05-14T11:17:17.573 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:17:17.573 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:17.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.720+0000 7f5791c00640 1 Processor -- start 2024-05-14T11:17:17.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.720+0000 7f5791c00640 1 -- start start 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5791c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f578c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5791c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c070080 con 0x7f578c072a30 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5791c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578c06cf10 con 0x7f578c0b0770 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5791c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f578c06db20 con 0x7f578c06f730 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f578c072a30 0x7f578c072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56930/0 (socket says 172.21.15.116:56930) 2024-05-14T11:17:17.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 -- 172.21.15.116:0/1506392691 learned_addr learned my addr 172.21.15.116:0/1506392691 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5782000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f578c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840031736 0 0) 0x7f5784001120 con 0x7f578c072a30 2024-05-14T11:17:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5770003450 con 0x7f578c072a30 2024-05-14T11:17:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 414401763 0 0) 0x7f5770003450 con 0x7f578c072a30 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 msgr2=0x7f578c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f578c06fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 >> v1:172.21.15.149:6789/0 conn(0x7f578c0b0770 legacy=0x7f578c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c06d1f0 con 0x7f578c072a30 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f57840018d0 con 0x7f578c072a30 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 >> v1:172.21.15.116:6789/0 conn(0x7f578c072a30 legacy=0x7f578c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5770003ae0 0x7f5770003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f577000a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f577000b060 0x7f577000d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f578c06db20 con 0x7f577000b060 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f577000ddc0 con 0x7f5770003ae0 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f577000dfc0 con 0x7f578c06f730 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f577000b060 0x7f577000d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578aa00640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5770003ae0 0x7f5770003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f5782000640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f577000a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 -- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 msgr2=0x7f577000a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c06f730 0x7f577000a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 -- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5770003ae0 msgr2=0x7f5770003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5770003ae0 0x7f5770003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.724+0000 7f578b400640 1 -- 172.21.15.116:0/1506392691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5770003450 con 0x7f577000b060 2024-05-14T11:17:17.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f578b400640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f577000b060 0x7f577000d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f578c0b0d40 tx=0x7f578401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d119038c3e1ef7b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:17.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784007e60 con 0x7f577000b060 2024-05-14T11:17:17.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57840a9020 con 0x7f577000b060 2024-05-14T11:17:17.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f578a000640 1 -- 172.21.15.116:0/1506392691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784005b20 con 0x7f577000b060 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 -- 172.21.15.116:0/1506392691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f577000b060 msgr2=0x7f577000d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 --2- 172.21.15.116:0/1506392691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f577000b060 0x7f577000d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f578c0b0d40 tx=0x7f578401ca90 comp rx=0 tx=0).stop 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f578b400640 1 -- 172.21.15.116:0/1506392691 reap_dead start 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 -- 172.21.15.116:0/1506392691 shutdown_connections 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 -- 172.21.15.116:0/1506392691 >> 172.21.15.116:0/1506392691 conn(0x7f578c0b4620 msgr2=0x7f578c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 -- 172.21.15.116:0/1506392691 shutdown_connections 2024-05-14T11:17:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.728+0000 7f5791c00640 1 -- 172.21.15.116:0/1506392691 wait complete. 2024-05-14T11:17:17.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 Processor -- start 2024-05-14T11:17:17.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- start start 2024-05-14T11:17:17.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f578c157490 0x7f578c09a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c1625f0 0x7f578c162a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f578c073ae0 con 0x7f578c166160 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f578c073960 con 0x7f578c157490 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f578c167fa0 con 0x7f578c1625f0 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42362/0 (socket says 172.21.15.116:42362) 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 -- 172.21.15.116:0/2890643718 learned_addr learned my addr 172.21.15.116:0/2890643718 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578aa00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f578c157490 0x7f578c09a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5782000640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c1625f0 0x7f578c162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 -- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c1625f0 msgr2=0x7f578c162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c1625f0 0x7f578c162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 -- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f578c157490 msgr2=0x7f578c09a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f578c157490 0x7f578c09a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 -- 172.21.15.116:0/2890643718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c166ab0 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f578b400640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f57840ab040 tx=0x7f5770003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784007e60 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57840b2030 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5784000e50 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c156150 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.732+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c1566d0 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578c06a9c0 con 0x7f578c166160 2024-05-14T11:17:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5784004ab0 con 0x7f578c166160 2024-05-14T11:17:17.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f5783400640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 0x7f5768071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:17.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5784127eb0 con 0x7f578c166160 2024-05-14T11:17:17.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f578aa00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 0x7f5768071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:17.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.736+0000 7f578aa00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 0x7f5768071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f578c0b2d00 tx=0x7f577c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:17.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.740+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f578c166ab0 con 0x7f578c166160 2024-05-14T11:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.932+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578c06f510 con 0x7f578c166160 2024-05-14T11:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.932+0000 7f5783400640 1 -- 172.21.15.116:0/2890643718 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f578c070080 con 0x7f578c166160 2024-05-14T11:17:17.935 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:17.935 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 msgr2=0x7f5768071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 0x7f5768071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f578c0b2d00 tx=0x7f577c009040 comp rx=0 tx=0).stop 2024-05-14T11:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 msgr2=0x7f578c166570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f57840ab040 tx=0x7f5770003740 comp rx=0 tx=0).stop 2024-05-14T11:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 shutdown_connections 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f576806f430 0x7f5768071940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f578c166160 0x7f578c166570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f578c1625f0 0x7f578c162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 --2- 172.21.15.116:0/2890643718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f578c157490 0x7f578c09a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 >> 172.21.15.116:0/2890643718 conn(0x7f578c0b4620 msgr2=0x7f578c15e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 shutdown_connections 2024-05-14T11:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:17.936+0000 7f5791c00640 1 -- 172.21.15.116:0/2890643718 wait complete. 2024-05-14T11:17:17.957 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:18.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.100+0000 7f52e1000640 1 Processor -- start 2024-05-14T11:17:18.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.100+0000 7f52e1000640 1 -- start start 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52e1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52e1000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc160510 con 0x7f52dc156fa0 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52e1000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc1607f0 con 0x7f52dc0d1670 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52e1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc0706f0 con 0x7f52dc0b0390 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d9e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f52dc156fa0 0x7f52dc06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56932/0 (socket says 172.21.15.116:56932) 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d9e00640 1 -- 172.21.15.116:0/162667794 learned_addr learned my addr 172.21.15.116:0/162667794 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52da800640 1 --2- 172.21.15.116:0/162667794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289918089 0 0) 0x7f52dc160510 con 0x7f52dc156fa0 2024-05-14T11:17:18.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52c0005740 con 0x7f52dc156fa0 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147529603 0 0) 0x7f52dc1607f0 con 0x7f52dc0d1670 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52dc160510 con 0x7f52dc0d1670 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52da800640 1 -- 172.21.15.116:0/162667794 >> v1:172.21.15.149:6789/0 conn(0x7f52dc0d1670 legacy=0x7f52dc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52da800640 1 -- 172.21.15.116:0/162667794 >> v1:172.21.15.116:6789/0 conn(0x7f52dc156fa0 legacy=0x7f52dc06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1839215024 0 0) 0x7f52c0005740 con 0x7f52dc156fa0 2024-05-14T11:17:18.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.104+0000 7f52da800640 1 -- 172.21.15.116:0/162667794 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc160ad0 con 0x7f52dc0b0390 2024-05-14T11:17:18.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52da800640 1 --2- 172.21.15.116:0/162667794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc167800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097f0 tx=0x7f52d401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1de96fad19ddfca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40b0070 con 0x7f52dc0b0390 2024-05-14T11:17:18.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d40ab400 con 0x7f52dc0b0390 2024-05-14T11:17:18.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52d8a00640 1 -- 172.21.15.116:0/162667794 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc001a80 con 0x7f52dc0b0390 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 -- 172.21.15.116:0/162667794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 msgr2=0x7f52dc167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 --2- 172.21.15.116:0/162667794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc167800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097f0 tx=0x7f52d401c400 comp rx=0 tx=0).stop 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 -- 172.21.15.116:0/162667794 shutdown_connections 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 --2- 172.21.15.116:0/162667794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc167800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 -- 172.21.15.116:0/162667794 >> 172.21.15.116:0/162667794 conn(0x7f52dc0b4620 msgr2=0x7f52dc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 -- 172.21.15.116:0/162667794 shutdown_connections 2024-05-14T11:17:18.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.108+0000 7f52e1000640 1 -- 172.21.15.116:0/162667794 wait complete. 2024-05-14T11:17:18.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 Processor -- start 2024-05-14T11:17:18.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- start start 2024-05-14T11:17:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 0x7f52dc09c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc06d2a0 con 0x7f52dc0d1670 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc06d120 con 0x7f52dc156fa0 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc06d420 con 0x7f52dc0b0390 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42370/0 (socket says 172.21.15.116:42370) 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 -- 172.21.15.116:0/2877782873 learned_addr learned my addr 172.21.15.116:0/2877782873 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9e00640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52da800640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 0x7f52dc09c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 -- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 msgr2=0x7f52dc09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 -- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 msgr2=0x7f52dc09c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 0x7f52dc09c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 -- 172.21.15.116:0/2877782873 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc154d90 con 0x7f52dc0d1670 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52da800640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 0x7f52dc09c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d9400640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c430 tx=0x7f52c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8017070 con 0x7f52dc0d1670 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc0739b0 con 0x7f52dc0d1670 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c80052d0 con 0x7f52dc0d1670 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8003840 con 0x7f52dc0d1670 2024-05-14T11:17:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.112+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc073f70 con 0x7f52dc0d1670 2024-05-14T11:17:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc06a9c0 con 0x7f52dc0d1670 2024-05-14T11:17:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52c801a070 con 0x7f52dc0d1670 2024-05-14T11:17:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52d2000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 0x7f52bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f52c808e860 con 0x7f52dc0d1670 2024-05-14T11:17:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52d9e00640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 0x7f52bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.116+0000 7f52d9e00640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 0x7f52bc071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52cc000f90 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.120+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f52c8005480 con 0x7f52dc0d1670 2024-05-14T11:17:18.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.280+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52dc06fdc0 con 0x7f52dc0d1670 2024-05-14T11:17:18.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52d2000640 1 -- 172.21.15.116:0/2877782873 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16990 (secure 0 0 0) 0x7f52c805b5c0 con 0x7f52dc0d1670 2024-05-14T11:17:18.285 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:18.285 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:15.527236+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:12.118135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:12.868729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:18.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 msgr2=0x7f52bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 0x7f52bc071910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52cc000f90 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 msgr2=0x7f52dc09bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c430 tx=0x7f52c800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 shutdown_connections 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52bc06f400 0x7f52bc071910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52dc156fa0 0x7f52dc09c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52dc0d1670 0x7f52dc09bd00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 --2- 172.21.15.116:0/2877782873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52dc0b0390 0x7f52dc09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 >> 172.21.15.116:0/2877782873 conn(0x7f52dc0b4620 msgr2=0x7f52dc0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:18.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 shutdown_connections 2024-05-14T11:17:18.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.284+0000 7f52e1000640 1 -- 172.21.15.116:0/2877782873 wait complete. 2024-05-14T11:17:18.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:17:18.307 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:17:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 Processor -- start 2024-05-14T11:17:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 -- start start 2024-05-14T11:17:18.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f94740d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f947406ff20 con 0x7f947415db30 2024-05-14T11:17:18.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94740d1fc0 con 0x7f947406c700 2024-05-14T11:17:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f947b800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94740ae100 con 0x7f94740d1670 2024-05-14T11:17:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f9478e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f947415db30 0x7f94740aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56934/0 (socket says 172.21.15.116:56934) 2024-05-14T11:17:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.452+0000 7f9478e00640 1 -- 172.21.15.116:0/3031787764 learned_addr learned my addr 172.21.15.116:0/3031787764 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f946be00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f94740d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002105866 0 0) 0x7f947406ff20 con 0x7f947415db30 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9458003530 con 0x7f947415db30 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2515191141 0 0) 0x7f9458003530 con 0x7f947415db30 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 msgr2=0x7f94740d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f94740d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 >> v1:172.21.15.149:6789/0 conn(0x7f947406c700 legacy=0x7f947406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9474070470 con 0x7f947415db30 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9464001700 con 0x7f947415db30 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 >> v1:172.21.15.116:6789/0 conn(0x7f947415db30 legacy=0x7f94740aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f9458003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9458004010 0x7f9458004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9458004e70 0x7f94580052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94740ae100 con 0x7f9458004e70 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f945800bd30 con 0x7f9458004010 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f945800bf30 con 0x7f94740d1670 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9458004e70 0x7f94580052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9478e00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f9458003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f946be00640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9458004010 0x7f9458004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 -- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 msgr2=0x7f9458003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94740d1670 0x7f9458003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 -- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9458004010 msgr2=0x7f9458004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9458004010 0x7f9458004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.456+0000 7f9479800640 1 -- 172.21.15.116:0/3031787764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458003530 con 0x7f9458004e70 2024-05-14T11:17:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f9479800640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9458004e70 0x7f94580052a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94741626c0 tx=0x7f946c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d54a55e4b1995be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c001a80 con 0x7f9458004e70 2024-05-14T11:17:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946c003b80 con 0x7f9458004e70 2024-05-14T11:17:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f9473e00640 1 -- 172.21.15.116:0/3031787764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0050d0 con 0x7f9458004e70 2024-05-14T11:17:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- 172.21.15.116:0/3031787764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9458004e70 msgr2=0x7f94580052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 --2- 172.21.15.116:0/3031787764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9458004e70 0x7f94580052a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94741626c0 tx=0x7f946c01c520 comp rx=0 tx=0).stop 2024-05-14T11:17:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f9479800640 1 -- 172.21.15.116:0/3031787764 reap_dead start 2024-05-14T11:17:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- 172.21.15.116:0/3031787764 shutdown_connections 2024-05-14T11:17:18.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- 172.21.15.116:0/3031787764 >> 172.21.15.116:0/3031787764 conn(0x7f94740b4620 msgr2=0x7f94740b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:18.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- 172.21.15.116:0/3031787764 shutdown_connections 2024-05-14T11:17:18.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- 172.21.15.116:0/3031787764 wait complete. 2024-05-14T11:17:18.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 Processor -- start 2024-05-14T11:17:18.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.460+0000 7f947b800640 1 -- start start 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9458004e70 0x7f947409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94741f76a0 0x7f94741f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f947415df40 con 0x7f94740cd7e0 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94740ae870 con 0x7f94741f76a0 2024-05-14T11:17:18.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9474169f80 con 0x7f9458004e70 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42396/0 (socket says 172.21.15.116:42396) 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 -- 172.21.15.116:0/2402229535 learned_addr learned my addr 172.21.15.116:0/2402229535 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f9478e00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9458004e70 0x7f947409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f9479800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94741f76a0 0x7f94741f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 -- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9458004e70 msgr2=0x7f947409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9458004e70 0x7f947409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 -- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94741f76a0 msgr2=0x7f94741f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94741f76a0 0x7f94741f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 -- 172.21.15.116:0/2402229535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94740ce130 con 0x7f94740cd7e0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f946be00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f945c011a00 tx=0x7f945c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00fb50 con 0x7f94740cd7e0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c00fcf0 con 0x7f94740cd7e0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00fea0 con 0x7f94740cd7e0 2024-05-14T11:17:18.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947415a760 con 0x7f94740cd7e0 2024-05-14T11:17:18.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94740ce690 con 0x7f94740cd7e0 2024-05-14T11:17:18.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.464+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947406a9c0 con 0x7f94740cd7e0 2024-05-14T11:17:18.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.468+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f945c012040 con 0x7f94740cd7e0 2024-05-14T11:17:18.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.468+0000 7f9471600640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 0x7f9450071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.468+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f945c08fa00 con 0x7f94740cd7e0 2024-05-14T11:17:18.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.468+0000 7f9478e00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 0x7f9450071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.468+0000 7f9478e00640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 0x7f9450071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9464007d80 tx=0x7f9464007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.472+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f94740ce130 con 0x7f94740cd7e0 2024-05-14T11:17:18.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.628+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f947415a4f0 con 0x7f945006f400 2024-05-14T11:17:18.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.636+0000 7f9471600640 1 -- 172.21.15.116:0/2402229535 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251878 (secure 0 0 0) 0x7f947415a4f0 con 0x7f945006f400 2024-05-14T11:17:18.642 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 msgr2=0x7f9450071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 0x7f9450071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9464007d80 tx=0x7f9464007980 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 msgr2=0x7f94740cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f945c011a00 tx=0x7f945c011ec0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 shutdown_connections 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f945006f400 0x7f9450071910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94741f76a0 0x7f94741f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94740cd7e0 0x7f94740cdbf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 --2- 172.21.15.116:0/2402229535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9458004e70 0x7f947409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 >> 172.21.15.116:0/2402229535 conn(0x7f94740b4620 msgr2=0x7f947414ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 shutdown_connections 2024-05-14T11:17:18.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.640+0000 7f947b800640 1 -- 172.21.15.116:0/2402229535 wait complete. 2024-05-14T11:17:18.662 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-05-14T11:17:18.374373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220716,"kb_used_data":4520,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4628480,"data_stored":2637494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.868067"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538452+0000","last_change":"2024-05-14T11:17:14.533328+0000","last_active":"2024-05-14T11:17:14.538452+0000","last_peered":"2024-05-14T11:17:14.538452+0000","last_clean":"2024-05-14T11:17:14.538452+0000","last_became_active":"2024-05-14T11:17:14.533018+0000","last_became_peered":"2024-05-14T11:17:14.533018+0000","last_unstale":"2024-05-14T11:17:14.538452+0000","last_undegraded":"2024-05-14T11:17:14.538452+0000","last_fullsized":"2024-05-14T11:17:14.538452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:31.712327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544595+0000","last_change":"2024-05-14T11:17:14.538491+0000","last_active":"2024-05-14T11:17:14.544595+0000","last_peered":"2024-05-14T11:17:14.544595+0000","last_clean":"2024-05-14T11:17:14.544595+0000","last_became_active":"2024-05-14T11:17:14.536144+0000","last_became_peered":"2024-05-14T11:17:14.536144+0000","last_unstale":"2024-05-14T11:17:14.544595+0000","last_undegraded":"2024-05-14T11:17:14.544595+0000","last_fullsized":"2024-05-14T11:17:14.544595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:48.197288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550936+0000","last_change":"2024-05-14T11:17:14.545323+0000","last_active":"2024-05-14T11:17:14.550936+0000","last_peered":"2024-05-14T11:17:14.550936+0000","last_clean":"2024-05-14T11:17:14.550936+0000","last_became_active":"2024-05-14T11:17:14.544962+0000","last_became_peered":"2024-05-14T11:17:14.544962+0000","last_unstale":"2024-05-14T11:17:14.550936+0000","last_undegraded":"2024-05-14T11:17:14.550936+0000","last_fullsized":"2024-05-14T11:17:14.550936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534402+0000","last_change":"2024-05-14T11:17:14.528673+0000","last_active":"2024-05-14T11:17:14.534402+0000","last_peered":"2024-05-14T11:17:14.534402+0000","last_clean":"2024-05-14T11:17:14.534402+0000","last_became_active":"2024-05-14T11:17:14.528327+0000","last_became_peered":"2024-05-14T11:17:14.528327+0000","last_unstale":"2024-05-14T11:17:14.534402+0000","last_undegraded":"2024-05-14T11:17:14.534402+0000","last_fullsized":"2024-05-14T11:17:14.534402+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:56:33.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.540741+0000","last_change":"2024-05-14T11:17:14.530152+0000","last_active":"2024-05-14T11:17:14.540741+0000","last_peered":"2024-05-14T11:17:14.540741+0000","last_clean":"2024-05-14T11:17:14.540741+0000","last_became_active":"2024-05-14T11:17:14.529820+0000","last_became_peered":"2024-05-14T11:17:14.529820+0000","last_unstale":"2024-05-14T11:17:14.540741+0000","last_undegraded":"2024-05-14T11:17:14.540741+0000","last_fullsized":"2024-05-14T11:17:14.540741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:21.545048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559365+0000","last_change":"2024-05-14T11:17:14.552988+0000","last_active":"2024-05-14T11:17:14.559365+0000","last_peered":"2024-05-14T11:17:14.559365+0000","last_clean":"2024-05-14T11:17:14.559365+0000","last_became_active":"2024-05-14T11:17:14.552642+0000","last_became_peered":"2024-05-14T11:17:14.552642+0000","last_unstale":"2024-05-14T11:17:14.559365+0000","last_undegraded":"2024-05-14T11:17:14.559365+0000","last_fullsized":"2024-05-14T11:17:14.559365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550363+0000","last_change":"2024-05-14T11:17:14.546100+0000","last_active":"2024-05-14T11:17:14.550363+0000","last_peered":"2024-05-14T11:17:14.550363+0000","last_clean":"2024-05-14T11:17:14.550363+0000","last_became_active":"2024-05-14T11:17:14.545657+0000","last_became_peered":"2024-05-14T11:17:14.545657+0000","last_unstale":"2024-05-14T11:17:14.550363+0000","last_undegraded":"2024-05-14T11:17:14.550363+0000","last_fullsized":"2024-05-14T11:17:14.550363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553749+0000","last_change":"2024-05-14T11:17:14.548840+0000","last_active":"2024-05-14T11:17:14.553749+0000","last_peered":"2024-05-14T11:17:14.553749+0000","last_clean":"2024-05-14T11:17:14.553749+0000","last_became_active":"2024-05-14T11:17:14.548527+0000","last_became_peered":"2024-05-14T11:17:14.548527+0000","last_unstale":"2024-05-14T11:17:14.553749+0000","last_undegraded":"2024-05-14T11:17:14.553749+0000","last_fullsized":"2024-05-14T11:17:14.553749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:54:47.825110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548985+0000","last_change":"2024-05-14T11:17:14.545053+0000","last_active":"2024-05-14T11:17:14.548985+0000","last_peered":"2024-05-14T11:17:14.548985+0000","last_clean":"2024-05-14T11:17:14.548985+0000","last_became_active":"2024-05-14T11:17:14.544737+0000","last_became_peered":"2024-05-14T11:17:14.544737+0000","last_unstale":"2024-05-14T11:17:14.548985+0000","last_undegraded":"2024-05-14T11:17:14.548985+0000","last_fullsized":"2024-05-14T11:17:14.548985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561126+0000","last_change":"2024-05-14T11:17:14.548127+0000","last_active":"2024-05-14T11:17:14.561126+0000","last_peered":"2024-05-14T11:17:14.561126+0000","last_clean":"2024-05-14T11:17:14.561126+0000","last_became_active":"2024-05-14T11:17:14.547849+0000","last_became_peered":"2024-05-14T11:17:14.547849+0000","last_unstale":"2024-05-14T11:17:14.561126+0000","last_undegraded":"2024-05-14T11:17:14.561126+0000","last_fullsized":"2024-05-14T11:17:14.561126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:32.851299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550157+0000","last_change":"2024-05-14T11:17:14.543642+0000","last_active":"2024-05-14T11:17:14.550157+0000","last_peered":"2024-05-14T11:17:14.550157+0000","last_clean":"2024-05-14T11:17:14.550157+0000","last_became_active":"2024-05-14T11:17:14.543324+0000","last_became_peered":"2024-05-14T11:17:14.543324+0000","last_unstale":"2024-05-14T11:17:14.550157+0000","last_undegraded":"2024-05-14T11:17:14.550157+0000","last_fullsized":"2024-05-14T11:17:14.550157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550096+0000","last_change":"2024-05-14T11:17:14.543142+0000","last_active":"2024-05-14T11:17:14.550096+0000","last_peered":"2024-05-14T11:17:14.550096+0000","last_clean":"2024-05-14T11:17:14.550096+0000","last_became_active":"2024-05-14T11:17:14.542821+0000","last_became_peered":"2024-05-14T11:17:14.542821+0000","last_unstale":"2024-05-14T11:17:14.550096+0000","last_undegraded":"2024-05-14T11:17:14.550096+0000","last_fullsized":"2024-05-14T11:17:14.550096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561279+0000","last_change":"2024-05-14T11:17:14.550510+0000","last_active":"2024-05-14T11:17:14.561279+0000","last_peered":"2024-05-14T11:17:14.561279+0000","last_clean":"2024-05-14T11:17:14.561279+0000","last_became_active":"2024-05-14T11:17:14.550215+0000","last_became_peered":"2024-05-14T11:17:14.550215+0000","last_unstale":"2024-05-14T11:17:14.561279+0000","last_undegraded":"2024-05-14T11:17:14.561279+0000","last_fullsized":"2024-05-14T11:17:14.561279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538327+0000","last_change":"2024-05-14T11:17:14.528562+0000","last_active":"2024-05-14T11:17:14.538327+0000","last_peered":"2024-05-14T11:17:14.538327+0000","last_clean":"2024-05-14T11:17:14.538327+0000","last_became_active":"2024-05-14T11:17:14.528209+0000","last_became_peered":"2024-05-14T11:17:14.528209+0000","last_unstale":"2024-05-14T11:17:14.538327+0000","last_undegraded":"2024-05-14T11:17:14.538327+0000","last_fullsized":"2024-05-14T11:17:14.538327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550437+0000","last_change":"2024-05-14T11:17:14.546112+0000","last_active":"2024-05-14T11:17:14.550437+0000","last_peered":"2024-05-14T11:17:14.550437+0000","last_clean":"2024-05-14T11:17:14.550437+0000","last_became_active":"2024-05-14T11:17:14.545612+0000","last_became_peered":"2024-05-14T11:17:14.545612+0000","last_unstale":"2024-05-14T11:17:14.550437+0000","last_undegraded":"2024-05-14T11:17:14.550437+0000","last_fullsized":"2024-05-14T11:17:14.550437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:42.117037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553371+0000","last_change":"2024-05-14T11:17:14.550990+0000","last_active":"2024-05-14T11:17:14.553371+0000","last_peered":"2024-05-14T11:17:14.553371+0000","last_clean":"2024-05-14T11:17:14.553371+0000","last_became_active":"2024-05-14T11:17:14.550709+0000","last_became_peered":"2024-05-14T11:17:14.550709+0000","last_unstale":"2024-05-14T11:17:14.553371+0000","last_undegraded":"2024-05-14T11:17:14.553371+0000","last_fullsized":"2024-05-14T11:17:14.553371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:32.851299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562285+0000","last_change":"2024-05-14T11:17:14.559223+0000","last_active":"2024-05-14T11:17:14.562285+0000","last_peered":"2024-05-14T11:17:14.562285+0000","last_clean":"2024-05-14T11:17:14.562285+0000","last_became_active":"2024-05-14T11:17:14.558733+0000","last_became_peered":"2024-05-14T11:17:14.558733+0000","last_unstale":"2024-05-14T11:17:14.562285+0000","last_undegraded":"2024-05-14T11:17:14.562285+0000","last_fullsized":"2024-05-14T11:17:14.562285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.540271+0000","last_change":"2024-05-14T11:17:14.536802+0000","last_active":"2024-05-14T11:17:14.540271+0000","last_peered":"2024-05-14T11:17:14.540271+0000","last_clean":"2024-05-14T11:17:14.540271+0000","last_became_active":"2024-05-14T11:17:14.536469+0000","last_became_peered":"2024-05-14T11:17:14.536469+0000","last_unstale":"2024-05-14T11:17:14.540271+0000","last_undegraded":"2024-05-14T11:17:14.540271+0000","last_fullsized":"2024-05-14T11:17:14.540271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.546175+0000","last_change":"2024-05-14T11:17:14.540028+0000","last_active":"2024-05-14T11:17:14.546175+0000","last_peered":"2024-05-14T11:17:14.546175+0000","last_clean":"2024-05-14T11:17:14.546175+0000","last_became_active":"2024-05-14T11:17:14.539559+0000","last_became_peered":"2024-05-14T11:17:14.539559+0000","last_unstale":"2024-05-14T11:17:14.546175+0000","last_undegraded":"2024-05-14T11:17:14.546175+0000","last_fullsized":"2024-05-14T11:17:14.546175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:33:01.436450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537726+0000","last_change":"2024-05-14T11:17:14.528785+0000","last_active":"2024-05-14T11:17:14.537726+0000","last_peered":"2024-05-14T11:17:14.537726+0000","last_clean":"2024-05-14T11:17:14.537726+0000","last_became_active":"2024-05-14T11:17:14.528464+0000","last_became_peered":"2024-05-14T11:17:14.528464+0000","last_unstale":"2024-05-14T11:17:14.537726+0000","last_undegraded":"2024-05-14T11:17:14.537726+0000","last_fullsized":"2024-05-14T11:17:14.537726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:41.135647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559067+0000","last_change":"2024-05-14T11:17:14.546816+0000","last_active":"2024-05-14T11:17:14.559067+0000","last_peered":"2024-05-14T11:17:14.559067+0000","last_clean":"2024-05-14T11:17:14.559067+0000","last_became_active":"2024-05-14T11:17:14.546540+0000","last_became_peered":"2024-05-14T11:17:14.546540+0000","last_unstale":"2024-05-14T11:17:14.559067+0000","last_undegraded":"2024-05-14T11:17:14.559067+0000","last_fullsized":"2024-05-14T11:17:14.559067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:02.984707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.541539+0000","last_change":"2024-05-14T11:17:14.535434+0000","last_active":"2024-05-14T11:17:14.541539+0000","last_peered":"2024-05-14T11:17:14.541539+0000","last_clean":"2024-05-14T11:17:14.541539+0000","last_became_active":"2024-05-14T11:17:14.535127+0000","last_became_peered":"2024-05-14T11:17:14.535127+0000","last_unstale":"2024-05-14T11:17:14.541539+0000","last_undegraded":"2024-05-14T11:17:14.541539+0000","last_fullsized":"2024-05-14T11:17:14.541539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550838+0000","last_change":"2024-05-14T11:17:14.547635+0000","last_active":"2024-05-14T11:17:14.550838+0000","last_peered":"2024-05-14T11:17:14.550838+0000","last_clean":"2024-05-14T11:17:14.550838+0000","last_became_active":"2024-05-14T11:17:14.545919+0000","last_became_peered":"2024-05-14T11:17:14.545919+0000","last_unstale":"2024-05-14T11:17:14.550838+0000","last_undegraded":"2024-05-14T11:17:14.550838+0000","last_fullsized":"2024-05-14T11:17:14.550838+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:42.117037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.551991+0000","last_change":"2024-05-14T11:17:14.546409+0000","last_active":"2024-05-14T11:17:14.551991+0000","last_peered":"2024-05-14T11:17:14.551991+0000","last_clean":"2024-05-14T11:17:14.551991+0000","last_became_active":"2024-05-14T11:17:14.546125+0000","last_became_peered":"2024-05-14T11:17:14.546125+0000","last_unstale":"2024-05-14T11:17:14.551991+0000","last_undegraded":"2024-05-14T11:17:14.551991+0000","last_fullsized":"2024-05-14T11:17:14.551991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:30:45.239492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537480+0000","last_change":"2024-05-14T11:17:14.531524+0000","last_active":"2024-05-14T11:17:14.537480+0000","last_peered":"2024-05-14T11:17:14.537480+0000","last_clean":"2024-05-14T11:17:14.537480+0000","last_became_active":"2024-05-14T11:17:14.530905+0000","last_became_peered":"2024-05-14T11:17:14.530905+0000","last_unstale":"2024-05-14T11:17:14.537480+0000","last_undegraded":"2024-05-14T11:17:14.537480+0000","last_fullsized":"2024-05-14T11:17:14.537480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:50.227093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562143+0000","last_change":"2024-05-14T11:17:14.558633+0000","last_active":"2024-05-14T11:17:14.562143+0000","last_peered":"2024-05-14T11:17:14.562143+0000","last_clean":"2024-05-14T11:17:14.562143+0000","last_became_active":"2024-05-14T11:17:14.558199+0000","last_became_peered":"2024-05-14T11:17:14.558199+0000","last_unstale":"2024-05-14T11:17:14.562143+0000","last_undegraded":"2024-05-14T11:17:14.562143+0000","last_fullsized":"2024-05-14T11:17:14.562143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544886+0000","last_change":"2024-05-14T11:17:14.540081+0000","last_active":"2024-05-14T11:17:14.544886+0000","last_peered":"2024-05-14T11:17:14.544886+0000","last_clean":"2024-05-14T11:17:14.544886+0000","last_became_active":"2024-05-14T11:17:14.538679+0000","last_became_peered":"2024-05-14T11:17:14.538679+0000","last_unstale":"2024-05-14T11:17:14.544886+0000","last_undegraded":"2024-05-14T11:17:14.544886+0000","last_fullsized":"2024-05-14T11:17:14.544886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562072+0000","last_change":"2024-05-14T11:17:14.557901+0000","last_active":"2024-05-14T11:17:14.562072+0000","last_peered":"2024-05-14T11:17:14.562072+0000","last_clean":"2024-05-14T11:17:14.562072+0000","last_became_active":"2024-05-14T11:17:14.557673+0000","last_became_peered":"2024-05-14T11:17:14.557673+0000","last_unstale":"2024-05-14T11:17:14.562072+0000","last_undegraded":"2024-05-14T11:17:14.562072+0000","last_fullsized":"2024-05-14T11:17:14.562072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:31.712327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548090+0000","last_change":"2024-05-14T11:17:14.542679+0000","last_active":"2024-05-14T11:17:14.548090+0000","last_peered":"2024-05-14T11:17:14.548090+0000","last_clean":"2024-05-14T11:17:14.548090+0000","last_became_active":"2024-05-14T11:17:14.542415+0000","last_became_peered":"2024-05-14T11:17:14.542415+0000","last_unstale":"2024-05-14T11:17:14.548090+0000","last_undegraded":"2024-05-14T11:17:14.548090+0000","last_fullsized":"2024-05-14T11:17:14.548090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538962+0000","last_change":"2024-05-14T11:17:14.533318+0000","last_active":"2024-05-14T11:17:14.538962+0000","last_peered":"2024-05-14T11:17:14.538962+0000","last_clean":"2024-05-14T11:17:14.538962+0000","last_became_active":"2024-05-14T11:17:14.532977+0000","last_became_peered":"2024-05-14T11:17:14.532977+0000","last_unstale":"2024-05-14T11:17:14.538962+0000","last_undegraded":"2024-05-14T11:17:14.538962+0000","last_fullsized":"2024-05-14T11:17:14.538962+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:14.407202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550604+0000","last_change":"2024-05-14T11:17:14.546896+0000","last_active":"2024-05-14T11:17:14.550604+0000","last_peered":"2024-05-14T11:17:14.550604+0000","last_clean":"2024-05-14T11:17:14.550604+0000","last_became_active":"2024-05-14T11:17:14.546283+0000","last_became_peered":"2024-05-14T11:17:14.546283+0000","last_unstale":"2024-05-14T11:17:14.550604+0000","last_undegraded":"2024-05-14T11:17:14.550604+0000","last_fullsized":"2024-05-14T11:17:14.550604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.549530+0000","last_change":"2024-05-14T11:17:14.543663+0000","last_active":"2024-05-14T11:17:14.549530+0000","last_peered":"2024-05-14T11:17:14.549530+0000","last_clean":"2024-05-14T11:17:14.549530+0000","last_became_active":"2024-05-14T11:17:14.543375+0000","last_became_peered":"2024-05-14T11:17:14.543375+0000","last_unstale":"2024-05-14T11:17:14.549530+0000","last_undegraded":"2024-05-14T11:17:14.549530+0000","last_fullsized":"2024-05-14T11:17:14.549530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553473+0000","last_change":"2024-05-14T11:17:14.551357+0000","last_active":"2024-05-14T11:17:14.553473+0000","last_peered":"2024-05-14T11:17:14.553473+0000","last_clean":"2024-05-14T11:17:14.553473+0000","last_became_active":"2024-05-14T11:17:14.551080+0000","last_became_peered":"2024-05-14T11:17:14.551080+0000","last_unstale":"2024-05-14T11:17:14.553473+0000","last_undegraded":"2024-05-14T11:17:14.553473+0000","last_fullsized":"2024-05-14T11:17:14.553473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560876+0000","last_change":"2024-05-14T11:17:13.528783+0000","last_active":"2024-05-14T11:17:14.560876+0000","last_peered":"2024-05-14T11:17:14.560876+0000","last_clean":"2024-05-14T11:17:14.560876+0000","last_became_active":"2024-05-14T11:17:13.527734+0000","last_became_peered":"2024-05-14T11:17:13.527734+0000","last_unstale":"2024-05-14T11:17:14.560876+0000","last_undegraded":"2024-05-14T11:17:14.560876+0000","last_fullsized":"2024-05-14T11:17:14.560876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:47.194388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559346+0000","last_change":"2024-05-14T11:17:14.553459+0000","last_active":"2024-05-14T11:17:14.559346+0000","last_peered":"2024-05-14T11:17:14.559346+0000","last_clean":"2024-05-14T11:17:14.559346+0000","last_became_active":"2024-05-14T11:17:14.553179+0000","last_became_peered":"2024-05-14T11:17:14.553179+0000","last_unstale":"2024-05-14T11:17:14.559346+0000","last_undegraded":"2024-05-14T11:17:14.559346+0000","last_fullsized":"2024-05-14T11:17:14.559346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561365+0000","last_change":"2024-05-14T11:17:13.532401+0000","last_active":"2024-05-14T11:17:14.561365+0000","last_peered":"2024-05-14T11:17:14.561365+0000","last_clean":"2024-05-14T11:17:14.561365+0000","last_became_active":"2024-05-14T11:17:13.532080+0000","last_became_peered":"2024-05-14T11:17:13.532080+0000","last_unstale":"2024-05-14T11:17:14.561365+0000","last_undegraded":"2024-05-14T11:17:14.561365+0000","last_fullsized":"2024-05-14T11:17:14.561365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539944+0000","last_change":"2024-05-14T11:17:14.529094+0000","last_active":"2024-05-14T11:17:14.539944+0000","last_peered":"2024-05-14T11:17:14.539944+0000","last_clean":"2024-05-14T11:17:14.539944+0000","last_became_active":"2024-05-14T11:17:14.528783+0000","last_became_peered":"2024-05-14T11:17:14.528783+0000","last_unstale":"2024-05-14T11:17:14.539944+0000","last_undegraded":"2024-05-14T11:17:14.539944+0000","last_fullsized":"2024-05-14T11:17:14.539944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.556635+0000","last_change":"2024-05-14T11:17:13.527879+0000","last_active":"2024-05-14T11:17:14.556635+0000","last_peered":"2024-05-14T11:17:14.556635+0000","last_clean":"2024-05-14T11:17:14.556635+0000","last_became_active":"2024-05-14T11:17:13.527509+0000","last_became_peered":"2024-05-14T11:17:13.527509+0000","last_unstale":"2024-05-14T11:17:14.556635+0000","last_undegraded":"2024-05-14T11:17:14.556635+0000","last_fullsized":"2024-05-14T11:17:14.556635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544592+0000","last_change":"2024-05-14T11:17:14.538586+0000","last_active":"2024-05-14T11:17:14.544592+0000","last_peered":"2024-05-14T11:17:14.544592+0000","last_clean":"2024-05-14T11:17:14.544592+0000","last_became_active":"2024-05-14T11:17:14.538187+0000","last_became_peered":"2024-05-14T11:17:14.538187+0000","last_unstale":"2024-05-14T11:17:14.544592+0000","last_undegraded":"2024-05-14T11:17:14.544592+0000","last_fullsized":"2024-05-14T11:17:14.544592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:47:39.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560654+0000","last_change":"2024-05-14T11:17:13.537106+0000","last_active":"2024-05-14T11:17:14.560654+0000","last_peered":"2024-05-14T11:17:14.560654+0000","last_clean":"2024-05-14T11:17:14.560654+0000","last_became_active":"2024-05-14T11:17:13.536580+0000","last_became_peered":"2024-05-14T11:17:13.536580+0000","last_unstale":"2024-05-14T11:17:14.560654+0000","last_undegraded":"2024-05-14T11:17:14.560654+0000","last_fullsized":"2024-05-14T11:17:14.560654+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545586+0000","last_change":"2024-05-14T11:17:14.541427+0000","last_active":"2024-05-14T11:17:14.545586+0000","last_peered":"2024-05-14T11:17:14.545586+0000","last_clean":"2024-05-14T11:17:14.545586+0000","last_became_active":"2024-05-14T11:17:14.541134+0000","last_became_peered":"2024-05-14T11:17:14.541134+0000","last_unstale":"2024-05-14T11:17:14.545586+0000","last_undegraded":"2024-05-14T11:17:14.545586+0000","last_fullsized":"2024-05-14T11:17:14.545586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562894+0000","last_change":"2024-05-14T11:17:13.535608+0000","last_active":"2024-05-14T11:17:14.562894+0000","last_peered":"2024-05-14T11:17:14.562894+0000","last_clean":"2024-05-14T11:17:14.562894+0000","last_became_active":"2024-05-14T11:17:13.535056+0000","last_became_peered":"2024-05-14T11:17:13.535056+0000","last_unstale":"2024-05-14T11:17:14.562894+0000","last_undegraded":"2024-05-14T11:17:14.562894+0000","last_fullsized":"2024-05-14T11:17:14.562894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:47.296773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.558364+0000","last_change":"2024-05-14T11:17:14.547072+0000","last_active":"2024-05-14T11:17:14.558364+0000","last_peered":"2024-05-14T11:17:14.558364+0000","last_clean":"2024-05-14T11:17:14.558364+0000","last_became_active":"2024-05-14T11:17:14.546550+0000","last_became_peered":"2024-05-14T11:17:14.546550+0000","last_unstale":"2024-05-14T11:17:14.558364+0000","last_undegraded":"2024-05-14T11:17:14.558364+0000","last_fullsized":"2024-05-14T11:17:14.558364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:20.068069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560861+0000","last_change":"2024-05-14T11:17:13.529732+0000","last_active":"2024-05-14T11:17:14.560861+0000","last_peered":"2024-05-14T11:17:14.560861+0000","last_clean":"2024-05-14T11:17:14.560861+0000","last_became_active":"2024-05-14T11:17:13.529425+0000","last_became_peered":"2024-05-14T11:17:13.529425+0000","last_unstale":"2024-05-14T11:17:14.560861+0000","last_undegraded":"2024-05-14T11:17:14.560861+0000","last_fullsized":"2024-05-14T11:17:14.560861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:24.343277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545788+0000","last_change":"2024-05-14T11:17:14.544342+0000","last_active":"2024-05-14T11:17:14.545788+0000","last_peered":"2024-05-14T11:17:14.545788+0000","last_clean":"2024-05-14T11:17:14.545788+0000","last_became_active":"2024-05-14T11:17:14.543072+0000","last_became_peered":"2024-05-14T11:17:14.543072+0000","last_unstale":"2024-05-14T11:17:14.545788+0000","last_undegraded":"2024-05-14T11:17:14.545788+0000","last_fullsized":"2024-05-14T11:17:14.545788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.527364+0000","last_change":"2024-05-14T11:17:13.530126+0000","last_active":"2024-05-14T11:17:14.527364+0000","last_peered":"2024-05-14T11:17:14.527364+0000","last_clean":"2024-05-14T11:17:14.527364+0000","last_became_active":"2024-05-14T11:17:13.529703+0000","last_became_peered":"2024-05-14T11:17:13.529703+0000","last_unstale":"2024-05-14T11:17:14.527364+0000","last_undegraded":"2024-05-14T11:17:14.527364+0000","last_fullsized":"2024-05-14T11:17:14.527364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:38.631345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544570+0000","last_change":"2024-05-14T11:17:14.537269+0000","last_active":"2024-05-14T11:17:14.544570+0000","last_peered":"2024-05-14T11:17:14.544570+0000","last_clean":"2024-05-14T11:17:14.544570+0000","last_became_active":"2024-05-14T11:17:14.536736+0000","last_became_peered":"2024-05-14T11:17:14.536736+0000","last_unstale":"2024-05-14T11:17:14.544570+0000","last_undegraded":"2024-05-14T11:17:14.544570+0000","last_fullsized":"2024-05-14T11:17:14.544570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:39.780118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525622+0000","last_change":"2024-05-14T11:17:13.528654+0000","last_active":"2024-05-14T11:17:14.525622+0000","last_peered":"2024-05-14T11:17:14.525622+0000","last_clean":"2024-05-14T11:17:14.525622+0000","last_became_active":"2024-05-14T11:17:13.528351+0000","last_became_peered":"2024-05-14T11:17:13.528351+0000","last_unstale":"2024-05-14T11:17:14.525622+0000","last_undegraded":"2024-05-14T11:17:14.525622+0000","last_fullsized":"2024-05-14T11:17:14.525622+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534373+0000","last_change":"2024-05-14T11:17:14.527495+0000","last_active":"2024-05-14T11:17:14.534373+0000","last_peered":"2024-05-14T11:17:14.534373+0000","last_clean":"2024-05-14T11:17:14.534373+0000","last_became_active":"2024-05-14T11:17:14.527118+0000","last_became_peered":"2024-05-14T11:17:14.527118+0000","last_unstale":"2024-05-14T11:17:14.534373+0000","last_undegraded":"2024-05-14T11:17:14.534373+0000","last_fullsized":"2024-05-14T11:17:14.534373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:00:07.518389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.528034+0000","last_change":"2024-05-14T11:17:13.531054+0000","last_active":"2024-05-14T11:17:14.528034+0000","last_peered":"2024-05-14T11:17:14.528034+0000","last_clean":"2024-05-14T11:17:14.528034+0000","last_became_active":"2024-05-14T11:17:13.530552+0000","last_became_peered":"2024-05-14T11:17:13.530552+0000","last_unstale":"2024-05-14T11:17:14.528034+0000","last_undegraded":"2024-05-14T11:17:14.528034+0000","last_fullsized":"2024-05-14T11:17:14.528034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.842738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.552793+0000","last_change":"2024-05-14T11:17:14.549782+0000","last_active":"2024-05-14T11:17:14.552793+0000","last_peered":"2024-05-14T11:17:14.552793+0000","last_clean":"2024-05-14T11:17:14.552793+0000","last_became_active":"2024-05-14T11:17:14.549143+0000","last_became_peered":"2024-05-14T11:17:14.549143+0000","last_unstale":"2024-05-14T11:17:14.552793+0000","last_undegraded":"2024-05-14T11:17:14.552793+0000","last_fullsized":"2024-05-14T11:17:14.552793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:02.984707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.542134+0000","last_change":"2024-05-14T11:17:13.535408+0000","last_active":"2024-05-14T11:17:14.542134+0000","last_peered":"2024-05-14T11:17:14.542134+0000","last_clean":"2024-05-14T11:17:14.542134+0000","last_became_active":"2024-05-14T11:17:13.535085+0000","last_became_peered":"2024-05-14T11:17:13.535085+0000","last_unstale":"2024-05-14T11:17:14.542134+0000","last_undegraded":"2024-05-14T11:17:14.542134+0000","last_fullsized":"2024-05-14T11:17:14.542134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:01.121816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545516+0000","last_change":"2024-05-14T11:17:14.540451+0000","last_active":"2024-05-14T11:17:14.545516+0000","last_peered":"2024-05-14T11:17:14.545516+0000","last_clean":"2024-05-14T11:17:14.545516+0000","last_became_active":"2024-05-14T11:17:14.540108+0000","last_became_peered":"2024-05-14T11:17:14.540108+0000","last_unstale":"2024-05-14T11:17:14.545516+0000","last_undegraded":"2024-05-14T11:17:14.545516+0000","last_fullsized":"2024-05-14T11:17:14.545516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564010+0000","last_change":"2024-05-14T11:17:13.531691+0000","last_active":"2024-05-14T11:17:14.564010+0000","last_peered":"2024-05-14T11:17:14.564010+0000","last_clean":"2024-05-14T11:17:14.564010+0000","last_became_active":"2024-05-14T11:17:13.531378+0000","last_became_peered":"2024-05-14T11:17:13.531378+0000","last_unstale":"2024-05-14T11:17:14.564010+0000","last_undegraded":"2024-05-14T11:17:14.564010+0000","last_fullsized":"2024-05-14T11:17:14.564010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:24.343277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.541671+0000","last_change":"2024-05-14T11:17:14.535513+0000","last_active":"2024-05-14T11:17:14.541671+0000","last_peered":"2024-05-14T11:17:14.541671+0000","last_clean":"2024-05-14T11:17:14.541671+0000","last_became_active":"2024-05-14T11:17:14.535161+0000","last_became_peered":"2024-05-14T11:17:14.535161+0000","last_unstale":"2024-05-14T11:17:14.541671+0000","last_undegraded":"2024-05-14T11:17:14.541671+0000","last_fullsized":"2024-05-14T11:17:14.541671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.567097+0000","last_change":"2024-05-14T11:17:13.535648+0000","last_active":"2024-05-14T11:17:14.567097+0000","last_peered":"2024-05-14T11:17:14.567097+0000","last_clean":"2024-05-14T11:17:14.567097+0000","last_became_active":"2024-05-14T11:17:13.535270+0000","last_became_peered":"2024-05-14T11:17:13.535270+0000","last_unstale":"2024-05-14T11:17:14.567097+0000","last_undegraded":"2024-05-14T11:17:14.567097+0000","last_fullsized":"2024-05-14T11:17:14.567097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:19.059507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.556265+0000","last_change":"2024-05-14T11:17:14.552298+0000","last_active":"2024-05-14T11:17:14.556265+0000","last_peered":"2024-05-14T11:17:14.556265+0000","last_clean":"2024-05-14T11:17:14.556265+0000","last_became_active":"2024-05-14T11:17:14.551997+0000","last_became_peered":"2024-05-14T11:17:14.551997+0000","last_unstale":"2024-05-14T11:17:14.556265+0000","last_undegraded":"2024-05-14T11:17:14.556265+0000","last_fullsized":"2024-05-14T11:17:14.556265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.563709+0000","last_change":"2024-05-14T11:17:13.530162+0000","last_active":"2024-05-14T11:17:14.563709+0000","last_peered":"2024-05-14T11:17:14.563709+0000","last_clean":"2024-05-14T11:17:14.563709+0000","last_became_active":"2024-05-14T11:17:13.529873+0000","last_became_peered":"2024-05-14T11:17:13.529873+0000","last_unstale":"2024-05-14T11:17:14.563709+0000","last_undegraded":"2024-05-14T11:17:14.563709+0000","last_fullsized":"2024-05-14T11:17:14.563709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:01.976145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538125+0000","last_change":"2024-05-14T11:17:14.532437+0000","last_active":"2024-05-14T11:17:14.538125+0000","last_peered":"2024-05-14T11:17:14.538125+0000","last_clean":"2024-05-14T11:17:14.538125+0000","last_became_active":"2024-05-14T11:17:14.532094+0000","last_became_peered":"2024-05-14T11:17:14.532094+0000","last_unstale":"2024-05-14T11:17:14.538125+0000","last_undegraded":"2024-05-14T11:17:14.538125+0000","last_fullsized":"2024-05-14T11:17:14.538125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.526525+0000","last_change":"2024-05-14T11:17:13.536727+0000","last_active":"2024-05-14T11:17:14.526525+0000","last_peered":"2024-05-14T11:17:14.526525+0000","last_clean":"2024-05-14T11:17:14.526525+0000","last_became_active":"2024-05-14T11:17:13.536390+0000","last_became_peered":"2024-05-14T11:17:13.536390+0000","last_unstale":"2024-05-14T11:17:14.526525+0000","last_undegraded":"2024-05-14T11:17:14.526525+0000","last_fullsized":"2024-05-14T11:17:14.526525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548167+0000","last_change":"2024-05-14T11:17:14.545128+0000","last_active":"2024-05-14T11:17:14.548167+0000","last_peered":"2024-05-14T11:17:14.548167+0000","last_clean":"2024-05-14T11:17:14.548167+0000","last_became_active":"2024-05-14T11:17:14.544835+0000","last_became_peered":"2024-05-14T11:17:14.544835+0000","last_unstale":"2024-05-14T11:17:14.548167+0000","last_undegraded":"2024-05-14T11:17:14.548167+0000","last_fullsized":"2024-05-14T11:17:14.548167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.523125+0000","last_change":"2024-05-14T11:17:13.529090+0000","last_active":"2024-05-14T11:17:14.523125+0000","last_peered":"2024-05-14T11:17:14.523125+0000","last_clean":"2024-05-14T11:17:14.523125+0000","last_became_active":"2024-05-14T11:17:13.528250+0000","last_became_peered":"2024-05-14T11:17:13.528250+0000","last_unstale":"2024-05-14T11:17:14.523125+0000","last_undegraded":"2024-05-14T11:17:14.523125+0000","last_fullsized":"2024-05-14T11:17:14.523125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:16.011697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539369+0000","last_change":"2024-05-14T11:17:14.533854+0000","last_active":"2024-05-14T11:17:14.539369+0000","last_peered":"2024-05-14T11:17:14.539369+0000","last_clean":"2024-05-14T11:17:14.539369+0000","last_became_active":"2024-05-14T11:17:14.533561+0000","last_became_peered":"2024-05-14T11:17:14.533561+0000","last_unstale":"2024-05-14T11:17:14.539369+0000","last_undegraded":"2024-05-14T11:17:14.539369+0000","last_fullsized":"2024-05-14T11:17:14.539369+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:07.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562624+0000","last_change":"2024-05-14T11:17:13.533231+0000","last_active":"2024-05-14T11:17:14.562624+0000","last_peered":"2024-05-14T11:17:14.562624+0000","last_clean":"2024-05-14T11:17:14.562624+0000","last_became_active":"2024-05-14T11:17:13.530949+0000","last_became_peered":"2024-05-14T11:17:13.530949+0000","last_unstale":"2024-05-14T11:17:14.562624+0000","last_undegraded":"2024-05-14T11:17:14.562624+0000","last_fullsized":"2024-05-14T11:17:14.562624+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:38.631345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544622+0000","last_change":"2024-05-14T11:17:14.534865+0000","last_active":"2024-05-14T11:17:14.544622+0000","last_peered":"2024-05-14T11:17:14.544622+0000","last_clean":"2024-05-14T11:17:14.544622+0000","last_became_active":"2024-05-14T11:17:14.534438+0000","last_became_peered":"2024-05-14T11:17:14.534438+0000","last_unstale":"2024-05-14T11:17:14.544622+0000","last_undegraded":"2024-05-14T11:17:14.544622+0000","last_fullsized":"2024-05-14T11:17:14.544622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:56:54.178936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564717+0000","last_change":"2024-05-14T11:17:13.533122+0000","last_active":"2024-05-14T11:17:14.564717+0000","last_peered":"2024-05-14T11:17:14.564717+0000","last_clean":"2024-05-14T11:17:14.564717+0000","last_became_active":"2024-05-14T11:17:13.532093+0000","last_became_peered":"2024-05-14T11:17:13.532093+0000","last_unstale":"2024-05-14T11:17:14.564717+0000","last_undegraded":"2024-05-14T11:17:14.564717+0000","last_fullsized":"2024-05-14T11:17:14.564717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:09.662910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562189+0000","last_change":"2024-05-14T11:17:14.558778+0000","last_active":"2024-05-14T11:17:14.562189+0000","last_peered":"2024-05-14T11:17:14.562189+0000","last_clean":"2024-05-14T11:17:14.562189+0000","last_became_active":"2024-05-14T11:17:14.558297+0000","last_became_peered":"2024-05-14T11:17:14.558297+0000","last_unstale":"2024-05-14T11:17:14.562189+0000","last_undegraded":"2024-05-14T11:17:14.562189+0000","last_fullsized":"2024-05-14T11:17:14.562189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:07.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564894+0000","last_change":"2024-05-14T11:17:13.536741+0000","last_active":"2024-05-14T11:17:14.564894+0000","last_peered":"2024-05-14T11:17:14.564894+0000","last_clean":"2024-05-14T11:17:14.564894+0000","last_became_active":"2024-05-14T11:17:13.536409+0000","last_became_peered":"2024-05-14T11:17:13.536409+0000","last_unstale":"2024-05-14T11:17:14.564894+0000","last_undegraded":"2024-05-14T11:17:14.564894+0000","last_fullsized":"2024-05-14T11:17:14.564894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545646+0000","last_change":"2024-05-14T11:17:14.543303+0000","last_active":"2024-05-14T11:17:14.545646+0000","last_peered":"2024-05-14T11:17:14.545646+0000","last_clean":"2024-05-14T11:17:14.545646+0000","last_became_active":"2024-05-14T11:17:14.541489+0000","last_became_peered":"2024-05-14T11:17:14.541489+0000","last_unstale":"2024-05-14T11:17:14.545646+0000","last_undegraded":"2024-05-14T11:17:14.545646+0000","last_fullsized":"2024-05-14T11:17:14.545646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559584+0000","last_change":"2024-05-14T11:17:13.519077+0000","last_active":"2024-05-14T11:17:14.559584+0000","last_peered":"2024-05-14T11:17:14.559584+0000","last_clean":"2024-05-14T11:17:14.559584+0000","last_became_active":"2024-05-14T11:17:13.514204+0000","last_became_peered":"2024-05-14T11:17:13.514204+0000","last_unstale":"2024-05-14T11:17:14.559584+0000","last_undegraded":"2024-05-14T11:17:14.559584+0000","last_fullsized":"2024-05-14T11:17:14.559584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:40.300485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539524+0000","last_change":"2024-05-14T11:17:14.530457+0000","last_active":"2024-05-14T11:17:14.539524+0000","last_peered":"2024-05-14T11:17:14.539524+0000","last_clean":"2024-05-14T11:17:14.539524+0000","last_became_active":"2024-05-14T11:17:14.530099+0000","last_became_peered":"2024-05-14T11:17:14.530099+0000","last_unstale":"2024-05-14T11:17:14.539524+0000","last_undegraded":"2024-05-14T11:17:14.539524+0000","last_fullsized":"2024-05-14T11:17:14.539524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:46:15.798416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.527156+0000","last_change":"2024-05-14T11:17:13.527520+0000","last_active":"2024-05-14T11:17:14.527156+0000","last_peered":"2024-05-14T11:17:14.527156+0000","last_clean":"2024-05-14T11:17:14.527156+0000","last_became_active":"2024-05-14T11:17:13.526648+0000","last_became_peered":"2024-05-14T11:17:13.526648+0000","last_unstale":"2024-05-14T11:17:14.527156+0000","last_undegraded":"2024-05-14T11:17:14.527156+0000","last_fullsized":"2024-05-14T11:17:14.527156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.546384+0000","last_change":"2024-05-14T11:17:14.543662+0000","last_active":"2024-05-14T11:17:14.546384+0000","last_peered":"2024-05-14T11:17:14.546384+0000","last_clean":"2024-05-14T11:17:14.546384+0000","last_became_active":"2024-05-14T11:17:14.543362+0000","last_became_peered":"2024-05-14T11:17:14.543362+0000","last_unstale":"2024-05-14T11:17:14.546384+0000","last_undegraded":"2024-05-14T11:17:14.546384+0000","last_fullsized":"2024-05-14T11:17:14.546384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559942+0000","last_change":"2024-05-14T11:17:13.526039+0000","last_active":"2024-05-14T11:17:14.559942+0000","last_peered":"2024-05-14T11:17:14.559942+0000","last_clean":"2024-05-14T11:17:14.559942+0000","last_became_active":"2024-05-14T11:17:13.523193+0000","last_became_peered":"2024-05-14T11:17:13.523193+0000","last_unstale":"2024-05-14T11:17:14.559942+0000","last_undegraded":"2024-05-14T11:17:14.559942+0000","last_fullsized":"2024-05-14T11:17:14.559942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:19:44.678648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.551135+0000","last_change":"2024-05-14T11:17:14.547868+0000","last_active":"2024-05-14T11:17:14.551135+0000","last_peered":"2024-05-14T11:17:14.551135+0000","last_clean":"2024-05-14T11:17:14.551135+0000","last_became_active":"2024-05-14T11:17:14.547399+0000","last_became_peered":"2024-05-14T11:17:14.547399+0000","last_unstale":"2024-05-14T11:17:14.551135+0000","last_undegraded":"2024-05-14T11:17:14.551135+0000","last_fullsized":"2024-05-14T11:17:14.551135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.519581+0000","last_change":"2024-05-14T11:17:13.527212+0000","last_active":"2024-05-14T11:17:14.519581+0000","last_peered":"2024-05-14T11:17:14.519581+0000","last_clean":"2024-05-14T11:17:14.519581+0000","last_became_active":"2024-05-14T11:17:13.526900+0000","last_became_peered":"2024-05-14T11:17:13.526900+0000","last_unstale":"2024-05-14T11:17:14.519581+0000","last_undegraded":"2024-05-14T11:17:14.519581+0000","last_fullsized":"2024-05-14T11:17:14.519581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:02:11.352543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561705+0000","last_change":"2024-05-14T11:17:14.549984+0000","last_active":"2024-05-14T11:17:14.561705+0000","last_peered":"2024-05-14T11:17:14.561705+0000","last_clean":"2024-05-14T11:17:14.561705+0000","last_became_active":"2024-05-14T11:17:14.549645+0000","last_became_peered":"2024-05-14T11:17:14.549645+0000","last_unstale":"2024-05-14T11:17:14.561705+0000","last_undegraded":"2024-05-14T11:17:14.561705+0000","last_fullsized":"2024-05-14T11:17:14.561705+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.535546+0000","last_change":"2024-05-14T11:17:13.520481+0000","last_active":"2024-05-14T11:17:14.535546+0000","last_peered":"2024-05-14T11:17:14.535546+0000","last_clean":"2024-05-14T11:17:14.535546+0000","last_became_active":"2024-05-14T11:17:13.520154+0000","last_became_peered":"2024-05-14T11:17:13.520154+0000","last_unstale":"2024-05-14T11:17:14.535546+0000","last_undegraded":"2024-05-14T11:17:14.535546+0000","last_fullsized":"2024-05-14T11:17:14.535546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:47.296773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544543+0000","last_change":"2024-05-14T11:17:14.537035+0000","last_active":"2024-05-14T11:17:14.544543+0000","last_peered":"2024-05-14T11:17:14.544543+0000","last_clean":"2024-05-14T11:17:14.544543+0000","last_became_active":"2024-05-14T11:17:14.536319+0000","last_became_peered":"2024-05-14T11:17:14.536319+0000","last_unstale":"2024-05-14T11:17:14.544543+0000","last_undegraded":"2024-05-14T11:17:14.544543+0000","last_fullsized":"2024-05-14T11:17:14.544543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:19.703304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.519655+0000","last_change":"2024-05-14T11:17:13.525724+0000","last_active":"2024-05-14T11:17:14.519655+0000","last_peered":"2024-05-14T11:17:14.519655+0000","last_clean":"2024-05-14T11:17:14.519655+0000","last_became_active":"2024-05-14T11:17:13.525417+0000","last_became_peered":"2024-05-14T11:17:13.525417+0000","last_unstale":"2024-05-14T11:17:14.519655+0000","last_undegraded":"2024-05-14T11:17:14.519655+0000","last_fullsized":"2024-05-14T11:17:14.519655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:58.483791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559316+0000","last_change":"2024-05-14T11:17:14.552971+0000","last_active":"2024-05-14T11:17:14.559316+0000","last_peered":"2024-05-14T11:17:14.559316+0000","last_clean":"2024-05-14T11:17:14.559316+0000","last_became_active":"2024-05-14T11:17:14.552604+0000","last_became_peered":"2024-05-14T11:17:14.552604+0000","last_unstale":"2024-05-14T11:17:14.559316+0000","last_undegraded":"2024-05-14T11:17:14.559316+0000","last_fullsized":"2024-05-14T11:17:14.559316+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.565391+0000","last_change":"2024-05-14T11:17:13.529960+0000","last_active":"2024-05-14T11:17:14.565391+0000","last_peered":"2024-05-14T11:17:14.565391+0000","last_clean":"2024-05-14T11:17:14.565391+0000","last_became_active":"2024-05-14T11:17:13.529643+0000","last_became_peered":"2024-05-14T11:17:13.529643+0000","last_unstale":"2024-05-14T11:17:14.565391+0000","last_undegraded":"2024-05-14T11:17:14.565391+0000","last_fullsized":"2024-05-14T11:17:14.565391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:22.007689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.552818+0000","last_change":"2024-05-14T11:17:14.549932+0000","last_active":"2024-05-14T11:17:14.552818+0000","last_peered":"2024-05-14T11:17:14.552818+0000","last_clean":"2024-05-14T11:17:14.552818+0000","last_became_active":"2024-05-14T11:17:14.549128+0000","last_became_peered":"2024-05-14T11:17:14.549128+0000","last_unstale":"2024-05-14T11:17:14.552818+0000","last_undegraded":"2024-05-14T11:17:14.552818+0000","last_fullsized":"2024-05-14T11:17:14.552818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:20.068069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.520549+0000","last_change":"2024-05-14T11:17:13.533692+0000","last_active":"2024-05-14T11:17:14.520549+0000","last_peered":"2024-05-14T11:17:14.520549+0000","last_clean":"2024-05-14T11:17:14.520549+0000","last_became_active":"2024-05-14T11:17:13.533367+0000","last_became_peered":"2024-05-14T11:17:13.533367+0000","last_unstale":"2024-05-14T11:17:14.520549+0000","last_undegraded":"2024-05-14T11:17:14.520549+0000","last_fullsized":"2024-05-14T11:17:14.520549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:56:00.482471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538642+0000","last_change":"2024-05-14T11:17:14.533285+0000","last_active":"2024-05-14T11:17:14.538642+0000","last_peered":"2024-05-14T11:17:14.538642+0000","last_clean":"2024-05-14T11:17:14.538642+0000","last_became_active":"2024-05-14T11:17:14.532919+0000","last_became_peered":"2024-05-14T11:17:14.532919+0000","last_unstale":"2024-05-14T11:17:14.538642+0000","last_undegraded":"2024-05-14T11:17:14.538642+0000","last_fullsized":"2024-05-14T11:17:14.538642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525856+0000","last_change":"2024-05-14T11:17:13.521159+0000","last_active":"2024-05-14T11:17:14.525856+0000","last_peered":"2024-05-14T11:17:14.525856+0000","last_clean":"2024-05-14T11:17:14.525856+0000","last_became_active":"2024-05-14T11:17:13.520863+0000","last_became_peered":"2024-05-14T11:17:13.520863+0000","last_unstale":"2024-05-14T11:17:14.525856+0000","last_undegraded":"2024-05-14T11:17:14.525856+0000","last_fullsized":"2024-05-14T11:17:14.525856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537164+0000","last_change":"2024-05-14T11:17:14.531820+0000","last_active":"2024-05-14T11:17:14.537164+0000","last_peered":"2024-05-14T11:17:14.537164+0000","last_clean":"2024-05-14T11:17:14.537164+0000","last_became_active":"2024-05-14T11:17:14.531479+0000","last_became_peered":"2024-05-14T11:17:14.531479+0000","last_unstale":"2024-05-14T11:17:14.537164+0000","last_undegraded":"2024-05-14T11:17:14.537164+0000","last_fullsized":"2024-05-14T11:17:14.537164+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525613+0000","last_change":"2024-05-14T11:17:13.524118+0000","last_active":"2024-05-14T11:17:14.525613+0000","last_peered":"2024-05-14T11:17:14.525613+0000","last_clean":"2024-05-14T11:17:14.525613+0000","last_became_active":"2024-05-14T11:17:13.523786+0000","last_became_peered":"2024-05-14T11:17:13.523786+0000","last_unstale":"2024-05-14T11:17:14.525613+0000","last_undegraded":"2024-05-14T11:17:14.525613+0000","last_fullsized":"2024-05-14T11:17:14.525613+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545784+0000","last_change":"2024-05-14T11:17:14.542166+0000","last_active":"2024-05-14T11:17:14.545784+0000","last_peered":"2024-05-14T11:17:14.545784+0000","last_clean":"2024-05-14T11:17:14.545784+0000","last_became_active":"2024-05-14T11:17:14.541273+0000","last_became_peered":"2024-05-14T11:17:14.541273+0000","last_unstale":"2024-05-14T11:17:14.545784+0000","last_undegraded":"2024-05-14T11:17:14.545784+0000","last_fullsized":"2024-05-14T11:17:14.545784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.524376+0000","last_change":"2024-05-14T11:17:13.529060+0000","last_active":"2024-05-14T11:17:14.524376+0000","last_peered":"2024-05-14T11:17:14.524376+0000","last_clean":"2024-05-14T11:17:14.524376+0000","last_became_active":"2024-05-14T11:17:13.524550+0000","last_became_peered":"2024-05-14T11:17:13.524550+0000","last_unstale":"2024-05-14T11:17:14.524376+0000","last_undegraded":"2024-05-14T11:17:14.524376+0000","last_fullsized":"2024-05-14T11:17:14.524376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:55.537589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539884+0000","last_change":"2024-05-14T11:17:14.530629+0000","last_active":"2024-05-14T11:17:14.539884+0000","last_peered":"2024-05-14T11:17:14.539884+0000","last_clean":"2024-05-14T11:17:14.539884+0000","last_became_active":"2024-05-14T11:17:14.530279+0000","last_became_peered":"2024-05-14T11:17:14.530279+0000","last_unstale":"2024-05-14T11:17:14.539884+0000","last_undegraded":"2024-05-14T11:17:14.539884+0000","last_fullsized":"2024-05-14T11:17:14.539884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.524574+0000","last_change":"2024-05-14T11:17:13.536338+0000","last_active":"2024-05-14T11:17:14.524574+0000","last_peered":"2024-05-14T11:17:14.524574+0000","last_clean":"2024-05-14T11:17:14.524574+0000","last_became_active":"2024-05-14T11:17:13.535745+0000","last_became_peered":"2024-05-14T11:17:13.535745+0000","last_unstale":"2024-05-14T11:17:14.524574+0000","last_undegraded":"2024-05-14T11:17:14.524574+0000","last_fullsized":"2024-05-14T11:17:14.524574+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:47.194388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.549970+0000","last_change":"2024-05-14T11:17:14.547300+0000","last_active":"2024-05-14T11:17:14.549970+0000","last_peered":"2024-05-14T11:17:14.549970+0000","last_clean":"2024-05-14T11:17:14.549970+0000","last_became_active":"2024-05-14T11:17:14.547007+0000","last_became_peered":"2024-05-14T11:17:14.547007+0000","last_unstale":"2024-05-14T11:17:14.549970+0000","last_undegraded":"2024-05-14T11:17:14.549970+0000","last_fullsized":"2024-05-14T11:17:14.549970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.523500+0000","last_change":"2024-05-14T11:17:13.531988+0000","last_active":"2024-05-14T11:17:14.523500+0000","last_peered":"2024-05-14T11:17:14.523500+0000","last_clean":"2024-05-14T11:17:14.523500+0000","last_became_active":"2024-05-14T11:17:13.531629+0000","last_became_peered":"2024-05-14T11:17:13.531629+0000","last_unstale":"2024-05-14T11:17:14.523500+0000","last_undegraded":"2024-05-14T11:17:14.523500+0000","last_fullsized":"2024-05-14T11:17:14.523500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:41.108476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564674+0000","last_change":"2024-05-14T11:17:14.563007+0000","last_active":"2024-05-14T11:17:14.564674+0000","last_peered":"2024-05-14T11:17:14.564674+0000","last_clean":"2024-05-14T11:17:14.564674+0000","last_became_active":"2024-05-14T11:17:14.562754+0000","last_became_peered":"2024-05-14T11:17:14.562754+0000","last_unstale":"2024-05-14T11:17:14.564674+0000","last_undegraded":"2024-05-14T11:17:14.564674+0000","last_fullsized":"2024-05-14T11:17:14.564674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:56:54.178936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537303+0000","last_change":"2024-05-14T11:17:13.527350+0000","last_active":"2024-05-14T11:17:14.537303+0000","last_peered":"2024-05-14T11:17:14.537303+0000","last_clean":"2024-05-14T11:17:14.537303+0000","last_became_active":"2024-05-14T11:17:13.527016+0000","last_became_peered":"2024-05-14T11:17:13.527016+0000","last_unstale":"2024-05-14T11:17:14.537303+0000","last_undegraded":"2024-05-14T11:17:14.537303+0000","last_fullsized":"2024-05-14T11:17:14.537303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:01.121816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":139,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534538+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:14.534538+0000","last_peered":"2024-05-14T11:17:14.534538+0000","last_clean":"2024-05-14T11:17:14.534538+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:14.534538+0000","last_undegraded":"2024-05-14T11:17:14.534538+0000","last_fullsized":"2024-05-14T11:17:14.534538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":775418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":183972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:17:18.666 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:17:18.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 Processor -- start 2024-05-14T11:17:18.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 -- start start 2024-05-14T11:17:18.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd3340b9160 0x7fd3340d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3340cfeb0 con 0x7fd33415fe00 2024-05-14T11:17:18.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3340cf110 con 0x7fd3340b70a0 2024-05-14T11:17:18.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd33a800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd3340b6d80 con 0x7fd3340b9160 2024-05-14T11:17:18.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd333e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd33415fe00 0x7fd3340d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56938/0 (socket says 172.21.15.116:56938) 2024-05-14T11:17:18.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd333e00640 1 -- 172.21.15.116:0/3508142967 learned_addr learned my addr 172.21.15.116:0/3508142967 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:18.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.848+0000 7fd32aa00640 1 --2- 172.21.15.116:0/3508142967 >> v2:172.21.15.149:3300/0 conn(0x7fd3340b9160 0x7fd3340d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224440904 0 0) 0x7fd3340cfeb0 con 0x7fd33415fe00 2024-05-14T11:17:18.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd318003530 con 0x7fd33415fe00 2024-05-14T11:17:18.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd32aa00640 1 -- 172.21.15.116:0/3508142967 >> v1:172.21.15.149:6789/0 conn(0x7fd3340b70a0 legacy=0x7fd3340b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4024984416 0 0) 0x7fd318003530 con 0x7fd33415fe00 2024-05-14T11:17:18.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd32aa00640 1 -- 172.21.15.116:0/3508142967 >> v1:172.21.15.116:6789/0 conn(0x7fd33415fe00 legacy=0x7fd3340d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd32aa00640 1 -- 172.21.15.116:0/3508142967 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3340cf3d0 con 0x7fd3340b9160 2024-05-14T11:17:18.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd32aa00640 1 --2- 172.21.15.116:0/3508142967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3340b9160 0x7fd3340d1310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097c0 tx=0x7fd31c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86bb8f04eca05ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0b0070 con 0x7fd3340b9160 2024-05-14T11:17:18.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c0ab400 con 0x7fd3340b9160 2024-05-14T11:17:18.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.852+0000 7fd332a00640 1 -- 172.21.15.116:0/3508142967 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c001a80 con 0x7fd3340b9160 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- 172.21.15.116:0/3508142967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3340b9160 msgr2=0x7fd3340d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 --2- 172.21.15.116:0/3508142967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3340b9160 0x7fd3340d1310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097c0 tx=0x7fd31c01c3c0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- 172.21.15.116:0/3508142967 shutdown_connections 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 --2- 172.21.15.116:0/3508142967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3340b9160 0x7fd3340d1310 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- 172.21.15.116:0/3508142967 >> 172.21.15.116:0/3508142967 conn(0x7fd3340d3740 msgr2=0x7fd3340d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- 172.21.15.116:0/3508142967 shutdown_connections 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- 172.21.15.116:0/3508142967 wait complete. 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 Processor -- start 2024-05-14T11:17:18.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.856+0000 7fd33a800640 1 -- start start 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3340b70a0 0x7fd334158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33415fe00 0x7fd3340cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3340b6b40 con 0x7fd3340b9160 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3340b69c0 con 0x7fd33415fe00 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd33416e640 con 0x7fd3340b70a0 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42412/0 (socket says 172.21.15.116:42412) 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 -- 172.21.15.116:0/1330277760 learned_addr learned my addr 172.21.15.116:0/1330277760 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd333e00640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33415fe00 0x7fd3340cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd333400640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3340b70a0 0x7fd334158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 -- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3340b70a0 msgr2=0x7fd334158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3340b70a0 0x7fd334158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 -- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33415fe00 msgr2=0x7fd3340cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33415fe00 0x7fd3340cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 -- 172.21.15.116:0/1330277760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3341582e0 con 0x7fd3340b9160 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32aa00640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c10 tx=0x7fd31c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0ba070 con 0x7fd3340b9160 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c003c50 con 0x7fd3340b9160 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3340713a0 con 0x7fd3340b9160 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0b0040 con 0x7fd3340b9160 2024-05-14T11:17:18.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd334071960 con 0x7fd3340b9160 2024-05-14T11:17:18.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.860+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd334071110 con 0x7fd3340b9160 2024-05-14T11:17:18.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.864+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd31c0bd070 con 0x7fd3340b9160 2024-05-14T11:17:18.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.864+0000 7fd32be00640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 0x7fd310071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:18.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.864+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd31c1286d0 con 0x7fd3340b9160 2024-05-14T11:17:18.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.864+0000 7fd333400640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 0x7fd310071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:18.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.864+0000 7fd333400640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 0x7fd310071910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd324000f70 tx=0x7fd324009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:18.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:18.868+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd31c015fb0 con 0x7fd3340b9160 2024-05-14T11:17:19.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.020+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd33406a9c0 con 0x7fd31006f400 2024-05-14T11:17:19.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.028+0000 7fd32be00640 1 -- 172.21.15.116:0/1330277760 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251878 (secure 0 0 0) 0x7fd33406a9c0 con 0x7fd31006f400 2024-05-14T11:17:19.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 msgr2=0x7fd310071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 0x7fd310071910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd324000f70 tx=0x7fd324009040 comp rx=0 tx=0).stop 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 msgr2=0x7fd334157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000c10 tx=0x7fd31c019160 comp rx=0 tx=0).stop 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 shutdown_connections 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd31006f400 0x7fd310071910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33415fe00 0x7fd3340cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3340b9160 0x7fd334157da0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 --2- 172.21.15.116:0/1330277760 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3340b70a0 0x7fd334158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 >> 172.21.15.116:0/1330277760 conn(0x7fd3340d3740 msgr2=0x7fd3340b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 shutdown_connections 2024-05-14T11:17:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.032+0000 7fd33a800640 1 -- 172.21.15.116:0/1330277760 wait complete. 2024-05-14T11:17:19.054 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-05-14T11:17:18.374373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220716,"kb_used_data":4520,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4628480,"data_stored":2637494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.868067"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538452+0000","last_change":"2024-05-14T11:17:14.533328+0000","last_active":"2024-05-14T11:17:14.538452+0000","last_peered":"2024-05-14T11:17:14.538452+0000","last_clean":"2024-05-14T11:17:14.538452+0000","last_became_active":"2024-05-14T11:17:14.533018+0000","last_became_peered":"2024-05-14T11:17:14.533018+0000","last_unstale":"2024-05-14T11:17:14.538452+0000","last_undegraded":"2024-05-14T11:17:14.538452+0000","last_fullsized":"2024-05-14T11:17:14.538452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:31.712327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544595+0000","last_change":"2024-05-14T11:17:14.538491+0000","last_active":"2024-05-14T11:17:14.544595+0000","last_peered":"2024-05-14T11:17:14.544595+0000","last_clean":"2024-05-14T11:17:14.544595+0000","last_became_active":"2024-05-14T11:17:14.536144+0000","last_became_peered":"2024-05-14T11:17:14.536144+0000","last_unstale":"2024-05-14T11:17:14.544595+0000","last_undegraded":"2024-05-14T11:17:14.544595+0000","last_fullsized":"2024-05-14T11:17:14.544595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:48.197288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550936+0000","last_change":"2024-05-14T11:17:14.545323+0000","last_active":"2024-05-14T11:17:14.550936+0000","last_peered":"2024-05-14T11:17:14.550936+0000","last_clean":"2024-05-14T11:17:14.550936+0000","last_became_active":"2024-05-14T11:17:14.544962+0000","last_became_peered":"2024-05-14T11:17:14.544962+0000","last_unstale":"2024-05-14T11:17:14.550936+0000","last_undegraded":"2024-05-14T11:17:14.550936+0000","last_fullsized":"2024-05-14T11:17:14.550936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534402+0000","last_change":"2024-05-14T11:17:14.528673+0000","last_active":"2024-05-14T11:17:14.534402+0000","last_peered":"2024-05-14T11:17:14.534402+0000","last_clean":"2024-05-14T11:17:14.534402+0000","last_became_active":"2024-05-14T11:17:14.528327+0000","last_became_peered":"2024-05-14T11:17:14.528327+0000","last_unstale":"2024-05-14T11:17:14.534402+0000","last_undegraded":"2024-05-14T11:17:14.534402+0000","last_fullsized":"2024-05-14T11:17:14.534402+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:56:33.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.540741+0000","last_change":"2024-05-14T11:17:14.530152+0000","last_active":"2024-05-14T11:17:14.540741+0000","last_peered":"2024-05-14T11:17:14.540741+0000","last_clean":"2024-05-14T11:17:14.540741+0000","last_became_active":"2024-05-14T11:17:14.529820+0000","last_became_peered":"2024-05-14T11:17:14.529820+0000","last_unstale":"2024-05-14T11:17:14.540741+0000","last_undegraded":"2024-05-14T11:17:14.540741+0000","last_fullsized":"2024-05-14T11:17:14.540741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:21.545048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559365+0000","last_change":"2024-05-14T11:17:14.552988+0000","last_active":"2024-05-14T11:17:14.559365+0000","last_peered":"2024-05-14T11:17:14.559365+0000","last_clean":"2024-05-14T11:17:14.559365+0000","last_became_active":"2024-05-14T11:17:14.552642+0000","last_became_peered":"2024-05-14T11:17:14.552642+0000","last_unstale":"2024-05-14T11:17:14.559365+0000","last_undegraded":"2024-05-14T11:17:14.559365+0000","last_fullsized":"2024-05-14T11:17:14.559365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550363+0000","last_change":"2024-05-14T11:17:14.546100+0000","last_active":"2024-05-14T11:17:14.550363+0000","last_peered":"2024-05-14T11:17:14.550363+0000","last_clean":"2024-05-14T11:17:14.550363+0000","last_became_active":"2024-05-14T11:17:14.545657+0000","last_became_peered":"2024-05-14T11:17:14.545657+0000","last_unstale":"2024-05-14T11:17:14.550363+0000","last_undegraded":"2024-05-14T11:17:14.550363+0000","last_fullsized":"2024-05-14T11:17:14.550363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553749+0000","last_change":"2024-05-14T11:17:14.548840+0000","last_active":"2024-05-14T11:17:14.553749+0000","last_peered":"2024-05-14T11:17:14.553749+0000","last_clean":"2024-05-14T11:17:14.553749+0000","last_became_active":"2024-05-14T11:17:14.548527+0000","last_became_peered":"2024-05-14T11:17:14.548527+0000","last_unstale":"2024-05-14T11:17:14.553749+0000","last_undegraded":"2024-05-14T11:17:14.553749+0000","last_fullsized":"2024-05-14T11:17:14.553749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:54:47.825110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548985+0000","last_change":"2024-05-14T11:17:14.545053+0000","last_active":"2024-05-14T11:17:14.548985+0000","last_peered":"2024-05-14T11:17:14.548985+0000","last_clean":"2024-05-14T11:17:14.548985+0000","last_became_active":"2024-05-14T11:17:14.544737+0000","last_became_peered":"2024-05-14T11:17:14.544737+0000","last_unstale":"2024-05-14T11:17:14.548985+0000","last_undegraded":"2024-05-14T11:17:14.548985+0000","last_fullsized":"2024-05-14T11:17:14.548985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561126+0000","last_change":"2024-05-14T11:17:14.548127+0000","last_active":"2024-05-14T11:17:14.561126+0000","last_peered":"2024-05-14T11:17:14.561126+0000","last_clean":"2024-05-14T11:17:14.561126+0000","last_became_active":"2024-05-14T11:17:14.547849+0000","last_became_peered":"2024-05-14T11:17:14.547849+0000","last_unstale":"2024-05-14T11:17:14.561126+0000","last_undegraded":"2024-05-14T11:17:14.561126+0000","last_fullsized":"2024-05-14T11:17:14.561126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:32.851299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550157+0000","last_change":"2024-05-14T11:17:14.543642+0000","last_active":"2024-05-14T11:17:14.550157+0000","last_peered":"2024-05-14T11:17:14.550157+0000","last_clean":"2024-05-14T11:17:14.550157+0000","last_became_active":"2024-05-14T11:17:14.543324+0000","last_became_peered":"2024-05-14T11:17:14.543324+0000","last_unstale":"2024-05-14T11:17:14.550157+0000","last_undegraded":"2024-05-14T11:17:14.550157+0000","last_fullsized":"2024-05-14T11:17:14.550157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550096+0000","last_change":"2024-05-14T11:17:14.543142+0000","last_active":"2024-05-14T11:17:14.550096+0000","last_peered":"2024-05-14T11:17:14.550096+0000","last_clean":"2024-05-14T11:17:14.550096+0000","last_became_active":"2024-05-14T11:17:14.542821+0000","last_became_peered":"2024-05-14T11:17:14.542821+0000","last_unstale":"2024-05-14T11:17:14.550096+0000","last_undegraded":"2024-05-14T11:17:14.550096+0000","last_fullsized":"2024-05-14T11:17:14.550096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561279+0000","last_change":"2024-05-14T11:17:14.550510+0000","last_active":"2024-05-14T11:17:14.561279+0000","last_peered":"2024-05-14T11:17:14.561279+0000","last_clean":"2024-05-14T11:17:14.561279+0000","last_became_active":"2024-05-14T11:17:14.550215+0000","last_became_peered":"2024-05-14T11:17:14.550215+0000","last_unstale":"2024-05-14T11:17:14.561279+0000","last_undegraded":"2024-05-14T11:17:14.561279+0000","last_fullsized":"2024-05-14T11:17:14.561279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538327+0000","last_change":"2024-05-14T11:17:14.528562+0000","last_active":"2024-05-14T11:17:14.538327+0000","last_peered":"2024-05-14T11:17:14.538327+0000","last_clean":"2024-05-14T11:17:14.538327+0000","last_became_active":"2024-05-14T11:17:14.528209+0000","last_became_peered":"2024-05-14T11:17:14.528209+0000","last_unstale":"2024-05-14T11:17:14.538327+0000","last_undegraded":"2024-05-14T11:17:14.538327+0000","last_fullsized":"2024-05-14T11:17:14.538327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550437+0000","last_change":"2024-05-14T11:17:14.546112+0000","last_active":"2024-05-14T11:17:14.550437+0000","last_peered":"2024-05-14T11:17:14.550437+0000","last_clean":"2024-05-14T11:17:14.550437+0000","last_became_active":"2024-05-14T11:17:14.545612+0000","last_became_peered":"2024-05-14T11:17:14.545612+0000","last_unstale":"2024-05-14T11:17:14.550437+0000","last_undegraded":"2024-05-14T11:17:14.550437+0000","last_fullsized":"2024-05-14T11:17:14.550437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:42.117037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553371+0000","last_change":"2024-05-14T11:17:14.550990+0000","last_active":"2024-05-14T11:17:14.553371+0000","last_peered":"2024-05-14T11:17:14.553371+0000","last_clean":"2024-05-14T11:17:14.553371+0000","last_became_active":"2024-05-14T11:17:14.550709+0000","last_became_peered":"2024-05-14T11:17:14.550709+0000","last_unstale":"2024-05-14T11:17:14.553371+0000","last_undegraded":"2024-05-14T11:17:14.553371+0000","last_fullsized":"2024-05-14T11:17:14.553371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:32.851299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562285+0000","last_change":"2024-05-14T11:17:14.559223+0000","last_active":"2024-05-14T11:17:14.562285+0000","last_peered":"2024-05-14T11:17:14.562285+0000","last_clean":"2024-05-14T11:17:14.562285+0000","last_became_active":"2024-05-14T11:17:14.558733+0000","last_became_peered":"2024-05-14T11:17:14.558733+0000","last_unstale":"2024-05-14T11:17:14.562285+0000","last_undegraded":"2024-05-14T11:17:14.562285+0000","last_fullsized":"2024-05-14T11:17:14.562285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.540271+0000","last_change":"2024-05-14T11:17:14.536802+0000","last_active":"2024-05-14T11:17:14.540271+0000","last_peered":"2024-05-14T11:17:14.540271+0000","last_clean":"2024-05-14T11:17:14.540271+0000","last_became_active":"2024-05-14T11:17:14.536469+0000","last_became_peered":"2024-05-14T11:17:14.536469+0000","last_unstale":"2024-05-14T11:17:14.540271+0000","last_undegraded":"2024-05-14T11:17:14.540271+0000","last_fullsized":"2024-05-14T11:17:14.540271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.546175+0000","last_change":"2024-05-14T11:17:14.540028+0000","last_active":"2024-05-14T11:17:14.546175+0000","last_peered":"2024-05-14T11:17:14.546175+0000","last_clean":"2024-05-14T11:17:14.546175+0000","last_became_active":"2024-05-14T11:17:14.539559+0000","last_became_peered":"2024-05-14T11:17:14.539559+0000","last_unstale":"2024-05-14T11:17:14.546175+0000","last_undegraded":"2024-05-14T11:17:14.546175+0000","last_fullsized":"2024-05-14T11:17:14.546175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:33:01.436450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537726+0000","last_change":"2024-05-14T11:17:14.528785+0000","last_active":"2024-05-14T11:17:14.537726+0000","last_peered":"2024-05-14T11:17:14.537726+0000","last_clean":"2024-05-14T11:17:14.537726+0000","last_became_active":"2024-05-14T11:17:14.528464+0000","last_became_peered":"2024-05-14T11:17:14.528464+0000","last_unstale":"2024-05-14T11:17:14.537726+0000","last_undegraded":"2024-05-14T11:17:14.537726+0000","last_fullsized":"2024-05-14T11:17:14.537726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:41.135647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559067+0000","last_change":"2024-05-14T11:17:14.546816+0000","last_active":"2024-05-14T11:17:14.559067+0000","last_peered":"2024-05-14T11:17:14.559067+0000","last_clean":"2024-05-14T11:17:14.559067+0000","last_became_active":"2024-05-14T11:17:14.546540+0000","last_became_peered":"2024-05-14T11:17:14.546540+0000","last_unstale":"2024-05-14T11:17:14.559067+0000","last_undegraded":"2024-05-14T11:17:14.559067+0000","last_fullsized":"2024-05-14T11:17:14.559067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:02.984707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.541539+0000","last_change":"2024-05-14T11:17:14.535434+0000","last_active":"2024-05-14T11:17:14.541539+0000","last_peered":"2024-05-14T11:17:14.541539+0000","last_clean":"2024-05-14T11:17:14.541539+0000","last_became_active":"2024-05-14T11:17:14.535127+0000","last_became_peered":"2024-05-14T11:17:14.535127+0000","last_unstale":"2024-05-14T11:17:14.541539+0000","last_undegraded":"2024-05-14T11:17:14.541539+0000","last_fullsized":"2024-05-14T11:17:14.541539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550838+0000","last_change":"2024-05-14T11:17:14.547635+0000","last_active":"2024-05-14T11:17:14.550838+0000","last_peered":"2024-05-14T11:17:14.550838+0000","last_clean":"2024-05-14T11:17:14.550838+0000","last_became_active":"2024-05-14T11:17:14.545919+0000","last_became_peered":"2024-05-14T11:17:14.545919+0000","last_unstale":"2024-05-14T11:17:14.550838+0000","last_undegraded":"2024-05-14T11:17:14.550838+0000","last_fullsized":"2024-05-14T11:17:14.550838+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:42.117037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.551991+0000","last_change":"2024-05-14T11:17:14.546409+0000","last_active":"2024-05-14T11:17:14.551991+0000","last_peered":"2024-05-14T11:17:14.551991+0000","last_clean":"2024-05-14T11:17:14.551991+0000","last_became_active":"2024-05-14T11:17:14.546125+0000","last_became_peered":"2024-05-14T11:17:14.546125+0000","last_unstale":"2024-05-14T11:17:14.551991+0000","last_undegraded":"2024-05-14T11:17:14.551991+0000","last_fullsized":"2024-05-14T11:17:14.551991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:30:45.239492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537480+0000","last_change":"2024-05-14T11:17:14.531524+0000","last_active":"2024-05-14T11:17:14.537480+0000","last_peered":"2024-05-14T11:17:14.537480+0000","last_clean":"2024-05-14T11:17:14.537480+0000","last_became_active":"2024-05-14T11:17:14.530905+0000","last_became_peered":"2024-05-14T11:17:14.530905+0000","last_unstale":"2024-05-14T11:17:14.537480+0000","last_undegraded":"2024-05-14T11:17:14.537480+0000","last_fullsized":"2024-05-14T11:17:14.537480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:50.227093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562143+0000","last_change":"2024-05-14T11:17:14.558633+0000","last_active":"2024-05-14T11:17:14.562143+0000","last_peered":"2024-05-14T11:17:14.562143+0000","last_clean":"2024-05-14T11:17:14.562143+0000","last_became_active":"2024-05-14T11:17:14.558199+0000","last_became_peered":"2024-05-14T11:17:14.558199+0000","last_unstale":"2024-05-14T11:17:14.562143+0000","last_undegraded":"2024-05-14T11:17:14.562143+0000","last_fullsized":"2024-05-14T11:17:14.562143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544886+0000","last_change":"2024-05-14T11:17:14.540081+0000","last_active":"2024-05-14T11:17:14.544886+0000","last_peered":"2024-05-14T11:17:14.544886+0000","last_clean":"2024-05-14T11:17:14.544886+0000","last_became_active":"2024-05-14T11:17:14.538679+0000","last_became_peered":"2024-05-14T11:17:14.538679+0000","last_unstale":"2024-05-14T11:17:14.544886+0000","last_undegraded":"2024-05-14T11:17:14.544886+0000","last_fullsized":"2024-05-14T11:17:14.544886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562072+0000","last_change":"2024-05-14T11:17:14.557901+0000","last_active":"2024-05-14T11:17:14.562072+0000","last_peered":"2024-05-14T11:17:14.562072+0000","last_clean":"2024-05-14T11:17:14.562072+0000","last_became_active":"2024-05-14T11:17:14.557673+0000","last_became_peered":"2024-05-14T11:17:14.557673+0000","last_unstale":"2024-05-14T11:17:14.562072+0000","last_undegraded":"2024-05-14T11:17:14.562072+0000","last_fullsized":"2024-05-14T11:17:14.562072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:31.712327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548090+0000","last_change":"2024-05-14T11:17:14.542679+0000","last_active":"2024-05-14T11:17:14.548090+0000","last_peered":"2024-05-14T11:17:14.548090+0000","last_clean":"2024-05-14T11:17:14.548090+0000","last_became_active":"2024-05-14T11:17:14.542415+0000","last_became_peered":"2024-05-14T11:17:14.542415+0000","last_unstale":"2024-05-14T11:17:14.548090+0000","last_undegraded":"2024-05-14T11:17:14.548090+0000","last_fullsized":"2024-05-14T11:17:14.548090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538962+0000","last_change":"2024-05-14T11:17:14.533318+0000","last_active":"2024-05-14T11:17:14.538962+0000","last_peered":"2024-05-14T11:17:14.538962+0000","last_clean":"2024-05-14T11:17:14.538962+0000","last_became_active":"2024-05-14T11:17:14.532977+0000","last_became_peered":"2024-05-14T11:17:14.532977+0000","last_unstale":"2024-05-14T11:17:14.538962+0000","last_undegraded":"2024-05-14T11:17:14.538962+0000","last_fullsized":"2024-05-14T11:17:14.538962+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:14.407202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.550604+0000","last_change":"2024-05-14T11:17:14.546896+0000","last_active":"2024-05-14T11:17:14.550604+0000","last_peered":"2024-05-14T11:17:14.550604+0000","last_clean":"2024-05-14T11:17:14.550604+0000","last_became_active":"2024-05-14T11:17:14.546283+0000","last_became_peered":"2024-05-14T11:17:14.546283+0000","last_unstale":"2024-05-14T11:17:14.550604+0000","last_undegraded":"2024-05-14T11:17:14.550604+0000","last_fullsized":"2024-05-14T11:17:14.550604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.549530+0000","last_change":"2024-05-14T11:17:14.543663+0000","last_active":"2024-05-14T11:17:14.549530+0000","last_peered":"2024-05-14T11:17:14.549530+0000","last_clean":"2024-05-14T11:17:14.549530+0000","last_became_active":"2024-05-14T11:17:14.543375+0000","last_became_peered":"2024-05-14T11:17:14.543375+0000","last_unstale":"2024-05-14T11:17:14.549530+0000","last_undegraded":"2024-05-14T11:17:14.549530+0000","last_fullsized":"2024-05-14T11:17:14.549530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.553473+0000","last_change":"2024-05-14T11:17:14.551357+0000","last_active":"2024-05-14T11:17:14.553473+0000","last_peered":"2024-05-14T11:17:14.553473+0000","last_clean":"2024-05-14T11:17:14.553473+0000","last_became_active":"2024-05-14T11:17:14.551080+0000","last_became_peered":"2024-05-14T11:17:14.551080+0000","last_unstale":"2024-05-14T11:17:14.553473+0000","last_undegraded":"2024-05-14T11:17:14.553473+0000","last_fullsized":"2024-05-14T11:17:14.553473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560876+0000","last_change":"2024-05-14T11:17:13.528783+0000","last_active":"2024-05-14T11:17:14.560876+0000","last_peered":"2024-05-14T11:17:14.560876+0000","last_clean":"2024-05-14T11:17:14.560876+0000","last_became_active":"2024-05-14T11:17:13.527734+0000","last_became_peered":"2024-05-14T11:17:13.527734+0000","last_unstale":"2024-05-14T11:17:14.560876+0000","last_undegraded":"2024-05-14T11:17:14.560876+0000","last_fullsized":"2024-05-14T11:17:14.560876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:47.194388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559346+0000","last_change":"2024-05-14T11:17:14.553459+0000","last_active":"2024-05-14T11:17:14.559346+0000","last_peered":"2024-05-14T11:17:14.559346+0000","last_clean":"2024-05-14T11:17:14.559346+0000","last_became_active":"2024-05-14T11:17:14.553179+0000","last_became_peered":"2024-05-14T11:17:14.553179+0000","last_unstale":"2024-05-14T11:17:14.559346+0000","last_undegraded":"2024-05-14T11:17:14.559346+0000","last_fullsized":"2024-05-14T11:17:14.559346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561365+0000","last_change":"2024-05-14T11:17:13.532401+0000","last_active":"2024-05-14T11:17:14.561365+0000","last_peered":"2024-05-14T11:17:14.561365+0000","last_clean":"2024-05-14T11:17:14.561365+0000","last_became_active":"2024-05-14T11:17:13.532080+0000","last_became_peered":"2024-05-14T11:17:13.532080+0000","last_unstale":"2024-05-14T11:17:14.561365+0000","last_undegraded":"2024-05-14T11:17:14.561365+0000","last_fullsized":"2024-05-14T11:17:14.561365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539944+0000","last_change":"2024-05-14T11:17:14.529094+0000","last_active":"2024-05-14T11:17:14.539944+0000","last_peered":"2024-05-14T11:17:14.539944+0000","last_clean":"2024-05-14T11:17:14.539944+0000","last_became_active":"2024-05-14T11:17:14.528783+0000","last_became_peered":"2024-05-14T11:17:14.528783+0000","last_unstale":"2024-05-14T11:17:14.539944+0000","last_undegraded":"2024-05-14T11:17:14.539944+0000","last_fullsized":"2024-05-14T11:17:14.539944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.556635+0000","last_change":"2024-05-14T11:17:13.527879+0000","last_active":"2024-05-14T11:17:14.556635+0000","last_peered":"2024-05-14T11:17:14.556635+0000","last_clean":"2024-05-14T11:17:14.556635+0000","last_became_active":"2024-05-14T11:17:13.527509+0000","last_became_peered":"2024-05-14T11:17:13.527509+0000","last_unstale":"2024-05-14T11:17:14.556635+0000","last_undegraded":"2024-05-14T11:17:14.556635+0000","last_fullsized":"2024-05-14T11:17:14.556635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544592+0000","last_change":"2024-05-14T11:17:14.538586+0000","last_active":"2024-05-14T11:17:14.544592+0000","last_peered":"2024-05-14T11:17:14.544592+0000","last_clean":"2024-05-14T11:17:14.544592+0000","last_became_active":"2024-05-14T11:17:14.538187+0000","last_became_peered":"2024-05-14T11:17:14.538187+0000","last_unstale":"2024-05-14T11:17:14.544592+0000","last_undegraded":"2024-05-14T11:17:14.544592+0000","last_fullsized":"2024-05-14T11:17:14.544592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:47:39.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560654+0000","last_change":"2024-05-14T11:17:13.537106+0000","last_active":"2024-05-14T11:17:14.560654+0000","last_peered":"2024-05-14T11:17:14.560654+0000","last_clean":"2024-05-14T11:17:14.560654+0000","last_became_active":"2024-05-14T11:17:13.536580+0000","last_became_peered":"2024-05-14T11:17:13.536580+0000","last_unstale":"2024-05-14T11:17:14.560654+0000","last_undegraded":"2024-05-14T11:17:14.560654+0000","last_fullsized":"2024-05-14T11:17:14.560654+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545586+0000","last_change":"2024-05-14T11:17:14.541427+0000","last_active":"2024-05-14T11:17:14.545586+0000","last_peered":"2024-05-14T11:17:14.545586+0000","last_clean":"2024-05-14T11:17:14.545586+0000","last_became_active":"2024-05-14T11:17:14.541134+0000","last_became_peered":"2024-05-14T11:17:14.541134+0000","last_unstale":"2024-05-14T11:17:14.545586+0000","last_undegraded":"2024-05-14T11:17:14.545586+0000","last_fullsized":"2024-05-14T11:17:14.545586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562894+0000","last_change":"2024-05-14T11:17:13.535608+0000","last_active":"2024-05-14T11:17:14.562894+0000","last_peered":"2024-05-14T11:17:14.562894+0000","last_clean":"2024-05-14T11:17:14.562894+0000","last_became_active":"2024-05-14T11:17:13.535056+0000","last_became_peered":"2024-05-14T11:17:13.535056+0000","last_unstale":"2024-05-14T11:17:14.562894+0000","last_undegraded":"2024-05-14T11:17:14.562894+0000","last_fullsized":"2024-05-14T11:17:14.562894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:47.296773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.558364+0000","last_change":"2024-05-14T11:17:14.547072+0000","last_active":"2024-05-14T11:17:14.558364+0000","last_peered":"2024-05-14T11:17:14.558364+0000","last_clean":"2024-05-14T11:17:14.558364+0000","last_became_active":"2024-05-14T11:17:14.546550+0000","last_became_peered":"2024-05-14T11:17:14.546550+0000","last_unstale":"2024-05-14T11:17:14.558364+0000","last_undegraded":"2024-05-14T11:17:14.558364+0000","last_fullsized":"2024-05-14T11:17:14.558364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:20.068069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.560861+0000","last_change":"2024-05-14T11:17:13.529732+0000","last_active":"2024-05-14T11:17:14.560861+0000","last_peered":"2024-05-14T11:17:14.560861+0000","last_clean":"2024-05-14T11:17:14.560861+0000","last_became_active":"2024-05-14T11:17:13.529425+0000","last_became_peered":"2024-05-14T11:17:13.529425+0000","last_unstale":"2024-05-14T11:17:14.560861+0000","last_undegraded":"2024-05-14T11:17:14.560861+0000","last_fullsized":"2024-05-14T11:17:14.560861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:24.343277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545788+0000","last_change":"2024-05-14T11:17:14.544342+0000","last_active":"2024-05-14T11:17:14.545788+0000","last_peered":"2024-05-14T11:17:14.545788+0000","last_clean":"2024-05-14T11:17:14.545788+0000","last_became_active":"2024-05-14T11:17:14.543072+0000","last_became_peered":"2024-05-14T11:17:14.543072+0000","last_unstale":"2024-05-14T11:17:14.545788+0000","last_undegraded":"2024-05-14T11:17:14.545788+0000","last_fullsized":"2024-05-14T11:17:14.545788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.527364+0000","last_change":"2024-05-14T11:17:13.530126+0000","last_active":"2024-05-14T11:17:14.527364+0000","last_peered":"2024-05-14T11:17:14.527364+0000","last_clean":"2024-05-14T11:17:14.527364+0000","last_became_active":"2024-05-14T11:17:13.529703+0000","last_became_peered":"2024-05-14T11:17:13.529703+0000","last_unstale":"2024-05-14T11:17:14.527364+0000","last_undegraded":"2024-05-14T11:17:14.527364+0000","last_fullsized":"2024-05-14T11:17:14.527364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:38.631345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544570+0000","last_change":"2024-05-14T11:17:14.537269+0000","last_active":"2024-05-14T11:17:14.544570+0000","last_peered":"2024-05-14T11:17:14.544570+0000","last_clean":"2024-05-14T11:17:14.544570+0000","last_became_active":"2024-05-14T11:17:14.536736+0000","last_became_peered":"2024-05-14T11:17:14.536736+0000","last_unstale":"2024-05-14T11:17:14.544570+0000","last_undegraded":"2024-05-14T11:17:14.544570+0000","last_fullsized":"2024-05-14T11:17:14.544570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:39.780118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525622+0000","last_change":"2024-05-14T11:17:13.528654+0000","last_active":"2024-05-14T11:17:14.525622+0000","last_peered":"2024-05-14T11:17:14.525622+0000","last_clean":"2024-05-14T11:17:14.525622+0000","last_became_active":"2024-05-14T11:17:13.528351+0000","last_became_peered":"2024-05-14T11:17:13.528351+0000","last_unstale":"2024-05-14T11:17:14.525622+0000","last_undegraded":"2024-05-14T11:17:14.525622+0000","last_fullsized":"2024-05-14T11:17:14.525622+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534373+0000","last_change":"2024-05-14T11:17:14.527495+0000","last_active":"2024-05-14T11:17:14.534373+0000","last_peered":"2024-05-14T11:17:14.534373+0000","last_clean":"2024-05-14T11:17:14.534373+0000","last_became_active":"2024-05-14T11:17:14.527118+0000","last_became_peered":"2024-05-14T11:17:14.527118+0000","last_unstale":"2024-05-14T11:17:14.534373+0000","last_undegraded":"2024-05-14T11:17:14.534373+0000","last_fullsized":"2024-05-14T11:17:14.534373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:00:07.518389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.528034+0000","last_change":"2024-05-14T11:17:13.531054+0000","last_active":"2024-05-14T11:17:14.528034+0000","last_peered":"2024-05-14T11:17:14.528034+0000","last_clean":"2024-05-14T11:17:14.528034+0000","last_became_active":"2024-05-14T11:17:13.530552+0000","last_became_peered":"2024-05-14T11:17:13.530552+0000","last_unstale":"2024-05-14T11:17:14.528034+0000","last_undegraded":"2024-05-14T11:17:14.528034+0000","last_fullsized":"2024-05-14T11:17:14.528034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:44:31.842738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.552793+0000","last_change":"2024-05-14T11:17:14.549782+0000","last_active":"2024-05-14T11:17:14.552793+0000","last_peered":"2024-05-14T11:17:14.552793+0000","last_clean":"2024-05-14T11:17:14.552793+0000","last_became_active":"2024-05-14T11:17:14.549143+0000","last_became_peered":"2024-05-14T11:17:14.549143+0000","last_unstale":"2024-05-14T11:17:14.552793+0000","last_undegraded":"2024-05-14T11:17:14.552793+0000","last_fullsized":"2024-05-14T11:17:14.552793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:02.984707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.542134+0000","last_change":"2024-05-14T11:17:13.535408+0000","last_active":"2024-05-14T11:17:14.542134+0000","last_peered":"2024-05-14T11:17:14.542134+0000","last_clean":"2024-05-14T11:17:14.542134+0000","last_became_active":"2024-05-14T11:17:13.535085+0000","last_became_peered":"2024-05-14T11:17:13.535085+0000","last_unstale":"2024-05-14T11:17:14.542134+0000","last_undegraded":"2024-05-14T11:17:14.542134+0000","last_fullsized":"2024-05-14T11:17:14.542134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:01.121816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545516+0000","last_change":"2024-05-14T11:17:14.540451+0000","last_active":"2024-05-14T11:17:14.545516+0000","last_peered":"2024-05-14T11:17:14.545516+0000","last_clean":"2024-05-14T11:17:14.545516+0000","last_became_active":"2024-05-14T11:17:14.540108+0000","last_became_peered":"2024-05-14T11:17:14.540108+0000","last_unstale":"2024-05-14T11:17:14.545516+0000","last_undegraded":"2024-05-14T11:17:14.545516+0000","last_fullsized":"2024-05-14T11:17:14.545516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564010+0000","last_change":"2024-05-14T11:17:13.531691+0000","last_active":"2024-05-14T11:17:14.564010+0000","last_peered":"2024-05-14T11:17:14.564010+0000","last_clean":"2024-05-14T11:17:14.564010+0000","last_became_active":"2024-05-14T11:17:13.531378+0000","last_became_peered":"2024-05-14T11:17:13.531378+0000","last_unstale":"2024-05-14T11:17:14.564010+0000","last_undegraded":"2024-05-14T11:17:14.564010+0000","last_fullsized":"2024-05-14T11:17:14.564010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:24.343277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.541671+0000","last_change":"2024-05-14T11:17:14.535513+0000","last_active":"2024-05-14T11:17:14.541671+0000","last_peered":"2024-05-14T11:17:14.541671+0000","last_clean":"2024-05-14T11:17:14.541671+0000","last_became_active":"2024-05-14T11:17:14.535161+0000","last_became_peered":"2024-05-14T11:17:14.535161+0000","last_unstale":"2024-05-14T11:17:14.541671+0000","last_undegraded":"2024-05-14T11:17:14.541671+0000","last_fullsized":"2024-05-14T11:17:14.541671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.567097+0000","last_change":"2024-05-14T11:17:13.535648+0000","last_active":"2024-05-14T11:17:14.567097+0000","last_peered":"2024-05-14T11:17:14.567097+0000","last_clean":"2024-05-14T11:17:14.567097+0000","last_became_active":"2024-05-14T11:17:13.535270+0000","last_became_peered":"2024-05-14T11:17:13.535270+0000","last_unstale":"2024-05-14T11:17:14.567097+0000","last_undegraded":"2024-05-14T11:17:14.567097+0000","last_fullsized":"2024-05-14T11:17:14.567097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:19.059507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.556265+0000","last_change":"2024-05-14T11:17:14.552298+0000","last_active":"2024-05-14T11:17:14.556265+0000","last_peered":"2024-05-14T11:17:14.556265+0000","last_clean":"2024-05-14T11:17:14.556265+0000","last_became_active":"2024-05-14T11:17:14.551997+0000","last_became_peered":"2024-05-14T11:17:14.551997+0000","last_unstale":"2024-05-14T11:17:14.556265+0000","last_undegraded":"2024-05-14T11:17:14.556265+0000","last_fullsized":"2024-05-14T11:17:14.556265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.563709+0000","last_change":"2024-05-14T11:17:13.530162+0000","last_active":"2024-05-14T11:17:14.563709+0000","last_peered":"2024-05-14T11:17:14.563709+0000","last_clean":"2024-05-14T11:17:14.563709+0000","last_became_active":"2024-05-14T11:17:13.529873+0000","last_became_peered":"2024-05-14T11:17:13.529873+0000","last_unstale":"2024-05-14T11:17:14.563709+0000","last_undegraded":"2024-05-14T11:17:14.563709+0000","last_fullsized":"2024-05-14T11:17:14.563709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:38:01.976145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538125+0000","last_change":"2024-05-14T11:17:14.532437+0000","last_active":"2024-05-14T11:17:14.538125+0000","last_peered":"2024-05-14T11:17:14.538125+0000","last_clean":"2024-05-14T11:17:14.538125+0000","last_became_active":"2024-05-14T11:17:14.532094+0000","last_became_peered":"2024-05-14T11:17:14.532094+0000","last_unstale":"2024-05-14T11:17:14.538125+0000","last_undegraded":"2024-05-14T11:17:14.538125+0000","last_fullsized":"2024-05-14T11:17:14.538125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.526525+0000","last_change":"2024-05-14T11:17:13.536727+0000","last_active":"2024-05-14T11:17:14.526525+0000","last_peered":"2024-05-14T11:17:14.526525+0000","last_clean":"2024-05-14T11:17:14.526525+0000","last_became_active":"2024-05-14T11:17:13.536390+0000","last_became_peered":"2024-05-14T11:17:13.536390+0000","last_unstale":"2024-05-14T11:17:14.526525+0000","last_undegraded":"2024-05-14T11:17:14.526525+0000","last_fullsized":"2024-05-14T11:17:14.526525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.548167+0000","last_change":"2024-05-14T11:17:14.545128+0000","last_active":"2024-05-14T11:17:14.548167+0000","last_peered":"2024-05-14T11:17:14.548167+0000","last_clean":"2024-05-14T11:17:14.548167+0000","last_became_active":"2024-05-14T11:17:14.544835+0000","last_became_peered":"2024-05-14T11:17:14.544835+0000","last_unstale":"2024-05-14T11:17:14.548167+0000","last_undegraded":"2024-05-14T11:17:14.548167+0000","last_fullsized":"2024-05-14T11:17:14.548167+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.523125+0000","last_change":"2024-05-14T11:17:13.529090+0000","last_active":"2024-05-14T11:17:14.523125+0000","last_peered":"2024-05-14T11:17:14.523125+0000","last_clean":"2024-05-14T11:17:14.523125+0000","last_became_active":"2024-05-14T11:17:13.528250+0000","last_became_peered":"2024-05-14T11:17:13.528250+0000","last_unstale":"2024-05-14T11:17:14.523125+0000","last_undegraded":"2024-05-14T11:17:14.523125+0000","last_fullsized":"2024-05-14T11:17:14.523125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:21:16.011697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539369+0000","last_change":"2024-05-14T11:17:14.533854+0000","last_active":"2024-05-14T11:17:14.539369+0000","last_peered":"2024-05-14T11:17:14.539369+0000","last_clean":"2024-05-14T11:17:14.539369+0000","last_became_active":"2024-05-14T11:17:14.533561+0000","last_became_peered":"2024-05-14T11:17:14.533561+0000","last_unstale":"2024-05-14T11:17:14.539369+0000","last_undegraded":"2024-05-14T11:17:14.539369+0000","last_fullsized":"2024-05-14T11:17:14.539369+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:07.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562624+0000","last_change":"2024-05-14T11:17:13.533231+0000","last_active":"2024-05-14T11:17:14.562624+0000","last_peered":"2024-05-14T11:17:14.562624+0000","last_clean":"2024-05-14T11:17:14.562624+0000","last_became_active":"2024-05-14T11:17:13.530949+0000","last_became_peered":"2024-05-14T11:17:13.530949+0000","last_unstale":"2024-05-14T11:17:14.562624+0000","last_undegraded":"2024-05-14T11:17:14.562624+0000","last_fullsized":"2024-05-14T11:17:14.562624+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:38.631345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544622+0000","last_change":"2024-05-14T11:17:14.534865+0000","last_active":"2024-05-14T11:17:14.544622+0000","last_peered":"2024-05-14T11:17:14.544622+0000","last_clean":"2024-05-14T11:17:14.544622+0000","last_became_active":"2024-05-14T11:17:14.534438+0000","last_became_peered":"2024-05-14T11:17:14.534438+0000","last_unstale":"2024-05-14T11:17:14.544622+0000","last_undegraded":"2024-05-14T11:17:14.544622+0000","last_fullsized":"2024-05-14T11:17:14.544622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:56:54.178936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564717+0000","last_change":"2024-05-14T11:17:13.533122+0000","last_active":"2024-05-14T11:17:14.564717+0000","last_peered":"2024-05-14T11:17:14.564717+0000","last_clean":"2024-05-14T11:17:14.564717+0000","last_became_active":"2024-05-14T11:17:13.532093+0000","last_became_peered":"2024-05-14T11:17:13.532093+0000","last_unstale":"2024-05-14T11:17:14.564717+0000","last_undegraded":"2024-05-14T11:17:14.564717+0000","last_fullsized":"2024-05-14T11:17:14.564717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:09.662910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.562189+0000","last_change":"2024-05-14T11:17:14.558778+0000","last_active":"2024-05-14T11:17:14.562189+0000","last_peered":"2024-05-14T11:17:14.562189+0000","last_clean":"2024-05-14T11:17:14.562189+0000","last_became_active":"2024-05-14T11:17:14.558297+0000","last_became_peered":"2024-05-14T11:17:14.558297+0000","last_unstale":"2024-05-14T11:17:14.562189+0000","last_undegraded":"2024-05-14T11:17:14.562189+0000","last_fullsized":"2024-05-14T11:17:14.562189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:29:07.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564894+0000","last_change":"2024-05-14T11:17:13.536741+0000","last_active":"2024-05-14T11:17:14.564894+0000","last_peered":"2024-05-14T11:17:14.564894+0000","last_clean":"2024-05-14T11:17:14.564894+0000","last_became_active":"2024-05-14T11:17:13.536409+0000","last_became_peered":"2024-05-14T11:17:13.536409+0000","last_unstale":"2024-05-14T11:17:14.564894+0000","last_undegraded":"2024-05-14T11:17:14.564894+0000","last_fullsized":"2024-05-14T11:17:14.564894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545646+0000","last_change":"2024-05-14T11:17:14.543303+0000","last_active":"2024-05-14T11:17:14.545646+0000","last_peered":"2024-05-14T11:17:14.545646+0000","last_clean":"2024-05-14T11:17:14.545646+0000","last_became_active":"2024-05-14T11:17:14.541489+0000","last_became_peered":"2024-05-14T11:17:14.541489+0000","last_unstale":"2024-05-14T11:17:14.545646+0000","last_undegraded":"2024-05-14T11:17:14.545646+0000","last_fullsized":"2024-05-14T11:17:14.545646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:10.671472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559584+0000","last_change":"2024-05-14T11:17:13.519077+0000","last_active":"2024-05-14T11:17:14.559584+0000","last_peered":"2024-05-14T11:17:14.559584+0000","last_clean":"2024-05-14T11:17:14.559584+0000","last_became_active":"2024-05-14T11:17:13.514204+0000","last_became_peered":"2024-05-14T11:17:13.514204+0000","last_unstale":"2024-05-14T11:17:14.559584+0000","last_undegraded":"2024-05-14T11:17:14.559584+0000","last_fullsized":"2024-05-14T11:17:14.559584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:40.300485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539524+0000","last_change":"2024-05-14T11:17:14.530457+0000","last_active":"2024-05-14T11:17:14.539524+0000","last_peered":"2024-05-14T11:17:14.539524+0000","last_clean":"2024-05-14T11:17:14.539524+0000","last_became_active":"2024-05-14T11:17:14.530099+0000","last_became_peered":"2024-05-14T11:17:14.530099+0000","last_unstale":"2024-05-14T11:17:14.539524+0000","last_undegraded":"2024-05-14T11:17:14.539524+0000","last_fullsized":"2024-05-14T11:17:14.539524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:46:15.798416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.527156+0000","last_change":"2024-05-14T11:17:13.527520+0000","last_active":"2024-05-14T11:17:14.527156+0000","last_peered":"2024-05-14T11:17:14.527156+0000","last_clean":"2024-05-14T11:17:14.527156+0000","last_became_active":"2024-05-14T11:17:13.526648+0000","last_became_peered":"2024-05-14T11:17:13.526648+0000","last_unstale":"2024-05-14T11:17:14.527156+0000","last_undegraded":"2024-05-14T11:17:14.527156+0000","last_fullsized":"2024-05-14T11:17:14.527156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:14:13.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.546384+0000","last_change":"2024-05-14T11:17:14.543662+0000","last_active":"2024-05-14T11:17:14.546384+0000","last_peered":"2024-05-14T11:17:14.546384+0000","last_clean":"2024-05-14T11:17:14.546384+0000","last_became_active":"2024-05-14T11:17:14.543362+0000","last_became_peered":"2024-05-14T11:17:14.543362+0000","last_unstale":"2024-05-14T11:17:14.546384+0000","last_undegraded":"2024-05-14T11:17:14.546384+0000","last_fullsized":"2024-05-14T11:17:14.546384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:23.016251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559942+0000","last_change":"2024-05-14T11:17:13.526039+0000","last_active":"2024-05-14T11:17:14.559942+0000","last_peered":"2024-05-14T11:17:14.559942+0000","last_clean":"2024-05-14T11:17:14.559942+0000","last_became_active":"2024-05-14T11:17:13.523193+0000","last_became_peered":"2024-05-14T11:17:13.523193+0000","last_unstale":"2024-05-14T11:17:14.559942+0000","last_undegraded":"2024-05-14T11:17:14.559942+0000","last_fullsized":"2024-05-14T11:17:14.559942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:19:44.678648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.551135+0000","last_change":"2024-05-14T11:17:14.547868+0000","last_active":"2024-05-14T11:17:14.551135+0000","last_peered":"2024-05-14T11:17:14.551135+0000","last_clean":"2024-05-14T11:17:14.551135+0000","last_became_active":"2024-05-14T11:17:14.547399+0000","last_became_peered":"2024-05-14T11:17:14.547399+0000","last_unstale":"2024-05-14T11:17:14.551135+0000","last_undegraded":"2024-05-14T11:17:14.551135+0000","last_fullsized":"2024-05-14T11:17:14.551135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:22:51.647048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.519581+0000","last_change":"2024-05-14T11:17:13.527212+0000","last_active":"2024-05-14T11:17:14.519581+0000","last_peered":"2024-05-14T11:17:14.519581+0000","last_clean":"2024-05-14T11:17:14.519581+0000","last_became_active":"2024-05-14T11:17:13.526900+0000","last_became_peered":"2024-05-14T11:17:13.526900+0000","last_unstale":"2024-05-14T11:17:14.519581+0000","last_undegraded":"2024-05-14T11:17:14.519581+0000","last_fullsized":"2024-05-14T11:17:14.519581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:02:11.352543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.561705+0000","last_change":"2024-05-14T11:17:14.549984+0000","last_active":"2024-05-14T11:17:14.561705+0000","last_peered":"2024-05-14T11:17:14.561705+0000","last_clean":"2024-05-14T11:17:14.561705+0000","last_became_active":"2024-05-14T11:17:14.549645+0000","last_became_peered":"2024-05-14T11:17:14.549645+0000","last_unstale":"2024-05-14T11:17:14.561705+0000","last_undegraded":"2024-05-14T11:17:14.561705+0000","last_fullsized":"2024-05-14T11:17:14.561705+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:21.600689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.535546+0000","last_change":"2024-05-14T11:17:13.520481+0000","last_active":"2024-05-14T11:17:14.535546+0000","last_peered":"2024-05-14T11:17:14.535546+0000","last_clean":"2024-05-14T11:17:14.535546+0000","last_became_active":"2024-05-14T11:17:13.520154+0000","last_became_peered":"2024-05-14T11:17:13.520154+0000","last_unstale":"2024-05-14T11:17:14.535546+0000","last_undegraded":"2024-05-14T11:17:14.535546+0000","last_fullsized":"2024-05-14T11:17:14.535546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:47.296773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.544543+0000","last_change":"2024-05-14T11:17:14.537035+0000","last_active":"2024-05-14T11:17:14.544543+0000","last_peered":"2024-05-14T11:17:14.544543+0000","last_clean":"2024-05-14T11:17:14.544543+0000","last_became_active":"2024-05-14T11:17:14.536319+0000","last_became_peered":"2024-05-14T11:17:14.536319+0000","last_unstale":"2024-05-14T11:17:14.544543+0000","last_undegraded":"2024-05-14T11:17:14.544543+0000","last_fullsized":"2024-05-14T11:17:14.544543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:08:19.703304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.519655+0000","last_change":"2024-05-14T11:17:13.525724+0000","last_active":"2024-05-14T11:17:14.519655+0000","last_peered":"2024-05-14T11:17:14.519655+0000","last_clean":"2024-05-14T11:17:14.519655+0000","last_became_active":"2024-05-14T11:17:13.525417+0000","last_became_peered":"2024-05-14T11:17:13.525417+0000","last_unstale":"2024-05-14T11:17:14.519655+0000","last_undegraded":"2024-05-14T11:17:14.519655+0000","last_fullsized":"2024-05-14T11:17:14.519655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:58.483791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.559316+0000","last_change":"2024-05-14T11:17:14.552971+0000","last_active":"2024-05-14T11:17:14.559316+0000","last_peered":"2024-05-14T11:17:14.559316+0000","last_clean":"2024-05-14T11:17:14.559316+0000","last_became_active":"2024-05-14T11:17:14.552604+0000","last_became_peered":"2024-05-14T11:17:14.552604+0000","last_unstale":"2024-05-14T11:17:14.559316+0000","last_undegraded":"2024-05-14T11:17:14.559316+0000","last_fullsized":"2024-05-14T11:17:14.559316+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:02.130378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.565391+0000","last_change":"2024-05-14T11:17:13.529960+0000","last_active":"2024-05-14T11:17:14.565391+0000","last_peered":"2024-05-14T11:17:14.565391+0000","last_clean":"2024-05-14T11:17:14.565391+0000","last_became_active":"2024-05-14T11:17:13.529643+0000","last_became_peered":"2024-05-14T11:17:13.529643+0000","last_unstale":"2024-05-14T11:17:14.565391+0000","last_undegraded":"2024-05-14T11:17:14.565391+0000","last_fullsized":"2024-05-14T11:17:14.565391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:22.007689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.552818+0000","last_change":"2024-05-14T11:17:14.549932+0000","last_active":"2024-05-14T11:17:14.552818+0000","last_peered":"2024-05-14T11:17:14.552818+0000","last_clean":"2024-05-14T11:17:14.552818+0000","last_became_active":"2024-05-14T11:17:14.549128+0000","last_became_peered":"2024-05-14T11:17:14.549128+0000","last_unstale":"2024-05-14T11:17:14.552818+0000","last_undegraded":"2024-05-14T11:17:14.552818+0000","last_fullsized":"2024-05-14T11:17:14.552818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:35:20.068069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.520549+0000","last_change":"2024-05-14T11:17:13.533692+0000","last_active":"2024-05-14T11:17:14.520549+0000","last_peered":"2024-05-14T11:17:14.520549+0000","last_clean":"2024-05-14T11:17:14.520549+0000","last_became_active":"2024-05-14T11:17:13.533367+0000","last_became_peered":"2024-05-14T11:17:13.533367+0000","last_unstale":"2024-05-14T11:17:14.520549+0000","last_undegraded":"2024-05-14T11:17:14.520549+0000","last_fullsized":"2024-05-14T11:17:14.520549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:56:00.482471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.538642+0000","last_change":"2024-05-14T11:17:14.533285+0000","last_active":"2024-05-14T11:17:14.538642+0000","last_peered":"2024-05-14T11:17:14.538642+0000","last_clean":"2024-05-14T11:17:14.538642+0000","last_became_active":"2024-05-14T11:17:14.532919+0000","last_became_peered":"2024-05-14T11:17:14.532919+0000","last_unstale":"2024-05-14T11:17:14.538642+0000","last_undegraded":"2024-05-14T11:17:14.538642+0000","last_fullsized":"2024-05-14T11:17:14.538642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:26.369946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525856+0000","last_change":"2024-05-14T11:17:13.521159+0000","last_active":"2024-05-14T11:17:14.525856+0000","last_peered":"2024-05-14T11:17:14.525856+0000","last_clean":"2024-05-14T11:17:14.525856+0000","last_became_active":"2024-05-14T11:17:13.520863+0000","last_became_peered":"2024-05-14T11:17:13.520863+0000","last_unstale":"2024-05-14T11:17:14.525856+0000","last_undegraded":"2024-05-14T11:17:14.525856+0000","last_fullsized":"2024-05-14T11:17:14.525856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:49.218531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537164+0000","last_change":"2024-05-14T11:17:14.531820+0000","last_active":"2024-05-14T11:17:14.537164+0000","last_peered":"2024-05-14T11:17:14.537164+0000","last_clean":"2024-05-14T11:17:14.537164+0000","last_became_active":"2024-05-14T11:17:14.531479+0000","last_became_peered":"2024-05-14T11:17:14.531479+0000","last_unstale":"2024-05-14T11:17:14.537164+0000","last_undegraded":"2024-05-14T11:17:14.537164+0000","last_fullsized":"2024-05-14T11:17:14.537164+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:20:39.639906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.525613+0000","last_change":"2024-05-14T11:17:13.524118+0000","last_active":"2024-05-14T11:17:14.525613+0000","last_peered":"2024-05-14T11:17:14.525613+0000","last_clean":"2024-05-14T11:17:14.525613+0000","last_became_active":"2024-05-14T11:17:13.523786+0000","last_became_peered":"2024-05-14T11:17:13.523786+0000","last_unstale":"2024-05-14T11:17:14.525613+0000","last_undegraded":"2024-05-14T11:17:14.525613+0000","last_fullsized":"2024-05-14T11:17:14.525613+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:12:40.127085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.545784+0000","last_change":"2024-05-14T11:17:14.542166+0000","last_active":"2024-05-14T11:17:14.545784+0000","last_peered":"2024-05-14T11:17:14.545784+0000","last_clean":"2024-05-14T11:17:14.545784+0000","last_became_active":"2024-05-14T11:17:14.541273+0000","last_became_peered":"2024-05-14T11:17:14.541273+0000","last_unstale":"2024-05-14T11:17:14.545784+0000","last_undegraded":"2024-05-14T11:17:14.545784+0000","last_fullsized":"2024-05-14T11:17:14.545784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:59:48.305335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.524376+0000","last_change":"2024-05-14T11:17:13.529060+0000","last_active":"2024-05-14T11:17:14.524376+0000","last_peered":"2024-05-14T11:17:14.524376+0000","last_clean":"2024-05-14T11:17:14.524376+0000","last_became_active":"2024-05-14T11:17:13.524550+0000","last_became_peered":"2024-05-14T11:17:13.524550+0000","last_unstale":"2024-05-14T11:17:14.524376+0000","last_undegraded":"2024-05-14T11:17:14.524376+0000","last_fullsized":"2024-05-14T11:17:14.524376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:55.537589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.539884+0000","last_change":"2024-05-14T11:17:14.530629+0000","last_active":"2024-05-14T11:17:14.539884+0000","last_peered":"2024-05-14T11:17:14.539884+0000","last_clean":"2024-05-14T11:17:14.539884+0000","last_became_active":"2024-05-14T11:17:14.530279+0000","last_became_peered":"2024-05-14T11:17:14.530279+0000","last_unstale":"2024-05-14T11:17:14.539884+0000","last_undegraded":"2024-05-14T11:17:14.539884+0000","last_fullsized":"2024-05-14T11:17:14.539884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:03:25.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.524574+0000","last_change":"2024-05-14T11:17:13.536338+0000","last_active":"2024-05-14T11:17:14.524574+0000","last_peered":"2024-05-14T11:17:14.524574+0000","last_clean":"2024-05-14T11:17:14.524574+0000","last_became_active":"2024-05-14T11:17:13.535745+0000","last_became_peered":"2024-05-14T11:17:13.535745+0000","last_unstale":"2024-05-14T11:17:14.524574+0000","last_undegraded":"2024-05-14T11:17:14.524574+0000","last_fullsized":"2024-05-14T11:17:14.524574+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:47.194388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.549970+0000","last_change":"2024-05-14T11:17:14.547300+0000","last_active":"2024-05-14T11:17:14.549970+0000","last_peered":"2024-05-14T11:17:14.549970+0000","last_clean":"2024-05-14T11:17:14.549970+0000","last_became_active":"2024-05-14T11:17:14.547007+0000","last_became_peered":"2024-05-14T11:17:14.547007+0000","last_unstale":"2024-05-14T11:17:14.549970+0000","last_undegraded":"2024-05-14T11:17:14.549970+0000","last_fullsized":"2024-05-14T11:17:14.549970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:23:06.020615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.523500+0000","last_change":"2024-05-14T11:17:13.531988+0000","last_active":"2024-05-14T11:17:14.523500+0000","last_peered":"2024-05-14T11:17:14.523500+0000","last_clean":"2024-05-14T11:17:14.523500+0000","last_became_active":"2024-05-14T11:17:13.531629+0000","last_became_peered":"2024-05-14T11:17:13.531629+0000","last_unstale":"2024-05-14T11:17:14.523500+0000","last_undegraded":"2024-05-14T11:17:14.523500+0000","last_fullsized":"2024-05-14T11:17:14.523500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:41.108476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.564674+0000","last_change":"2024-05-14T11:17:14.563007+0000","last_active":"2024-05-14T11:17:14.564674+0000","last_peered":"2024-05-14T11:17:14.564674+0000","last_clean":"2024-05-14T11:17:14.564674+0000","last_became_active":"2024-05-14T11:17:14.562754+0000","last_became_peered":"2024-05-14T11:17:14.562754+0000","last_unstale":"2024-05-14T11:17:14.564674+0000","last_undegraded":"2024-05-14T11:17:14.564674+0000","last_fullsized":"2024-05-14T11:17:14.564674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:13.500057+0000","last_clean_scrub_stamp":"2024-05-14T11:17:13.500057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:56:54.178936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.537303+0000","last_change":"2024-05-14T11:17:13.527350+0000","last_active":"2024-05-14T11:17:14.537303+0000","last_peered":"2024-05-14T11:17:14.537303+0000","last_clean":"2024-05-14T11:17:14.537303+0000","last_became_active":"2024-05-14T11:17:13.527016+0000","last_became_peered":"2024-05-14T11:17:13.527016+0000","last_unstale":"2024-05-14T11:17:14.537303+0000","last_undegraded":"2024-05-14T11:17:14.537303+0000","last_fullsized":"2024-05-14T11:17:14.537303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:12.491495+0000","last_clean_scrub_stamp":"2024-05-14T11:17:12.491495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:01.121816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":139,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-14T11:17:14.534538+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:14.534538+0000","last_peered":"2024-05-14T11:17:14.534538+0000","last_clean":"2024-05-14T11:17:14.534538+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:14.534538+0000","last_undegraded":"2024-05-14T11:17:14.534538+0000","last_fullsized":"2024-05-14T11:17:14.534538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":775418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":183972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":773770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:17:19.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:17:19.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:17:19.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.248+0000 7f52f3e00640 1 Processor -- start 2024-05-14T11:17:19.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.248+0000 7f52f3e00640 1 -- start start 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52f41601e0 0x7f52f41605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f406dfa0 0x7f52f406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f52f4148d00 0x7f52f4149110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f52f406f730 con 0x7f52f41601e0 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f52f406e8f0 con 0x7f52f4148d00 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52f4160ca0 con 0x7f52f406dfa0 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52f41601e0 0x7f52f41605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52f41601e0 0x7f52f41605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42416/0 (socket says 172.21.15.116:42416) 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 -- 172.21.15.116:0/3178732517 learned_addr learned my addr 172.21.15.116:0/3178732517 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52ea000640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f406dfa0 0x7f52f406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f3400640 1 --2- 172.21.15.116:0/3178732517 >> v2:172.21.15.149:3300/0 conn(0x7f52f4148d00 0x7f52f4149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 -- 172.21.15.116:0/3178732517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f406dfa0 msgr2=0x7f52f406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f406dfa0 0x7f52f406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 -- 172.21.15.116:0/3178732517 >> v2:172.21.15.149:3300/0 conn(0x7f52f4148d00 msgr2=0x7f52f4149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 --2- 172.21.15.116:0/3178732517 >> v2:172.21.15.149:3300/0 conn(0x7f52f4148d00 0x7f52f4149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.252+0000 7f52f2a00640 1 -- 172.21.15.116:0/3178732517 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f4160f80 con 0x7f52f41601e0 2024-05-14T11:17:19.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f2a00640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f41601e0 0x7f52f41605f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52f40b51e0 tx=0x7f52e401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d6b97fbf0185b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f2000640 1 -- 172.21.15.116:0/3178732517 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40aa660 con 0x7f52f41601e0 2024-05-14T11:17:19.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f2000640 1 -- 172.21.15.116:0/3178732517 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e40af070 con 0x7f52f41601e0 2024-05-14T11:17:19.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f2000640 1 -- 172.21.15.116:0/3178732517 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40aaa20 con 0x7f52f41601e0 2024-05-14T11:17:19.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 -- 172.21.15.116:0/3178732517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f41601e0 msgr2=0x7f52f41605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f41601e0 0x7f52f41605f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52f40b51e0 tx=0x7f52e401cf80 comp rx=0 tx=0).stop 2024-05-14T11:17:19.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 -- 172.21.15.116:0/3178732517 shutdown_connections 2024-05-14T11:17:19.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 --2- 172.21.15.116:0/3178732517 >> v2:172.21.15.149:3300/0 conn(0x7f52f4148d00 0x7f52f4149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f406dfa0 0x7f52f406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 --2- 172.21.15.116:0/3178732517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f41601e0 0x7f52f41605f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 -- 172.21.15.116:0/3178732517 >> 172.21.15.116:0/3178732517 conn(0x7f52f40731e0 msgr2=0x7f52f40735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:19.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 -- 172.21.15.116:0/3178732517 shutdown_connections 2024-05-14T11:17:19.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.256+0000 7f52f3e00640 1 -- 172.21.15.116:0/3178732517 wait complete. 2024-05-14T11:17:19.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 Processor -- start 2024-05-14T11:17:19.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- start start 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f4148d00 0x7f52f415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f41601e0 0x7f52f4161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52f4074320 con 0x7f52f406dfa0 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40741a0 con 0x7f52f4148d00 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52f4163fe0 con 0x7f52f41601e0 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42420/0 (socket says 172.21.15.116:42420) 2024-05-14T11:17:19.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 -- 172.21.15.116:0/742028849 learned_addr learned my addr 172.21.15.116:0/742028849 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52ea000640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f4148d00 0x7f52f415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3400640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f41601e0 0x7f52f4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 -- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f41601e0 msgr2=0x7f52f4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f41601e0 0x7f52f4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 -- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f4148d00 msgr2=0x7f52f415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f4148d00 0x7f52f415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 -- 172.21.15.116:0/742028849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f4144050 con 0x7f52f406dfa0 2024-05-14T11:17:19.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f2a00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52e401abd0 tx=0x7f52e401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40bd5f0 con 0x7f52f406dfa0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e40bd790 con 0x7f52f406dfa0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f4144330 con 0x7f52f406dfa0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40bdb80 con 0x7f52f406dfa0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.260+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f41448f0 con 0x7f52f406dfa0 2024-05-14T11:17:19.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f40508f0 con 0x7f52f406dfa0 2024-05-14T11:17:19.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52e40b33f0 con 0x7f52f406dfa0 2024-05-14T11:17:19.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52eb400640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 0x7f52d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f52e4129ca0 con 0x7f52f406dfa0 2024-05-14T11:17:19.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52ea000640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 0x7f52d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.264+0000 7f52ea000640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 0x7f52d0071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52f406e520 tx=0x7f52dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.268+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f52e40c3060 con 0x7f52f406dfa0 2024-05-14T11:17:19.483 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:17:19.480+0000 7fd9b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:17:19.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.484+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f52f40d14e0 con 0x7f52f406dfa0 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.484+0000 7f52eb400640 1 -- 172.21.15.116:0/742028849 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f52e4016110 con 0x7f52f406dfa0 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:17:19.488 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:17:19.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 msgr2=0x7f52d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 0x7f52d0071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52f406e520 tx=0x7f52dc008040 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 msgr2=0x7f52f41457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52e401abd0 tx=0x7f52e401ad60 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 shutdown_connections 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52d006f430 0x7f52d0071940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f4148d00 0x7f52f415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52f41601e0 0x7f52f4161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 --2- 172.21.15.116:0/742028849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52f406dfa0 0x7f52f41457c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 >> 172.21.15.116:0/742028849 conn(0x7f52f40731e0 msgr2=0x7f52f40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 shutdown_connections 2024-05-14T11:17:19.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.488+0000 7f52f3e00640 1 -- 172.21.15.116:0/742028849 wait complete. 2024-05-14T11:17:19.510 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.656+0000 7fcadb600640 1 Processor -- start 2024-05-14T11:17:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.656+0000 7fcadb600640 1 -- start start 2024-05-14T11:17:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcadb600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcad40b0770 0x7fcad406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcadb600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad4072a30 0x7fcad4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcadb600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad406cc40 con 0x7fcad406f730 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcadb600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcad406db20 con 0x7fcad40b0770 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcadb600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcad406cf10 con 0x7fcad4072a30 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcad40b0770 0x7fcad406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcad40b0770 0x7fcad406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42432/0 (socket says 172.21.15.116:42432) 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 -- 172.21.15.116:0/879301678 learned_addr learned my addr 172.21.15.116:0/879301678 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad9600640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad4072a30 0x7fcad4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 -- 172.21.15.116:0/879301678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad4072a30 msgr2=0x7fcad4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad4072a30 0x7fcad4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 -- 172.21.15.116:0/879301678 >> v1:172.21.15.149:6789/0 conn(0x7fcad406f730 legacy=0x7fcad406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 -- 172.21.15.116:0/879301678 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad406d1f0 con 0x7fcad40b0770 2024-05-14T11:17:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.660+0000 7fcad8c00640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad40b0770 0x7fcad406c700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009870 tx=0x7fcac401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50ed356cc721798 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcad3e00640 1 -- 172.21.15.116:0/879301678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40aa600 con 0x7fcad40b0770 2024-05-14T11:17:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcad3e00640 1 -- 172.21.15.116:0/879301678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac40af070 con 0x7fcad40b0770 2024-05-14T11:17:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcad3e00640 1 -- 172.21.15.116:0/879301678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40aa9c0 con 0x7fcad40b0770 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- 172.21.15.116:0/879301678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad40b0770 msgr2=0x7fcad406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad40b0770 0x7fcad406c700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009870 tx=0x7fcac401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- 172.21.15.116:0/879301678 shutdown_connections 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad4072a30 0x7fcad4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 --2- 172.21.15.116:0/879301678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad40b0770 0x7fcad406c700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- 172.21.15.116:0/879301678 >> 172.21.15.116:0/879301678 conn(0x7fcad40b4620 msgr2=0x7fcad40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- 172.21.15.116:0/879301678 shutdown_connections 2024-05-14T11:17:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- 172.21.15.116:0/879301678 wait complete. 2024-05-14T11:17:19.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 Processor -- start 2024-05-14T11:17:19.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.664+0000 7fcadb600640 1 -- start start 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 0x7fcad4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad40b0770 0x7fcad4168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40d17f0 con 0x7fcad406f730 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40d1670 con 0x7fcad4072a30 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40d1970 con 0x7fcad40b0770 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42446/0 (socket says 172.21.15.116:42446) 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 -- 172.21.15.116:0/4046233780 learned_addr learned my addr 172.21.15.116:0/4046233780 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcacbe00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 0x7fcad4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad9600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad40b0770 0x7fcad4168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 -- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad40b0770 msgr2=0x7fcad4168fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad40b0770 0x7fcad4168fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 -- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 msgr2=0x7fcad4154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 0x7fcad4154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 -- 172.21.15.116:0/4046233780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad415a320 con 0x7fcad406f730 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcacbe00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 0x7fcad4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad8c00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcac40ac040 tx=0x7fcac401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40bd5f0 con 0x7fcad406f730 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac40bd790 con 0x7fcad406f730 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad415a5e0 con 0x7fcad406f730 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40bdb80 con 0x7fcad406f730 2024-05-14T11:17:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad415aba0 con 0x7fcad406f730 2024-05-14T11:17:19.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.668+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad406a9c0 con 0x7fcad406f730 2024-05-14T11:17:19.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.672+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcac40a9070 con 0x7fcad406f730 2024-05-14T11:17:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.672+0000 7fcad1600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 0x7fcab0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.672+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fcac4129de0 con 0x7fcad406f730 2024-05-14T11:17:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.672+0000 7fcacbe00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 0x7fcab0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:19.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.672+0000 7fcacbe00640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 0x7fcab0071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcad40b2900 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:19.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.676+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcac40c3060 con 0x7fcad406f730 2024-05-14T11:17:19.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.864+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcad4074d30 con 0x7fcad406f730 2024-05-14T11:17:19.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.864+0000 7fcad1600640 1 -- 172.21.15.116:0/4046233780 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fcac4016150 con 0x7fcad406f730 2024-05-14T11:17:19.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:19.870 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 msgr2=0x7fcab0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 0x7fcab0071910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcad40b2900 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 msgr2=0x7fcad415e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcac40ac040 tx=0x7fcac401ace0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 shutdown_connections 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcab006f400 0x7fcab0071910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad4072a30 0x7fcad4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad40b0770 0x7fcad4168fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 --2- 172.21.15.116:0/4046233780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcad406f730 0x7fcad415e870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 >> 172.21.15.116:0/4046233780 conn(0x7fcad40b4620 msgr2=0x7fcad40b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 shutdown_connections 2024-05-14T11:17:19.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:19.868+0000 7fcadb600640 1 -- 172.21.15.116:0/4046233780 wait complete. 2024-05-14T11:17:20.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:17:13.909695+0000', 'modified': '2024-05-14T11:17:16.938510+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4926': {'gid': 4926, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6837/2525866958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 2525866958}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 2525866958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4926, 'qdb_cluster': [4926]} 2024-05-14T11:17:20.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:17:20.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:17:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.440+0000 7f97c9400640 1 Processor -- start 2024-05-14T11:17:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.440+0000 7f97c9400640 1 -- start start 2024-05-14T11:17:20.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c9400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97c4156fa0 0x7f97c406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c40d1670 0x7f97c40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c9400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c40d1fc0 con 0x7f97c406e9e0 2024-05-14T11:17:20.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c9400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f97c41573b0 con 0x7f97c4156fa0 2024-05-14T11:17:20.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f97c4161120 con 0x7f97c40d1670 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c3200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f97c406e9e0 0x7f97c406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56954/0 (socket says 172.21.15.116:56954) 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c2800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97c4156fa0 0x7f97c406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c3200640 1 -- 172.21.15.116:0/3602331376 learned_addr learned my addr 172.21.15.116:0/3602331376 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97b9e00640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c40d1670 0x7f97c40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c1e00640 1 -- 172.21.15.116:0/3602331376 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593284421 0 0) 0x7f97bc001120 con 0x7f97c406e9e0 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c1e00640 1 -- 172.21.15.116:0/3602331376 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97a8003530 con 0x7f97c406e9e0 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c2800640 1 -- 172.21.15.116:0/3602331376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c40d1670 msgr2=0x7f97c40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c2800640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c40d1670 0x7f97c40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c2800640 1 -- 172.21.15.116:0/3602331376 >> v1:172.21.15.116:6789/0 conn(0x7f97c406e9e0 legacy=0x7f97c406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.444+0000 7f97c2800640 1 -- 172.21.15.116:0/3602331376 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c4160510 con 0x7f97c4156fa0 2024-05-14T11:17:20.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c2800640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c4156fa0 0x7f97c406ffe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009d90 tx=0x7f97b401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d994a7dbfa6edf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c1e00640 1 -- 172.21.15.116:0/3602331376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40b2070 con 0x7f97c4156fa0 2024-05-14T11:17:20.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c1e00640 1 -- 172.21.15.116:0/3602331376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b40a5a00 con 0x7f97c4156fa0 2024-05-14T11:17:20.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c1e00640 1 -- 172.21.15.116:0/3602331376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40a5dd0 con 0x7f97c4156fa0 2024-05-14T11:17:20.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- 172.21.15.116:0/3602331376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c4156fa0 msgr2=0x7f97c406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c4156fa0 0x7f97c406ffe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009d90 tx=0x7f97b401c940 comp rx=0 tx=0).stop 2024-05-14T11:17:20.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- 172.21.15.116:0/3602331376 shutdown_connections 2024-05-14T11:17:20.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c40d1670 0x7f97c40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 --2- 172.21.15.116:0/3602331376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c4156fa0 0x7f97c406ffe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- 172.21.15.116:0/3602331376 >> 172.21.15.116:0/3602331376 conn(0x7f97c40b4620 msgr2=0x7f97c40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- 172.21.15.116:0/3602331376 shutdown_connections 2024-05-14T11:17:20.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- 172.21.15.116:0/3602331376 wait complete. 2024-05-14T11:17:20.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 Processor -- start 2024-05-14T11:17:20.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.448+0000 7f97c9400640 1 -- start start 2024-05-14T11:17:20.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 0x7f97c414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97c4156fa0 0x7f97c4073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4070520 con 0x7f97c40d1670 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40b07a0 con 0x7f97c4156fa0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f97c40706a0 con 0x7f97c406e9e0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42464/0 (socket says 172.21.15.116:42464) 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 -- 172.21.15.116:0/1512040762 learned_addr learned my addr 172.21.15.116:0/1512040762 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c2800640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 0x7f97c414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c3200640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97c4156fa0 0x7f97c4073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 -- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 msgr2=0x7f97c414e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 0x7f97c414e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 -- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97c4156fa0 msgr2=0x7f97c4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97c4156fa0 0x7f97c4073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 -- 172.21.15.116:0/1512040762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c415f910 con 0x7f97c40d1670 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c2800640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 0x7f97c414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00d5f0 tx=0x7f97ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97bb400640 1 -- 172.21.15.116:0/1512040762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0125f0 con 0x7f97c40d1670 2024-05-14T11:17:20.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97bb400640 1 -- 172.21.15.116:0/1512040762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac019070 con 0x7f97c40d1670 2024-05-14T11:17:20.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97bb400640 1 -- 172.21.15.116:0/1512040762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0129b0 con 0x7f97c40d1670 2024-05-14T11:17:20.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c415fbf0 con 0x7f97c40d1670 2024-05-14T11:17:20.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.452+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c4151530 con 0x7f97c40d1670 2024-05-14T11:17:20.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97bb400640 1 -- 172.21.15.116:0/1512040762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f97ac012b50 con 0x7f97c40d1670 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97bb400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 0x7f97a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97bb400640 1 -- 172.21.15.116:0/1512040762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f97ac08f920 con 0x7f97c40d1670 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97c2800640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 0x7f97a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 Processor -- start 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 -- start start 2024-05-14T11:17:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97c2800640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 0x7f97a0071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97b40af000 tx=0x7f97b40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87a7ce7c0 0x55a87ad08320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87abb6f80 0x55a87ad08860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a87aba5620 con 0x55a87aba06b0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a87aac0740 con 0x55a87a7ce7c0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.456+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a87ab0e090 con 0x55a87abb6f80 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42468/0 (socket says 172.21.15.116:42468) 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 -- 172.21.15.116:0/3647118328 learned_addr learned my addr 172.21.15.116:0/3647118328 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c2800640 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87a7ce7c0 0x55a87ad08320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97b9e00640 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87abb6f80 0x55a87ad08860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 -- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87abb6f80 msgr2=0x55a87ad08860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87abb6f80 0x55a87ad08860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 -- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87a7ce7c0 msgr2=0x55a87ad08320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87a7ce7c0 0x55a87ad08320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 -- 172.21.15.116:0/3647118328 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87aca20a0 con 0x55a87aba06b0 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97c3200640 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001500 tx=0x7f97bc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b4b3f008502c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97b3000640 1 -- 172.21.15.116:0/3647118328 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc019070 con 0x55a87aba06b0 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97b3000640 1 -- 172.21.15.116:0/3647118328 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc005ad0 con 0x55a87aba06b0 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97b3000640 1 -- 172.21.15.116:0/3647118328 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0070d0 con 0x55a87aba06b0 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- 172.21.15.116:0/3647118328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 msgr2=0x55a87ad08da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001500 tx=0x7f97bc0019c0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- 172.21.15.116:0/3647118328 shutdown_connections 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad08da0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87abb6f80 0x55a87ad08860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 --2- 172.21.15.116:0/3647118328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87a7ce7c0 0x55a87ad08320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- 172.21.15.116:0/3647118328 >> 172.21.15.116:0/3647118328 conn(0x55a87ac8d340 msgr2=0x55a87acbcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- 172.21.15.116:0/3647118328 shutdown_connections 2024-05-14T11:17:20.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- 172.21.15.116:0/3647118328 wait complete. 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 -- start start 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.460+0000 7f97d0a74000 1 Processor -- start 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acc33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87aba06b0 0x55a87acc38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a87ab0a050 con 0x55a87abb6f80 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a87aac6ec0 con 0x55a87aba06b0 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a87aaadc90 con 0x55a87a7ce7c0 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 10 client.0 _ll_register_callbacks cb 0x55a87aca9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42484/0 (socket says 172.21.15.116:42484) 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 -- 172.21.15.116:0/573395963 learned_addr learned my addr 172.21.15.116:0/573395963 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c2800640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acc33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97b9e00640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87aba06b0 0x55a87acc38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 -- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 msgr2=0x55a87acc33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acc33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 -- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87aba06b0 msgr2=0x55a87acc38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87aba06b0 0x55a87acc38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 -- 172.21.15.116:0/573395963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87abb86f0 con 0x55a87abb6f80 2024-05-14T11:17:20.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c3200640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001a00 tx=0x7f97bc001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69031f84f409df68 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc022020 con 0x55a87abb6f80 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f97bc007480 con 0x55a87abb6f80 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97bc005ad0 con 0x55a87abb6f80 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f97bc02c050 con 0x55a87abb6f80 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 10 client.5082 fetch_fsmap learned FSMap version 18 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55a87abb46a0 con 0x55a87abb6f80 2024-05-14T11:17:20.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc005ad0 con 0x55a87abb6f80 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f97bc02f020 con 0x55a87abb6f80 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 10 client.5082 fetch_fsmap finished waiting for FSMap version 18 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 10 client.5082 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 20 client.5082 populate_metadata read hostname 'smithi116' 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7b37d0 0x55a87ad198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 4 client.5082 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:17:20.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 --> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87aaa6be0 con 0x55a87a7b37d0 2024-05-14T11:17:20.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c2800640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7b37d0 0x55a87ad198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97c2800640 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7b37d0 0x55a87ad198e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf358d05ace3258 server_cookie=de41e490770577bd in_seq=0 out_seq=0 2024-05-14T11:17:20.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.464+0000 7f97a7400640 10 client.5082 ms_handle_connect on v2:172.21.15.116:6836/2525866958 2024-05-14T11:17:20.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97a7400640 1 -- 172.21.15.116:0/573395963 <== mds.0 v2:172.21.15.116:6836/2525866958 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a87aaa6be0 con 0x55a87a7b37d0 2024-05-14T11:17:20.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97a7400640 10 client.5082 handle_command_reply: tid=0 2024-05-14T11:17:20.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 client.5082 shutdown 2024-05-14T11:17:20.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 msgr2=0x55a87aced770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001a00 tx=0x7f97bc001360 comp rx=0 tx=0).stop 2024-05-14T11:17:20.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 shutdown_connections 2024-05-14T11:17:20.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7b37d0 0x55a87ad198e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87aba06b0 0x55a87acc38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97a7400640 0 client.5082 ms_handle_reset on v2:172.21.15.116:6836/2525866958 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87abb6f80 0x55a87aced770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 --2- 172.21.15.116:0/573395963 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acc33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 >> 172.21.15.116:0/573395963 conn(0x55a87ac8d340 msgr2=0x55a87acbd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 shutdown_connections 2024-05-14T11:17:20.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 1 -- 172.21.15.116:0/573395963 wait complete. 2024-05-14T11:17:20.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.468+0000 7f97d0a74000 20 client.5082 trim_cache size 0 max 16384 2024-05-14T11:17:20.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.484+0000 7f97d0a74000 1 Processor -- start 2024-05-14T11:17:20.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.484+0000 7f97d0a74000 1 -- start start 2024-05-14T11:17:20.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87acbd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4205920 con 0x55a87aba06b0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4155d40 con 0x55a87abb6f80 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f97c4156390 con 0x55a87a7ce7c0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42486/0 (socket says 172.21.15.116:42486) 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 -- 172.21.15.116:0/1486185551 learned_addr learned my addr 172.21.15.116:0/1486185551 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97c3200640 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87acbd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97c2800640 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 -- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 msgr2=0x55a87acece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 -- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 msgr2=0x55a87acbd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87acbd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 -- 172.21.15.116:0/1486185551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87aae7af0 con 0x55a87aba06b0 2024-05-14T11:17:20.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00dfb0 tx=0x7f97ac007380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d64749e06ce979 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b2600640 1 -- 172.21.15.116:0/1486185551 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac08a070 con 0x55a87aba06b0 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b2600640 1 -- 172.21.15.116:0/1486185551 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac05bee0 con 0x55a87aba06b0 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97b2600640 1 -- 172.21.15.116:0/1486185551 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac060290 con 0x55a87aba06b0 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- 172.21.15.116:0/1486185551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 msgr2=0x55a87aced370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00dfb0 tx=0x7f97ac007380 comp rx=0 tx=0).stop 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- 172.21.15.116:0/1486185551 shutdown_connections 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87acbd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87aced370 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1486185551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87acece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- 172.21.15.116:0/1486185551 >> 172.21.15.116:0/1486185551 conn(0x55a87ac8d340 msgr2=0x55a87acec720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- 172.21.15.116:0/1486185551 shutdown_connections 2024-05-14T11:17:20.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- 172.21.15.116:0/1486185551 wait complete. 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 -- start start 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.488+0000 7f97d0a74000 1 Processor -- start 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87ad7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87ad357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a87ab17c30 con 0x55a87aba06b0 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a87aab17d0 con 0x55a87abb6f80 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a87aaa8220 con 0x55a87a7ce7c0 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 10 client.0 _ll_register_callbacks cb 0x7f97bc017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42490/0 (socket says 172.21.15.116:42490) 2024-05-14T11:17:20.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 -- 172.21.15.116:0/1226913358 learned_addr learned my addr 172.21.15.116:0/1226913358 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97c2800640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87ad7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97c3200640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87ad357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 -- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 msgr2=0x55a87ad7a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87ad7a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 -- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 msgr2=0x55a87ad357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87ad357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 -- 172.21.15.116:0/1226913358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87aae6e40 con 0x55a87aba06b0 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97b9e00640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f97ac057f60 tx=0x7f97ac087000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26accc299aa721b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:17:20.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac060290 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f97ac07c070 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac05bee0 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f97ac058170 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 10 client.5088 fetch_fsmap learned FSMap version 18 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55a87ab8e4e0 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac05bee0 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f97ac013070 con 0x55a87aba06b0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 10 client.5088 fetch_fsmap finished waiting for FSMap version 18 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 10 client.5088 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 20 client.5088 populate_metadata read hostname 'smithi116' 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7a1830 0x55a87ad36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 4 client.5088 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 --> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] -- command(tid 0: {"prefix": "status"}) -- 0x55a87ab1bca0 con 0x55a87a7a1830 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97c2800640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7a1830 0x55a87ad36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97c2800640 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7a1830 0x55a87ad36160 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f25cab297a9021 server_cookie=e4e3c7b1b90b4aba in_seq=0 out_seq=0 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.492+0000 7f97a7400640 10 client.5088 ms_handle_connect on v2:172.21.15.116:6836/2525866958 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97a7400640 1 -- 172.21.15.116:0/1226913358 <== mds.0 v2:172.21.15.116:6836/2525866958 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55a87ab1bca0 con 0x55a87a7a1830 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97a7400640 10 client.5088 handle_command_reply: tid=0 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 client.5088 shutdown 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 msgr2=0x55a87ad7aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f97ac057f60 tx=0x7f97ac087000 comp rx=0 tx=0).stop 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 shutdown_connections 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:6836/2525866958,v1:172.21.15.116:6837/2525866958] conn(0x55a87a7a1830 0x55a87ad36160 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a87abb6f80 0x55a87ad357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a87aba06b0 0x55a87ad7aea0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97a7400640 0 client.5088 ms_handle_reset on v2:172.21.15.116:6836/2525866958 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 --2- 172.21.15.116:0/1226913358 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a87a7ce7c0 0x55a87ad7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 >> 172.21.15.116:0/1226913358 conn(0x55a87ac8d340 msgr2=0x55a87ad14d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 shutdown_connections 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 1 -- 172.21.15.116:0/1226913358 wait complete. 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97d0a74000 20 client.5088 trim_cache size 0 max 16384 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:17:20.501 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4926, 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 5.9641165020000004, 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 18, 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 38, 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 38, 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 12.8244475 2024-05-14T11:17:20.502 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 msgr2=0x7f97a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 0x7f97a0071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97b40af000 tx=0x7f97b40a7070 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 msgr2=0x7f97c40733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00d5f0 tx=0x7f97ac00dab0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 shutdown_connections 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f97a006f400 0x7f97a0071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97c4156fa0 0x7f97c4073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97c40d1670 0x7f97c40733d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 --2- 172.21.15.116:0/1512040762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97c406e9e0 0x7f97c414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 >> 172.21.15.116:0/1512040762 conn(0x7f97c40b4620 msgr2=0x7f97c4150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 shutdown_connections 2024-05-14T11:17:20.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.496+0000 7f97c9400640 1 -- 172.21.15.116:0/1512040762 wait complete. 2024-05-14T11:17:20.517 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:17:20.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 Processor -- start 2024-05-14T11:17:20.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 -- start start 2024-05-14T11:17:20.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fec680cedd0 0x7fec6806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6806ec60 con 0x7fec6815fec0 2024-05-14T11:17:20.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6806ef40 con 0x7fec680d2230 2024-05-14T11:17:20.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec6dc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fec6806f220 con 0x7fec680cedd0 2024-05-14T11:17:20.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec5fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fec6815fec0 0x7fec680d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56968/0 (socket says 172.21.15.116:56968) 2024-05-14T11:17:20.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec5fe00640 1 -- 172.21.15.116:0/3214869028 learned_addr learned my addr 172.21.15.116:0/3214869028 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.664+0000 7fec66a00640 1 --2- 172.21.15.116:0/3214869028 >> v2:172.21.15.149:3300/0 conn(0x7fec680cedd0 0x7fec6806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576727787 0 0) 0x7fec6806ec60 con 0x7fec6815fec0 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec4c003530 con 0x7fec6815fec0 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638061227 0 0) 0x7fec4c003530 con 0x7fec6815fec0 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 >> v2:172.21.15.149:3300/0 conn(0x7fec680cedd0 msgr2=0x7fec6806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 --2- 172.21.15.116:0/3214869028 >> v2:172.21.15.149:3300/0 conn(0x7fec680cedd0 0x7fec6806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 >> v1:172.21.15.149:6789/0 conn(0x7fec680d2230 legacy=0x7fec680b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6806f500 con 0x7fec6815fec0 2024-05-14T11:17:20.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fec50001720 con 0x7fec6815fec0 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 >> v1:172.21.15.116:6789/0 conn(0x7fec6815fec0 legacy=0x7fec680d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec4c003a60 0x7fec4c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c004810 0x7fec4c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fec6806f220 con 0x7fec4c005360 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c00c290 con 0x7fec4c004810 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c00c490 con 0x7fec4c003a60 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec5fe00640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c004810 0x7fec4c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec66a00640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec4c003a60 0x7fec4c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 -- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec4c003a60 msgr2=0x7fec4c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec4c003a60 0x7fec4c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 -- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c004810 msgr2=0x7fec4c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:20.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c004810 0x7fec4c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.668+0000 7fec67400640 1 -- 172.21.15.116:0/3214869028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c003530 con 0x7fec4c005360 2024-05-14T11:17:20.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec67400640 1 -- 172.21.15.116:0/3214869028 reap_dead start 2024-05-14T11:17:20.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec67400640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec4c00b9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec60001e90 tx=0x7fec6001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f7c2a4983ab4a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60001ae0 con 0x7fec4c005360 2024-05-14T11:17:20.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60003b80 con 0x7fec4c005360 2024-05-14T11:17:20.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec66000640 1 -- 172.21.15.116:0/3214869028 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec600050d0 con 0x7fec4c005360 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 msgr2=0x7fec4c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec4c00b9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec60001e90 tx=0x7fec6001c720 comp rx=0 tx=0).stop 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- 172.21.15.116:0/3214869028 shutdown_connections 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3214869028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec4c00b9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- 172.21.15.116:0/3214869028 >> 172.21.15.116:0/3214869028 conn(0x7fec680d3670 msgr2=0x7fec680d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- 172.21.15.116:0/3214869028 shutdown_connections 2024-05-14T11:17:20.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- 172.21.15.116:0/3214869028 wait complete. 2024-05-14T11:17:20.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 Processor -- start 2024-05-14T11:17:20.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.672+0000 7fec6dc00640 1 -- start start 2024-05-14T11:17:20.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec68160f70 0x7fec68159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec6815e9f0 0x7fec6815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fec680cf1e0 con 0x7fec4c005360 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fec681604e0 con 0x7fec68160f70 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fec680cf360 con 0x7fec6815e9f0 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42506/0 (socket says 172.21.15.116:42506) 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 -- 172.21.15.116:0/3065610316 learned_addr learned my addr 172.21.15.116:0/3065610316 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec5fe00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec68160f70 0x7fec68159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec67400640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec6815e9f0 0x7fec6815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 -- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec6815e9f0 msgr2=0x7fec6815ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec6815e9f0 0x7fec6815ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 -- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec68160f70 msgr2=0x7fec68159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec68160f70 0x7fec68159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 -- 172.21.15.116:0/3065610316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec681635d0 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec66a00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec58011990 tx=0x7fec58011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800fa30 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5800fbd0 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58014440 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6815a110 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec68160870 con 0x7fec4c005360 2024-05-14T11:17:20.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.676+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6806a9c0 con 0x7fec4c005360 2024-05-14T11:17:20.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.680+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec58012040 con 0x7fec4c005360 2024-05-14T11:17:20.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.680+0000 7fec5ea00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 0x7fec44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:20.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.680+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fec5808f830 con 0x7fec4c005360 2024-05-14T11:17:20.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.680+0000 7fec5fe00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 0x7fec44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:20.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.680+0000 7fec5fe00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 0x7fec44071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec680b3be0 tx=0x7fec50007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:20.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.684+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fec58019030 con 0x7fec4c005360 2024-05-14T11:17:20.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fec6806e770 con 0x7fec4c005360 2024-05-14T11:17:20.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec5ea00640 1 -- 172.21.15.116:0/3065610316 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fec5805c5c0 con 0x7fec4c005360 2024-05-14T11:17:20.887 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:17:20.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 msgr2=0x7fec44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 0x7fec44071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec680b3be0 tx=0x7fec50007980 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 msgr2=0x7fec68164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec58011990 tx=0x7fec58011d90 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 shutdown_connections 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fec4406f400 0x7fec44071910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec68160f70 0x7fec68159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fec6815e9f0 0x7fec6815ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 --2- 172.21.15.116:0/3065610316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c005360 0x7fec68164750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 >> 172.21.15.116:0/3065610316 conn(0x7fec680d3670 msgr2=0x7fec68071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 shutdown_connections 2024-05-14T11:17:20.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:20.884+0000 7fec6dc00640 1 -- 172.21.15.116:0/3065610316 wait complete. 2024-05-14T11:17:20.908 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:17:20.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:17:20.908 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:17:20.908 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:17:20.908 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:20.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:17:20.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:17:20.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:17:20.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:17:20.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:20.909 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:17:20.918 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:17:20.918 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:17:20.918 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:17:20.919 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42362sec preferred_lft 42362sec 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:17:20.920 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:17:20.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:20.921 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:17:20.977 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:17:20.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:20.980 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:17:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:17:21.033 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:17:21.033 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:17:21.033 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:17:21.033 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:17:21.033 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:17:21.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:17:21.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:21.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:21.165 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:17:21.220 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:17:21.220 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:17:21.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:17:21.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:21.222 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:17:21.288 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:17:21.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:17:21.288 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:21.340 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:17:21.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:17:21.341 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:21.396 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:17:21.457 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:17:21.510 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:17:21.511 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:17:21.512 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:17:21.513 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:17:21.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:21.514 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:17:21.565 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:17:21.565 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:17:21.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:17:21.631 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:17:21.682 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:17:21.683 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:17:21.684 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:17:21.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:21.685 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:17:21.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:17:21.716+0000 7f98d95d9580 -1 init, newargv = 0x55a04c1680f0 newargc=15 2024-05-14T11:17:21.718 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[37058]: starting ceph client 2024-05-14T11:17:21.725 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:17:21.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[37058]: starting fuse 2024-05-14T11:17:21.727 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:21.727 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:17:21.728 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:17:21.729 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:21.848 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.37058.asok 2024-05-14T11:17:21.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:21.859 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37058.asok 2024-05-14T11:17:21.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:21.859 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37058.asok status 2024-05-14T11:17:22.037 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:17:22.037 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:17:22.037 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "37058", 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5100, 2024-05-14T11:17:22.038 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 5100 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3600722032 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3600722032 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:22.039 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.5100 192.168.0.1:0/3600722032", 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/3600722032", 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 18, 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 38, 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:17:22.040 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:17:22.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.052 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:17:22.052 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:22.061 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:17:22.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:22.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:22.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.124 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-14T11:17:22.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.185 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-14T11:17:22.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.240 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-14T11:17:22.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.296 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-14T11:17:22.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.356 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-14T11:17:22.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:22.408 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./ 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./rfile 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./parentdir 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./parentdir/pfile 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./parentdir/childdir 2024-05-14T11:17:22.473 INFO:teuthology.orchestra.run.smithi116.stdout:./parentdir/childdir/cfile 2024-05-14T11:17:22.474 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:22.474 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:22.474 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:22.475 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:22.475 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-14T11:17:22.475 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:22.588 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:17:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:22.597 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:22.597 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:22.597 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:22.598 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:22.598 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-14T11:17:22.598 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:22.707 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627778 2024-05-14T11:17:22.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:22.716 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:22.716 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:22.717 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:22.717 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:22.717 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-14T11:17:22.717 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:22.824 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:17:22.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-14T11:17:22.834 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:22.941 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628279 2024-05-14T11:17:22.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-14T11:17:22.950 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:23.056 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:17:23.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-14T11:17:23.065 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:17:23.175 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628280 2024-05-14T11:17:23.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:17:23.185 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:23.185 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:23.241 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:17:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.428+0000 7fb190e00640 1 Processor -- start 2024-05-14T11:17:23.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.428+0000 7fb190e00640 1 -- start start 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb190e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb18c156fa0 0x7fb18c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb190e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c160510 con 0x7fb18c0af380 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb190e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18c1607f0 con 0x7fb18c0d1670 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb190e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb18c1573b0 con 0x7fb18c156fa0 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb18c156fa0 0x7fb18c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb18a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb18c0af380 0x7fb18c0af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56976/0 (socket says 172.21.15.116:56976) 2024-05-14T11:17:23.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb18a800640 1 -- 172.21.15.116:0/1252477738 learned_addr learned my addr 172.21.15.116:0/1252477738 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:23.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189e00640 1 -- 172.21.15.116:0/1252477738 >> v1:172.21.15.149:6789/0 conn(0x7fb18c0d1670 legacy=0x7fb18c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189e00640 1 -- 172.21.15.116:0/1252477738 >> v1:172.21.15.116:6789/0 conn(0x7fb18c0af380 legacy=0x7fb18c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189e00640 1 -- 172.21.15.116:0/1252477738 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c160ad0 con 0x7fb18c156fa0 2024-05-14T11:17:23.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189400640 1 -- 172.21.15.116:0/1252477738 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789721983 0 0) 0x7fb18c160510 con 0x7fb18c0af380 2024-05-14T11:17:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.432+0000 7fb189e00640 1 --2- 172.21.15.116:0/1252477738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c156fa0 0x7fb18c06ffe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0097c0 tx=0x7fb17c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbea7a31ce63683 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb189400640 1 -- 172.21.15.116:0/1252477738 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b1070 con 0x7fb18c156fa0 2024-05-14T11:17:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb189400640 1 -- 172.21.15.116:0/1252477738 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c0ac410 con 0x7fb18c156fa0 2024-05-14T11:17:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb189400640 1 -- 172.21.15.116:0/1252477738 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0ac7e0 con 0x7fb18c156fa0 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- 172.21.15.116:0/1252477738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c156fa0 msgr2=0x7fb18c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 --2- 172.21.15.116:0/1252477738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c156fa0 0x7fb18c06ffe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0097c0 tx=0x7fb17c01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- 172.21.15.116:0/1252477738 shutdown_connections 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 --2- 172.21.15.116:0/1252477738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c156fa0 0x7fb18c06ffe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- 172.21.15.116:0/1252477738 >> 172.21.15.116:0/1252477738 conn(0x7fb18c0b4620 msgr2=0x7fb18c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- 172.21.15.116:0/1252477738 shutdown_connections 2024-05-14T11:17:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- 172.21.15.116:0/1252477738 wait complete. 2024-05-14T11:17:23.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 Processor -- start 2024-05-14T11:17:23.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.436+0000 7fb190e00640 1 -- start start 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb18c0d1670 0x7fb18c14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c156fa0 0x7fb18c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c06ca90 con 0x7fb18c0af380 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c06c910 con 0x7fb18c156fa0 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb18c06cc10 con 0x7fb18c0d1670 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42522/0 (socket says 172.21.15.116:42522) 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 -- 172.21.15.116:0/922379830 learned_addr learned my addr 172.21.15.116:0/922379830 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:23.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb18a800640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c156fa0 0x7fb18c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb181400640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb18c0d1670 0x7fb18c14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 -- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb18c0d1670 msgr2=0x7fb18c14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb18c0d1670 0x7fb18c14a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 -- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c156fa0 msgr2=0x7fb18c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c156fa0 0x7fb18c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 -- 172.21.15.116:0/922379830 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c09ae60 con 0x7fb18c0af380 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb189e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0190b0 tx=0x7fb17c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b1040 con 0x7fb18c0af380 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c0bb070 con 0x7fb18c0af380 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c14af20 con 0x7fb18c0af380 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0b5630 con 0x7fb18c0af380 2024-05-14T11:17:23.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c14b4e0 con 0x7fb18c0af380 2024-05-14T11:17:23.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.440+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c06a9c0 con 0x7fb18c0af380 2024-05-14T11:17:23.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.444+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb17c0b57d0 con 0x7fb18c0af380 2024-05-14T11:17:23.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.444+0000 7fb182a00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 0x7fb168071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.444+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb17c128fa0 con 0x7fb18c0af380 2024-05-14T11:17:23.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.444+0000 7fb181400640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 0x7fb168071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.444+0000 7fb181400640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 0x7fb168071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb18c1fcfe0 tx=0x7fb174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.448+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb17c0c5020 con 0x7fb18c0af380 2024-05-14T11:17:23.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.616+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb18c06fdc0 con 0x7fb18c0af380 2024-05-14T11:17:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb182a00640 1 -- 172.21.15.116:0/922379830 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7fb17c016090 con 0x7fb18c0af380 2024-05-14T11:17:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 msgr2=0x7fb168071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 0x7fb168071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb18c1fcfe0 tx=0x7fb174009040 comp rx=0 tx=0).stop 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 msgr2=0x7fb18c09a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0190b0 tx=0x7fb17c019240 comp rx=0 tx=0).stop 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 shutdown_connections 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb16806f400 0x7fb168071910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb18c156fa0 0x7fb18c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb18c0d1670 0x7fb18c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 --2- 172.21.15.116:0/922379830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb18c0af380 0x7fb18c09a920 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 >> 172.21.15.116:0/922379830 conn(0x7fb18c0b4620 msgr2=0x7fb18c153490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 shutdown_connections 2024-05-14T11:17:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.620+0000 7fb190e00640 1 -- 172.21.15.116:0/922379830 wait complete. 2024-05-14T11:17:23.643 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:23.643 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:23.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:17:23.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.836+0000 7f8ec9400640 1 Processor -- start 2024-05-14T11:17:23.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.836+0000 7f8ec9400640 1 -- start start 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec4156fa0 0x7f8ec406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec9400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ec40d1670 0x7f8ec40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec9400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec40d1fc0 con 0x7f8ec406e9e0 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec9400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec41573b0 con 0x7f8ec40d1670 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec4161120 con 0x7f8ec4156fa0 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ec40d1670 0x7f8ec40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ec40d1670 0x7f8ec40d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42528/0 (socket says 172.21.15.116:42528) 2024-05-14T11:17:23.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3232740228 learned_addr learned my addr 172.21.15.116:0/3232740228 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ec2800640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec4156fa0 0x7f8ec406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3232740228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec4156fa0 msgr2=0x7f8ec406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec4156fa0 0x7f8ec406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3232740228 >> v1:172.21.15.149:6789/0 conn(0x7f8ec406e9e0 legacy=0x7f8ec406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.840+0000 7f8ebbe00640 1 -- 172.21.15.116:0/3232740228 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4160510 con 0x7f8ec40d1670 2024-05-14T11:17:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec40d1a80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00bb10 tx=0x7f8eac01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49df917ac246176 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec1e00640 1 -- 172.21.15.116:0/3232740228 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b4070 con 0x7f8ec40d1670 2024-05-14T11:17:23.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec1e00640 1 -- 172.21.15.116:0/3232740228 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0a7b70 con 0x7f8ec40d1670 2024-05-14T11:17:23.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec1e00640 1 -- 172.21.15.116:0/3232740228 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0af440 con 0x7f8ec40d1670 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 -- 172.21.15.116:0/3232740228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 msgr2=0x7f8ec40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec40d1a80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00bb10 tx=0x7f8eac01eba0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 -- 172.21.15.116:0/3232740228 shutdown_connections 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec40d1a80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 --2- 172.21.15.116:0/3232740228 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec4156fa0 0x7f8ec406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 -- 172.21.15.116:0/3232740228 >> 172.21.15.116:0/3232740228 conn(0x7f8ec40b4620 msgr2=0x7f8ec40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 -- 172.21.15.116:0/3232740228 shutdown_connections 2024-05-14T11:17:23.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.844+0000 7f8ec9400640 1 -- 172.21.15.116:0/3232740228 wait complete. 2024-05-14T11:17:23.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 Processor -- start 2024-05-14T11:17:23.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- start start 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec406e9e0 0x7f8ec414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 0x7f8ec415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4070520 con 0x7f8ec40d1670 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40b07a0 con 0x7f8ec4156fa0 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec40706a0 con 0x7f8ec406e9e0 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42532/0 (socket says 172.21.15.116:42532) 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2890819291 learned_addr learned my addr 172.21.15.116:0/2890819291 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec2800640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec406e9e0 0x7f8ec414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec3200640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 0x7f8ec415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec406e9e0 msgr2=0x7f8ec414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec406e9e0 0x7f8ec414c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 msgr2=0x7f8ec415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 0x7f8ec415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 -- 172.21.15.116:0/2890819291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4152990 con 0x7f8ec40d1670 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec3200640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 0x7f8ec415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebbe00640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b2c0 tx=0x7f8eac01b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b4030 con 0x7f8ec40d1670 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac0b6070 con 0x7f8ec40d1670 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0ad7b0 con 0x7f8ec40d1670 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec41518f0 con 0x7f8ec40d1670 2024-05-14T11:17:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.848+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec4151eb0 con 0x7f8ec40d1670 2024-05-14T11:17:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec406a9c0 con 0x7f8ec40d1670 2024-05-14T11:17:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8eac0c1060 con 0x7f8ec40d1670 2024-05-14T11:17:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ebaa00640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 0x7f8ea0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8eac129870 con 0x7f8ec40d1670 2024-05-14T11:17:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ec2800640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 0x7f8ea0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.852+0000 7f8ec2800640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 0x7f8ea0071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40b0d60 tx=0x7f8eb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:23.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:23.856+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8ec4152990 con 0x7f8ec40d1670 2024-05-14T11:17:24.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.028+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ec40b3f80 con 0x7f8ec40d1670 2024-05-14T11:17:24.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.028+0000 7f8ebaa00640 1 -- 172.21.15.116:0/2890819291 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) ==== 80+0+0 (secure 0 0 0) 0x7f8eac018820 con 0x7f8ec40d1670 2024-05-14T11:17:24.032 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 msgr2=0x7f8ea0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 0x7f8ea0071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40b0d60 tx=0x7f8eb4008040 comp rx=0 tx=0).stop 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 msgr2=0x7f8ec4154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b2c0 tx=0x7f8eac01b9a0 comp rx=0 tx=0).stop 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 shutdown_connections 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8ea006f430 0x7f8ea0071940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ec4156fa0 0x7f8ec415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ec40d1670 0x7f8ec4154f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 --2- 172.21.15.116:0/2890819291 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ec406e9e0 0x7f8ec414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 >> 172.21.15.116:0/2890819291 conn(0x7f8ec40b4620 msgr2=0x7f8ec406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 shutdown_connections 2024-05-14T11:17:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:24.032+0000 7f8ec9400640 1 -- 172.21.15.116:0/2890819291 wait complete. 2024-05-14T11:17:24.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:17:24.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:24.054 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:17:24.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[37058]: fuse finished with error 0 and tester_r 0 2024-05-14T11:17:24.074 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:30.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:17:30.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:17:30.177 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:17:30.187 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:30.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.384+0000 7fa0ce000640 1 Processor -- start 2024-05-14T11:17:30.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.384+0000 7fa0ce000640 1 -- start start 2024-05-14T11:17:30.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 0x7fa0c815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0ce000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0c81549f0 0x7fa0c80b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0ce000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0c80706f0 con 0x7fa0c80b75b0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0ce000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c80b62c0 con 0x7fa0c81549f0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0ce000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c80b79c0 con 0x7fa0c80702e0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0c81549f0 0x7fa0c80b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0c81549f0 0x7fa0c80b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41906/0 (socket says 172.21.15.116:41906) 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 -- 172.21.15.116:0/2150443237 learned_addr learned my addr 172.21.15.116:0/2150443237 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 -- 172.21.15.116:0/2150443237 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 msgr2=0x7fa0c815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c7400640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 0x7fa0c815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 0x7fa0c815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 -- 172.21.15.116:0/2150443237 >> v1:172.21.15.149:6789/0 conn(0x7fa0c80b75b0 legacy=0x7fa0c81615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c6a00640 1 -- 172.21.15.116:0/2150443237 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c80b59d0 con 0x7fa0c81549f0 2024-05-14T11:17:30.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.388+0000 7fa0c7400640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 0x7fa0c815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:30.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0c6a00640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c80b7070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec12b64dd0bed74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0c6000640 1 -- 172.21.15.116:0/2150443237 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b0070 con 0x7fa0c81549f0 2024-05-14T11:17:30.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0c6000640 1 -- 172.21.15.116:0/2150443237 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b40ab600 con 0x7fa0c81549f0 2024-05-14T11:17:30.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0c6000640 1 -- 172.21.15.116:0/2150443237 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40ab9d0 con 0x7fa0c81549f0 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 -- 172.21.15.116:0/2150443237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 msgr2=0x7fa0c80b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c80b7070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b401c380 comp rx=0 tx=0).stop 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 -- 172.21.15.116:0/2150443237 shutdown_connections 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c80b7070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 --2- 172.21.15.116:0/2150443237 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80702e0 0x7fa0c815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 -- 172.21.15.116:0/2150443237 >> 172.21.15.116:0/2150443237 conn(0x7fa0c8071270 msgr2=0x7fa0c8071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 -- 172.21.15.116:0/2150443237 shutdown_connections 2024-05-14T11:17:30.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 -- 172.21.15.116:0/2150443237 wait complete. 2024-05-14T11:17:30.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.392+0000 7fa0ce000640 1 Processor -- start 2024-05-14T11:17:30.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- start start 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0c80702e0 0x7fa0c8146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80b75b0 0x7fa0c8159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c80cd9f0 con 0x7fa0c81549f0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8070970 con 0x7fa0c80702e0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c80cdb70 con 0x7fa0c80b75b0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c6a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80b75b0 0x7fa0c8159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41914/0 (socket says 172.21.15.116:41914) 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 -- 172.21.15.116:0/3053471070 learned_addr learned my addr 172.21.15.116:0/3053471070 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7400640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0c80702e0 0x7fa0c8146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 -- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80b75b0 msgr2=0x7fa0c8159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80b75b0 0x7fa0c8159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 -- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0c80702e0 msgr2=0x7fa0c8146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0c80702e0 0x7fa0c8146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 -- 172.21.15.116:0/3053471070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c815a6a0 con 0x7fa0c81549f0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0c7e00640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000be60 tx=0x7fa0c000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0019070 con 0x7fa0c81549f0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c0014440 con 0x7fa0c81549f0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0014870 con 0x7fa0c81549f0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c815a980 con 0x7fa0c81549f0 2024-05-14T11:17:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.396+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c815af40 con 0x7fa0c81549f0 2024-05-14T11:17:30.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c806a9c0 con 0x7fa0c81549f0 2024-05-14T11:17:30.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa0c001c070 con 0x7fa0c81549f0 2024-05-14T11:17:30.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0bf400640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 0x7fa0a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa0c008f360 con 0x7fa0c81549f0 2024-05-14T11:17:30.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0c7400640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 0x7fa0a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.400+0000 7fa0c7400640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 0x7fa0a8071940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006fd0 tx=0x7fa0b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.404+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa0c005ba60 con 0x7fa0c81549f0 2024-05-14T11:17:30.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.604+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c815a430 con 0x7fa0c81549f0 2024-05-14T11:17:30.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.604+0000 7fa0bf400640 1 -- 172.21.15.116:0/3053471070 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fa0c005fd60 con 0x7fa0c81549f0 2024-05-14T11:17:30.609 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:30.609 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 msgr2=0x7fa0a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 0x7fa0a8071940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006fd0 tx=0x7fa0b8008040 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 msgr2=0x7fa0c8159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000be60 tx=0x7fa0c000bf50 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 shutdown_connections 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa0a806f430 0x7fa0a8071940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c81549f0 0x7fa0c8159e60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa0c80b75b0 0x7fa0c8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 --2- 172.21.15.116:0/3053471070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0c80702e0 0x7fa0c8146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 >> 172.21.15.116:0/3053471070 conn(0x7fa0c8071270 msgr2=0x7fa0c815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 shutdown_connections 2024-05-14T11:17:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.608+0000 7fa0ce000640 1 -- 172.21.15.116:0/3053471070 wait complete. 2024-05-14T11:17:30.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-14T11:17:30.806 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:17:30.807 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:17:30.807 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:30.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 Processor -- start 2024-05-14T11:17:30.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 -- start start 2024-05-14T11:17:30.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc31415db30 0x7fc3140aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc31406c700 0x7fc31406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc31406ff20 con 0x7fc3140d1670 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc3140ae100 con 0x7fc31415db30 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc31a600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc3140ae870 con 0x7fc31406c700 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc31415db30 0x7fc3140aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc30aa00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc3140d1670 0x7fc3140d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35830/0 (socket says 172.21.15.116:35830) 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc30aa00640 1 -- 172.21.15.116:0/1143114438 learned_addr learned my addr 172.21.15.116:0/1143114438 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313e00640 1 --2- 172.21.15.116:0/1143114438 >> v2:172.21.15.149:3300/0 conn(0x7fc31406c700 0x7fc31406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313400640 1 -- 172.21.15.116:0/1143114438 >> v2:172.21.15.149:3300/0 conn(0x7fc31406c700 msgr2=0x7fc31406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313400640 1 --2- 172.21.15.116:0/1143114438 >> v2:172.21.15.149:3300/0 conn(0x7fc31406c700 0x7fc31406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313400640 1 -- 172.21.15.116:0/1143114438 >> v1:172.21.15.116:6789/0 conn(0x7fc3140d1670 legacy=0x7fc3140d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc313400640 1 -- 172.21.15.116:0/1143114438 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc314070470 con 0x7fc31415db30 2024-05-14T11:17:30.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.952+0000 7fc312a00640 1 -- 172.21.15.116:0/1143114438 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291934880 0 0) 0x7fc31406ff20 con 0x7fc3140d1670 2024-05-14T11:17:30.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc313400640 1 --2- 172.21.15.116:0/1143114438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc31415db30 0x7fc3140aeaa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3141626a0 tx=0x7fc30401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f62254c0b87a56 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc312a00640 1 -- 172.21.15.116:0/1143114438 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040b0070 con 0x7fc31415db30 2024-05-14T11:17:30.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc312a00640 1 -- 172.21.15.116:0/1143114438 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3040ab600 con 0x7fc31415db30 2024-05-14T11:17:30.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc312a00640 1 -- 172.21.15.116:0/1143114438 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040ab9d0 con 0x7fc31415db30 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 -- 172.21.15.116:0/1143114438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc31415db30 msgr2=0x7fc3140aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 --2- 172.21.15.116:0/1143114438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc31415db30 0x7fc3140aeaa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3141626a0 tx=0x7fc30401c3b0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 -- 172.21.15.116:0/1143114438 shutdown_connections 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 --2- 172.21.15.116:0/1143114438 >> v2:172.21.15.149:3300/0 conn(0x7fc31406c700 0x7fc31406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 --2- 172.21.15.116:0/1143114438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc31415db30 0x7fc3140aeaa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 -- 172.21.15.116:0/1143114438 >> 172.21.15.116:0/1143114438 conn(0x7fc3140b4620 msgr2=0x7fc3140b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 -- 172.21.15.116:0/1143114438 shutdown_connections 2024-05-14T11:17:30.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.956+0000 7fc31a600640 1 -- 172.21.15.116:0/1143114438 wait complete. 2024-05-14T11:17:30.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 Processor -- start 2024-05-14T11:17:30.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 -- start start 2024-05-14T11:17:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc31406c700 0x7fc314148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 0x7fc314167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc31416a770 con 0x7fc3140d1670 2024-05-14T11:17:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc31415df40 con 0x7fc31406c700 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc31416a8f0 con 0x7fc31415db30 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41938/0 (socket says 172.21.15.116:41938) 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 -- 172.21.15.116:0/4073141914 learned_addr learned my addr 172.21.15.116:0/4073141914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc313400640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc31406c700 0x7fc314148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc313e00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 0x7fc314167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 -- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 msgr2=0x7fc314167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 0x7fc314167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 -- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc31406c700 msgr2=0x7fc314148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc31406c700 0x7fc314148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 -- 172.21.15.116:0/4073141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc314159980 con 0x7fc3140d1670 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc313e00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 0x7fc314167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30aa00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c430 tx=0x7fc2fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0074e0 con 0x7fc3140d1670 2024-05-14T11:17:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0047f0 con 0x7fc3140d1670 2024-05-14T11:17:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.960+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc314159c60 con 0x7fc3140d1670 2024-05-14T11:17:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc004be0 con 0x7fc3140d1670 2024-05-14T11:17:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc314148ed0 con 0x7fc3140d1670 2024-05-14T11:17:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31406a9c0 con 0x7fc3140d1670 2024-05-14T11:17:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc2fc011070 con 0x7fc3140d1670 2024-05-14T11:17:30.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc30be00640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 0x7fc2f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:30.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc2fc08efd0 con 0x7fc3140d1670 2024-05-14T11:17:30.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.964+0000 7fc313400640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 0x7fc2f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:30.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.968+0000 7fc313400640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 0x7fc2f0071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ad000 tx=0x7fc3040a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:30.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:30.968+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc2fc05bd30 con 0x7fc3140d1670 2024-05-14T11:17:31.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.164+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc314159e30 con 0x7fc3140d1670 2024-05-14T11:17:31.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.164+0000 7fc30be00640 1 -- 172.21.15.116:0/4073141914 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fc2fc05b630 con 0x7fc3140d1670 2024-05-14T11:17:31.168 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:31.168 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:31.168 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:31.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 msgr2=0x7fc2f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:31.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 0x7fc2f0071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ad000 tx=0x7fc3040a6000 comp rx=0 tx=0).stop 2024-05-14T11:17:31.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 msgr2=0x7fc314166b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:31.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c430 tx=0x7fc2fc00c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:31.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 shutdown_connections 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc2f006f400 0x7fc2f0071910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc31415db30 0x7fc314167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3140d1670 0x7fc314166b40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 --2- 172.21.15.116:0/4073141914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc31406c700 0x7fc314148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 >> 172.21.15.116:0/4073141914 conn(0x7fc3140b4620 msgr2=0x7fc31415b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 shutdown_connections 2024-05-14T11:17:31.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:31.168+0000 7fc31a600640 1 -- 172.21.15.116:0/4073141914 wait complete. 2024-05-14T11:17:31.190 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-05-14T11:17:31.360 INFO:teuthology.orchestra.run.smithi116.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-14T11:17:31.360 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-14T11:17:40.978 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:17:40.982 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:17:41.362 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-14T11:17:41.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.424+0000 7f30a6315080 1 Processor -- start 2024-05-14T11:17:41.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.424+0000 7f30a6315080 1 -- start start 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc511410 0x5573bc5e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5573bc4b5aa0 con 0x5573bc53bbd0 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5573bc510a30 con 0x5573bc559bb0 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5573bc5e8820 con 0x5573bc511410 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45964/0 (socket says 172.21.15.116:45964) 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 -- 172.21.15.116:0/2083484153 learned_addr learned my addr 172.21.15.116:0/2083484153 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309fe00640 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309f400640 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc511410 0x5573bc5e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 -- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc511410 msgr2=0x5573bc5e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc511410 0x5573bc5e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 -- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 msgr2=0x5573bc5e4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5e4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 -- 172.21.15.116:0/2083484153 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bc5e8ac0 con 0x5573bc53bbd0 2024-05-14T11:17:41.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309ea00640 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f308c00a460 tx=0x7f308c00a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de813e20742a0b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309d600640 1 -- 172.21.15.116:0/2083484153 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00b920 con 0x5573bc53bbd0 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309d600640 1 -- 172.21.15.116:0/2083484153 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308c00f070 con 0x5573bc53bbd0 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f309d600640 1 -- 172.21.15.116:0/2083484153 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00bd20 con 0x5573bc53bbd0 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- 172.21.15.116:0/2083484153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 msgr2=0x5573bc5e47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f308c00a460 tx=0x7f308c00a920 comp rx=0 tx=0).stop 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- 172.21.15.116:0/2083484153 shutdown_connections 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc53bbd0 0x5573bc5e47d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 --2- 172.21.15.116:0/2083484153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc511410 0x5573bc5e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- 172.21.15.116:0/2083484153 >> 172.21.15.116:0/2083484153 conn(0x5573bc4fae10 msgr2=0x5573bc512620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- 172.21.15.116:0/2083484153 shutdown_connections 2024-05-14T11:17:41.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.428+0000 7f30a6315080 1 -- 172.21.15.116:0/2083484153 wait complete. 2024-05-14T11:17:41.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 Processor -- start 2024-05-14T11:17:41.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 -- start start 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 0x5573bc556630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc53bbd0 0x5573bc5ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5573bc5e22b0 con 0x5573bc511410 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5573bc5e2010 con 0x5573bc559bb0 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f30a6315080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5573bc5e8d60 con 0x5573bc53bbd0 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 0x5573bc556630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 0x5573bc556630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45968/0 (socket says 172.21.15.116:45968) 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 -- 172.21.15.116:0/2490560664 learned_addr learned my addr 172.21.15.116:0/2490560664 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309ea00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc53bbd0 0x5573bc5ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309fe00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 -- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc53bbd0 msgr2=0x5573bc5ec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5573bc53bbd0 0x5573bc5ec7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 -- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 msgr2=0x5573bc5ecce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573bc559bb0 0x5573bc5ecce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 -- 172.21.15.116:0/2490560664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bc5c9be0 con 0x5573bc511410 2024-05-14T11:17:41.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f309f400640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 0x5573bc556630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3090008790 tx=0x7f3090008c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f3097e00640 1 -- 172.21.15.116:0/2490560664 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309001b070 con 0x5573bc511410 2024-05-14T11:17:41.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f3097e00640 1 -- 172.21.15.116:0/2490560664 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3090006d20 con 0x5573bc511410 2024-05-14T11:17:41.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.432+0000 7f3097e00640 1 -- 172.21.15.116:0/2490560664 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090016680 con 0x5573bc511410 2024-05-14T11:17:41.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573bc5b4e20 con 0x5573bc511410 2024-05-14T11:17:41.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bc5b5360 con 0x5573bc511410 2024-05-14T11:17:41.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f3097e00640 1 -- 172.21.15.116:0/2490560664 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f309001e070 con 0x5573bc511410 2024-05-14T11:17:41.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f3097e00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308006dde0 0x7f30800702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f3097e00640 1 -- 172.21.15.116:0/2490560664 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f309008e0e0 con 0x5573bc511410 2024-05-14T11:17:41.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f309ea00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308006dde0 0x7f30800702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f30a6315080 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5573bc5b6d50 0x5573bc5b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x5573bc5b33c0 con 0x5573bc5b6d50 2024-05-14T11:17:41.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.436+0000 7f309fe00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5573bc5b6d50 0x5573bc5b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f309ea00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308006dde0 0x7f30800702f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f308c00a960 tx=0x7f308c002000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f309fe00640 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5573bc5b6d50 0x5573bc5b2e80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f309fe00640 1 -- 172.21.15.116:0/2490560664 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f30980aaa70 con 0x5573bc5b6d50 2024-05-14T11:17:41.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5573bc5b6d50 msgr2=0x5573bc5b2e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5573bc5b6d50 0x5573bc5b2e80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308006dde0 msgr2=0x7f30800702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308006dde0 0x7f30800702f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f308c00a960 tx=0x7f308c002000 comp rx=0 tx=0).stop 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 msgr2=0x5573bc556630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 --2- 172.21.15.116:0/2490560664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5573bc511410 0x5573bc556630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3090008790 tx=0x7f3090008c50 comp rx=0 tx=0).stop 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f309fe00640 1 -- 172.21.15.116:0/2490560664 reap_dead start 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 shutdown_connections 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 >> 172.21.15.116:0/2490560664 conn(0x5573bc4fae10 msgr2=0x5573bc5e9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 shutdown_connections 2024-05-14T11:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.440+0000 7f30a6315080 1 -- 172.21.15.116:0/2490560664 wait complete. 2024-05-14T11:17:41.447 INFO:teuthology.orchestra.run.smithi116.stdout:mytag 2024-05-14T11:17:41.448 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-14T11:17:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 Processor -- start 2024-05-14T11:17:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 -- start start 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b6dde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b6de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55742b62e610 con 0x55742b574410 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55742b518aa0 con 0x55742b5bcbb0 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb728708080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55742b62e790 con 0x55742b59ebd0 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45988/0 (socket says 172.21.15.116:45988) 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 -- 172.21.15.116:0/3478301811 learned_addr learned my addr 172.21.15.116:0/3478301811 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 -- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 msgr2=0x55742b6dde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b6dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 -- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 msgr2=0x55742b6de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b6de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 -- 172.21.15.116:0/3478301811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742b571550 con 0x55742b574410 2024-05-14T11:17:41.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.504+0000 7fb721800640 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b1f0 tx=0x7fb71400b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927b853d8ce55e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb71b400640 1 -- 172.21.15.116:0/3478301811 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400a040 con 0x55742b574410 2024-05-14T11:17:41.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb71b400640 1 -- 172.21.15.116:0/3478301811 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71400f620 con 0x55742b574410 2024-05-14T11:17:41.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb71b400640 1 -- 172.21.15.116:0/3478301811 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400fa10 con 0x55742b574410 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- 172.21.15.116:0/3478301811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 msgr2=0x55742b571f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b1f0 tx=0x7fb71400b6b0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- 172.21.15.116:0/3478301811 shutdown_connections 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b6de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b6dde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- 172.21.15.116:0/3478301811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b571f80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- 172.21.15.116:0/3478301811 >> 172.21.15.116:0/3478301811 conn(0x55742b55de10 msgr2=0x55742b59db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- 172.21.15.116:0/3478301811 shutdown_connections 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- 172.21.15.116:0/3478301811 wait complete. 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 Processor -- start 2024-05-14T11:17:41.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- start start 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b5b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b571f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b5724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55742b6451d0 con 0x55742b574410 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55742b645470 con 0x55742b5bcbb0 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb728708080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55742b5717f0 con 0x55742b59ebd0 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb721800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b5b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb721800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b5b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45996/0 (socket says 172.21.15.116:45996) 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb721800640 1 -- 172.21.15.116:0/2435060579 learned_addr learned my addr 172.21.15.116:0/2435060579 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb722200640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b5724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.508+0000 7fb720e00640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b571f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 -- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 msgr2=0x55742b571f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55742b59ebd0 0x55742b571f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 -- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 msgr2=0x55742b5724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55742b5bcbb0 0x55742b5724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 -- 172.21.15.116:0/2435060579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742b645e70 con 0x55742b574410 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb721800640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b5b9e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb71400bbb0 tx=0x7fb714000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb71a000640 1 -- 172.21.15.116:0/2435060579 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714020070 con 0x55742b574410 2024-05-14T11:17:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb71a000640 1 -- 172.21.15.116:0/2435060579 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71401c890 con 0x55742b574410 2024-05-14T11:17:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55742b646110 con 0x55742b574410 2024-05-14T11:17:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb71a000640 1 -- 172.21.15.116:0/2435060579 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71401cc80 con 0x55742b574410 2024-05-14T11:17:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742b646650 con 0x55742b574410 2024-05-14T11:17:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb71a000640 1 -- 172.21.15.116:0/2435060579 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb714023070 con 0x55742b574410 2024-05-14T11:17:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.512+0000 7fb71a000640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb70406de10 0x7fb704070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb71a000640 1 -- 172.21.15.116:0/2435060579 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fb71408fed0 con 0x55742b574410 2024-05-14T11:17:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb720e00640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb70406de10 0x7fb704070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55742b646af0 0x55742b646f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55742b647440 con 0x55742b646af0 2024-05-14T11:17:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb722200640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55742b646af0 0x55742b646f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb720e00640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb70406de10 0x7fb704070320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb710005fd0 tx=0x7fb71000e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb722200640 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55742b646af0 0x55742b646f00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb722200640 1 -- 172.21.15.116:0/2435060579 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fb71c0aa4d0 con 0x55742b646af0 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55742b646af0 msgr2=0x55742b646f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55742b646af0 0x55742b646f00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb70406de10 msgr2=0x7fb704070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb70406de10 0x7fb704070320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb710005fd0 tx=0x7fb71000e040 comp rx=0 tx=0).stop 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 msgr2=0x55742b5b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 --2- 172.21.15.116:0/2435060579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55742b574410 0x55742b5b9e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb71400bbb0 tx=0x7fb714000bd0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb722200640 1 -- 172.21.15.116:0/2435060579 reap_dead start 2024-05-14T11:17:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 shutdown_connections 2024-05-14T11:17:41.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 >> 172.21.15.116:0/2435060579 conn(0x55742b55de10 msgr2=0x55742b5a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 shutdown_connections 2024-05-14T11:17:41.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.516+0000 7fb728708080 1 -- 172.21.15.116:0/2435060579 wait complete. 2024-05-14T11:17:41.524 INFO:teuthology.orchestra.run.smithi116.stdout:mytag 2024-05-14T11:17:41.524 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-14T11:17:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 Processor -- start 2024-05-14T11:17:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 -- start start 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 0x55fb5fa68620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 0x55fb5fa68b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fb5f9e0aa0 con 0x55fb5fa3d100 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb5f9e4ae0 con 0x55fb5fa83570 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9579317080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55fb5fa84200 con 0x55fb5faf4560 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46012/0 (socket says 172.21.15.116:46012) 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 -- 172.21.15.116:0/915052329 learned_addr learned my addr 172.21.15.116:0/915052329 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 -- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 msgr2=0x55fb5fa68b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572e00640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 0x55fb5fa68b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9571a00640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 0x55fb5fa68620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 0x55fb5fa68b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 -- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 msgr2=0x55fb5fa68620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 0x55fb5fa68620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 -- 172.21.15.116:0/915052329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5fa7e980 con 0x55fb5fa3d100 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572e00640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 0x55fb5fa68b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9571a00640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 0x55fb5fa68620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f9572400640 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95640b2040 tx=0x7f9564003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59a5ea03d0ee77d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f956be00640 1 -- 172.21.15.116:0/915052329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640be070 con 0x55fb5fa3d100 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f956be00640 1 -- 172.21.15.116:0/915052329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95640b5040 con 0x55fb5fa3d100 2024-05-14T11:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.580+0000 7f956be00640 1 -- 172.21.15.116:0/915052329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640b4070 con 0x55fb5fa3d100 2024-05-14T11:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- 172.21.15.116:0/915052329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 msgr2=0x55fb5fb131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95640b2040 tx=0x7f9564003030 comp rx=0 tx=0).stop 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- 172.21.15.116:0/915052329 shutdown_connections 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5fa83570 0x55fb5fa68620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5faf4560 0x55fb5fa68b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- 172.21.15.116:0/915052329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb131d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- 172.21.15.116:0/915052329 >> 172.21.15.116:0/915052329 conn(0x55fb5fa25e10 msgr2=0x55fb5fa7df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- 172.21.15.116:0/915052329 shutdown_connections 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- 172.21.15.116:0/915052329 wait complete. 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 Processor -- start 2024-05-14T11:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- start start 2024-05-14T11:17:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb0ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5fa83570 0x55fb5fba6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5faf4560 0x55fb5fb0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fb5fa65a60 con 0x55fb5fa3d100 2024-05-14T11:17:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb5fa65800 con 0x55fb5faf4560 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9579317080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55fb5fa65d00 con 0x55fb5fa83570 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb0ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb0ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46022/0 (socket says 172.21.15.116:46022) 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 -- 172.21.15.116:0/1315429470 learned_addr learned my addr 172.21.15.116:0/1315429470 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9571a00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5fa83570 0x55fb5fba6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572e00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5faf4560 0x55fb5fb0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 -- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5fa83570 msgr2=0x55fb5fba6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5fa83570 0x55fb5fba6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 -- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5faf4560 msgr2=0x55fb5fb0f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5faf4560 0x55fb5fb0f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 -- 172.21.15.116:0/1315429470 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5fba77c0 con 0x55fb5fa3d100 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9571a00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55fb5fa83570 0x55fb5fba6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.584+0000 7f9572400640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb0ecf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f956401a6d0 tx=0x7f956401a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 -- 172.21.15.116:0/1315429470 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640b4070 con 0x55fb5fa3d100 2024-05-14T11:17:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fb5fba7a60 con 0x55fb5fa3d100 2024-05-14T11:17:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 -- 172.21.15.116:0/1315429470 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95640a7bc0 con 0x55fb5fa3d100 2024-05-14T11:17:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 -- 172.21.15.116:0/1315429470 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640be040 con 0x55fb5fa3d100 2024-05-14T11:17:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5fa5ecc0 con 0x55fb5fa3d100 2024-05-14T11:17:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 -- 172.21.15.116:0/1315429470 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95640c9060 con 0x55fb5fa3d100 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f955406dde0 0x7f95540702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f956aa00640 1 -- 172.21.15.116:0/1315429470 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f956412c5a0 con 0x55fb5fa3d100 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9571a00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f955406dde0 0x7f95540702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9579317080 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55fb5fa7f5a0 0x55fb5fa7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55fb5fba7380 con 0x55fb5fa7f5a0 2024-05-14T11:17:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9572e00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55fb5fa7f5a0 0x55fb5fa7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.588+0000 7f9571a00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f955406dde0 0x7f95540702f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c10 tx=0x7f95600036f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9572e00640 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55fb5fa7f5a0 0x55fb5fa7f9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9572e00640 1 -- 172.21.15.116:0/1315429470 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f956c0065f0 con 0x55fb5fa7f5a0 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55fb5fa7f5a0 msgr2=0x55fb5fa7f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55fb5fa7f5a0 0x55fb5fa7f9b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f955406dde0 msgr2=0x7f95540702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f955406dde0 0x7f95540702f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c10 tx=0x7f95600036f0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 msgr2=0x55fb5fb0ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 --2- 172.21.15.116:0/1315429470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fb5fa3d100 0x55fb5fb0ecf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f956401a6d0 tx=0x7f956401a860 comp rx=0 tx=0).stop 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9572e00640 1 -- 172.21.15.116:0/1315429470 reap_dead start 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 shutdown_connections 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 >> 172.21.15.116:0/1315429470 conn(0x55fb5fa25e10 msgr2=0x55fb5fb127c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 shutdown_connections 2024-05-14T11:17:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.592+0000 7f9579317080 1 -- 172.21.15.116:0/1315429470 wait complete. 2024-05-14T11:17:41.598 INFO:teuthology.orchestra.run.smithi116.stdout:mytag 2024-05-14T11:17:41.599 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-14T11:17:41.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.652+0000 7f0f94970080 1 Processor -- start 2024-05-14T11:17:41.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.652+0000 7f0f94970080 1 -- start start 2024-05-14T11:17:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9da890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b9e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b9b8b3aa0 con 0x556b9b93a110 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b9b8b7ae0 con 0x556b9b957ba0 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556b9b911690 con 0x556b9b9e3d10 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46036/0 (socket says 172.21.15.116:46036) 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 -- 172.21.15.116:0/3483821931 learned_addr learned my addr 172.21.15.116:0/3483821931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8e600640 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b9e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8d200640 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9da890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 -- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 msgr2=0x556b9b9e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b9e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 -- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 msgr2=0x556b9b9da890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9da890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 -- 172.21.15.116:0/3483821931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9b9e1110 con 0x556b9b93a110 2024-05-14T11:17:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0f80017ea0 tx=0x7f0f80003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ece26e5858068be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f87400640 1 -- 172.21.15.116:0/3483821931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800af070 con 0x556b9b93a110 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f87400640 1 -- 172.21.15.116:0/3483821931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f800ba070 con 0x556b9b93a110 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f87400640 1 -- 172.21.15.116:0/3483821931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800b5420 con 0x556b9b93a110 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- 172.21.15.116:0/3483821931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 msgr2=0x556b9b8fbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0f80017ea0 tx=0x7f0f80003030 comp rx=0 tx=0).stop 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- 172.21.15.116:0/3483821931 shutdown_connections 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9da890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b9e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 --2- 172.21.15.116:0/3483821931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b8fbe80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- 172.21.15.116:0/3483821931 >> 172.21.15.116:0/3483821931 conn(0x556b9b8f8e10 msgr2=0x556b9b9395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- 172.21.15.116:0/3483821931 shutdown_connections 2024-05-14T11:17:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- 172.21.15.116:0/3483821931 wait complete. 2024-05-14T11:17:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 Processor -- start 2024-05-14T11:17:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.656+0000 7f0f94970080 1 -- start start 2024-05-14T11:17:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b9d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b90dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b9b9dadd0 con 0x556b9b93a110 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b9b90d670 con 0x556b9b957ba0 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556b9b9e0bd0 con 0x556b9b9e3d10 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b9d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b9d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46038/0 (socket says 172.21.15.116:46038) 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 -- 172.21.15.116:0/760511072 learned_addr learned my addr 172.21.15.116:0/760511072 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8d200640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8e600640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b90dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 -- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 msgr2=0x556b9b90dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b90dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 -- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 msgr2=0x556b9b9dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9dafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 -- 172.21.15.116:0/760511072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9b9e03f0 con 0x556b9b93a110 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8e600640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556b9b9e3d10 0x556b9b90dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8d200640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b9b957ba0 0x556b9b9dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f8dc00640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b9d80d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0f80017ea0 tx=0x7f0f80018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f86000640 1 -- 172.21.15.116:0/760511072 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800af040 con 0x556b9b93a110 2024-05-14T11:17:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f86000640 1 -- 172.21.15.116:0/760511072 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f800b0040 con 0x556b9b93a110 2024-05-14T11:17:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f86000640 1 -- 172.21.15.116:0/760511072 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800b5830 con 0x556b9b93a110 2024-05-14T11:17:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b9b9e0690 con 0x556b9b93a110 2024-05-14T11:17:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.660+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b9b9dea10 con 0x556b9b93a110 2024-05-14T11:17:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f86000640 1 -- 172.21.15.116:0/760511072 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f800c7020 con 0x556b9b93a110 2024-05-14T11:17:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f86000640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f7006dde0 0x7f0f700702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f86000640 1 -- 172.21.15.116:0/760511072 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0f8012a4e0 con 0x556b9b93a110 2024-05-14T11:17:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8d200640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f7006dde0 0x7f0f700702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x556b9b9e5ac0 0x556b9b9deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x556b9b9df3f0 con 0x556b9b9e5ac0 2024-05-14T11:17:41.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8d200640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f7006dde0 0x7f0f700702f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c002ef0 tx=0x7f0f7c0026f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8e600640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x556b9b9e5ac0 0x556b9b9deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8e600640 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x556b9b9e5ac0 0x556b9b9deeb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8e600640 1 -- 172.21.15.116:0/760511072 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f0f880015f0 con 0x556b9b9e5ac0 2024-05-14T11:17:41.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x556b9b9e5ac0 msgr2=0x556b9b9deeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x556b9b9e5ac0 0x556b9b9deeb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f7006dde0 msgr2=0x7f0f700702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f7006dde0 0x7f0f700702f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c002ef0 tx=0x7f0f7c0026f0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 msgr2=0x556b9b9d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 --2- 172.21.15.116:0/760511072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b9b93a110 0x556b9b9d80d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0f80017ea0 tx=0x7f0f80018030 comp rx=0 tx=0).stop 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f8e600640 1 -- 172.21.15.116:0/760511072 reap_dead start 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 shutdown_connections 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 >> 172.21.15.116:0/760511072 conn(0x556b9b8f8e10 msgr2=0x556b9b9e2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 shutdown_connections 2024-05-14T11:17:41.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.664+0000 7f0f94970080 1 -- 172.21.15.116:0/760511072 wait complete. 2024-05-14T11:17:41.673 INFO:teuthology.orchestra.run.smithi116.stdout:mytag 2024-05-14T11:17:41.673 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-14T11:17:41.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 Processor -- start 2024-05-14T11:17:41.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 -- start start 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b400750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b48b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56302b360aa0 con 0x56302b4021f0 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56302b364ae0 con 0x56302b403020 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f6985bd4080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56302b401570 con 0x56302b3eaa20 2024-05-14T11:17:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46066/0 (socket says 172.21.15.116:46066) 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 -- 172.21.15.116:0/1331666872 learned_addr learned my addr 172.21.15.116:0/1331666872 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697f800640 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b48b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697ee00640 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b400750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 -- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 msgr2=0x56302b400750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b400750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 -- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 msgr2=0x56302b48b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b48b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 -- 172.21.15.116:0/1331666872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302b48ce40 con 0x56302b4021f0 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697e400640 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f696c01b7d0 tx=0x7f696c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0fe0ff7cbc9673 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697d000640 1 -- 172.21.15.116:0/1331666872 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0be070 con 0x56302b4021f0 2024-05-14T11:17:41.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697d000640 1 -- 172.21.15.116:0/1331666872 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c0b5040 con 0x56302b4021f0 2024-05-14T11:17:41.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.728+0000 7f697d000640 1 -- 172.21.15.116:0/1331666872 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0b4070 con 0x56302b4021f0 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- 172.21.15.116:0/1331666872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 msgr2=0x56302b48b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f696c01b7d0 tx=0x7f696c003030 comp rx=0 tx=0).stop 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- 172.21.15.116:0/1331666872 shutdown_connections 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b48b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b48b0d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- 172.21.15.116:0/1331666872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b400750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- 172.21.15.116:0/1331666872 >> 172.21.15.116:0/1331666872 conn(0x56302b3a5e10 msgr2=0x56302b491d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- 172.21.15.116:0/1331666872 shutdown_connections 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- 172.21.15.116:0/1331666872 wait complete. 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 Processor -- start 2024-05-14T11:17:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- start start 2024-05-14T11:17:41.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b3fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b478f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b479490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56302b51d7d0 con 0x56302b4021f0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56302b3bc920 con 0x56302b403020 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f6985bd4080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56302b48d0e0 con 0x56302b3eaa20 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b478f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b478f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46080/0 (socket says 172.21.15.116:46080) 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 -- 172.21.15.116:0/3451427356 learned_addr learned my addr 172.21.15.116:0/3451427356 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697ee00640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b3fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697f800640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b479490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 -- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 msgr2=0x56302b3fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56302b3eaa20 0x56302b3fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 -- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 msgr2=0x56302b479490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56302b403020 0x56302b479490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 -- 172.21.15.116:0/3451427356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302b48afd0 con 0x56302b4021f0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.732+0000 7f697e400640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b478f50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f696c01b7d0 tx=0x7f696c01a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 -- 172.21.15.116:0/3451427356 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0c8070 con 0x56302b4021f0 2024-05-14T11:17:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 -- 172.21.15.116:0/3451427356 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f696c0b0a00 con 0x56302b4021f0 2024-05-14T11:17:41.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56302b478570 con 0x56302b4021f0 2024-05-14T11:17:41.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 -- 172.21.15.116:0/3451427356 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c0b4070 con 0x56302b4021f0 2024-05-14T11:17:41.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302b478ab0 con 0x56302b4021f0 2024-05-14T11:17:41.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 -- 172.21.15.116:0/3451427356 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f696c0cb070 con 0x56302b4021f0 2024-05-14T11:17:41.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f696006dde0 0x7f69600702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6977400640 1 -- 172.21.15.116:0/3451427356 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f696c12c6e0 con 0x56302b4021f0 2024-05-14T11:17:41.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f697ee00640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f696006dde0 0x7f69600702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6985bd4080 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56302b529ac0 0x56302b51f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:41.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.736+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x56302b51faf0 con 0x56302b529ac0 2024-05-14T11:17:41.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f697f800640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56302b529ac0 0x56302b51f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:41.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f697ee00640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f696006dde0 0x7f69600702f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6970000c10 tx=0x7f6970003860 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f697f800640 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56302b529ac0 0x56302b51f5b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:41.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f697f800640 1 -- 172.21.15.116:0/3451427356 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f6978004550 con 0x56302b529ac0 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56302b529ac0 msgr2=0x56302b51f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56302b529ac0 0x56302b51f5b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f696006dde0 msgr2=0x7f69600702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f696006dde0 0x7f69600702f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6970000c10 tx=0x7f6970003860 comp rx=0 tx=0).stop 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 msgr2=0x56302b478f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 --2- 172.21.15.116:0/3451427356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56302b4021f0 0x56302b478f50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f696c01b7d0 tx=0x7f696c01a940 comp rx=0 tx=0).stop 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f697f800640 1 -- 172.21.15.116:0/3451427356 reap_dead start 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 shutdown_connections 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 >> 172.21.15.116:0/3451427356 conn(0x56302b3a5e10 msgr2=0x56302b491d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 shutdown_connections 2024-05-14T11:17:41.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:41.740+0000 7f6985bd4080 1 -- 172.21.15.116:0/3451427356 wait complete. 2024-05-14T11:17:41.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:17:41.747 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:17:41.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:17:41.831 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:17:41.877 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:17:41.877 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:17:41.878 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:41.878 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:41.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:17:41.886 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:17:41.887 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:17:41.887 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:41.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:17:41.927 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:17:41.927 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:41.927 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:41.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:17:41.936 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:17:41.936 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:17:41.936 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:41.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:17:41.978 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 Processor -- start 2024-05-14T11:17:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 -- start start 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f18a4166750 0x7f18a4168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a406ff20 con 0x7f18a415db30 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a4070470 con 0x7f18a40d1670 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18aaa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f18a406d710 con 0x7f18a4166750 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18a3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f18a415db30 0x7f18a40aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47472/0 (socket says 172.21.15.116:47472) 2024-05-14T11:17:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18a3e00640 1 -- 172.21.15.116:0/2975470279 learned_addr learned my addr 172.21.15.116:0/2975470279 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18a8a00640 1 --2- 172.21.15.116:0/2975470279 >> v2:172.21.15.149:3300/0 conn(0x7f18a4166750 0x7f18a4168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346446768 0 0) 0x7f18a406ff20 con 0x7f18a415db30 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.124+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1888003530 con 0x7f18a415db30 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 870017789 0 0) 0x7f1888003530 con 0x7f18a415db30 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 >> v2:172.21.15.149:3300/0 conn(0x7f18a4166750 msgr2=0x7f18a4168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 --2- 172.21.15.116:0/2975470279 >> v2:172.21.15.149:3300/0 conn(0x7f18a4166750 0x7f18a4168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 >> v1:172.21.15.149:6789/0 conn(0x7f18a40d1670 legacy=0x7f18a40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a4070750 con 0x7f18a415db30 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f1894001700 con 0x7f18a415db30 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 >> v1:172.21.15.116:6789/0 conn(0x7f18a415db30 legacy=0x7f18a40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 0x7f1888003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1888004810 0x7f1888004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18880053b0 0x7f188800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f188800c310 con 0x7f1888003a60 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f188800c4c0 con 0x7f1888004810 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f188800c6c0 con 0x7f18880053b0 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 0x7f1888003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3400640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1888004810 0x7f1888004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a8a00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18880053b0 0x7f188800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 -- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18880053b0 msgr2=0x7f188800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18880053b0 0x7f188800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 -- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1888004810 msgr2=0x7f1888004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1888004810 0x7f1888004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 -- 172.21.15.116:0/2975470279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1888003530 con 0x7f1888003a60 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3400640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1888004810 0x7f1888004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a8a00640 1 -- 172.21.15.116:0/2975470279 reap_dead start 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 0x7f1888003ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f189400c240 tx=0x7f189401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c70a8551ef2748 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894007d70 con 0x7f1888003a60 2024-05-14T11:17:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18940a9020 con 0x7f1888003a60 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.128+0000 7f18a2a00640 1 -- 172.21.15.116:0/2975470279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894005b00 con 0x7f1888003a60 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 msgr2=0x7f1888003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 0x7f1888003ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f189400c240 tx=0x7f189401cb80 comp rx=0 tx=0).stop 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- 172.21.15.116:0/2975470279 shutdown_connections 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2975470279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1888003a60 0x7f1888003ed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- 172.21.15.116:0/2975470279 >> 172.21.15.116:0/2975470279 conn(0x7f18a40b4620 msgr2=0x7f18a40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- 172.21.15.116:0/2975470279 shutdown_connections 2024-05-14T11:17:42.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- 172.21.15.116:0/2975470279 wait complete. 2024-05-14T11:17:42.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 Processor -- start 2024-05-14T11:17:42.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.132+0000 7f18aaa00640 1 -- start start 2024-05-14T11:17:42.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1888003a60 0x7f18a40d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a40b0d70 0x7f18a40b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40ae870 con 0x7f18a414b180 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40ae100 con 0x7f18a40b0d70 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f18a40d1e90 con 0x7f1888003a60 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46102/0 (socket says 172.21.15.116:46102) 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 -- 172.21.15.116:0/2721003311 learned_addr learned my addr 172.21.15.116:0/2721003311 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1888003a60 0x7f18a40d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a3400640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a40b0d70 0x7f18a40b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 -- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1888003a60 msgr2=0x7f18a40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1888003a60 0x7f18a40d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 -- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a40b0d70 msgr2=0x7f18a40b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a40b0d70 0x7f18a40b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 -- 172.21.15.116:0/2721003311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a40b16c0 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18a8a00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18a40b0d40 tx=0x7f189c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c011de0 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c01b070 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a409cc00 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0039a0 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a409d1c0 con 0x7f18a414b180 2024-05-14T11:17:42.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.136+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a406a9c0 con 0x7f18a414b180 2024-05-14T11:17:42.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.140+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f189c003b40 con 0x7f18a414b180 2024-05-14T11:17:42.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.140+0000 7f189be00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 0x7f1884071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.140+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f189c08fb20 con 0x7f18a414b180 2024-05-14T11:17:42.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.140+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 0x7f1884071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.140+0000 7f18a3e00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 0x7f1884071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f18940aa000 tx=0x7f18a40b0b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.144+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f18a40b16c0 con 0x7f18a414b180 2024-05-14T11:17:42.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18a40b4e40 con 0x7f18a414b180 2024-05-14T11:17:42.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f189be00640 1 -- 172.21.15.116:0/2721003311 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f189c05c220 con 0x7f18a414b180 2024-05-14T11:17:42.338 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:42.338 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:42.338 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 msgr2=0x7f1884071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 0x7f1884071910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f18940aa000 tx=0x7f18a40b0b70 comp rx=0 tx=0).stop 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 msgr2=0x7f18a414b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18a40b0d40 tx=0x7f189c0115a0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 shutdown_connections 2024-05-14T11:17:42.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f188406f400 0x7f1884071910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a40b0d70 0x7f18a40b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18a414b180 0x7f18a414b590 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 --2- 172.21.15.116:0/2721003311 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1888003a60 0x7f18a40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 >> 172.21.15.116:0/2721003311 conn(0x7f18a40b4620 msgr2=0x7f18a4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 shutdown_connections 2024-05-14T11:17:42.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.336+0000 7f18aaa00640 1 -- 172.21.15.116:0/2721003311 wait complete. 2024-05-14T11:17:42.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:42.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 Processor -- start 2024-05-14T11:17:42.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 -- start start 2024-05-14T11:17:42.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 0x7f4a6406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a640d1670 0x7f4a640d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a640d1fc0 con 0x7f4a6406e9e0 2024-05-14T11:17:42.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4a641573b0 con 0x7f4a640d1670 2024-05-14T11:17:42.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.508+0000 7f4a6c800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4a64161120 con 0x7f4a64156fa0 2024-05-14T11:17:42.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a640d1670 0x7f4a640d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a640d1670 0x7f4a640d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46112/0 (socket says 172.21.15.116:46112) 2024-05-14T11:17:42.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 -- 172.21.15.116:0/3510509202 learned_addr learned my addr 172.21.15.116:0/3510509202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:42.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69e00640 1 --2- 172.21.15.116:0/3510509202 >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 0x7f4a6406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 -- 172.21.15.116:0/3510509202 >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 msgr2=0x7f4a6406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:42.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 --2- 172.21.15.116:0/3510509202 >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 0x7f4a6406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 -- 172.21.15.116:0/3510509202 >> v1:172.21.15.116:6789/0 conn(0x7f4a6406e9e0 legacy=0x7f4a6406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 -- 172.21.15.116:0/3510509202 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64160510 con 0x7f4a640d1670 2024-05-14T11:17:42.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69e00640 1 --2- 172.21.15.116:0/3510509202 >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 0x7f4a6406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:42.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a69400640 1 --2- 172.21.15.116:0/3510509202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a640d1670 0x7f4a640d1a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4a64071f20 tx=0x7f4a5401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7103eee32e62f2c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a68a00640 1 -- 172.21.15.116:0/3510509202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540aa650 con 0x7f4a640d1670 2024-05-14T11:17:42.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a68a00640 1 -- 172.21.15.116:0/3510509202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a540af070 con 0x7f4a640d1670 2024-05-14T11:17:42.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.512+0000 7f4a68a00640 1 -- 172.21.15.116:0/3510509202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540aaa10 con 0x7f4a640d1670 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- 172.21.15.116:0/3510509202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a640d1670 msgr2=0x7f4a640d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 --2- 172.21.15.116:0/3510509202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a640d1670 0x7f4a640d1a80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4a64071f20 tx=0x7f4a5401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- 172.21.15.116:0/3510509202 shutdown_connections 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 --2- 172.21.15.116:0/3510509202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a640d1670 0x7f4a640d1a80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 --2- 172.21.15.116:0/3510509202 >> v2:172.21.15.149:3300/0 conn(0x7f4a64156fa0 0x7f4a6406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- 172.21.15.116:0/3510509202 >> 172.21.15.116:0/3510509202 conn(0x7f4a640b4620 msgr2=0x7f4a640b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- 172.21.15.116:0/3510509202 shutdown_connections 2024-05-14T11:17:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- 172.21.15.116:0/3510509202 wait complete. 2024-05-14T11:17:42.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 Processor -- start 2024-05-14T11:17:42.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.516+0000 7f4a6c800640 1 -- start start 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a6406e9e0 0x7f4a64148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a640d1670 0x7f4a64152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64070520 con 0x7f4a64156fa0 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b07a0 con 0x7f4a6406e9e0 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a640706a0 con 0x7f4a640d1670 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46122/0 (socket says 172.21.15.116:46122) 2024-05-14T11:17:42.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 -- 172.21.15.116:0/2816981334 learned_addr learned my addr 172.21.15.116:0/2816981334 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a69400640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a640d1670 0x7f4a64152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a69e00640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a6406e9e0 0x7f4a64148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 -- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a640d1670 msgr2=0x7f4a64152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a640d1670 0x7f4a64152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 -- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a6406e9e0 msgr2=0x7f4a64148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a6406e9e0 0x7f4a64148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 -- 172.21.15.116:0/2816981334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6415e180 con 0x7f4a64156fa0 2024-05-14T11:17:42.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6a800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c730 tx=0x7f4a6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60003b80 con 0x7f4a64156fa0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60003d20 con 0x7f4a64156fa0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6415e460 con 0x7f4a64156fa0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000aa70 con 0x7f4a64156fa0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6415ea20 con 0x7f4a64156fa0 2024-05-14T11:17:42.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.520+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6406a9c0 con 0x7f4a64156fa0 2024-05-14T11:17:42.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.524+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a6000ac10 con 0x7f4a64156fa0 2024-05-14T11:17:42.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.524+0000 7f4a5e000640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 0x7f4a48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.524+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4a6008f220 con 0x7f4a64156fa0 2024-05-14T11:17:42.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.524+0000 7f4a69e00640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 0x7f4a48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.524+0000 7f4a69e00640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 0x7f4a48071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a640b0d60 tx=0x7f4a58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.528+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4a640d1fc0 con 0x7f4a64156fa0 2024-05-14T11:17:42.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.716+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a640cd4e0 con 0x7f4a64156fa0 2024-05-14T11:17:42.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.716+0000 7f4a5e000640 1 -- 172.21.15.116:0/2816981334 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f4a6005b910 con 0x7f4a64156fa0 2024-05-14T11:17:42.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:42.719 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:42.720 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 msgr2=0x7f4a48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 0x7f4a48071940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a640b0d60 tx=0x7f4a58008040 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 msgr2=0x7f4a64155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c730 tx=0x7f4a6000cbf0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 shutdown_connections 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a4806f430 0x7f4a48071940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a64156fa0 0x7f4a64155870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a640d1670 0x7f4a64152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 --2- 172.21.15.116:0/2816981334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a6406e9e0 0x7f4a64148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 >> 172.21.15.116:0/2816981334 conn(0x7f4a640b4620 msgr2=0x7f4a640b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 shutdown_connections 2024-05-14T11:17:42.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.720+0000 7f4a6c800640 1 -- 172.21.15.116:0/2816981334 wait complete. 2024-05-14T11:17:42.741 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:42.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.884+0000 7ff951c00640 1 Processor -- start 2024-05-14T11:17:42.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.884+0000 7ff951c00640 1 -- start start 2024-05-14T11:17:42.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff951c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff94c0d1670 0x7ff94c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff951c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff94c0af380 0x7ff94c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff951c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94c0d1fc0 con 0x7ff94c156fa0 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff951c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c0706f0 con 0x7ff94c0af380 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff951c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c1573b0 con 0x7ff94c0d1670 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff94c0af380 0x7ff94c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff94c0af380 0x7ff94c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46130/0 (socket says 172.21.15.116:46130) 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 -- 172.21.15.116:0/3229678981 learned_addr learned my addr 172.21.15.116:0/3229678981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff942000640 1 --2- 172.21.15.116:0/3229678981 >> v2:172.21.15.149:3300/0 conn(0x7ff94c0d1670 0x7ff94c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 -- 172.21.15.116:0/3229678981 >> v2:172.21.15.149:3300/0 conn(0x7ff94c0d1670 msgr2=0x7ff94c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 --2- 172.21.15.116:0/3229678981 >> v2:172.21.15.149:3300/0 conn(0x7ff94c0d1670 0x7ff94c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 -- 172.21.15.116:0/3229678981 >> v1:172.21.15.149:6789/0 conn(0x7ff94c156fa0 legacy=0x7ff94c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 -- 172.21.15.116:0/3229678981 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c160510 con 0x7ff94c0af380 2024-05-14T11:17:42.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.888+0000 7ff94b400640 1 --2- 172.21.15.116:0/3229678981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c0af790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff94c070c70 tx=0x7ff94401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235d8af59726730a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff94a000640 1 -- 172.21.15.116:0/3229678981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440a57a0 con 0x7ff94c0af380 2024-05-14T11:17:42.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff94a000640 1 -- 172.21.15.116:0/3229678981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9440a5940 con 0x7ff94c0af380 2024-05-14T11:17:42.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff94a000640 1 -- 172.21.15.116:0/3229678981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440a5d10 con 0x7ff94c0af380 2024-05-14T11:17:42.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- 172.21.15.116:0/3229678981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 msgr2=0x7ff94c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 --2- 172.21.15.116:0/3229678981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c0af790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff94c070c70 tx=0x7ff94401c9f0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- 172.21.15.116:0/3229678981 shutdown_connections 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 --2- 172.21.15.116:0/3229678981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c0af790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 --2- 172.21.15.116:0/3229678981 >> v2:172.21.15.149:3300/0 conn(0x7ff94c0d1670 0x7ff94c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- 172.21.15.116:0/3229678981 >> 172.21.15.116:0/3229678981 conn(0x7ff94c0b4620 msgr2=0x7ff94c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- 172.21.15.116:0/3229678981 shutdown_connections 2024-05-14T11:17:42.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- 172.21.15.116:0/3229678981 wait complete. 2024-05-14T11:17:42.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 Processor -- start 2024-05-14T11:17:42.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.892+0000 7ff951c00640 1 -- start start 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 0x7ff94c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 0x7ff94c072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c0b0660 con 0x7ff94c0af380 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c0b04e0 con 0x7ff94c0d1670 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c169810 con 0x7ff94c156fa0 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46136/0 (socket says 172.21.15.116:46136) 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 -- 172.21.15.116:0/3631291266 learned_addr learned my addr 172.21.15.116:0/3631291266 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94b400640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 0x7ff94c072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff942000640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 0x7ff94c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 -- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 msgr2=0x7ff94c072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 0x7ff94c072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 -- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 msgr2=0x7ff94c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 0x7ff94c146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 -- 172.21.15.116:0/3631291266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c146a90 con 0x7ff94c0af380 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94b400640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 0x7ff94c072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff942000640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 0x7ff94c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff94aa00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00c3f0 tx=0x7ff93c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c017070 con 0x7ff94c0af380 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c0052d0 con 0x7ff94c0af380 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003840 con 0x7ff94c0af380 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c14bdd0 con 0x7ff94c0af380 2024-05-14T11:17:42.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c14c390 con 0x7ff94c0af380 2024-05-14T11:17:42.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.896+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94c06a9c0 con 0x7ff94c0af380 2024-05-14T11:17:42.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.900+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff93c01a070 con 0x7ff94c0af380 2024-05-14T11:17:42.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.900+0000 7ff943400640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 0x7ff928071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:42.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.900+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff93c08e820 con 0x7ff94c0af380 2024-05-14T11:17:42.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.900+0000 7ff942000640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 0x7ff928071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:42.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.900+0000 7ff942000640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 0x7ff928071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1530c0 tx=0x7ff934008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:42.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:42.904+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff94c0d1fc0 con 0x7ff94c0af380 2024-05-14T11:17:43.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.060+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff94c15fc90 con 0x7ff94c0af380 2024-05-14T11:17:43.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff943400640 1 -- 172.21.15.116:0/3631291266 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16990 (secure 0 0 0) 0x7ff93c005480 con 0x7ff94c0af380 2024-05-14T11:17:43.065 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:43.065 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:15.527236+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:12.118135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:12.868729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 msgr2=0x7ff928071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 0x7ff928071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1530c0 tx=0x7ff934008040 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 msgr2=0x7ff94c14b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00c3f0 tx=0x7ff93c00c8b0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 shutdown_connections 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92806f430 0x7ff928071940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff94c0d1670 0x7ff94c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94c156fa0 0x7ff94c072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 --2- 172.21.15.116:0/3631291266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff94c0af380 0x7ff94c14b4a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 >> 172.21.15.116:0/3631291266 conn(0x7ff94c0b4620 msgr2=0x7ff94c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 shutdown_connections 2024-05-14T11:17:43.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.064+0000 7ff951c00640 1 -- 172.21.15.116:0/3631291266 wait complete. 2024-05-14T11:17:43.087 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:17:43.088 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:43.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 Processor -- start 2024-05-14T11:17:43.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 -- start start 2024-05-14T11:17:43.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe304073cd0 0x7fe3040740e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe304156560 0x7fe304156970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe304074620 con 0x7fe3040d0900 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe3040cfeb0 con 0x7fe304156560 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe308a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe3040ceab0 con 0x7fe304073cd0 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe304156560 0x7fe304156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe304156560 0x7fe304156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46140/0 (socket says 172.21.15.116:46140) 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 -- 172.21.15.116:0/881339065 learned_addr learned my addr 172.21.15.116:0/881339065 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe302a00640 1 --2- 172.21.15.116:0/881339065 >> v2:172.21.15.149:3300/0 conn(0x7fe304073cd0 0x7fe3040740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 -- 172.21.15.116:0/881339065 >> v2:172.21.15.149:3300/0 conn(0x7fe304073cd0 msgr2=0x7fe3040740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 --2- 172.21.15.116:0/881339065 >> v2:172.21.15.149:3300/0 conn(0x7fe304073cd0 0x7fe3040740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 -- 172.21.15.116:0/881339065 >> v1:172.21.15.149:6789/0 conn(0x7fe3040d0900 legacy=0x7fe3040b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.232+0000 7fe303e00640 1 -- 172.21.15.116:0/881339065 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3040b5db0 con 0x7fe304156560 2024-05-14T11:17:43.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe303e00640 1 --2- 172.21.15.116:0/881339065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304156560 0x7fe304156970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe304074c50 tx=0x7fe2fc01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5a22fb6ff7a0ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe302000640 1 -- 172.21.15.116:0/881339065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0a55d0 con 0x7fe304156560 2024-05-14T11:17:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe302000640 1 -- 172.21.15.116:0/881339065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0a5770 con 0x7fe304156560 2024-05-14T11:17:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe302000640 1 -- 172.21.15.116:0/881339065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0a5b40 con 0x7fe304156560 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 -- 172.21.15.116:0/881339065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304156560 msgr2=0x7fe304156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 --2- 172.21.15.116:0/881339065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304156560 0x7fe304156970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe304074c50 tx=0x7fe2fc01c750 comp rx=0 tx=0).stop 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 -- 172.21.15.116:0/881339065 shutdown_connections 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 --2- 172.21.15.116:0/881339065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304156560 0x7fe304156970 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 --2- 172.21.15.116:0/881339065 >> v2:172.21.15.149:3300/0 conn(0x7fe304073cd0 0x7fe3040740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 -- 172.21.15.116:0/881339065 >> 172.21.15.116:0/881339065 conn(0x7fe3040d26e0 msgr2=0x7fe3040d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 -- 172.21.15.116:0/881339065 shutdown_connections 2024-05-14T11:17:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.236+0000 7fe308a00640 1 -- 172.21.15.116:0/881339065 wait complete. 2024-05-14T11:17:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 Processor -- start 2024-05-14T11:17:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 -- start start 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3040d0900 0x7fe3040b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe304156560 0x7fe3040b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe304156eb0 con 0x7fe304073cd0 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3041603e0 con 0x7fe3040d0900 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe308a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe3040b5a00 con 0x7fe304156560 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46156/0 (socket says 172.21.15.116:46156) 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 -- 172.21.15.116:0/3059590642 learned_addr learned my addr 172.21.15.116:0/3059590642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 -- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe304156560 msgr2=0x7fe3040b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe304156560 0x7fe3040b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 -- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3040d0900 msgr2=0x7fe3040b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3040d0900 0x7fe3040b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 -- 172.21.15.116:0/3059590642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3040afbb0 con 0x7fe304073cd0 2024-05-14T11:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe303400640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400b3f0 tx=0x7fe2f400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.240+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4015020 con 0x7fe304073cd0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f4011850 con 0x7fe304073cd0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3040afe90 con 0x7fe304073cd0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4011c40 con 0x7fe304073cd0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3040b0450 con 0x7fe304073cd0 2024-05-14T11:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe304050880 con 0x7fe304073cd0 2024-05-14T11:17:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2f4018020 con 0x7fe304073cd0 2024-05-14T11:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe2fb400640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 0x7fe2e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe2f408f0a0 con 0x7fe304073cd0 2024-05-14T11:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.244+0000 7fe302a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 0x7fe2e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.248+0000 7fe302a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 0x7fe2e4071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe30406fcb0 tx=0x7fe2f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.248+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe2f405be90 con 0x7fe304073cd0 2024-05-14T11:17:43.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.440+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3040b6530 con 0x7fe304073cd0 2024-05-14T11:17:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.440+0000 7fe2fb400640 1 -- 172.21.15.116:0/3059590642 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fe2f405b790 con 0x7fe304073cd0 2024-05-14T11:17:43.443 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:43.443 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 msgr2=0x7fe2e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 0x7fe2e4071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe30406fcb0 tx=0x7fe2f0008040 comp rx=0 tx=0).stop 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 msgr2=0x7fe3041f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400b3f0 tx=0x7fe2f400b8b0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 shutdown_connections 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2e406f400 0x7fe2e4071910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3040d0900 0x7fe3040b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe304156560 0x7fe3040b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 --2- 172.21.15.116:0/3059590642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe304073cd0 0x7fe3041f8c00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 >> 172.21.15.116:0/3059590642 conn(0x7fe3040d26e0 msgr2=0x7fe30416ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 shutdown_connections 2024-05-14T11:17:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.444+0000 7fe308a00640 1 -- 172.21.15.116:0/3059590642 wait complete. 2024-05-14T11:17:43.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:43.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.608+0000 7f6e58600640 1 Processor -- start 2024-05-14T11:17:43.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.608+0000 7f6e58600640 1 -- start start 2024-05-14T11:17:43.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e58600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 0x7f6e500d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e58600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6e500d1bb0 0x7f6e5006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e58600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e500d1fc0 con 0x7f6e5006cc40 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e58600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6e501600a0 con 0x7f6e500d1bb0 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e58600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5006f5d0 con 0x7f6e50164220 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e56600640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6e5006cc40 0x7f6e500ae2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47496/0 (socket says 172.21.15.116:47496) 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e56600640 1 -- 172.21.15.116:0/2430443156 learned_addr learned my addr 172.21.15.116:0/2430443156 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55200640 1 --2- 172.21.15.116:0/2430443156 >> v2:172.21.15.149:3300/0 conn(0x7f6e500d1bb0 0x7f6e5006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55c00640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 0x7f6e500d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32060792 0 0) 0x7f6e480014a0 con 0x7f6e5006cc40 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e38003530 con 0x7f6e5006cc40 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55200640 1 -- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 msgr2=0x7f6e500d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55200640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 0x7f6e500d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55200640 1 -- 172.21.15.116:0/2430443156 >> v1:172.21.15.116:6789/0 conn(0x7f6e5006cc40 legacy=0x7f6e500ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4112661395 0 0) 0x7f6e38003530 con 0x7f6e5006cc40 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55200640 1 -- 172.21.15.116:0/2430443156 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5006f8b0 con 0x7f6e500d1bb0 2024-05-14T11:17:43.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.612+0000 7f6e55c00640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 0x7f6e500d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:43.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e55200640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e500d1bb0 0x7f6e5006c700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0097c0 tx=0x7f6e3c01c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280d25824dff09f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0b0070 con 0x7f6e500d1bb0 2024-05-14T11:17:43.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0ab400 con 0x7f6e500d1bb0 2024-05-14T11:17:43.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e4fe00640 1 -- 172.21.15.116:0/2430443156 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48001c50 con 0x7f6e500d1bb0 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 -- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e500d1bb0 msgr2=0x7f6e5006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e500d1bb0 0x7f6e5006c700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0097c0 tx=0x7f6e3c01c300 comp rx=0 tx=0).stop 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 -- 172.21.15.116:0/2430443156 shutdown_connections 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e500d1bb0 0x7f6e5006c700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 --2- 172.21.15.116:0/2430443156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e50164220 0x7f6e500d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 -- 172.21.15.116:0/2430443156 >> 172.21.15.116:0/2430443156 conn(0x7f6e500b4620 msgr2=0x7f6e500b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 -- 172.21.15.116:0/2430443156 shutdown_connections 2024-05-14T11:17:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.616+0000 7f6e58600640 1 -- 172.21.15.116:0/2430443156 wait complete. 2024-05-14T11:17:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 Processor -- start 2024-05-14T11:17:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- start start 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 0x7f6e50145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 0x7f6e50149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5006d460 con 0x7f6e500d1bb0 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500701d0 con 0x7f6e50164220 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6e50164630 con 0x7f6e5006cc40 2024-05-14T11:17:43.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46168/0 (socket says 172.21.15.116:46168) 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 -- 172.21.15.116:0/1930221744 learned_addr learned my addr 172.21.15.116:0/1930221744 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55c00640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 0x7f6e50145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 -- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 msgr2=0x7f6e50145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 0x7f6e50145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 -- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 msgr2=0x7f6e50149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e56600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 0x7f6e50149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 0x7f6e50149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 -- 172.21.15.116:0/1930221744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50162130 con 0x7f6e500d1bb0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55c00640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 0x7f6e50145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e56600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 0x7f6e50149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e55200640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0097c0 tx=0x7f6e3c0190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0b0040 con 0x7f6e500d1bb0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0abc90 con 0x7f6e500d1bb0 2024-05-14T11:17:43.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501fde70 con 0x7f6e500d1bb0 2024-05-14T11:17:43.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004220 con 0x7f6e500d1bb0 2024-05-14T11:17:43.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.620+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501fe430 con 0x7f6e500d1bb0 2024-05-14T11:17:43.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5006a9c0 con 0x7f6e500d1bb0 2024-05-14T11:17:43.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6e3c0bd020 con 0x7f6e500d1bb0 2024-05-14T11:17:43.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e4d600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 0x7f6e30071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f6e3c128a30 con 0x7f6e500d1bb0 2024-05-14T11:17:43.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e55c00640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 0x7f6e30071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.624+0000 7f6e55c00640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 0x7f6e30071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e50074920 tx=0x7f6e44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.628+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6e3c015fb0 con 0x7f6e500d1bb0 2024-05-14T11:17:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.820+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e501fe600 con 0x7f6e500d1bb0 2024-05-14T11:17:43.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.820+0000 7f6e4d600640 1 -- 172.21.15.116:0/1930221744 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f6e3c0f56f0 con 0x7f6e500d1bb0 2024-05-14T11:17:43.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:43.823 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:13.909695+0000","modified":"2024-05-14T11:17:16.938510+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2525866958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2525866958},{"type":"v1","addr":"172.21.15.116:6837","nonce":2525866958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-05-14T11:17:43.824 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 18 2024-05-14T11:17:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 msgr2=0x7f6e30071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 0x7f6e30071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e50074920 tx=0x7f6e44008040 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 msgr2=0x7f6e501545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0097c0 tx=0x7f6e3c0190a0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 shutdown_connections 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6e3006f430 0x7f6e30071940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e50164220 0x7f6e50149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e500d1bb0 0x7f6e501545a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 --2- 172.21.15.116:0/1930221744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e5006cc40 0x7f6e50145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 >> 172.21.15.116:0/1930221744 conn(0x7f6e500b4620 msgr2=0x7f6e50162460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 shutdown_connections 2024-05-14T11:17:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.824+0000 7f6e58600640 1 -- 172.21.15.116:0/1930221744 wait complete. 2024-05-14T11:17:43.846 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:43.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 Processor -- start 2024-05-14T11:17:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 -- start start 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae80cfeb0 con 0x7f0ae80b9160 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae80cf110 con 0x7f0ae80b70a0 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0aee400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae80b6d80 con 0x7f0ae815fe00 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0adea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0ae80b9160 0x7f0ae80d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47510/0 (socket says 172.21.15.116:47510) 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.988+0000 7f0adea00640 1 -- 172.21.15.116:0/3108806624 learned_addr learned my addr 172.21.15.116:0/3108806624 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae7e00640 1 --2- 172.21.15.116:0/3108806624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:43.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888157487 0 0) 0x7f0ae80cfeb0 con 0x7f0ae80b9160 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0acc003530 con 0x7f0ae80b9160 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289279200 0 0) 0x7f0ae80cf110 con 0x7f0ae80b70a0 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ae80cfeb0 con 0x7f0ae80b70a0 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae7e00640 1 -- 172.21.15.116:0/3108806624 >> v1:172.21.15.149:6789/0 conn(0x7f0ae80b70a0 legacy=0x7f0ae80b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae7e00640 1 -- 172.21.15.116:0/3108806624 >> v1:172.21.15.116:6789/0 conn(0x7f0ae80b9160 legacy=0x7f0ae80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae7e00640 1 -- 172.21.15.116:0/3108806624 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae80cf3d0 con 0x7f0ae815fe00 2024-05-14T11:17:43.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.992+0000 7f0ae7e00640 1 --2- 172.21.15.116:0/3108806624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80d1a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097f0 tx=0x7f0ae001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc96fc5066e592 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:43.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00b0070 con 0x7f0ae815fe00 2024-05-14T11:17:43.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae00ab600 con 0x7f0ae815fe00 2024-05-14T11:17:43.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0ae6a00640 1 -- 172.21.15.116:0/3108806624 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00ab9b0 con 0x7f0ae815fe00 2024-05-14T11:17:43.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- 172.21.15.116:0/3108806624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 msgr2=0x7f0ae80d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:43.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 --2- 172.21.15.116:0/3108806624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80d1a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097f0 tx=0x7f0ae001c4e0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- 172.21.15.116:0/3108806624 shutdown_connections 2024-05-14T11:17:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 --2- 172.21.15.116:0/3108806624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80d1a20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- 172.21.15.116:0/3108806624 >> 172.21.15.116:0/3108806624 conn(0x7f0ae80d3740 msgr2=0x7f0ae80d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- 172.21.15.116:0/3108806624 shutdown_connections 2024-05-14T11:17:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- 172.21.15.116:0/3108806624 wait complete. 2024-05-14T11:17:44.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 Processor -- start 2024-05-14T11:17:44.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:43.996+0000 7f0aee400640 1 -- start start 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 0x7f0ae80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b6b40 con 0x7f0ae80b70a0 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b69c0 con 0x7f0ae80b9160 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae816e640 con 0x7f0ae815fe00 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46172/0 (socket says 172.21.15.116:46172) 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 -- 172.21.15.116:0/3178997916 learned_addr learned my addr 172.21.15.116:0/3178997916 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:44.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0adea00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 0x7f0ae80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7e00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 -- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 msgr2=0x7f0ae80b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 -- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 msgr2=0x7f0ae80aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 0x7f0ae80aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 -- 172.21.15.116:0/3178997916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae81589d0 con 0x7f0ae80b70a0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0adea00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 0x7f0ae80aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0ae7400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c780 tx=0x7f0ad800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8003b80 con 0x7f0ae80b70a0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad8003d20 con 0x7f0ae80b70a0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800aa90 con 0x7f0ae80b70a0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae8158cb0 con 0x7f0ae80b70a0 2024-05-14T11:17:44.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae815a9f0 con 0x7f0ae80b70a0 2024-05-14T11:17:44.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.000+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae806a9c0 con 0x7f0ae80b70a0 2024-05-14T11:17:44.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.004+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ad800ac30 con 0x7f0ae80b70a0 2024-05-14T11:17:44.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.004+0000 7f0adfe00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 0x7f0ac4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.004+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0ad808fd30 con 0x7f0ae80b70a0 2024-05-14T11:17:44.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.004+0000 7f0adea00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 0x7f0ac4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.004+0000 7f0adea00640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 0x7f0ac4071940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c10 tx=0x7f0ad0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.008+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0ad805cb70 con 0x7f0ae80b70a0 2024-05-14T11:17:44.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.164+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ae80cede0 con 0x7f0ae80b70a0 2024-05-14T11:17:44.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.164+0000 7f0adfe00640 1 -- 172.21.15.116:0/3178997916 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16990 (secure 0 0 0) 0x7f0ad805c470 con 0x7f0ae80b70a0 2024-05-14T11:17:44.167 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:44.167 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:15.527236+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:12.118135+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:12.868729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 msgr2=0x7f0ac4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 0x7f0ac4071940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c10 tx=0x7f0ad0009040 comp rx=0 tx=0).stop 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 msgr2=0x7f0ae8158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c780 tx=0x7f0ad800cc40 comp rx=0 tx=0).stop 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 shutdown_connections 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ac406f430 0x7f0ac4071940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae80b9160 0x7f0ae80aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0ae815fe00 0x7f0ae80b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 --2- 172.21.15.116:0/3178997916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ae80b70a0 0x7f0ae8158080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 >> 172.21.15.116:0/3178997916 conn(0x7f0ae80d3740 msgr2=0x7f0ae80cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:44.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 shutdown_connections 2024-05-14T11:17:44.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.168+0000 7f0aee400640 1 -- 172.21.15.116:0/3178997916 wait complete. 2024-05-14T11:17:44.189 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:17:44.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.332+0000 7fe16d200640 1 Processor -- start 2024-05-14T11:17:44.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.332+0000 7fe16d200640 1 -- start start 2024-05-14T11:17:44.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe16d200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe16d200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe16d200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1680d1fc0 con 0x7fe16806e9e0 2024-05-14T11:17:44.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe16d200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe1681573b0 con 0x7fe168156fa0 2024-05-14T11:17:44.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe16d200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe168161120 con 0x7fe1680d1670 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe165e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:55198/0 (socket says 172.21.15.116:55198) 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 -- 172.21.15.116:0/3494247049 learned_addr learned my addr 172.21.15.116:0/3494247049 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe165e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:32862/0 (socket says 172.21.15.116:32862) 2024-05-14T11:17:44.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 -- 172.21.15.116:0/3494247049 >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 msgr2=0x7fe16806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:44.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 --2- 172.21.15.116:0/3494247049 >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 -- 172.21.15.116:0/3494247049 >> v1:172.21.15.149:6789/0 conn(0x7fe16806e9e0 legacy=0x7fe16806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe15d400640 1 -- 172.21.15.116:0/3494247049 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe168160510 con 0x7fe1680d1670 2024-05-14T11:17:44.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.336+0000 7fe165e00640 1 --2- 172.21.15.116:0/3494247049 >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:44.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe15d400640 1 --2- 172.21.15.116:0/3494247049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe150009870 tx=0x7fe15001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765bcd30b08b24d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe165400640 1 -- 172.21.15.116:0/3494247049 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500a95f0 con 0x7fe1680d1670 2024-05-14T11:17:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe165400640 1 -- 172.21.15.116:0/3494247049 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1500a9790 con 0x7fe1680d1670 2024-05-14T11:17:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe165400640 1 -- 172.21.15.116:0/3494247049 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500a9b60 con 0x7fe1680d1670 2024-05-14T11:17:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 -- 172.21.15.116:0/3494247049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 msgr2=0x7fe1680d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 --2- 172.21.15.116:0/3494247049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe150009870 tx=0x7fe15001c530 comp rx=0 tx=0).stop 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 -- 172.21.15.116:0/3494247049 shutdown_connections 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 --2- 172.21.15.116:0/3494247049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680d1a80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 --2- 172.21.15.116:0/3494247049 >> v2:172.21.15.149:3300/0 conn(0x7fe168156fa0 0x7fe16806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 -- 172.21.15.116:0/3494247049 >> 172.21.15.116:0/3494247049 conn(0x7fe1680b4620 msgr2=0x7fe1680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 -- 172.21.15.116:0/3494247049 shutdown_connections 2024-05-14T11:17:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.340+0000 7fe16d200640 1 -- 172.21.15.116:0/3494247049 wait complete. 2024-05-14T11:17:44.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 Processor -- start 2024-05-14T11:17:44.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 -- start start 2024-05-14T11:17:44.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe168156fa0 0x7fe168149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe168070520 con 0x7fe16806e9e0 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1680b07a0 con 0x7fe168156fa0 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe16d200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe1680706a0 con 0x7fe1680d1670 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46184/0 (socket says 172.21.15.116:46184) 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 -- 172.21.15.116:0/523656332 learned_addr learned my addr 172.21.15.116:0/523656332 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe15d400640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe166800640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe168156fa0 0x7fe168149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 -- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 msgr2=0x7fe1680735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 -- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe168156fa0 msgr2=0x7fe168149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe168156fa0 0x7fe168149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 -- 172.21.15.116:0/523656332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16814fd70 con 0x7fe16806e9e0 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.344+0000 7fe165e00640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e960 tx=0x7fe15800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158017070 con 0x7fe16806e9e0 2024-05-14T11:17:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15800ca00 con 0x7fe16806e9e0 2024-05-14T11:17:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe168150050 con 0x7fe16806e9e0 2024-05-14T11:17:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15800cdd0 con 0x7fe16806e9e0 2024-05-14T11:17:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe168150610 con 0x7fe16806e9e0 2024-05-14T11:17:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16806a9c0 con 0x7fe16806e9e0 2024-05-14T11:17:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe15801a070 con 0x7fe16806e9e0 2024-05-14T11:17:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.348+0000 7fe15ea00640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 0x7fe144071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.352+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe15808f660 con 0x7fe16806e9e0 2024-05-14T11:17:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.352+0000 7fe15d400640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 0x7fe144071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.352+0000 7fe15d400640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 0x7fe144071910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1500ae000 tx=0x7fe1500b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.356+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe15805bda0 con 0x7fe16806e9e0 2024-05-14T11:17:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.548+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe1680b4b60 con 0x7fe16806e9e0 2024-05-14T11:17:44.553 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:17:44.552+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:17:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.552+0000 7fe15ea00640 1 -- 172.21.15.116:0/523656332 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fe1580600a0 con 0x7fe16806e9e0 2024-05-14T11:17:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:17:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 msgr2=0x7fe144071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 0x7fe144071910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1500ae000 tx=0x7fe1500b3040 comp rx=0 tx=0).stop 2024-05-14T11:17:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 msgr2=0x7fe1680730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e960 tx=0x7fe15800ee20 comp rx=0 tx=0).stop 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 shutdown_connections 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe14406f400 0x7fe144071910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe168156fa0 0x7fe168149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1680d1670 0x7fe1680735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 --2- 172.21.15.116:0/523656332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe16806e9e0 0x7fe1680730a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 >> 172.21.15.116:0/523656332 conn(0x7fe1680b4620 msgr2=0x7fe16815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:44.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 shutdown_connections 2024-05-14T11:17:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.556+0000 7fe16d200640 1 -- 172.21.15.116:0/523656332 wait complete. 2024-05-14T11:17:44.569 INFO:tasks.ceph.mds.c.smithi116.stderr: -5793> 2024-05-14T11:17:19.480+0000 7fd9b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:17:44.580 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:17:44.690 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:17:44.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 Processor -- start 2024-05-14T11:17:44.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 -- start start 2024-05-14T11:17:44.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb0d40d1670 0x7fb0d40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d40d1fc0 con 0x7fb0d4156fa0 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d4160510 con 0x7fb0d40af380 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0dbe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d40706f0 con 0x7fb0d40d1670 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d9400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb0d4156fa0 0x7fb0d406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47536/0 (socket says 172.21.15.116:47536) 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d9400640 1 -- 172.21.15.116:0/4042073428 learned_addr learned my addr 172.21.15.116:0/4042073428 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d8a00640 1 --2- 172.21.15.116:0/4042073428 >> v2:172.21.15.116:3300/0 conn(0x7fb0d40d1670 0x7fb0d40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d8a00640 1 -- 172.21.15.116:0/4042073428 >> v1:172.21.15.149:6789/0 conn(0x7fb0d40af380 legacy=0x7fb0d40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d8a00640 1 -- 172.21.15.116:0/4042073428 >> v1:172.21.15.116:6789/0 conn(0x7fb0d4156fa0 legacy=0x7fb0d406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.732+0000 7fb0d8a00640 1 -- 172.21.15.116:0/4042073428 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41607f0 con 0x7fb0d40d1670 2024-05-14T11:17:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0d8a00640 1 --2- 172.21.15.116:0/4042073428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d40d1670 0x7fb0d40d1a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f3c4e9e453fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0d3e00640 1 -- 172.21.15.116:0/4042073428 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00b0070 con 0x7fb0d40d1670 2024-05-14T11:17:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0d3e00640 1 -- 172.21.15.116:0/4042073428 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c00ab600 con 0x7fb0d40d1670 2024-05-14T11:17:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0d3e00640 1 -- 172.21.15.116:0/4042073428 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00ab9d0 con 0x7fb0d40d1670 2024-05-14T11:17:44.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 -- 172.21.15.116:0/4042073428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d40d1670 msgr2=0x7fb0d40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/4042073428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d40d1670 0x7fb0d40d1a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 -- 172.21.15.116:0/4042073428 shutdown_connections 2024-05-14T11:17:44.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/4042073428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d40d1670 0x7fb0d40d1a80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 -- 172.21.15.116:0/4042073428 >> 172.21.15.116:0/4042073428 conn(0x7fb0d40b4620 msgr2=0x7fb0d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:44.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 -- 172.21.15.116:0/4042073428 shutdown_connections 2024-05-14T11:17:44.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.736+0000 7fb0dbe00640 1 -- 172.21.15.116:0/4042073428 wait complete. 2024-05-14T11:17:44.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 Processor -- start 2024-05-14T11:17:44.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- start start 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d40af380 0x7fb0d414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 0x7fb0d414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d416abb0 con 0x7fb0d4156fa0 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d416aa30 con 0x7fb0d40af380 2024-05-14T11:17:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d416ad30 con 0x7fb0d40d1670 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46210/0 (socket says 172.21.15.116:46210) 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 -- 172.21.15.116:0/3887454977 learned_addr learned my addr 172.21.15.116:0/3887454977 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d8a00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 0x7fb0d414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9400640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d40af380 0x7fb0d414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 -- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 msgr2=0x7fb0d414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 0x7fb0d414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 -- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d40af380 msgr2=0x7fb0d414ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d40af380 0x7fb0d414ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 -- 172.21.15.116:0/3887454977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d414a040 con 0x7fb0d4156fa0 2024-05-14T11:17:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d8a00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 0x7fb0d414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d9e00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00ba10 tx=0x7fb0cc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc019070 con 0x7fb0d4156fa0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc00cb40 con 0x7fb0d4156fa0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d409ae60 con 0x7fb0d4156fa0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc014440 con 0x7fb0d4156fa0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.740+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d414a5c0 con 0x7fb0d4156fa0 2024-05-14T11:17:44.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d414d3a0 con 0x7fb0d4156fa0 2024-05-14T11:17:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb0cc01c070 con 0x7fb0d4156fa0 2024-05-14T11:17:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0d1600640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 0x7fb0b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fb0cc08f0a0 con 0x7fb0d4156fa0 2024-05-14T11:17:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0d9400640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 0x7fb0b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:44.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.744+0000 7fb0d9400640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 0x7fb0b4071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000f70 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.748+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb0cc05bd50 con 0x7fb0d4156fa0 2024-05-14T11:17:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:44.940+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb0d406a9c0 con 0x7fb0d4156fa0 2024-05-14T11:17:45.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.556+0000 7fb0d1600640 1 -- 172.21.15.116:0/3887454977 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fb0cc05b650 con 0x7fb0d4156fa0 2024-05-14T11:17:45.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 msgr2=0x7fb0b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 0x7fb0b4071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000f70 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2024-05-14T11:17:45.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 msgr2=0x7fb0d409a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00ba10 tx=0x7fb0cc00bed0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 shutdown_connections 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb0b406f430 0x7fb0b4071940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0d4156fa0 0x7fb0d409a920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0d40d1670 0x7fb0d414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 --2- 172.21.15.116:0/3887454977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0d40af380 0x7fb0d414ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 >> 172.21.15.116:0/3887454977 conn(0x7fb0d40b4620 msgr2=0x7fb0d40b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 shutdown_connections 2024-05-14T11:17:45.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.560+0000 7fb0dbe00640 1 -- 172.21.15.116:0/3887454977 wait complete. 2024-05-14T11:17:45.583 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:17:45.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 Processor -- start 2024-05-14T11:17:45.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 -- start start 2024-05-14T11:17:45.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcfc806ef80 0x7fcfc80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfc80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc815d360 con 0x7fcfc8070640 2024-05-14T11:17:45.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc815dc90 con 0x7fcfc806ef80 2024-05-14T11:17:45.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcf200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc815d640 con 0x7fcfc8070a50 2024-05-14T11:17:45.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcfc8070640 0x7fcfc806e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47538/0 (socket says 172.21.15.116:47538) 2024-05-14T11:17:45.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc7e00640 1 -- 172.21.15.116:0/4147071694 learned_addr learned my addr 172.21.15.116:0/4147071694 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfcd200640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfc80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc7400640 1 --2- 172.21.15.116:0/4147071694 >> v2:172.21.15.149:3300/0 conn(0x7fcfc806ef80 0x7fcfc80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470090742 0 0) 0x7fcfc815d360 con 0x7fcfc8070640 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfac005740 con 0x7fcfc8070640 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1336636163 0 0) 0x7fcfac005740 con 0x7fcfc8070640 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 msgr2=0x7fcfc80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfc80cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 >> v2:172.21.15.149:3300/0 conn(0x7fcfc806ef80 msgr2=0x7fcfc80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 --2- 172.21.15.116:0/4147071694 >> v2:172.21.15.149:3300/0 conn(0x7fcfc806ef80 0x7fcfc80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc815d920 con 0x7fcfc8070640 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fcfb8001700 con 0x7fcfc8070640 2024-05-14T11:17:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.732+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 >> v1:172.21.15.116:6789/0 conn(0x7fcfc8070640 legacy=0x7fcfc806e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfac005c20 0x7fcfac006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfac006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfac007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc815dc90 con 0x7fcfac006a20 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc815d640 con 0x7fcfac005c20 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac00dc90 con 0x7fcfc8070a50 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfac006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7e00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfac005c20 0x7fcfac006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfcd200640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfac007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 -- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 msgr2=0x7fcfac007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfac007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 -- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfac005c20 msgr2=0x7fcfac006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfac005c20 0x7fcfac006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 -- 172.21.15.116:0/4147071694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac005740 con 0x7fcfac006a20 2024-05-14T11:17:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7e00640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfac005c20 0x7fcfac006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfcd200640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc8070a50 0x7fcfac007370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc7400640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfac006e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009ae0 tx=0x7fcfb401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f772c74745004a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b2070 con 0x7fcfac006a20 2024-05-14T11:17:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb40a56a0 con 0x7fcfac006a20 2024-05-14T11:17:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.736+0000 7fcfc6a00640 1 -- 172.21.15.116:0/4147071694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40096b0 con 0x7fcfac006a20 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- 172.21.15.116:0/4147071694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 msgr2=0x7fcfac006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 --2- 172.21.15.116:0/4147071694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfac006e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009ae0 tx=0x7fcfb401cab0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcd200640 1 -- 172.21.15.116:0/4147071694 reap_dead start 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- 172.21.15.116:0/4147071694 shutdown_connections 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- 172.21.15.116:0/4147071694 >> 172.21.15.116:0/4147071694 conn(0x7fcfc80cde80 msgr2=0x7fcfc80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- 172.21.15.116:0/4147071694 shutdown_connections 2024-05-14T11:17:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- 172.21.15.116:0/4147071694 wait complete. 2024-05-14T11:17:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 Processor -- start 2024-05-14T11:17:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.740+0000 7fcfcf200640 1 -- start start 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 0x7fcfc81f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 0x7fcfc81f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc80cd800 con 0x7fcfac006a20 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc813fda0 con 0x7fcfc813ce30 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc80cd980 con 0x7fcfc813d240 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46228/0 (socket says 172.21.15.116:46228) 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 -- 172.21.15.116:0/1048725704 learned_addr learned my addr 172.21.15.116:0/1048725704 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7400640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 0x7fcfc81f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcd200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 0x7fcfc81f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 -- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 msgr2=0x7fcfc81f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 0x7fcfc81f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 -- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 msgr2=0x7fcfc81f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 0x7fcfc81f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 -- 172.21.15.116:0/1048725704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc81586f0 con 0x7fcfac006a20 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcd200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 0x7fcfc81f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7400640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 0x7fcfc81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfc7e00640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8011840 tx=0x7fcfb8011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8010040 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb801a070 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800fb50 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc81589d0 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc8158f90 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.744+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc806aa30 con 0x7fcfac006a20 2024-05-14T11:17:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.748+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcfb800fcf0 con 0x7fcfac006a20 2024-05-14T11:17:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.748+0000 7fcfbfe00640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 0x7fcfa8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.748+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7fcfb80925d0 con 0x7fcfac006a20 2024-05-14T11:17:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.748+0000 7fcfc7400640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 0x7fcfa8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.748+0000 7fcfc7400640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 0x7fcfa8071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009ae0 tx=0x7fcfb40046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.752+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcfb805f240 con 0x7fcfac006a20 2024-05-14T11:17:45.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:45.908+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcfc806d4f0 con 0x7fcfac006a20 2024-05-14T11:17:46.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfbfe00640 1 -- 172.21.15.116:0/1048725704 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7fcfb805eb40 con 0x7fcfac006a20 2024-05-14T11:17:46.565 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:17:46.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 msgr2=0x7fcfa8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:46.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 0x7fcfa8071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009ae0 tx=0x7fcfb40046d0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 msgr2=0x7fcfc8157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8011840 tx=0x7fcfb8011d00 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.564+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 shutdown_connections 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcfa806f400 0x7fcfa8071910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfc813ce30 0x7fcfc81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfc813d240 0x7fcfc81f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 --2- 172.21.15.116:0/1048725704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfac006a20 0x7fcfc8157e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 >> 172.21.15.116:0/1048725704 conn(0x7fcfc80cde80 msgr2=0x7fcfc813ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:46.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 shutdown_connections 2024-05-14T11:17:46.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.568+0000 7fcfcf200640 1 -- 172.21.15.116:0/1048725704 wait complete. 2024-05-14T11:17:46.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.736+0000 7fa8e7400640 1 Processor -- start 2024-05-14T11:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.736+0000 7fa8e7400640 1 -- start start 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa8e00b5f30 0x7fa8e00b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e00cf280 0x7fa8e00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e015d0f0 con 0x7fa8e00cf280 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e00b6880 con 0x7fa8e00b5f30 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e7400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e006d8e0 con 0x7fa8e006cf90 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e00cf280 0x7fa8e00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e00cf280 0x7fa8e00cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46230/0 (socket says 172.21.15.116:46230) 2024-05-14T11:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 -- 172.21.15.116:0/287088189 learned_addr learned my addr 172.21.15.116:0/287088189 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8dfe00640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e4a00640 1 --2- 172.21.15.116:0/287088189 >> v2:172.21.15.149:3300/0 conn(0x7fa8e00b5f30 0x7fa8e00b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 -- 172.21.15.116:0/287088189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 msgr2=0x7fa8e006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 -- 172.21.15.116:0/287088189 >> v2:172.21.15.149:3300/0 conn(0x7fa8e00b5f30 msgr2=0x7fa8e00b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 --2- 172.21.15.116:0/287088189 >> v2:172.21.15.149:3300/0 conn(0x7fa8e00b5f30 0x7fa8e00b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.740+0000 7fa8e5400640 1 -- 172.21.15.116:0/287088189 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e015c800 con 0x7fa8e00cf280 2024-05-14T11:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e5400640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e00cf690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ceb40 tx=0x7fa8d801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adde37ad78bca030 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8df400640 1 -- 172.21.15.116:0/287088189 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80a56e0 con 0x7fa8e00cf280 2024-05-14T11:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8df400640 1 -- 172.21.15.116:0/287088189 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d80a5880 con 0x7fa8e00cf280 2024-05-14T11:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8df400640 1 -- 172.21.15.116:0/287088189 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80a5c50 con 0x7fa8e00cf280 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 -- 172.21.15.116:0/287088189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 msgr2=0x7fa8e00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e00cf690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00ceb40 tx=0x7fa8d801c9f0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 -- 172.21.15.116:0/287088189 shutdown_connections 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e00cf690 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 --2- 172.21.15.116:0/287088189 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 --2- 172.21.15.116:0/287088189 >> v2:172.21.15.149:3300/0 conn(0x7fa8e00b5f30 0x7fa8e00b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 -- 172.21.15.116:0/287088189 >> 172.21.15.116:0/287088189 conn(0x7fa8e00cde80 msgr2=0x7fa8e00ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 -- 172.21.15.116:0/287088189 shutdown_connections 2024-05-14T11:17:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.744+0000 7fa8e7400640 1 -- 172.21.15.116:0/287088189 wait complete. 2024-05-14T11:17:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 Processor -- start 2024-05-14T11:17:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- start start 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e00b5f30 0x7fa8e014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e015d580 con 0x7fa8e00cf280 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00cffe0 con 0x7fa8e00b5f30 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b1f30 con 0x7fa8e006cf90 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e4a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46240/0 (socket says 172.21.15.116:46240) 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 -- 172.21.15.116:0/2081468045 learned_addr learned my addr 172.21.15.116:0/2081468045 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8dfe00640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e00b5f30 0x7fa8e014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 -- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 msgr2=0x7fa8e0156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e0156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 -- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e00b5f30 msgr2=0x7fa8e014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e00b5f30 0x7fa8e014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 -- 172.21.15.116:0/2081468045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e0158fd0 con 0x7fa8e00cf280 2024-05-14T11:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e5400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009e50 tx=0x7fa8d8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80c4070 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d80b55d0 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8009510 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e01592b0 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.748+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e0159870 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.752+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e006aa30 con 0x7fa8e00cf280 2024-05-14T11:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.752+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa8d80b59a0 con 0x7fa8e00cf280 2024-05-14T11:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.756+0000 7fa8dcc00640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 0x7fa8bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.756+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5749+0+0 (secure 0 0 0) 0x7fa8d812d470 con 0x7fa8e00cf280 2024-05-14T11:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.756+0000 7fa8e4a00640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 0x7fa8bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.756+0000 7fa8e4a00640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 0x7fa8bc071940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007940 tx=0x7fa8d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.760+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa8d80ba070 con 0x7fa8e00cf280 2024-05-14T11:17:46.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:46.924+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8e014ab10 con 0x7fa8e00cf280 2024-05-14T11:17:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8dcc00640 1 -- 172.21.15.116:0/2081468045 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7fa8d80165e0 con 0x7fa8e00cf280 2024-05-14T11:17:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 msgr2=0x7fa8bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 0x7fa8bc071940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007940 tx=0x7fa8d0008040 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 msgr2=0x7fa8e014beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009e50 tx=0x7fa8d8019790 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 shutdown_connections 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa8bc06f430 0x7fa8bc071940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e00b5f30 0x7fa8e014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e00cf280 0x7fa8e014beb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 --2- 172.21.15.116:0/2081468045 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8e006cf90 0x7fa8e0156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.572+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 >> 172.21.15.116:0/2081468045 conn(0x7fa8e00cde80 msgr2=0x7fa8e00b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.576+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 shutdown_connections 2024-05-14T11:17:47.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.576+0000 7fa8e7400640 1 -- 172.21.15.116:0/2081468045 wait complete. 2024-05-14T11:17:47.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:17:47.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 Processor -- start 2024-05-14T11:17:47.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 -- start start 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f243815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24380cde90 con 0x7f2438161da0 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24380ce170 con 0x7f24380ce5f0 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f243cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f24380d2b30 con 0x7f24381621b0 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2435400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2438161da0 0x7f243815d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58090/0 (socket says 172.21.15.116:58090) 2024-05-14T11:17:47.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2435400640 1 -- 172.21.15.116:0/769573439 learned_addr learned my addr 172.21.15.116:0/769573439 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:47.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924425186 0 0) 0x7f24380cde90 con 0x7f2438161da0 2024-05-14T11:17:47.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2436800640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f243815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f241c003530 con 0x7f2438161da0 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 652759122 0 0) 0x7f241c003530 con 0x7f2438161da0 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 msgr2=0x7f243815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f243815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 >> v1:172.21.15.149:6789/0 conn(0x7f24380ce5f0 legacy=0x7f2438161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24380d27c0 con 0x7f2438161da0 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f2424001700 con 0x7f2438161da0 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 >> v1:172.21.15.116:6789/0 conn(0x7f2438161da0 legacy=0x7f243815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f241c003a40 0x7f241c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f241c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f241c004d50 0x7f241c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f24380d2b30 con 0x7f241c003a40 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f241c00bb90 con 0x7f241c004d50 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.756+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f241c00bd90 con 0x7f24381621b0 2024-05-14T11:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f241c003a40 0x7f241c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435400640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f241c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2436800640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f241c004d50 0x7f241c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 -- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 msgr2=0x7f241c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f24381621b0 0x7f241c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 -- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f241c004d50 msgr2=0x7f241c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f241c004d50 0x7f241c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 -- 172.21.15.116:0/769573439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c003530 con 0x7f241c003a40 2024-05-14T11:17:47.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2435e00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f241c003a40 0x7f241c003eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f24380cfb10 tx=0x7f242801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f202c1d75f74f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:47.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24280013e0 con 0x7f241c003a40 2024-05-14T11:17:47.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24280052d0 con 0x7f241c003a40 2024-05-14T11:17:47.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.760+0000 7f2434a00640 1 -- 172.21.15.116:0/769573439 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428003840 con 0x7f241c003a40 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- 172.21.15.116:0/769573439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f241c003a40 msgr2=0x7f241c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 --2- 172.21.15.116:0/769573439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f241c003a40 0x7f241c003eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f24380cfb10 tx=0x7f242801c3a0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f2436800640 1 -- 172.21.15.116:0/769573439 reap_dead start 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- 172.21.15.116:0/769573439 shutdown_connections 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- 172.21.15.116:0/769573439 >> 172.21.15.116:0/769573439 conn(0x7f24380d0670 msgr2=0x7f24380d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- 172.21.15.116:0/769573439 shutdown_connections 2024-05-14T11:17:47.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- 172.21.15.116:0/769573439 wait complete. 2024-05-14T11:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 Processor -- start 2024-05-14T11:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.764+0000 7f243cc00640 1 -- start start 2024-05-14T11:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f241c003a40 0x7f2438152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f243806fb90 0x7f24380b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2438169a30 con 0x7f243815b690 2024-05-14T11:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24380cec10 con 0x7f243806fb90 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2438169bb0 con 0x7f241c003a40 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47332/0 (socket says 172.21.15.116:47332) 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 -- 172.21.15.116:0/3902244304 learned_addr learned my addr 172.21.15.116:0/3902244304 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435e00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f241c003a40 0x7f2438152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2436800640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f243806fb90 0x7f24380b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 -- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f241c003a40 msgr2=0x7f2438152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f241c003a40 0x7f2438152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 -- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f243806fb90 msgr2=0x7f24380b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f243806fb90 0x7f24380b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 -- 172.21.15.116:0/3902244304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24380c70c0 con 0x7f243815b690 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f2435400640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2424007310 tx=0x7f24240077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242401a070 con 0x7f243815b690 2024-05-14T11:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242400f960 con 0x7f243815b690 2024-05-14T11:17:47.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400fd10 con 0x7f243815b690 2024-05-14T11:17:47.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243815b050 con 0x7f243815b690 2024-05-14T11:17:47.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24380b65e0 con 0x7f243815b690 2024-05-14T11:17:47.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243806a9c0 con 0x7f243815b690 2024-05-14T11:17:47.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.768+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2424012350 con 0x7f243815b690 2024-05-14T11:17:47.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.772+0000 7f242e000640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 0x7f2418071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:47.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.772+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f242408e880 con 0x7f243815b690 2024-05-14T11:17:47.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.772+0000 7f2435e00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 0x7f2418071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:47.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.772+0000 7f2435e00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 0x7f2418071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24280ac000 tx=0x7f24280011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:47.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.776+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f242405b3b0 con 0x7f243815b690 2024-05-14T11:17:47.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.932+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f24380d14c0 con 0x7f243815b690 2024-05-14T11:17:47.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f242e000640 1 -- 172.21.15.116:0/3902244304 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f242405f6b0 con 0x7f243815b690 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 msgr2=0x7f2418071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 0x7f2418071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24280ac000 tx=0x7f24280011a0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 msgr2=0x7f243806f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2424007310 tx=0x7f24240077d0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 shutdown_connections 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f241806f430 0x7f2418071940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f243806fb90 0x7f24380b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f243815b690 0x7f243806f650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 --2- 172.21.15.116:0/3902244304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f241c003a40 0x7f2438152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:47.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 >> 172.21.15.116:0/3902244304 conn(0x7f24380d0670 msgr2=0x7f24380b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:47.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 shutdown_connections 2024-05-14T11:17:47.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:47.940+0000 7f243cc00640 1 -- 172.21.15.116:0/3902244304 wait complete. 2024-05-14T11:17:47.962 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-14T11:17:48.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 Processor -- start 2024-05-14T11:17:48.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 -- start start 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2700d1670 0x7fe2700d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe2700af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2700d1fc0 con 0x7fe270156fa0 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe2700706f0 con 0x7fe2700d1670 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe275e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe2701573b0 con 0x7fe2700af380 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2700d1670 0x7fe2700d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2700d1670 0x7fe2700d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47334/0 (socket says 172.21.15.116:47334) 2024-05-14T11:17:48.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 -- 172.21.15.116:0/4049357563 learned_addr learned my addr 172.21.15.116:0/4049357563 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:48.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe26f400640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe2700af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 -- 172.21.15.116:0/4049357563 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 msgr2=0x7fe2700af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:48.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe2700af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 -- 172.21.15.116:0/4049357563 >> v1:172.21.15.149:6789/0 conn(0x7fe270156fa0 legacy=0x7fe27006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.108+0000 7fe266000640 1 -- 172.21.15.116:0/4049357563 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe270160510 con 0x7fe2700d1670 2024-05-14T11:17:48.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe266000640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe2700d1a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb496940941831a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe26e000640 1 -- 172.21.15.116:0/4049357563 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580ab650 con 0x7fe2700d1670 2024-05-14T11:17:48.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe26e000640 1 -- 172.21.15.116:0/4049357563 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580b0070 con 0x7fe2700d1670 2024-05-14T11:17:48.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe26e000640 1 -- 172.21.15.116:0/4049357563 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580aba10 con 0x7fe2700d1670 2024-05-14T11:17:48.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 -- 172.21.15.116:0/4049357563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 msgr2=0x7fe2700d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe2700d1a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25801c560 comp rx=0 tx=0).stop 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 -- 172.21.15.116:0/4049357563 shutdown_connections 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe2700af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 --2- 172.21.15.116:0/4049357563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe2700d1a80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 -- 172.21.15.116:0/4049357563 >> 172.21.15.116:0/4049357563 conn(0x7fe2700b4620 msgr2=0x7fe2700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 -- 172.21.15.116:0/4049357563 shutdown_connections 2024-05-14T11:17:48.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.112+0000 7fe275e00640 1 -- 172.21.15.116:0/4049357563 wait complete. 2024-05-14T11:17:48.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 Processor -- start 2024-05-14T11:17:48.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 -- start start 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe270151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe270156fa0 0x7fe270073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe2700b0660 con 0x7fe2700d1670 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe2700b04e0 con 0x7fe270156fa0 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe275e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe270169810 con 0x7fe2700af380 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47340/0 (socket says 172.21.15.116:47340) 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 -- 172.21.15.116:0/2257194875 learned_addr learned my addr 172.21.15.116:0/2257194875 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe26f400640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe270156fa0 0x7fe270073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe26ea00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe270151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 -- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 msgr2=0x7fe270151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe270151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 -- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe270156fa0 msgr2=0x7fe270073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe270156fa0 0x7fe270073a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 -- 172.21.15.116:0/2257194875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe270151150 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe266000640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe25801a190 tx=0x7fe25801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.116+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580be5f0 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580be790 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580beb80 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe270151430 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2700cf130 con 0x7fe2700d1670 2024-05-14T11:17:48.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe270152080 con 0x7fe2700d1670 2024-05-14T11:17:48.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2580b43f0 con 0x7fe2700d1670 2024-05-14T11:17:48.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe267e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 0x7fe24c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fe258129f30 con 0x7fe2700d1670 2024-05-14T11:17:48.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe26ea00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 0x7fe24c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.120+0000 7fe26ea00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 0x7fe24c071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe260007fd0 tx=0x7fe260009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.124+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe2580c4060 con 0x7fe2700d1670 2024-05-14T11:17:48.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.336+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fe27006a9c0 con 0x7fe2700d1670 2024-05-14T11:17:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe267e00640 1 -- 172.21.15.116:0/2257194875 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v110) ==== 142+0+0 (secure 0 0 0) 0x7fe258016150 con 0x7fe2700d1670 2024-05-14T11:17:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 msgr2=0x7fe24c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 0x7fe24c071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe260007fd0 tx=0x7fe260009040 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 msgr2=0x7fe27014fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe25801a190 tx=0x7fe25801ada0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 shutdown_connections 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe24c06f400 0x7fe24c071910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe270156fa0 0x7fe270073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2700d1670 0x7fe27014fb10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 --2- 172.21.15.116:0/2257194875 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2700af380 0x7fe270151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 >> 172.21.15.116:0/2257194875 conn(0x7fe2700b4620 msgr2=0x7fe2700b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 shutdown_connections 2024-05-14T11:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.576+0000 7fe275e00640 1 -- 172.21.15.116:0/2257194875 wait complete. 2024-05-14T11:17:48.598 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-14T11:17:48.598 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-14T11:17:48.598 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:17:48.599 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:48.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 Processor -- start 2024-05-14T11:17:48.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 -- start start 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8cf406e200 0x7f8cf40b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf40b5980 0x7f8cf406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406eff0 0x7f8cf406f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf4163790 con 0x7f8cf40b5980 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf40d2200 con 0x7f8cf406e200 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf40b5d90 con 0x7f8cf406eff0 2024-05-14T11:17:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf40b5980 0x7f8cf406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf40b5980 0x7f8cf406eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47350/0 (socket says 172.21.15.116:47350) 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 -- 172.21.15.116:0/4138033700 learned_addr learned my addr 172.21.15.116:0/4138033700 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3e00640 1 --2- 172.21.15.116:0/4138033700 >> v2:172.21.15.149:3300/0 conn(0x7f8cf406e200 0x7f8cf40b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf9200640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406eff0 0x7f8cf406f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 -- 172.21.15.116:0/4138033700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406eff0 msgr2=0x7f8cf406f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406eff0 0x7f8cf406f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 -- 172.21.15.116:0/4138033700 >> v2:172.21.15.149:3300/0 conn(0x7f8cf406e200 msgr2=0x7f8cf40b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 --2- 172.21.15.116:0/4138033700 >> v2:172.21.15.149:3300/0 conn(0x7f8cf406e200 0x7f8cf40b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.744+0000 7f8cf3400640 1 -- 172.21.15.116:0/4138033700 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41639e0 con 0x7f8cf40b5980 2024-05-14T11:17:48.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf3400640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf406eab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40b5f40 tx=0x7f8ce001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46bc8fd6f052bbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf2a00640 1 -- 172.21.15.116:0/4138033700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00b0070 con 0x7f8cf40b5980 2024-05-14T11:17:48.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf2a00640 1 -- 172.21.15.116:0/4138033700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce00ab600 con 0x7f8cf40b5980 2024-05-14T11:17:48.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf2a00640 1 -- 172.21.15.116:0/4138033700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00ab9d0 con 0x7f8cf40b5980 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 -- 172.21.15.116:0/4138033700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 msgr2=0x7f8cf406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf406eab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40b5f40 tx=0x7f8ce001c400 comp rx=0 tx=0).stop 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 -- 172.21.15.116:0/4138033700 shutdown_connections 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf406eab0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/4138033700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406eff0 0x7f8cf406f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/4138033700 >> v2:172.21.15.149:3300/0 conn(0x7f8cf406e200 0x7f8cf40b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 -- 172.21.15.116:0/4138033700 >> 172.21.15.116:0/4138033700 conn(0x7f8cf40d0550 msgr2=0x7f8cf40d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 -- 172.21.15.116:0/4138033700 shutdown_connections 2024-05-14T11:17:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.748+0000 7f8cf9c00640 1 -- 172.21.15.116:0/4138033700 wait complete. 2024-05-14T11:17:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 Processor -- start 2024-05-14T11:17:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 -- start start 2024-05-14T11:17:48.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 0x7f8cf40b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf406eff0 0x7f8cf40b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf406e610 con 0x7f8cf40b5980 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf40cdd00 con 0x7f8cf406eff0 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf406f940 con 0x7f8cf406e200 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47352/0 (socket says 172.21.15.116:47352) 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 -- 172.21.15.116:0/588947769 learned_addr learned my addr 172.21.15.116:0/588947769 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf3400640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf406eff0 0x7f8cf40b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf3e00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 0x7f8cf40b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 -- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 msgr2=0x7f8cf40b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 0x7f8cf40b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 -- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf406eff0 msgr2=0x7f8cf40b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf406eff0 0x7f8cf40b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf9200640 1 -- 172.21.15.116:0/588947769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf40caff0 con 0x7f8cf40b5980 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.752+0000 7f8cf3e00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 0x7f8cf40b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:48.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cf9200640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00efc0 tx=0x7f8cec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec018020 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec014480 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec014870 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf40cb2d0 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf40cb890 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf4050880 con 0x7f8cf40b5980 2024-05-14T11:17:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8cec01b020 con 0x7f8cf40b5980 2024-05-14T11:17:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 0x7f8cd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f8cec08fe70 con 0x7f8cf40b5980 2024-05-14T11:17:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.756+0000 7f8cf3e00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 0x7f8cd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:48.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.760+0000 7f8cf3e00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 0x7f8cd4071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009a90 tx=0x7f8ce4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:48.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.760+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8cec022030 con 0x7f8cf40b5980 2024-05-14T11:17:48.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.920+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cf4150f40 con 0x7f8cf40b5980 2024-05-14T11:17:48.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.920+0000 7f8cebe00640 1 -- 172.21.15.116:0/588947769 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13343 (secure 0 0 0) 0x7f8cec010040 con 0x7f8cf40b5980 2024-05-14T11:17:48.924 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:48.924 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":41,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:47.568102+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6837/2525866958":"2024-05-15T11:17:44.555775+0000","172.21.15.116:6836/2525866958":"2024-05-15T11:17:44.555775+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:48.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 msgr2=0x7f8cd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 0x7f8cd4071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009a90 tx=0x7f8ce4008040 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 msgr2=0x7f8cf40ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00efc0 tx=0x7f8cec00c490 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 shutdown_connections 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cd406f400 0x7f8cd4071910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf406eff0 0x7f8cf40b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf40b5980 0x7f8cf40ca550 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 --2- 172.21.15.116:0/588947769 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8cf406e200 0x7f8cf40b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 >> 172.21.15.116:0/588947769 conn(0x7f8cf40d0550 msgr2=0x7f8cf40d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 shutdown_connections 2024-05-14T11:17:48.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:48.924+0000 7f8cf9c00640 1 -- 172.21.15.116:0/588947769 wait complete. 2024-05-14T11:17:48.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:17:48.947 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:17:49.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.092+0000 7f26a0c00640 1 Processor -- start 2024-05-14T11:17:49.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.092+0000 7f26a0c00640 1 -- start start 2024-05-14T11:17:49.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f26a0c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f269c06f450 0x7f269c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f26a0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f269c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f26a0c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c06f860 con 0x7f269c0cce60 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f26a0c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f269c06db80 con 0x7f269c06f450 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f26a0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0d2080 con 0x7f269c0d1730 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f269a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f269c0cce60 0x7f269c0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58102/0 (socket says 172.21.15.116:58102) 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f269a800640 1 -- 172.21.15.116:0/1135863030 learned_addr learned my addr 172.21.15.116:0/1135863030 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 --2- 172.21.15.116:0/1135863030 >> v2:172.21.15.149:3300/0 conn(0x7f269c06f450 0x7f269c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699400640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f269c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213682506 0 0) 0x7f2694001120 con 0x7f269c0cce60 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2680003530 con 0x7f269c0cce60 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 443053331 0 0) 0x7f2680003530 con 0x7f269c0cce60 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 msgr2=0x7f269c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f269c0d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 >> v2:172.21.15.149:3300/0 conn(0x7f269c06f450 msgr2=0x7f269c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 --2- 172.21.15.116:0/1135863030 >> v2:172.21.15.149:3300/0 conn(0x7f269c06f450 0x7f269c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c06fae0 con 0x7f269c0cce60 2024-05-14T11:17:49.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f26940018d0 con 0x7f269c0cce60 2024-05-14T11:17:49.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 >> v1:172.21.15.116:6789/0 conn(0x7f269c0cce60 legacy=0x7f269c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f2680003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f2680004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2680004d50 0x7f2680005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f269c06db80 con 0x7f2680003a10 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0d2080 con 0x7f2680004d50 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f268000ba80 con 0x7f269c0d1730 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f2680003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699400640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f2680004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f269a800640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2680004d50 0x7f2680005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 -- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 msgr2=0x7f2680004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f2680004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 -- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2680004d50 msgr2=0x7f2680005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2680004d50 0x7f2680005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699e00640 1 -- 172.21.15.116:0/1135863030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2680003530 con 0x7f2680003a10 2024-05-14T11:17:49.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.096+0000 7f2699400640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c0d1730 0x7f2680004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:49.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f2699e00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f2680003e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f268c009f90 tx=0x7f268c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a6b995bf2c2cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0b2070 con 0x7f2680003a10 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c0a5a70 con 0x7f2680003a10 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f2698a00640 1 -- 172.21.15.116:0/1135863030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c009c10 con 0x7f2680003a10 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 -- 172.21.15.116:0/1135863030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 msgr2=0x7f2680003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1135863030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f2680003e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f268c009f90 tx=0x7f268c01cc80 comp rx=0 tx=0).stop 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f269a800640 1 -- 172.21.15.116:0/1135863030 reap_dead start 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 -- 172.21.15.116:0/1135863030 shutdown_connections 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 -- 172.21.15.116:0/1135863030 >> 172.21.15.116:0/1135863030 conn(0x7f269c0b4720 msgr2=0x7f269c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 -- 172.21.15.116:0/1135863030 shutdown_connections 2024-05-14T11:17:49.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.100+0000 7f26a0c00640 1 -- 172.21.15.116:0/1135863030 wait complete. 2024-05-14T11:17:49.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 Processor -- start 2024-05-14T11:17:49.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 -- start start 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 0x7f269c1fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c0b1ab0 0x7f269c0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f269c168600 con 0x7f2680003a10 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f269c070660 con 0x7f269c0b1ab0 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f26a0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f269c168780 con 0x7f269c09a230 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f2699e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f2699e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47376/0 (socket says 172.21.15.116:47376) 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f2699e00640 1 -- 172.21.15.116:0/1087016321 learned_addr learned my addr 172.21.15.116:0/1087016321 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f269a800640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c0b1ab0 0x7f269c0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.104+0000 7f2699400640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 0x7f269c1fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 -- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 msgr2=0x7f269c1fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 0x7f269c1fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 -- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c0b1ab0 msgr2=0x7f269c0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c0b1ab0 0x7f269c0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 -- 172.21.15.116:0/1087016321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c1fdf00 con 0x7f2680003a10 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699400640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 0x7f269c1fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:49.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2699e00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f268c01cab0 tx=0x7f268c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0b2040 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c0bc070 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c1fe1e0 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0ce190 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c15ad80 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c0ce830 con 0x7f2680003a10 2024-05-14T11:17:49.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f268c0adce0 con 0x7f2680003a10 2024-05-14T11:17:49.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.108+0000 7f2692000640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.112+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f268c129aa0 con 0x7f2680003a10 2024-05-14T11:17:49.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.112+0000 7f2699400640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.112+0000 7f2699400640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 0x7f267c071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f269c15a740 tx=0x7f2688008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.116+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f268c016990 con 0x7f2680003a10 2024-05-14T11:17:49.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.272+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f269c06a9c0 con 0x7f2680003a10 2024-05-14T11:17:49.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.272+0000 7f2692000640 1 -- 172.21.15.116:0/1087016321 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f268c0f6520 con 0x7f2680003a10 2024-05-14T11:17:49.275 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 msgr2=0x7f267c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 0x7f267c071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f269c15a740 tx=0x7f2688008040 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 msgr2=0x7f269c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f268c01cab0 tx=0x7f268c009750 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 shutdown_connections 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f267c06f400 0x7f267c071910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f269c0b1ab0 0x7f269c0b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f269c09a230 0x7f269c1fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 --2- 172.21.15.116:0/1087016321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2680003a10 0x7f269c15b580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 >> 172.21.15.116:0/1087016321 conn(0x7f269c0b4720 msgr2=0x7f269c1677b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 shutdown_connections 2024-05-14T11:17:49.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.276+0000 7f26a0c00640 1 -- 172.21.15.116:0/1087016321 wait complete. 2024-05-14T11:17:49.296 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-14T11:17:49.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 Processor -- start 2024-05-14T11:17:49.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 -- start start 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe7580b70a0 0x7fe7580b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75815fe00 0x7fe7580b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7581638a0 0x7fe758163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe7580b94d0 con 0x7fe75815fe00 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe7580cfeb0 con 0x7fe7580b70a0 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe75e400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe7580cf110 con 0x7fe7581638a0 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe74ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75815fe00 0x7fe7580b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe74ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75815fe00 0x7fe7580b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47388/0 (socket says 172.21.15.116:47388) 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.440+0000 7fe74ea00640 1 -- 172.21.15.116:0/3498642336 learned_addr learned my addr 172.21.15.116:0/3498642336 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe757e00640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7581638a0 0x7fe758163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe757400640 1 --2- 172.21.15.116:0/3498642336 >> v2:172.21.15.149:3300/0 conn(0x7fe7580b70a0 0x7fe7580b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 -- 172.21.15.116:0/3498642336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7581638a0 msgr2=0x7fe758163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7581638a0 0x7fe758163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 -- 172.21.15.116:0/3498642336 >> v2:172.21.15.149:3300/0 conn(0x7fe7580b70a0 msgr2=0x7fe7580b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 --2- 172.21.15.116:0/3498642336 >> v2:172.21.15.149:3300/0 conn(0x7fe7580b70a0 0x7fe7580b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 -- 172.21.15.116:0/3498642336 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7580b6d80 con 0x7fe75815fe00 2024-05-14T11:17:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe74ea00640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580b8f90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe7580d4410 tx=0x7fe74001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cc0c3a453dcb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe756a00640 1 -- 172.21.15.116:0/3498642336 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400b1070 con 0x7fe75815fe00 2024-05-14T11:17:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe756a00640 1 -- 172.21.15.116:0/3498642336 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7400ac630 con 0x7fe75815fe00 2024-05-14T11:17:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.444+0000 7fe756a00640 1 -- 172.21.15.116:0/3498642336 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400a8050 con 0x7fe75815fe00 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- 172.21.15.116:0/3498642336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 msgr2=0x7fe7580b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580b8f90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe7580d4410 tx=0x7fe74001c520 comp rx=0 tx=0).stop 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- 172.21.15.116:0/3498642336 shutdown_connections 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580b8f90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 --2- 172.21.15.116:0/3498642336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7581638a0 0x7fe758163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 --2- 172.21.15.116:0/3498642336 >> v2:172.21.15.149:3300/0 conn(0x7fe7580b70a0 0x7fe7580b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- 172.21.15.116:0/3498642336 >> 172.21.15.116:0/3498642336 conn(0x7fe7580d3740 msgr2=0x7fe7580d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:49.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- 172.21.15.116:0/3498642336 shutdown_connections 2024-05-14T11:17:49.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- 172.21.15.116:0/3498642336 wait complete. 2024-05-14T11:17:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 Processor -- start 2024-05-14T11:17:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 -- start start 2024-05-14T11:17:49.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.448+0000 7fe75e400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7580b70a0 0x7fe758157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7581638a0 0x7fe7580cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7580b69c0 con 0x7fe75815fe00 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe758164600 con 0x7fe7581638a0 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe7580b6b40 con 0x7fe7580b70a0 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47398/0 (socket says 172.21.15.116:47398) 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 -- 172.21.15.116:0/583359061 learned_addr learned my addr 172.21.15.116:0/583359061 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe757400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7580b70a0 0x7fe758157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe757e00640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7581638a0 0x7fe7580cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 -- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7580b70a0 msgr2=0x7fe758157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7580b70a0 0x7fe758157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 -- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7581638a0 msgr2=0x7fe7580cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7581638a0 0x7fe7580cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 -- 172.21.15.116:0/583359061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581579b0 con 0x7fe75815fe00 2024-05-14T11:17:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74ea00640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe74001abd0 tx=0x7fe7400192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400bc070 con 0x7fe75815fe00 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7400ab5e0 con 0x7fe75815fe00 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400b1070 con 0x7fe75815fe00 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe758070ea0 con 0x7fe75815fe00 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe758162d80 con 0x7fe75815fe00 2024-05-14T11:17:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75806a9c0 con 0x7fe75815fe00 2024-05-14T11:17:49.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.452+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7400ab9d0 con 0x7fe75815fe00 2024-05-14T11:17:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.456+0000 7fe74fe00640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 0x7fe734071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.456+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fe740128b50 con 0x7fe75815fe00 2024-05-14T11:17:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.456+0000 7fe757400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 0x7fe734071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.456+0000 7fe757400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 0x7fe734071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7480045f0 tx=0x7fe748008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:49.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.460+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe740016110 con 0x7fe75815fe00 2024-05-14T11:17:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:49.672+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fe7580b4c10 con 0x7fe75815fe00 2024-05-14T11:17:50.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.580+0000 7fe74fe00640 1 -- 172.21.15.116:0/583359061 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v112) ==== 152+0+0 (secure 0 0 0) 0x7fe7400f5550 con 0x7fe75815fe00 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 msgr2=0x7fe734071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 0x7fe734071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7480045f0 tx=0x7fe748008040 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 msgr2=0x7fe7580cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe74001abd0 tx=0x7fe7400192c0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 shutdown_connections 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe73406f430 0x7fe734071940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7581638a0 0x7fe7580cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75815fe00 0x7fe7580cb840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 --2- 172.21.15.116:0/583359061 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7580b70a0 0x7fe758157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 >> 172.21.15.116:0/583359061 conn(0x7fe7580d3740 msgr2=0x7fe7580b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 shutdown_connections 2024-05-14T11:17:50.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.584+0000 7fe75e400640 1 -- 172.21.15.116:0/583359061 wait complete. 2024-05-14T11:17:50.608 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:17:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 Processor -- start 2024-05-14T11:17:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 -- start start 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada806f450 0x7fada806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80d1730 0x7fada80d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada806f860 con 0x7fada80cce60 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fada806db80 con 0x7fada80d1730 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.756+0000 7fadaf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fada80d2080 con 0x7fada806f450 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80d1730 0x7fada80d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fada80d1730 0x7fada80d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47406/0 (socket says 172.21.15.116:47406) 2024-05-14T11:17:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 -- 172.21.15.116:0/4179452879 learned_addr learned my addr 172.21.15.116:0/4179452879 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fada7e00640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada806f450 0x7fada806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 -- 172.21.15.116:0/4179452879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada806f450 msgr2=0x7fada806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada806f450 0x7fada806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 -- 172.21.15.116:0/4179452879 >> v1:172.21.15.149:6789/0 conn(0x7fada80cce60 legacy=0x7fada80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 -- 172.21.15.116:0/4179452879 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada806fae0 con 0x7fada80d1730 2024-05-14T11:17:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fad9f400640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80d1730 0x7fada80d1b40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad90009870 tx=0x7fad9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cfe0c0361b9067 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fada7400640 1 -- 172.21.15.116:0/4179452879 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80ce190 con 0x7fada80d1730 2024-05-14T11:17:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fada7400640 1 -- 172.21.15.116:0/4179452879 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad900b0070 con 0x7fada80d1730 2024-05-14T11:17:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.760+0000 7fada7400640 1 -- 172.21.15.116:0/4179452879 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad900ab870 con 0x7fada80d1730 2024-05-14T11:17:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- 172.21.15.116:0/4179452879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80d1730 msgr2=0x7fada80d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80d1730 0x7fada80d1b40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad90009870 tx=0x7fad9001c560 comp rx=0 tx=0).stop 2024-05-14T11:17:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- 172.21.15.116:0/4179452879 shutdown_connections 2024-05-14T11:17:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada80d1730 0x7fada80d1b40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 --2- 172.21.15.116:0/4179452879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada806f450 0x7fada806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- 172.21.15.116:0/4179452879 >> 172.21.15.116:0/4179452879 conn(0x7fada80b4720 msgr2=0x7fada80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- 172.21.15.116:0/4179452879 shutdown_connections 2024-05-14T11:17:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- 172.21.15.116:0/4179452879 wait complete. 2024-05-14T11:17:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 Processor -- start 2024-05-14T11:17:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.764+0000 7fadaf000640 1 -- start start 2024-05-14T11:17:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada80cce60 0x7fada80cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 0x7fada815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fada816a610 con 0x7fada806f450 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fada8070660 con 0x7fada80cce60 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fada816a790 con 0x7fada80d1730 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47420/0 (socket says 172.21.15.116:47420) 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 -- 172.21.15.116:0/1936331013 learned_addr learned my addr 172.21.15.116:0/1936331013 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fad9f400640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada80cce60 0x7fada80cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadad000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 0x7fada815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 -- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 msgr2=0x7fada815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 0x7fada815bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 -- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada80cce60 msgr2=0x7fada80cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada80cce60 0x7fada80cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 -- 172.21.15.116:0/1936331013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada815ac10 con 0x7fada806f450 2024-05-14T11:17:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadad000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 0x7fada815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada7e00640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d600 tx=0x7fad9800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98004240 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad980043e0 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98003840 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada80d0130 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada8165bb0 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.768+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada806a9c0 con 0x7fada806f450 2024-05-14T11:17:50.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.772+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad980039e0 con 0x7fada806f450 2024-05-14T11:17:50.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.772+0000 7fada4c00640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:50.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.772+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fad9808f430 con 0x7fada806f450 2024-05-14T11:17:50.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.772+0000 7fad9f400640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:50.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.772+0000 7fad9f400640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad900af000 tx=0x7fad900a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:50.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.776+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fad9805c3f0 con 0x7fada806f450 2024-05-14T11:17:50.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.932+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fada8069de0 con 0x7fada806f450 2024-05-14T11:17:50.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.936+0000 7fada4c00640 1 -- 172.21.15.116:0/1936331013 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fad9805bcf0 con 0x7fada806f450 2024-05-14T11:17:50.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 msgr2=0x7fad84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad900af000 tx=0x7fad900a7000 comp rx=0 tx=0).stop 2024-05-14T11:17:50.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 msgr2=0x7fada80cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:50.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d600 tx=0x7fad9800dac0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 shutdown_connections 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fad8406f400 0x7fad84071910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada80cce60 0x7fada80cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fada80d1730 0x7fada815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 --2- 172.21.15.116:0/1936331013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fada806f450 0x7fada80cf130 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 >> 172.21.15.116:0/1936331013 conn(0x7fada80b4720 msgr2=0x7fada8073db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 shutdown_connections 2024-05-14T11:17:50.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:50.940+0000 7fadaf000640 1 -- 172.21.15.116:0/1936331013 wait complete. 2024-05-14T11:17:50.962 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:17:50.962 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:17:51.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:17:51.141 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:17:51.141 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:51.141 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:51.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:17:51.150 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:17:51.150 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:51.159 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:17:51.159 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:17:51.159 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:51.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:17:51.205 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:17:51.205 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:17:51.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:17:51.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:17:51.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:51.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 Processor -- start 2024-05-14T11:17:51.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 -- start start 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5c4167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c4160510 con 0x7fe5c4156fa0 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c41607f0 con 0x7fe5c40d1670 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c9200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c40706f0 con 0x7fe5c40b0390 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe5c4156fa0 0x7fe5c406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58104/0 (socket says 172.21.15.116:58104) 2024-05-14T11:17:51.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1e00640 1 -- 172.21.15.116:0/634396558 learned_addr learned my addr 172.21.15.116:0/634396558 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:51.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780996910 0 0) 0x7fe5c4160510 con 0x7fe5c4156fa0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c2800640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5c4167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5a8003530 con 0x7fe5c4156fa0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1380876994 0 0) 0x7fe5a8003530 con 0x7fe5c4156fa0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 msgr2=0x7fe5c4167800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.396+0000 7fe5c1400640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5c4167800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 >> v1:172.21.15.149:6789/0 conn(0x7fe5c40d1670 legacy=0x7fe5c40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c4160ad0 con 0x7fe5c4156fa0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fe5b4001a80 con 0x7fe5c4156fa0 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 >> v1:172.21.15.116:6789/0 conn(0x7fe5c4156fa0 legacy=0x7fe5c406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8003a40 0x7fe5a8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5a8004d50 0x7fe5a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c40706f0 con 0x7fe5a8004d50 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a800bb90 con 0x7fe5a8003a40 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a800dfa0 con 0x7fe5c40b0390 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1e00640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8003a40 0x7fe5a8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5a8004d50 0x7fe5a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5b9400640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 -- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 msgr2=0x7fe5a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c40b0390 0x7fe5a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 -- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8003a40 msgr2=0x7fe5a8003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8003a40 0x7fe5a8003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 -- 172.21.15.116:0/634396558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a8003530 con 0x7fe5a8004d50 2024-05-14T11:17:51.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c2800640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5a8004d50 0x7fe5a8005200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009ae0 tx=0x7fe5bc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f2c27879b1a5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0b2070 con 0x7fe5a8004d50 2024-05-14T11:17:51.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.400+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0a5680 con 0x7fe5a8004d50 2024-05-14T11:17:51.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c1400640 1 -- 172.21.15.116:0/634396558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0096b0 con 0x7fe5a8004d50 2024-05-14T11:17:51.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- 172.21.15.116:0/634396558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5a8004d50 msgr2=0x7fe5a8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 --2- 172.21.15.116:0/634396558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5a8004d50 0x7fe5a8005200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009ae0 tx=0x7fe5bc01ca90 comp rx=0 tx=0).stop 2024-05-14T11:17:51.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c2800640 1 -- 172.21.15.116:0/634396558 reap_dead start 2024-05-14T11:17:51.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- 172.21.15.116:0/634396558 shutdown_connections 2024-05-14T11:17:51.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- 172.21.15.116:0/634396558 >> 172.21.15.116:0/634396558 conn(0x7fe5c40b4620 msgr2=0x7fe5c40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:51.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- 172.21.15.116:0/634396558 shutdown_connections 2024-05-14T11:17:51.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- 172.21.15.116:0/634396558 wait complete. 2024-05-14T11:17:51.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 Processor -- start 2024-05-14T11:17:51.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.404+0000 7fe5c9200640 1 -- start start 2024-05-14T11:17:51.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 0x7fe5c4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c414fc90 0x7fe5c40cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4161120 con 0x7fe5c40d0520 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c40b07a0 con 0x7fe5a8004d50 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c406c700 con 0x7fe5c414fc90 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47440/0 (socket says 172.21.15.116:47440) 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 -- 172.21.15.116:0/2305062029 learned_addr learned my addr 172.21.15.116:0/2305062029 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 -- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c414fc90 msgr2=0x7fe5c40cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c2800640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c414fc90 0x7fe5c40cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c1e00640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 0x7fe5c4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c414fc90 0x7fe5c40cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 -- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 msgr2=0x7fe5c4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 0x7fe5c4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 -- 172.21.15.116:0/2305062029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41536a0 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c1e00640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 0x7fe5c4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5b9400640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00f820 tx=0x7fe5ac00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0050d0 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac018070 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00d730 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c409edf0 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c4151890 con 0x7fe5c40d0520 2024-05-14T11:17:51.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.408+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c406a9c0 con 0x7fe5c40d0520 2024-05-14T11:17:51.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.412+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe5ac00d8d0 con 0x7fe5c40d0520 2024-05-14T11:17:51.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.412+0000 7fe5baa00640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 0x7fe5a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.412+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7fe5ac08f110 con 0x7fe5c40d0520 2024-05-14T11:17:51.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.412+0000 7fe5c1e00640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 0x7fe5a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.412+0000 7fe5c1e00640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 0x7fe5a0071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40b0d60 tx=0x7fe5b4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.416+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe5ac05c0d0 con 0x7fe5c40d0520 2024-05-14T11:17:51.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.604+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5c409eb80 con 0x7fe5c40d0520 2024-05-14T11:17:51.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.604+0000 7fe5baa00640 1 -- 172.21.15.116:0/2305062029 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3878 (secure 0 0 0) 0x7fe5ac05b9d0 con 0x7fe5c40d0520 2024-05-14T11:17:51.609 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:51.609 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4658,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2565499573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2565499573},{"type":"v1","addr":"172.21.15.149:6833","nonce":2565499573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1732045326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1732045326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1732045326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3048933994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3048933994},{"type":"v1","addr":"172.21.15.149:6835","nonce":3048933994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/325851977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":325851977},{"type":"v1","addr":"172.21.15.116:6837","nonce":325851977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-05-14T11:17:51.609 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 msgr2=0x7fe5a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 0x7fe5a0071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40b0d60 tx=0x7fe5b4006040 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 msgr2=0x7fe5c414f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00f820 tx=0x7fe5ac00fce0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 shutdown_connections 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe5a006f400 0x7fe5a0071910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5c414fc90 0x7fe5c40cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c40d0520 0x7fe5c414f750 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 --2- 172.21.15.116:0/2305062029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5a8004d50 0x7fe5c4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 >> 172.21.15.116:0/2305062029 conn(0x7fe5c40b4620 msgr2=0x7fe5c4150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 shutdown_connections 2024-05-14T11:17:51.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.608+0000 7fe5c9200640 1 -- 172.21.15.116:0/2305062029 wait complete. 2024-05-14T11:17:51.631 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:17:51.631 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:17:51.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:17:51.631 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:51.631 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:17:51.631 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:17:51.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:17:51.631 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:51.632 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:17:51.632 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:17:51.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:17:51.632 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:51.632 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:17:51.632 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:17:51.632 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:17:51.632 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:17:51.644 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:17:51.644 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:17:51.646 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:17:51.647 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:17:51.647 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:17:51.647 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:17:51.652 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:17:51.652 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:17:51.694 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:17:51.694 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:51.696 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:17:51.696 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:51.697 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:17:51.697 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:51.699 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:17:51.699 DEBUG:teuthology.parallel:result is None 2024-05-14T11:17:51.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:17:51.795 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:17:51.798 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:17:51.807 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:17:51.807 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:17:51.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 Processor -- start 2024-05-14T11:17:51.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 -- start start 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f81380706c0 0x7f8138161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813815e270 con 0x7f8138143fc0 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81381623a0 con 0x7f8138161bb0 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f813ea00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f813815e8a0 con 0x7f81380706c0 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f8137e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8138143fc0 0x7f813806ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58120/0 (socket says 172.21.15.116:58120) 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f8137e00640 1 -- 172.21.15.116:0/1744072675 learned_addr learned my addr 172.21.15.116:0/1744072675 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:51.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f8137400640 1 --2- 172.21.15.116:0/1744072675 >> v2:172.21.15.149:3300/0 conn(0x7f81380706c0 0x7f8138161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f8136a00640 1 -- 172.21.15.116:0/1744072675 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547705356 0 0) 0x7f813815e270 con 0x7f8138143fc0 2024-05-14T11:17:51.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.856+0000 7f8136a00640 1 -- 172.21.15.116:0/1744072675 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f811c003530 con 0x7f8138143fc0 2024-05-14T11:17:51.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8137400640 1 -- 172.21.15.116:0/1744072675 >> v1:172.21.15.149:6789/0 conn(0x7f8138161bb0 legacy=0x7f813815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8137400640 1 -- 172.21.15.116:0/1744072675 >> v1:172.21.15.116:6789/0 conn(0x7f8138143fc0 legacy=0x7f813806ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8137400640 1 -- 172.21.15.116:0/1744072675 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813815e530 con 0x7f81380706c0 2024-05-14T11:17:51.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8137400640 1 --2- 172.21.15.116:0/1744072675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81380706c0 0x7f8138161670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f813815ea20 tx=0x7f812401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f127fc04ab88f04d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8136a00640 1 -- 172.21.15.116:0/1744072675 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240ab430 con 0x7f81380706c0 2024-05-14T11:17:51.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8136a00640 1 -- 172.21.15.116:0/1744072675 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81240b0070 con 0x7f81380706c0 2024-05-14T11:17:51.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.860+0000 7f8136a00640 1 -- 172.21.15.116:0/1744072675 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240ab7f0 con 0x7f81380706c0 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- 172.21.15.116:0/1744072675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81380706c0 msgr2=0x7f8138161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 --2- 172.21.15.116:0/1744072675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81380706c0 0x7f8138161670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f813815ea20 tx=0x7f812401c410 comp rx=0 tx=0).stop 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- 172.21.15.116:0/1744072675 shutdown_connections 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 --2- 172.21.15.116:0/1744072675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81380706c0 0x7f8138161670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- 172.21.15.116:0/1744072675 >> 172.21.15.116:0/1744072675 conn(0x7f8138071370 msgr2=0x7f8138071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:51.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- 172.21.15.116:0/1744072675 shutdown_connections 2024-05-14T11:17:51.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- 172.21.15.116:0/1744072675 wait complete. 2024-05-14T11:17:51.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 Processor -- start 2024-05-14T11:17:51.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.864+0000 7f813ea00640 1 -- start start 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 0x7f81381fa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8138161bb0 0x7f8138155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f813816bdb0 con 0x7f81380706c0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f813816bc30 con 0x7f8138143fc0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f813816bf30 con 0x7f8138161bb0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47492/0 (socket says 172.21.15.116:47492) 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 -- 172.21.15.116:0/4266883722 learned_addr learned my addr 172.21.15.116:0/4266883722 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137400640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 0x7f81381fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ca00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8138161bb0 0x7f8138155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 -- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8138161bb0 msgr2=0x7f8138155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8138161bb0 0x7f8138155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 -- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 msgr2=0x7f81381fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 0x7f81381fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 -- 172.21.15.116:0/4266883722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81380748f0 con 0x7f81380706c0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137400640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 0x7f81381fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f8137e00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f812800c430 tx=0x7f812800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128017070 con 0x7f81380706c0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8128005cf0 con 0x7f81380706c0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8138154860 con 0x7f81380706c0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280072f0 con 0x7f81380706c0 2024-05-14T11:17:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813815a7c0 con 0x7f81380706c0 2024-05-14T11:17:51.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813806a9c0 con 0x7f81380706c0 2024-05-14T11:17:51.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.868+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f812801a070 con 0x7f81380706c0 2024-05-14T11:17:51.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.872+0000 7f812fe00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 0x7f8118071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:51.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.872+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5335+0+0 (secure 0 0 0) 0x7f812808e5a0 con 0x7f81380706c0 2024-05-14T11:17:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.872+0000 7f8137400640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 0x7f8118071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.872+0000 7f8137400640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 0x7f8118071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81240af000 tx=0x7f81240a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:51.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:51.876+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f812805b620 con 0x7f81380706c0 2024-05-14T11:17:52.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.044+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8138159120 con 0x7f81380706c0 2024-05-14T11:17:52.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.600+0000 7f812fe00640 1 -- 172.21.15.116:0/4266883722 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7f812805af20 con 0x7f81380706c0 2024-05-14T11:17:52.603 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:17:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 msgr2=0x7f8118071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 0x7f8118071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81240af000 tx=0x7f81240a7000 comp rx=0 tx=0).stop 2024-05-14T11:17:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 msgr2=0x7f8138142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f812800c430 tx=0x7f812800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 shutdown_connections 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f811806f400 0x7f8118071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8138143fc0 0x7f81381fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8138161bb0 0x7f8138155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 --2- 172.21.15.116:0/4266883722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81380706c0 0x7f8138142b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 >> 172.21.15.116:0/4266883722 conn(0x7f8138071370 msgr2=0x7f81380b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 shutdown_connections 2024-05-14T11:17:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.604+0000 7f813ea00640 1 -- 172.21.15.116:0/4266883722 wait complete. 2024-05-14T11:17:52.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:17:52.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.776+0000 7f967fe00640 1 Processor -- start 2024-05-14T11:17:52.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 -- start start 2024-05-14T11:17:52.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f96780aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f96780d1670 0x7f96780d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967806ff20 con 0x7f9678161600 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f967806d710 con 0x7f96780d1670 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f96780ae100 con 0x7f967815db30 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9678161600 0x7f9678161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58122/0 (socket says 172.21.15.116:58122) 2024-05-14T11:17:52.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967de00640 1 -- 172.21.15.116:0/1767354346 learned_addr learned my addr 172.21.15.116:0/1767354346 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:52.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f96780aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967ca00640 1 --2- 172.21.15.116:0/1767354346 >> v2:172.21.15.149:3300/0 conn(0x7f96780d1670 0x7f96780d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923649984 0 0) 0x7f967806ff20 con 0x7f9678161600 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9660003530 con 0x7f9678161600 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3436071694 0 0) 0x7f9660003530 con 0x7f9678161600 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 msgr2=0x7f96780aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f96780aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 >> v2:172.21.15.149:3300/0 conn(0x7f96780d1670 msgr2=0x7f96780d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 --2- 172.21.15.116:0/1767354346 >> v2:172.21.15.149:3300/0 conn(0x7f96780d1670 0x7f96780d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96780d1fc0 con 0x7f9678161600 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9670001a80 con 0x7f9678161600 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 >> v1:172.21.15.116:6789/0 conn(0x7f9678161600 legacy=0x7f9678161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f9660003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f9660004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9660004d50 0x7f9660005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f96780ae100 con 0x7f9660004d50 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f966000ba80 con 0x7f9660003a10 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f966000bc80 con 0x7f967815db30 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967de00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9660004d50 0x7f9660005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f9660003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967ca00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f9660004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 -- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 msgr2=0x7f9660004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f9660004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 -- 172.21.15.116:0/1767354346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9660004d50 msgr2=0x7f9660005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9660004d50 0x7f9660005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967d400640 1 -- 172.21.15.116:0/1767354346 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9660003530 con 0x7f9660003a10 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967ca00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f967815db30 0x7f9660004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.780+0000 7f967de00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9660004d50 0x7f9660005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967d400640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f9660003e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f966c009ae0 tx=0x7f966c01ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18855d19556ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c0b2070 con 0x7f9660003a10 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c0a5680 con 0x7f9660003a10 2024-05-14T11:17:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f9677e00640 1 -- 172.21.15.116:0/1767354346 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c0096b0 con 0x7f9660003a10 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967fe00640 1 -- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 msgr2=0x7f9660003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967fe00640 1 --2- 172.21.15.116:0/1767354346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f9660003e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f966c009ae0 tx=0x7f966c01ca90 comp rx=0 tx=0).stop 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967de00640 1 -- 172.21.15.116:0/1767354346 reap_dead start 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967fe00640 1 -- 172.21.15.116:0/1767354346 shutdown_connections 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967fe00640 1 -- 172.21.15.116:0/1767354346 >> 172.21.15.116:0/1767354346 conn(0x7f96780b4620 msgr2=0x7f96780b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.784+0000 7f967fe00640 1 -- 172.21.15.116:0/1767354346 shutdown_connections 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 -- 172.21.15.116:0/1767354346 wait complete. 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 Processor -- start 2024-05-14T11:17:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 -- start start 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f96780cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f96780ce5d0 0x7f96781f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9678070750 con 0x7f96781e8c70 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f96780ae870 con 0x7f9660003a10 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f96780ae350 con 0x7f96780ce5d0 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.788+0000 7f967de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47508/0 (socket says 172.21.15.116:47508) 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 -- 172.21.15.116:0/3287607606 learned_addr learned my addr 172.21.15.116:0/3287607606 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967d400640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f96780cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967ca00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f96780ce5d0 0x7f96781f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 -- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f96780ce5d0 msgr2=0x7f96781f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f96780ce5d0 0x7f96781f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 -- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 msgr2=0x7f96780cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f96780cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 -- 172.21.15.116:0/3287607606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96780cd880 con 0x7f96781e8c70 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967de00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9678072800 tx=0x7f9670012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:52.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000a4c0 con 0x7f96781e8c70 2024-05-14T11:17:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9670007070 con 0x7f96781e8c70 2024-05-14T11:17:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96640096b0 con 0x7f96781e8c70 2024-05-14T11:17:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96780cdaf0 con 0x7f96781e8c70 2024-05-14T11:17:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96780ce0b0 con 0x7f96781e8c70 2024-05-14T11:17:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96780cfb80 con 0x7f96781e8c70 2024-05-14T11:17:52.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.792+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f967001e020 con 0x7f96781e8c70 2024-05-14T11:17:52.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.796+0000 7f9675600640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 0x7f9658071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:52.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.796+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f96700926d0 con 0x7f96781e8c70 2024-05-14T11:17:52.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.796+0000 7f967d400640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 0x7f9658071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:52.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.796+0000 7f967d400640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 0x7f9658071940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f966c0af000 tx=0x7f966c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:52.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:52.800+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f96780d1fc0 con 0x7f96781e8c70 2024-05-14T11:17:53.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.008+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f96780cf8d0 con 0x7f96781e8c70 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.008+0000 7f9675600640 1 -- 172.21.15.116:0/3287607606 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f967005f120 con 0x7f96781e8c70 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:17:53.012 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.013 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:17:53.014 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.015 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.016 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:17:53.017 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.018 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:17:53.019 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.020 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:53.021 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:17:53.022 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:17:53.023 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:17:53.024 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 msgr2=0x7f9658071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 0x7f9658071940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f966c0af000 tx=0x7f966c0046d0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 msgr2=0x7f96781e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9678072800 tx=0x7f9670012630 comp rx=0 tx=0).stop 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 shutdown_connections 2024-05-14T11:17:53.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965806f430 0x7f9658071940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96781e8c70 0x7f96781e9080 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f96780ce5d0 0x7f96781f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 --2- 172.21.15.116:0/3287607606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660003a10 0x7f96780cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 >> 172.21.15.116:0/3287607606 conn(0x7f96780b4620 msgr2=0x7f9678161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 shutdown_connections 2024-05-14T11:17:53.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.012+0000 7f967fe00640 1 -- 172.21.15.116:0/3287607606 wait complete. 2024-05-14T11:17:53.036 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:17:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 Processor -- start 2024-05-14T11:17:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 -- start start 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe824160080 0x7fe8240cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 0x7fe8240b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8240b2f40 con 0x7fe8240ce4c0 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe8240b3220 con 0x7fe824160080 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe82b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe8240b3500 con 0x7fe8240d1c20 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe823e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe824160080 0x7fe8240cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe81b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe8240ce4c0 0x7fe8240d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58130/0 (socket says 172.21.15.116:58130) 2024-05-14T11:17:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe81b400640 1 -- 172.21.15.116:0/1579724645 learned_addr learned my addr 172.21.15.116:0/1579724645 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe823e00640 1 -- 172.21.15.116:0/1579724645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 msgr2=0x7fe8240b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.180+0000 7fe829200640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 0x7fe8240b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823400640 1 -- 172.21.15.116:0/1579724645 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871288928 0 0) 0x7fe8240b2f40 con 0x7fe8240ce4c0 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823e00640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 0x7fe8240b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823e00640 1 -- 172.21.15.116:0/1579724645 >> v1:172.21.15.116:6789/0 conn(0x7fe8240ce4c0 legacy=0x7fe8240d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823e00640 1 -- 172.21.15.116:0/1579724645 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8240b37e0 con 0x7fe824160080 2024-05-14T11:17:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe829200640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 0x7fe8240b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:53.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823e00640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe824160080 0x7fe8240cdf80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8140097c0 tx=0x7fe81401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a494a8e52568b734 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823400640 1 -- 172.21.15.116:0/1579724645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140b0070 con 0x7fe824160080 2024-05-14T11:17:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823400640 1 -- 172.21.15.116:0/1579724645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8140ab600 con 0x7fe824160080 2024-05-14T11:17:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe823400640 1 -- 172.21.15.116:0/1579724645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140ab9d0 con 0x7fe824160080 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe82b200640 1 -- 172.21.15.116:0/1579724645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe824160080 msgr2=0x7fe8240cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.184+0000 7fe82b200640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe824160080 0x7fe8240cdf80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8140097c0 tx=0x7fe81401c350 comp rx=0 tx=0).stop 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- 172.21.15.116:0/1579724645 shutdown_connections 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe8240d1c20 0x7fe8240b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 --2- 172.21.15.116:0/1579724645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe824160080 0x7fe8240cdf80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- 172.21.15.116:0/1579724645 >> 172.21.15.116:0/1579724645 conn(0x7fe8240d0770 msgr2=0x7fe8240d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- 172.21.15.116:0/1579724645 shutdown_connections 2024-05-14T11:17:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- 172.21.15.116:0/1579724645 wait complete. 2024-05-14T11:17:53.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 Processor -- start 2024-05-14T11:17:53.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- start start 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe824160080 0x7fe8240b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8240af270 0x7fe8240af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe82416cd40 con 0x7fe8240ce4c0 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.188+0000 7fe82b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe82406ffe0 con 0x7fe8240af270 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe82b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe82416cec0 con 0x7fe824160080 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47524/0 (socket says 172.21.15.116:47524) 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 -- 172.21.15.116:0/1784618082 learned_addr learned my addr 172.21.15.116:0/1784618082 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe81b400640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe824160080 0x7fe8240b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe829200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8240af270 0x7fe8240af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 -- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe824160080 msgr2=0x7fe8240b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe824160080 0x7fe8240b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 -- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8240af270 msgr2=0x7fe8240af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8240af270 0x7fe8240af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 -- 172.21.15.116:0/1784618082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8240afbc0 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe823e00640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8140ae040 tx=0x7fe81401ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140b0050 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8140bc070 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140b5a70 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8240b8380 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8240b8940 con 0x7fe8240ce4c0 2024-05-14T11:17:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82406a9c0 con 0x7fe8240ce4c0 2024-05-14T11:17:53.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.192+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8140ab4c0 con 0x7fe8240ce4c0 2024-05-14T11:17:53.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.196+0000 7fe820c00640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 0x7fe800071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.196+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7fe81412a4b0 con 0x7fe8240ce4c0 2024-05-14T11:17:53.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.196+0000 7fe81b400640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 0x7fe800071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.196+0000 7fe81b400640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 0x7fe800071940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000f70 tx=0x7fe80c007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.196+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe8240afbc0 con 0x7fe8240ce4c0 2024-05-14T11:17:53.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe82416ab10 con 0x7fe8240ce4c0 2024-05-14T11:17:53.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe820c00640 1 -- 172.21.15.116:0/1784618082 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) ==== 84+0+65 (secure 0 0 0) 0x7fe814016090 con 0x7fe8240ce4c0 2024-05-14T11:17:53.358 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:53.358 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:17:53.358 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:17:53.359 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:17:53.359 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:17:53.359 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:17:53.359 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:17:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 msgr2=0x7fe800071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 0x7fe800071940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000f70 tx=0x7fe80c007c10 comp rx=0 tx=0).stop 2024-05-14T11:17:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 msgr2=0x7fe8241597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8140ae040 tx=0x7fe81401ab90 comp rx=0 tx=0).stop 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 shutdown_connections 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe80006f430 0x7fe800071940 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8240af270 0x7fe8240af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe824160080 0x7fe8240b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 --2- 172.21.15.116:0/1784618082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8240ce4c0 0x7fe8241597b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 >> 172.21.15.116:0/1784618082 conn(0x7fe8240d0770 msgr2=0x7fe82415c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 shutdown_connections 2024-05-14T11:17:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.356+0000 7fe82b200640 1 -- 172.21.15.116:0/1784618082 wait complete. 2024-05-14T11:17:53.380 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:17:53.380 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:17:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.528+0000 7f5b12a00640 1 Processor -- start 2024-05-14T11:17:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.528+0000 7f5b12a00640 1 -- start start 2024-05-14T11:17:53.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 0x7f5b0c161a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b12a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c161f80 con 0x7f5b0c070cc0 2024-05-14T11:17:53.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b12a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c0719b0 con 0x7f5b0c0b8ae0 2024-05-14T11:17:53.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b12a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c0b9600 con 0x7f5b0c161630 2024-05-14T11:17:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5b0c070cc0 0x7f5b0c0710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58138/0 (socket says 172.21.15.116:58138) 2024-05-14T11:17:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0b400640 1 -- 172.21.15.116:0/1368094487 learned_addr learned my addr 172.21.15.116:0/1368094487 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0be00640 1 --2- 172.21.15.116:0/1368094487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 0x7f5b0c161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236176212 0 0) 0x7f5af8001120 con 0x7f5b0c070cc0 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af0003530 con 0x7f5b0c070cc0 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269454864 0 0) 0x7f5b0c0719b0 con 0x7f5b0c0b8ae0 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af8001120 con 0x7f5b0c0b8ae0 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0be00640 1 -- 172.21.15.116:0/1368094487 >> v1:172.21.15.149:6789/0 conn(0x7f5b0c0b8ae0 legacy=0x7f5b0c0b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0be00640 1 -- 172.21.15.116:0/1368094487 >> v1:172.21.15.116:6789/0 conn(0x7f5b0c070cc0 legacy=0x7f5b0c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.532+0000 7f5b0be00640 1 -- 172.21.15.116:0/1368094487 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c0d14a0 con 0x7f5b0c161630 2024-05-14T11:17:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b0be00640 1 --2- 172.21.15.116:0/1368094487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 0x7f5b0c161a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009dc0 tx=0x7f5afc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c87ec942d9319a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0b2070 con 0x7f5b0c161630 2024-05-14T11:17:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc0a5960 con 0x7f5b0c161630 2024-05-14T11:17:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b0aa00640 1 -- 172.21.15.116:0/1368094487 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0a5d30 con 0x7f5b0c161630 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 -- 172.21.15.116:0/1368094487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 msgr2=0x7f5b0c161a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 --2- 172.21.15.116:0/1368094487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 0x7f5b0c161a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009dc0 tx=0x7f5afc01c990 comp rx=0 tx=0).stop 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 -- 172.21.15.116:0/1368094487 shutdown_connections 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 --2- 172.21.15.116:0/1368094487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c161630 0x7f5b0c161a40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 -- 172.21.15.116:0/1368094487 >> 172.21.15.116:0/1368094487 conn(0x7f5b0c0d35e0 msgr2=0x7f5b0c0d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 -- 172.21.15.116:0/1368094487 shutdown_connections 2024-05-14T11:17:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 -- 172.21.15.116:0/1368094487 wait complete. 2024-05-14T11:17:53.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.536+0000 7f5b12a00640 1 Processor -- start 2024-05-14T11:17:53.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- start start 2024-05-14T11:17:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c070cc0 0x7f5b0c1f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 0x7f5b0c0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c16bbd0 con 0x7f5b0c161630 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c0d1af0 con 0x7f5b0c0b8ae0 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c16bd50 con 0x7f5b0c070cc0 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47540/0 (socket says 172.21.15.116:47540) 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 -- 172.21.15.116:0/663151259 learned_addr learned my addr 172.21.15.116:0/663151259 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b0b400640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 0x7f5b0c0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b0be00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c070cc0 0x7f5b0c1f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 -- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c070cc0 msgr2=0x7f5b0c1f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c070cc0 0x7f5b0c1f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 -- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 msgr2=0x7f5b0c0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 0x7f5b0c0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 -- 172.21.15.116:0/663151259 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1f9f40 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b0b400640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 0x7f5b0c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b10a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c3c0 tx=0x7f5b0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04017070 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b040052d0 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c1fa450 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04003840 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c06e240 con 0x7f5b0c161630 2024-05-14T11:17:53.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.540+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b0c06aa30 con 0x7f5b0c161630 2024-05-14T11:17:53.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.544+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b0401a070 con 0x7f5b0c161630 2024-05-14T11:17:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.544+0000 7f5b03e00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 0x7f5aec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.544+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5249+0+0 (secure 0 0 0) 0x7f5b0408e450 con 0x7f5b0c161630 2024-05-14T11:17:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.544+0000 7f5b0be00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 0x7f5aec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.544+0000 7f5b0be00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 0x7f5aec071910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.548+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5b04005480 con 0x7f5b0c161630 2024-05-14T11:17:53.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:53.712+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5b0c1fa1e0 con 0x7f5b0c161630 2024-05-14T11:17:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.608+0000 7f5b03e00640 1 -- 172.21.15.116:0/663151259 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) ==== 138+0+0 (secure 0 0 0) 0x7f5b0405b4f0 con 0x7f5b0c161630 2024-05-14T11:17:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 msgr2=0x7f5aec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 0x7f5aec071910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc0a7070 comp rx=0 tx=0).stop 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 msgr2=0x7f5b0c0ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c3c0 tx=0x7f5b0400c880 comp rx=0 tx=0).stop 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 shutdown_connections 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5aec06f400 0x7f5aec071910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c0b8ae0 0x7f5b0c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0c161630 0x7f5b0c0ccd60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 --2- 172.21.15.116:0/663151259 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b0c070cc0 0x7f5b0c1f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 >> 172.21.15.116:0/663151259 conn(0x7f5b0c0d35e0 msgr2=0x7f5b0c0d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 shutdown_connections 2024-05-14T11:17:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.612+0000 7f5b12a00640 1 -- 172.21.15.116:0/663151259 wait complete. 2024-05-14T11:17:54.645 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:17:54.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 Processor -- start 2024-05-14T11:17:54.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 -- start start 2024-05-14T11:17:54.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f59580d1e40 0x7f59580cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595806f0a0 con 0x7f59580cd1c0 2024-05-14T11:17:54.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f59580cbb60 con 0x7f59580d1e40 2024-05-14T11:17:54.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f595d400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f595806f420 con 0x7f595806c700 2024-05-14T11:17:54.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5955e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f59580d1e40 0x7f59580cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:48266/0 (socket says 172.21.15.116:48266) 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 -- 172.21.15.116:0/2667861400 learned_addr learned my addr 172.21.15.116:0/2667861400 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5956800640 1 --1- 172.21.15.116:0/2667861400 >> v1:172.21.15.149:6789/0 conn(0x7f59580cd1c0 0x7f59580d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:41010/0 (socket says 172.21.15.116:41010) 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 -- 172.21.15.116:0/2667861400 >> v2:172.21.15.149:3300/0 conn(0x7f59580d1e40 msgr2=0x7f59580cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 --2- 172.21.15.116:0/2667861400 >> v2:172.21.15.149:3300/0 conn(0x7f59580d1e40 0x7f59580cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 -- 172.21.15.116:0/2667861400 >> v1:172.21.15.149:6789/0 conn(0x7f59580cd1c0 legacy=0x7f59580d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.800+0000 7f5957200640 1 -- 172.21.15.116:0/2667861400 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595806f700 con 0x7f595806c700 2024-05-14T11:17:54.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f5957200640 1 --2- 172.21.15.116:0/2667861400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e50 tx=0x7f595001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e4bc365e7105b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:54.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f5955400640 1 -- 172.21.15.116:0/2667861400 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500a5710 con 0x7f595806c700 2024-05-14T11:17:54.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f5955400640 1 -- 172.21.15.116:0/2667861400 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59500a58b0 con 0x7f595806c700 2024-05-14T11:17:54.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f5955400640 1 -- 172.21.15.116:0/2667861400 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500a5c80 con 0x7f595806c700 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 -- 172.21.15.116:0/2667861400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 msgr2=0x7f595806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 --2- 172.21.15.116:0/2667861400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e50 tx=0x7f595001c850 comp rx=0 tx=0).stop 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 -- 172.21.15.116:0/2667861400 shutdown_connections 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 --2- 172.21.15.116:0/2667861400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f595806cb10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 --2- 172.21.15.116:0/2667861400 >> v2:172.21.15.149:3300/0 conn(0x7f59580d1e40 0x7f59580cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 -- 172.21.15.116:0/2667861400 >> 172.21.15.116:0/2667861400 conn(0x7f59580b4720 msgr2=0x7f59580b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 -- 172.21.15.116:0/2667861400 shutdown_connections 2024-05-14T11:17:54.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.804+0000 7f595d400640 1 -- 172.21.15.116:0/2667861400 wait complete. 2024-05-14T11:17:54.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 Processor -- start 2024-05-14T11:17:54.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 -- start start 2024-05-14T11:17:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f5958165e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 0x7f59580b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59581699b0 con 0x7f595815b9d0 2024-05-14T11:17:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f595815cf10 con 0x7f59580cd1c0 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f595d400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5958169b30 con 0x7f595806c700 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47552/0 (socket says 172.21.15.116:47552) 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 -- 172.21.15.116:0/3117880946 learned_addr learned my addr 172.21.15.116:0/3117880946 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5955e00640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 0x7f59580b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5956800640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f5958165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 -- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 msgr2=0x7f5958165e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f5958165e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 -- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 msgr2=0x7f59580b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 0x7f59580b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 -- 172.21.15.116:0/3117880946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595815c320 con 0x7f595815b9d0 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5955e00640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 0x7f59580b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:54.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.808+0000 7f5957200640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f59500af040 tx=0x7f59500195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:54.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950009510 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59500be070 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500b88d0 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59580c9520 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59580c9ae0 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595806a9c0 con 0x7f595815b9d0 2024-05-14T11:17:54.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.812+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59500b8a70 con 0x7f595815b9d0 2024-05-14T11:17:54.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.816+0000 7f594ea00640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 0x7f5938071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:54.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.816+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5619+0+0 (secure 0 0 0) 0x7f5950129ff0 con 0x7f595815b9d0 2024-05-14T11:17:54.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.816+0000 7f5956800640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 0x7f5938071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:54.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.816+0000 7f5956800640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 0x7f5938071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f70 tx=0x7f5948007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:54.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.820+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f595815c320 con 0x7f595815b9d0 2024-05-14T11:17:54.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:54.980+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f59580cef00 con 0x7f595815b9d0 2024-05-14T11:17:55.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.616+0000 7f594ea00640 1 -- 172.21.15.116:0/3117880946 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) ==== 170+0+0 (secure 0 0 0) 0x7f595806d860 con 0x7f595815b9d0 2024-05-14T11:17:55.620 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:17:55.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 msgr2=0x7f5938071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 0x7f5938071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f70 tx=0x7f5948007e50 comp rx=0 tx=0).stop 2024-05-14T11:17:55.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 msgr2=0x7f595815bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f59500af040 tx=0x7f59500195f0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 shutdown_connections 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f593806f400 0x7f5938071910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59580cd1c0 0x7f59580b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f595815b9d0 0x7f595815bde0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 --2- 172.21.15.116:0/3117880946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f595806c700 0x7f5958165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 >> 172.21.15.116:0/3117880946 conn(0x7f59580b4720 msgr2=0x7f59580b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 shutdown_connections 2024-05-14T11:17:55.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.620+0000 7f595d400640 1 -- 172.21.15.116:0/3117880946 wait complete. 2024-05-14T11:17:55.661 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:17:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.808+0000 7f89ff200640 1 Processor -- start 2024-05-14T11:17:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.808+0000 7f89ff200640 1 -- start start 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89ff200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 0x7f8a001602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89ff200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000b07a0 con 0x7f8a001607e0 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89ff200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000d1d20 con 0x7f8a000b17e0 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89ff200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a00160ff0 con 0x7f8a000b6b10 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fe800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8a001607e0 0x7f8a000d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58152/0 (socket says 172.21.15.116:58152) 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fe800640 1 -- 172.21.15.116:0/2287008233 learned_addr learned my addr 172.21.15.116:0/2287008233 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89f7e00640 1 --2- 172.21.15.116:0/2287008233 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 0x7f8a001602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569744862 0 0) 0x7f89f80014a0 con 0x7f8a001607e0 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89e4003530 con 0x7f8a001607e0 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238952549 0 0) 0x7f8a000d1d20 con 0x7f8a000b17e0 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f80014a0 con 0x7f8a000b17e0 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89f7e00640 1 -- 172.21.15.116:0/2287008233 >> v1:172.21.15.149:6789/0 conn(0x7f8a000b17e0 legacy=0x7f8a000b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89f7e00640 1 -- 172.21.15.116:0/2287008233 >> v1:172.21.15.116:6789/0 conn(0x7f8a001607e0 legacy=0x7f8a000d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.812+0000 7f89f7e00640 1 -- 172.21.15.116:0/2287008233 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000b0a20 con 0x7f8a000b6b10 2024-05-14T11:17:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89f7e00640 1 --2- 172.21.15.116:0/2287008233 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 0x7f8a001602a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a00161170 tx=0x7f89e801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45824aeec4030256 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e80b0070 con 0x7f8a000b6b10 2024-05-14T11:17:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e80ab600 con 0x7f8a000b6b10 2024-05-14T11:17:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89fd400640 1 -- 172.21.15.116:0/2287008233 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e80ab9d0 con 0x7f8a000b6b10 2024-05-14T11:17:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 -- 172.21.15.116:0/2287008233 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 msgr2=0x7f8a001602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 --2- 172.21.15.116:0/2287008233 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 0x7f8a001602a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a00161170 tx=0x7f89e801c4c0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 -- 172.21.15.116:0/2287008233 shutdown_connections 2024-05-14T11:17:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 --2- 172.21.15.116:0/2287008233 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b6b10 0x7f8a001602a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 -- 172.21.15.116:0/2287008233 >> 172.21.15.116:0/2287008233 conn(0x7f8a00072080 msgr2=0x7f8a00073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 -- 172.21.15.116:0/2287008233 shutdown_connections 2024-05-14T11:17:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.816+0000 7f89ff200640 1 -- 172.21.15.116:0/2287008233 wait complete. 2024-05-14T11:17:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 Processor -- start 2024-05-14T11:17:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- start start 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b17e0 0x7f8a0016af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 0x7f8a0015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00160d70 con 0x7f8a0015f060 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00160bf0 con 0x7f8a001607e0 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b1bf0 con 0x7f8a000b17e0 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47558/0 (socket says 172.21.15.116:47558) 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 -- 172.21.15.116:0/3107409008 learned_addr learned my addr 172.21.15.116:0/3107409008 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fde00640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b17e0 0x7f8a0016af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89f7e00640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 0x7f8a0015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 -- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b17e0 msgr2=0x7f8a0016af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b17e0 0x7f8a0016af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 -- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 msgr2=0x7f8a0015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 0x7f8a0015eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 -- 172.21.15.116:0/3107409008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0015f470 con 0x7f8a0015f060 2024-05-14T11:17:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89f7e00640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 0x7f8a0015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89fe800640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89f8001f00 tx=0x7f89f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8012400 con 0x7f8a0015f060 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f80125a0 con 0x7f8a0015f060 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001f8e60 con 0x7f8a0015f060 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8012970 con 0x7f8a0015f060 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.820+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0015a9e0 con 0x7f8a0015f060 2024-05-14T11:17:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0006a9c0 con 0x7f8a0015f060 2024-05-14T11:17:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89f8012b10 con 0x7f8a0015f060 2024-05-14T11:17:55.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89f6000640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 0x7f89dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:55.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6001+0+0 (secure 0 0 0) 0x7f89f80905a0 con 0x7f8a0015f060 2024-05-14T11:17:55.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89fde00640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 0x7f89dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.824+0000 7f89fde00640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 0x7f89dc071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89f0007fd0 tx=0x7f89f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:55.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:55.828+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8a0015f470 con 0x7f8a0015f060 2024-05-14T11:17:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.020+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a00075dc0 con 0x7f8a0015f060 2024-05-14T11:17:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 7 and data pool 8 2024-05-14T11:17:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.636+0000 7f89f6000640 1 -- 172.21.15.116:0/3107409008 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7f8a000b07a0 con 0x7f8a0015f060 2024-05-14T11:17:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 msgr2=0x7f89dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 0x7f89dc071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89f0007fd0 tx=0x7f89f0009040 comp rx=0 tx=0).stop 2024-05-14T11:17:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 msgr2=0x7f8a0015b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f89f8001f00 tx=0x7f89f800a430 comp rx=0 tx=0).stop 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 shutdown_connections 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89dc06f400 0x7f89dc071910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a001607e0 0x7f8a0015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0015f060 0x7f8a0015b7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 --2- 172.21.15.116:0/3107409008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a000b17e0 0x7f8a0016af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 >> 172.21.15.116:0/3107409008 conn(0x7f8a00072080 msgr2=0x7f8a000b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 shutdown_connections 2024-05-14T11:17:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.648+0000 7f89ff200640 1 -- 172.21.15.116:0/3107409008 wait complete. 2024-05-14T11:17:56.686 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:56.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.832+0000 7f0e10e00640 1 Processor -- start 2024-05-14T11:17:56.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.832+0000 7f0e10e00640 1 -- start start 2024-05-14T11:17:56.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 0x7f0e0c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e10e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e0c06c700 0x7f0e0c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e10e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e0c06ff20 con 0x7f0e0c15db30 2024-05-14T11:17:56.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e10e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c0ae100 con 0x7f0e0c06c700 2024-05-14T11:17:56.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e10e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c0ae870 con 0x7f0e0c0d1670 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e0c06c700 0x7f0e0c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e0c06c700 0x7f0e0c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47574/0 (socket says 172.21.15.116:47574) 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 -- 172.21.15.116:0/2533603025 learned_addr learned my addr 172.21.15.116:0/2533603025 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e09400640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 0x7f0e0c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 -- 172.21.15.116:0/2533603025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 msgr2=0x7f0e0c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 0x7f0e0c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 -- 172.21.15.116:0/2533603025 >> v1:172.21.15.149:6789/0 conn(0x7f0e0c15db30 legacy=0x7f0e0c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:56.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e0a800640 1 -- 172.21.15.116:0/2533603025 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c0d1fc0 con 0x7f0e0c06c700 2024-05-14T11:17:56.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.836+0000 7f0e09400640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 0x7f0e0c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:17:56.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e0a800640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c06c700 0x7f0e0c06cb10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009870 tx=0x7f0e0401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08d211780bdfdb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:56.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e08a00640 1 -- 172.21.15.116:0/2533603025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b0070 con 0x7f0e0c06c700 2024-05-14T11:17:56.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e08a00640 1 -- 172.21.15.116:0/2533603025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e040ab600 con 0x7f0e0c06c700 2024-05-14T11:17:56.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e08a00640 1 -- 172.21.15.116:0/2533603025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040ab9d0 con 0x7f0e0c06c700 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- 172.21.15.116:0/2533603025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c06c700 msgr2=0x7f0e0c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c06c700 0x7f0e0c06cb10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009870 tx=0x7f0e0401c400 comp rx=0 tx=0).stop 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- 172.21.15.116:0/2533603025 shutdown_connections 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c06c700 0x7f0e0c06cb10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 --2- 172.21.15.116:0/2533603025 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c0d1670 0x7f0e0c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- 172.21.15.116:0/2533603025 >> 172.21.15.116:0/2533603025 conn(0x7f0e0c0b4620 msgr2=0x7f0e0c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- 172.21.15.116:0/2533603025 shutdown_connections 2024-05-14T11:17:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- 172.21.15.116:0/2533603025 wait complete. 2024-05-14T11:17:56.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 Processor -- start 2024-05-14T11:17:56.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.840+0000 7f0e10e00640 1 -- start start 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 0x7f0e0c09a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e0c0d1670 0x7f0e0c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c16a770 con 0x7f0e0c15db30 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c15df40 con 0x7f0e0c0d1670 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c16a8f0 con 0x7f0e0c06c700 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47576/0 (socket says 172.21.15.116:47576) 2024-05-14T11:17:56.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 -- 172.21.15.116:0/526813943 learned_addr learned my addr 172.21.15.116:0/526813943 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e09400640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e0c0d1670 0x7f0e0c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e09e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 0x7f0e0c09a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 -- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 msgr2=0x7f0e0c09a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 0x7f0e0c09a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 -- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e0c0d1670 msgr2=0x7f0e0c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e0c0d1670 0x7f0e0c099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 -- 172.21.15.116:0/526813943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c0b3940 con 0x7f0e0c15db30 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e09e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 0x7f0e0c09a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e0a800640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401a030 tx=0x7f0e040191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b0040 con 0x7f0e0c15db30 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e040b9070 con 0x7f0e0c15db30 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c145c30 con 0x7f0e0c15db30 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b4660 con 0x7f0e0c15db30 2024-05-14T11:17:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c0b3ec0 con 0x7f0e0c15db30 2024-05-14T11:17:56.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.844+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c06a9c0 con 0x7f0e0c15db30 2024-05-14T11:17:56.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.848+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e040b4800 con 0x7f0e0c15db30 2024-05-14T11:17:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.848+0000 7f0e02000640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 0x7f0de8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.848+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7f0e04129170 con 0x7f0e0c15db30 2024-05-14T11:17:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.848+0000 7f0e09e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 0x7f0de8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:56.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.848+0000 7f0e09e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 0x7f0de8071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc001db0 tx=0x7f0dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:56.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:56.852+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0e0c0d1fc0 con 0x7f0e0c15db30 2024-05-14T11:17:57.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.012+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e0c162450 con 0x7f0e0c15db30 2024-05-14T11:17:57.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e02000640 1 -- 172.21.15.116:0/526813943 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17003 (secure 0 0 0) 0x7f0e04016110 con 0x7f0e0c15db30 2024-05-14T11:17:57.017 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:57.017 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":48,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:56.025329+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:53.726817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:54.993539+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:57.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 msgr2=0x7f0de8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 0x7f0de8071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc001db0 tx=0x7f0dfc009040 comp rx=0 tx=0).stop 2024-05-14T11:17:57.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 msgr2=0x7f0e0c15f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401a030 tx=0x7f0e040191a0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 shutdown_connections 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0de806f400 0x7f0de8071910 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e0c0d1670 0x7f0e0c099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c15db30 0x7f0e0c15f110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 --2- 172.21.15.116:0/526813943 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e0c06c700 0x7f0e0c09a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 >> 172.21.15.116:0/526813943 conn(0x7f0e0c0b4620 msgr2=0x7f0e0c15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 shutdown_connections 2024-05-14T11:17:57.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.016+0000 7f0e10e00640 1 -- 172.21.15.116:0/526813943 wait complete. 2024-05-14T11:17:57.039 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:17:57.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 Processor -- start 2024-05-14T11:17:57.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 -- start start 2024-05-14T11:17:57.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff4e40d1670 0x7ff4e40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e40d1fc0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e40706f0 con 0x7ff4e40d1670 2024-05-14T11:17:57.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4ea600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e41573b0 con 0x7ff4e40af380 2024-05-14T11:17:57.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff4e40d1670 0x7ff4e40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff4e40d1670 0x7ff4e40d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:42884/0 (socket says 172.21.15.116:42884) 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e3400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7ff4e4156fa0 0x7ff4e406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:41034/0 (socket says 172.21.15.116:41034) 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 -- 172.21.15.116:0/47975023 learned_addr learned my addr 172.21.15.116:0/47975023 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:48314/0 (socket says 172.21.15.116:48314) 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e2a00640 1 -- 172.21.15.116:0/47975023 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714014907 0 0) 0x7ff4d4001120 con 0x7ff4e4156fa0 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e2a00640 1 -- 172.21.15.116:0/47975023 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4c8003530 con 0x7ff4e4156fa0 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 -- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 msgr2=0x7ff4e40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 -- 172.21.15.116:0/47975023 >> v1:172.21.15.149:6789/0 conn(0x7ff4e4156fa0 legacy=0x7ff4e406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4dbe00640 1 -- 172.21.15.116:0/47975023 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e4160510 con 0x7ff4e40d1670 2024-05-14T11:17:57.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.184+0000 7ff4e3e00640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:17:57.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4dbe00640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e40d1a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bb10 tx=0x7ff4cc01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9c48933c52f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4e2a00640 1 -- 172.21.15.116:0/47975023 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0b4070 con 0x7ff4e40d1670 2024-05-14T11:17:57.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4e2a00640 1 -- 172.21.15.116:0/47975023 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0a7bf0 con 0x7ff4e40d1670 2024-05-14T11:17:57.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4e2a00640 1 -- 172.21.15.116:0/47975023 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0af440 con 0x7ff4e40d1670 2024-05-14T11:17:57.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4ea600640 1 -- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 msgr2=0x7ff4e40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.188+0000 7ff4ea600640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e40d1a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00bb10 tx=0x7ff4cc01ec30 comp rx=0 tx=0).stop 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- 172.21.15.116:0/47975023 shutdown_connections 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 --2- 172.21.15.116:0/47975023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e40d1a80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- 172.21.15.116:0/47975023 >> 172.21.15.116:0/47975023 conn(0x7ff4e40b4620 msgr2=0x7ff4e40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- 172.21.15.116:0/47975023 shutdown_connections 2024-05-14T11:17:57.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- 172.21.15.116:0/47975023 wait complete. 2024-05-14T11:17:57.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 Processor -- start 2024-05-14T11:17:57.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- start start 2024-05-14T11:17:57.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e41559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e4148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b0660 con 0x7ff4e4156fa0 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b04e0 con 0x7ff4e40d1670 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.192+0000 7ff4ea600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e406c700 con 0x7ff4e40af380 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47578/0 (socket says 172.21.15.116:47578) 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 -- 172.21.15.116:0/4055222100 learned_addr learned my addr 172.21.15.116:0/4055222100 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4dbe00640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e4148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3400640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e41559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 -- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 msgr2=0x7ff4e41559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e41559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 -- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 msgr2=0x7ff4e4148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e4148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 -- 172.21.15.116:0/4055222100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e4150480 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4e3e00640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009e60 tx=0x7ff4dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc018030 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00cdf0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003bc0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e4154bf0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41551b0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e406a9c0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.196+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4dc01b070 con 0x7ff4e4156fa0 2024-05-14T11:17:57.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.200+0000 7ff4db400640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 0x7ff4c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.200+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7ff4dc08fe90 con 0x7ff4e4156fa0 2024-05-14T11:17:57.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.200+0000 7ff4e3400640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 0x7ff4c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.200+0000 7ff4e3400640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 0x7ff4c0071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007fd0 tx=0x7ff4d4007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.204+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff4dc010040 con 0x7ff4e4156fa0 2024-05-14T11:17:57.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.360+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4e415c950 con 0x7ff4e4156fa0 2024-05-14T11:17:57.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.360+0000 7ff4db400640 1 -- 172.21.15.116:0/4055222100 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17003 (secure 0 0 0) 0x7ff4dc05cbe0 con 0x7ff4e4156fa0 2024-05-14T11:17:57.365 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:17:57.365 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":48,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:56.025329+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:53.726817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:54.993539+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 msgr2=0x7ff4c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 0x7ff4c0071910 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007fd0 tx=0x7ff4d4007a40 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 msgr2=0x7ff4e414aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009e60 tx=0x7ff4dc00c490 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 shutdown_connections 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4c006f400 0x7ff4c0071910 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4e40d1670 0x7ff4e4148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e4156fa0 0x7ff4e414aa30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 --2- 172.21.15.116:0/4055222100 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4e40af380 0x7ff4e41559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 >> 172.21.15.116:0/4055222100 conn(0x7ff4e40b4620 msgr2=0x7ff4e40b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 shutdown_connections 2024-05-14T11:17:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.364+0000 7ff4ea600640 1 -- 172.21.15.116:0/4055222100 wait complete. 2024-05-14T11:17:57.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:17:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.532+0000 7facdae00640 1 Processor -- start 2024-05-14T11:17:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.532+0000 7facdae00640 1 -- start start 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facdae00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 0x7facd40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facdae00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd415d160 con 0x7facd40b6b10 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facdae00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd415d440 con 0x7facd4142e40 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facdae00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7facd415d720 con 0x7facd415f230 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd9000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7facd40b6b10 0x7facd4164320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37392/0 (socket says 172.21.15.116:37392) 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd9000640 1 -- 172.21.15.116:0/3018712162 learned_addr learned my addr 172.21.15.116:0/3018712162 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facda400640 1 --2- 172.21.15.116:0/3018712162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 0x7facd40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321020686 0 0) 0x7facd415d160 con 0x7facd40b6b10 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facbc003530 con 0x7facd40b6b10 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145720802 0 0) 0x7facd415d440 con 0x7facd4142e40 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facd415d160 con 0x7facd4142e40 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facda400640 1 -- 172.21.15.116:0/3018712162 >> v1:172.21.15.149:6789/0 conn(0x7facd4142e40 legacy=0x7facd40b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facda400640 1 -- 172.21.15.116:0/3018712162 >> v1:172.21.15.116:6789/0 conn(0x7facd40b6b10 legacy=0x7facd4164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.536+0000 7facda400640 1 -- 172.21.15.116:0/3018712162 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd415da00 con 0x7facd415f230 2024-05-14T11:17:57.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facda400640 1 --2- 172.21.15.116:0/3018712162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 0x7facd40cfad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faccc009dc0 tx=0x7faccc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e09ff7baca162d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0b2070 con 0x7facd415f230 2024-05-14T11:17:57.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc0a5850 con 0x7facd415f230 2024-05-14T11:17:57.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facd3e00640 1 -- 172.21.15.116:0/3018712162 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0a5c20 con 0x7facd415f230 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 -- 172.21.15.116:0/3018712162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 msgr2=0x7facd40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 --2- 172.21.15.116:0/3018712162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 0x7facd40cfad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faccc009dc0 tx=0x7faccc01c990 comp rx=0 tx=0).stop 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 -- 172.21.15.116:0/3018712162 shutdown_connections 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 --2- 172.21.15.116:0/3018712162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd415f230 0x7facd40cfad0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 -- 172.21.15.116:0/3018712162 >> 172.21.15.116:0/3018712162 conn(0x7facd4072080 msgr2=0x7facd4073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 -- 172.21.15.116:0/3018712162 shutdown_connections 2024-05-14T11:17:57.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.540+0000 7facdae00640 1 -- 172.21.15.116:0/3018712162 wait complete. 2024-05-14T11:17:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 Processor -- start 2024-05-14T11:17:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- start start 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd4142e40 0x7facd4141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facd40aed30 0x7facd40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7facd416aa10 con 0x7facd415f230 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7facd416a890 con 0x7facd40aed30 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7facd416ab90 con 0x7facd4142e40 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58224/0 (socket says 172.21.15.116:58224) 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 -- 172.21.15.116:0/485378810 learned_addr learned my addr 172.21.15.116:0/485378810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9a00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd4142e40 0x7facd4141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facda400640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facd40aed30 0x7facd40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 -- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd4142e40 msgr2=0x7facd4141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd4142e40 0x7facd4141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 -- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facd40aed30 msgr2=0x7facd40af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facd40aed30 0x7facd40af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 -- 172.21.15.116:0/485378810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41426e0 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd9000640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7facc000c430 tx=0x7facc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0017070 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc0005d10 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0007310 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd40af680 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.544+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd415b160 con 0x7facd415f230 2024-05-14T11:17:57.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd406a9c0 con 0x7facd415f230 2024-05-14T11:17:57.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7facc001a070 con 0x7facd415f230 2024-05-14T11:17:57.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facd1600640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 0x7facb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:57.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6101+0+0 (secure 0 0 0) 0x7facc008e880 con 0x7facd415f230 2024-05-14T11:17:57.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facd9a00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 0x7facb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:57.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.548+0000 7facd9a00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 0x7facb4071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facc8000c10 tx=0x7facc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:57.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.552+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7facd40af680 con 0x7facd415f230 2024-05-14T11:17:57.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:57.748+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7facd40b1890 con 0x7facd415f230 2024-05-14T11:17:58.607 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:17:58.608+0000 7f2ef1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:17:58.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facd1600640 1 -- 172.21.15.116:0/485378810 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7facd41426e0 con 0x7facd415f230 2024-05-14T11:17:58.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 msgr2=0x7facb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 0x7facb4071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7facc8000c10 tx=0x7facc8009040 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 msgr2=0x7facd40b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7facc000c430 tx=0x7facc000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 shutdown_connections 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7facb406f400 0x7facb4071910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facd40aed30 0x7facd40af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd415f230 0x7facd40b0f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 --2- 172.21.15.116:0/485378810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7facd4142e40 0x7facd4141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 >> 172.21.15.116:0/485378810 conn(0x7facd4072080 msgr2=0x7facd406f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.652+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 shutdown_connections 2024-05-14T11:17:58.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.656+0000 7facdae00640 1 -- 172.21.15.116:0/485378810 wait complete. 2024-05-14T11:17:58.680 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:17:58.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:17:58.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.832+0000 7fef87a00640 1 Processor -- start 2024-05-14T11:17:58.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.832+0000 7fef87a00640 1 -- start start 2024-05-14T11:17:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef87a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef800b1b60 0x7fef8006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef87a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fef800704f0 0x7fef80161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef87a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef80162450 con 0x7fef80161c60 2024-05-14T11:17:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef87a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fef800b1f70 con 0x7fef800704f0 2024-05-14T11:17:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef87a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fef8015e320 con 0x7fef800b1b60 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fef800704f0 0x7fef80161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef85a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fef80161c60 0x7fef8015f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37398/0 (socket says 172.21.15.116:37398) 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef85a00640 1 -- 172.21.15.116:0/3198872529 learned_addr learned my addr 172.21.15.116:0/3198872529 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef85000640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef800b1b60 0x7fef8006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7fe00640 1 -- 172.21.15.116:0/3198872529 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef800b1b60 msgr2=0x7fef8006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7fe00640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef800b1b60 0x7fef8006ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7fe00640 1 -- 172.21.15.116:0/3198872529 >> v1:172.21.15.116:6789/0 conn(0x7fef80161c60 legacy=0x7fef8015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7fe00640 1 -- 172.21.15.116:0/3198872529 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8015e600 con 0x7fef800704f0 2024-05-14T11:17:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.836+0000 7fef7f400640 1 -- 172.21.15.116:0/3198872529 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340700742 0 0) 0x7fef80162450 con 0x7fef80161c60 2024-05-14T11:17:58.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef7fe00640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80161720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0097c0 tx=0x7fef6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850ebc5a00a6d40b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:58.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef7f400640 1 -- 172.21.15.116:0/3198872529 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0b1070 con 0x7fef800704f0 2024-05-14T11:17:58.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef7f400640 1 -- 172.21.15.116:0/3198872529 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c0ac410 con 0x7fef800704f0 2024-05-14T11:17:58.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef7f400640 1 -- 172.21.15.116:0/3198872529 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0ac7e0 con 0x7fef800704f0 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- 172.21.15.116:0/3198872529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 msgr2=0x7fef80161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80161720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0097c0 tx=0x7fef6c01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- 172.21.15.116:0/3198872529 shutdown_connections 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80161720 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 --2- 172.21.15.116:0/3198872529 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef800b1b60 0x7fef8006ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- 172.21.15.116:0/3198872529 >> 172.21.15.116:0/3198872529 conn(0x7fef80071370 msgr2=0x7fef80071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- 172.21.15.116:0/3198872529 shutdown_connections 2024-05-14T11:17:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- 172.21.15.116:0/3198872529 wait complete. 2024-05-14T11:17:58.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 Processor -- start 2024-05-14T11:17:58.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.840+0000 7fef87a00640 1 -- start start 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef800b1b60 0x7fef8013f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef80161c60 0x7fef8013fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fef8016ac40 con 0x7fef800704f0 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef8016aac0 con 0x7fef800b1b60 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fef8016adc0 con 0x7fef80161c60 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58244/0 (socket says 172.21.15.116:58244) 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 -- 172.21.15.116:0/3011317192 learned_addr learned my addr 172.21.15.116:0/3011317192 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef80161c60 0x7fef8013fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef7fe00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef800b1b60 0x7fef8013f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 -- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef80161c60 msgr2=0x7fef8013fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef80161c60 0x7fef8013fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 -- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef800b1b60 msgr2=0x7fef8013f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef800b1b60 0x7fef8013f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 -- 172.21.15.116:0/3011317192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef80144d50 con 0x7fef800704f0 2024-05-14T11:17:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef85000640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef7400d600 tx=0x7fef7400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74004240 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef740043e0 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef80145030 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74003840 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef80158c30 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.844+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8006a9c0 con 0x7fef800704f0 2024-05-14T11:17:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.848+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fef740039e0 con 0x7fef800704f0 2024-05-14T11:17:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.848+0000 7fef7cc00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 0x7fef60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.848+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fef7408f690 con 0x7fef800704f0 2024-05-14T11:17:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.848+0000 7fef7fe00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 0x7fef60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.848+0000 7fef7fe00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 0x7fef60071940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0ae000 tx=0x7fef6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:58.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:58.852+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fef7405bd00 con 0x7fef800704f0 2024-05-14T11:17:59.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.044+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fef80160390 con 0x7fef800704f0 2024-05-14T11:17:59.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.652+0000 7fef7cc00640 1 -- 172.21.15.116:0/3011317192 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fef74060000 con 0x7fef800704f0 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 msgr2=0x7fef60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 0x7fef60071940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0ae000 tx=0x7fef6c008040 comp rx=0 tx=0).stop 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 msgr2=0x7fef80147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef7400d600 tx=0x7fef7400dac0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 shutdown_connections 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef6006f430 0x7fef60071940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef800b1b60 0x7fef8013f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef80161c60 0x7fef8013fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 --2- 172.21.15.116:0/3011317192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef800704f0 0x7fef80147210 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 >> 172.21.15.116:0/3011317192 conn(0x7fef80071370 msgr2=0x7fef80160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:59.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 shutdown_connections 2024-05-14T11:17:59.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.656+0000 7fef87a00640 1 -- 172.21.15.116:0/3011317192 wait complete. 2024-05-14T11:17:59.679 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:17:59.679 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:17:59.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 Processor -- start 2024-05-14T11:17:59.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 -- start start 2024-05-14T11:17:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94f00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f94f006c700 0x7f94f006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f006ff20 con 0x7f94f00d1670 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f94f00ae100 con 0x7f94f006c700 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94f6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94f00ae870 con 0x7f94f015db30 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.824+0000 7f94e7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f94f00d1670 0x7f94f00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37408/0 (socket says 172.21.15.116:37408) 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94e7e00640 1 -- 172.21.15.116:0/1989096459 learned_addr learned my addr 172.21.15.116:0/1989096459 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94efe00640 1 --2- 172.21.15.116:0/1989096459 >> v2:172.21.15.149:3300/0 conn(0x7f94f006c700 0x7f94f006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94f00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876853125 0 0) 0x7f94f006ff20 con 0x7f94f00d1670 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d4003530 con 0x7f94f00d1670 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4213639599 0 0) 0x7f94d4003530 con 0x7f94f00d1670 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 msgr2=0x7f94f00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94f00aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 >> v2:172.21.15.149:3300/0 conn(0x7f94f006c700 msgr2=0x7f94f006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 --2- 172.21.15.116:0/1989096459 >> v2:172.21.15.149:3300/0 conn(0x7f94f006c700 0x7f94f006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f0070470 con 0x7f94f00d1670 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f94d8001700 con 0x7f94f00d1670 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 >> v1:172.21.15.116:6789/0 conn(0x7f94f00d1670 legacy=0x7f94f00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94d4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d4004d50 0x7f94d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00ae100 con 0x7f94d4003a10 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00ae870 con 0x7f94d4004d50 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94d400bb00 con 0x7f94f015db30 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94d4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94e7e00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94efe00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d4004d50 0x7f94d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 -- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 msgr2=0x7f94d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f015db30 0x7f94d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 -- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d4004d50 msgr2=0x7f94d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:17:59.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d4004d50 0x7f94d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94ef400640 1 -- 172.21.15.116:0/1989096459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d4003530 con 0x7f94d4003a10 2024-05-14T11:17:59.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.828+0000 7f94efe00640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d4004d50 0x7f94d4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94ef400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94d4003e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009ae0 tx=0x7f94e001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a57aa5e9986498c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00b2070 con 0x7f94d4003a10 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e00a5510 con 0x7f94d4003a10 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94eea00640 1 -- 172.21.15.116:0/1989096459 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00096b0 con 0x7f94d4003a10 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- 172.21.15.116:0/1989096459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 msgr2=0x7f94d4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 --2- 172.21.15.116:0/1989096459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94d4003e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009ae0 tx=0x7f94e001ca90 comp rx=0 tx=0).stop 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94efe00640 1 -- 172.21.15.116:0/1989096459 reap_dead start 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- 172.21.15.116:0/1989096459 shutdown_connections 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- 172.21.15.116:0/1989096459 >> 172.21.15.116:0/1989096459 conn(0x7f94f00b4620 msgr2=0x7f94f00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- 172.21.15.116:0/1989096459 shutdown_connections 2024-05-14T11:17:59.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- 172.21.15.116:0/1989096459 wait complete. 2024-05-14T11:17:59.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 Processor -- start 2024-05-14T11:17:59.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.832+0000 7f94f6400640 1 -- start start 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f01483b0 0x7f94f0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94f01487c0 0x7f94f0159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00af030 con 0x7f94d4003a10 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94f00aeeb0 con 0x7f94f01487c0 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94f0168760 con 0x7f94f01483b0 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58254/0 (socket says 172.21.15.116:58254) 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 -- 172.21.15.116:0/1101398694 learned_addr learned my addr 172.21.15.116:0/1101398694 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94efe00640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94f01487c0 0x7f94f0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94e7e00640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f01483b0 0x7f94f0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 -- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f01483b0 msgr2=0x7f94f0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f01483b0 0x7f94f0158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 -- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94f01487c0 msgr2=0x7f94f0159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94f01487c0 0x7f94f0159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 -- 172.21.15.116:0/1101398694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f01fdb60 con 0x7f94d4003a10 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94ef400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f94e001c8c0 tx=0x7f94d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00b2040 con 0x7f94d4003a10 2024-05-14T11:17:59.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e00a5cc0 con 0x7f94d4003a10 2024-05-14T11:17:59.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00096b0 con 0x7f94d4003a10 2024-05-14T11:17:59.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f01fde40 con 0x7f94d4003a10 2024-05-14T11:17:59.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f00ce0a0 con 0x7f94d4003a10 2024-05-14T11:17:59.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.836+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f0165650 con 0x7f94d4003a10 2024-05-14T11:17:59.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.840+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f94e0004250 con 0x7f94d4003a10 2024-05-14T11:17:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.840+0000 7f94e7400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 0x7f94cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:17:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.840+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f94e01298c0 con 0x7f94d4003a10 2024-05-14T11:17:59.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.840+0000 7f94e7e00640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 0x7f94cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:17:59.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.840+0000 7f94e7e00640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 0x7f94cc071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94f01596c0 tx=0x7f94d8001a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:17:59.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:17:59.844+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f94f0165650 con 0x7f94d4003a10 2024-05-14T11:18:00.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.032+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94f00cdbb0 con 0x7f94d4003a10 2024-05-14T11:18:00.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94e7400640 1 -- 172.21.15.116:0/1101398694 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f94e00167a0 con 0x7f94d4003a10 2024-05-14T11:18:00.037 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:00.037 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:00.037 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:00.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 msgr2=0x7f94cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 0x7f94cc071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f94f01596c0 tx=0x7f94d8001a40 comp rx=0 tx=0).stop 2024-05-14T11:18:00.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 msgr2=0x7f94f01fd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f94e001c8c0 tx=0x7f94d4005840 comp rx=0 tx=0).stop 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 shutdown_connections 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94cc06f400 0x7f94cc071910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94f01487c0 0x7f94f0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94f01483b0 0x7f94f0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 --2- 172.21.15.116:0/1101398694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94d4003a10 0x7f94f01fd0a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 >> 172.21.15.116:0/1101398694 conn(0x7f94f00b4620 msgr2=0x7f94f0160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 shutdown_connections 2024-05-14T11:18:00.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.036+0000 7f94f6400640 1 -- 172.21.15.116:0/1101398694 wait complete. 2024-05-14T11:18:00.060 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:00.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.204+0000 7f4c09800640 1 Processor -- start 2024-05-14T11:18:00.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.204+0000 7f4c09800640 1 -- start start 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c09800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 0x7f4c04160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c09800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4c040b54b0 0x7f4c040b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c09800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0406f5c0 con 0x7f4c04070130 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c09800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4c04161200 con 0x7f4c040b54b0 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c09800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4c04161450 con 0x7f4c04070610 2024-05-14T11:18:00.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c02a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4c04070130 0x7f4c041605b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37424/0 (socket says 172.21.15.116:37424) 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c02a00640 1 -- 172.21.15.116:0/3799317214 learned_addr learned my addr 172.21.15.116:0/3799317214 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4bfbe00640 1 --2- 172.21.15.116:0/3799317214 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 0x7f4c04160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c03400640 1 --2- 172.21.15.116:0/3799317214 >> v2:172.21.15.149:3300/0 conn(0x7f4c040b54b0 0x7f4c040b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4c02000640 1 -- 172.21.15.116:0/3799317214 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727281593 0 0) 0x7f4c0406f5c0 con 0x7f4c04070130 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4bfbe00640 1 -- 172.21.15.116:0/3799317214 >> v2:172.21.15.149:3300/0 conn(0x7f4c040b54b0 msgr2=0x7f4c040b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4bfbe00640 1 --2- 172.21.15.116:0/3799317214 >> v2:172.21.15.149:3300/0 conn(0x7f4c040b54b0 0x7f4c040b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4bfbe00640 1 -- 172.21.15.116:0/3799317214 >> v1:172.21.15.116:6789/0 conn(0x7f4c04070130 legacy=0x7f4c041605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.208+0000 7f4bfbe00640 1 -- 172.21.15.116:0/3799317214 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c040d1fb0 con 0x7f4c04070610 2024-05-14T11:18:00.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4bfbe00640 1 --2- 172.21.15.116:0/3799317214 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 0x7f4c04160cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097c0 tx=0x7f4bec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285667dbe0204645 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c02000640 1 -- 172.21.15.116:0/3799317214 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0b2070 con 0x7f4c04070610 2024-05-14T11:18:00.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c02000640 1 -- 172.21.15.116:0/3799317214 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bec0ad640 con 0x7f4c04070610 2024-05-14T11:18:00.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c02000640 1 -- 172.21.15.116:0/3799317214 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0ada10 con 0x7f4c04070610 2024-05-14T11:18:00.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 -- 172.21.15.116:0/3799317214 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 msgr2=0x7f4c04160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 --2- 172.21.15.116:0/3799317214 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 0x7f4c04160cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097c0 tx=0x7f4bec01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 -- 172.21.15.116:0/3799317214 shutdown_connections 2024-05-14T11:18:00.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 --2- 172.21.15.116:0/3799317214 >> v2:172.21.15.149:3300/0 conn(0x7f4c040b54b0 0x7f4c040b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 --2- 172.21.15.116:0/3799317214 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070610 0x7f4c04160cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 -- 172.21.15.116:0/3799317214 >> 172.21.15.116:0/3799317214 conn(0x7f4c04071370 msgr2=0x7f4c04071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 -- 172.21.15.116:0/3799317214 shutdown_connections 2024-05-14T11:18:00.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.212+0000 7f4c09800640 1 -- 172.21.15.116:0/3799317214 wait complete. 2024-05-14T11:18:00.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 Processor -- start 2024-05-14T11:18:00.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- start start 2024-05-14T11:18:00.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 0x7f4c0415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 0x7f4c04143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c041698c0 con 0x7f4c040b54b0 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4c04169740 con 0x7f4c04070610 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4c04169a40 con 0x7f4c04070130 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58256/0 (socket says 172.21.15.116:58256) 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 -- 172.21.15.116:0/4065490286 learned_addr learned my addr 172.21.15.116:0/4065490286 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c02a00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 0x7f4c0415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4bfbe00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 0x7f4c04143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 -- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 msgr2=0x7f4c0415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 0x7f4c0415a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 -- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 msgr2=0x7f4c04143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 0x7f4c04143670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 -- 172.21.15.116:0/4065490286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c04159080 con 0x7f4c040b54b0 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4bfbe00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 0x7f4c04143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c02a00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 0x7f4c0415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c03400640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00efc0 tx=0x7f4bfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc018020 con 0x7f4c040b54b0 2024-05-14T11:18:00.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bfc014480 con 0x7f4c040b54b0 2024-05-14T11:18:00.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c04159360 con 0x7f4c040b54b0 2024-05-14T11:18:00.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc009e60 con 0x7f4c040b54b0 2024-05-14T11:18:00.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.216+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c04159920 con 0x7f4c040b54b0 2024-05-14T11:18:00.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c04158df0 con 0x7f4c040b54b0 2024-05-14T11:18:00.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4bfc01b020 con 0x7f4c040b54b0 2024-05-14T11:18:00.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4bfaa00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 0x7f4be0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f4bfc090100 con 0x7f4c040b54b0 2024-05-14T11:18:00.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4c02a00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 0x7f4be0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.220+0000 7f4c02a00640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 0x7f4be0071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000f70 tx=0x7f4bf4007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.224+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4bfc022030 con 0x7f4c040b54b0 2024-05-14T11:18:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.392+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c0406a9c0 con 0x7f4c040b54b0 2024-05-14T11:18:00.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.392+0000 7f4bfaa00640 1 -- 172.21.15.116:0/4065490286 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16990 (secure 0 0 0) 0x7f4bfc010040 con 0x7f4c040b54b0 2024-05-14T11:18:00.396 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:00.396 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":49,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:57.635905+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:53.726817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:54.993539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 msgr2=0x7f4be0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 0x7f4be0071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000f70 tx=0x7f4bf4007df0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 msgr2=0x7f4c04143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00efc0 tx=0x7f4bfc00c490 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 shutdown_connections 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4be006f400 0x7f4be0071910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c04070610 0x7f4c04143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c040b54b0 0x7f4c04143f80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 --2- 172.21.15.116:0/4065490286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c04070130 0x7f4c0415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 >> 172.21.15.116:0/4065490286 conn(0x7f4c04071370 msgr2=0x7f4c0415c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 shutdown_connections 2024-05-14T11:18:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.396+0000 7f4c09800640 1 -- 172.21.15.116:0/4065490286 wait complete. 2024-05-14T11:18:00.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:18:00.419 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:18:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.564+0000 7f1ca3200640 1 Processor -- start 2024-05-14T11:18:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.564+0000 7f1ca3200640 1 -- start start 2024-05-14T11:18:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca3200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 0x7f1c9c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca3200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c9c06f660 0x7f1c9c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca3200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9c1603b0 con 0x7f1c9c1598f0 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca3200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0d1f90 con 0x7f1c9c06f660 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca3200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c160fc0 con 0x7f1c9c06f250 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c9c06f660 0x7f1c9c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca1200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1c9c1598f0 0x7f1c9c159d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37440/0 (socket says 172.21.15.116:37440) 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1ca1200640 1 -- 172.21.15.116:0/2045103879 learned_addr learned my addr 172.21.15.116:0/2045103879 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9be00640 1 --2- 172.21.15.116:0/2045103879 >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 0x7f1c9c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9b400640 1 -- 172.21.15.116:0/2045103879 >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 msgr2=0x7f1c9c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9b400640 1 --2- 172.21.15.116:0/2045103879 >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 0x7f1c9c0714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9b400640 1 -- 172.21.15.116:0/2045103879 >> v1:172.21.15.116:6789/0 conn(0x7f1c9c1598f0 legacy=0x7f1c9c159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9aa00640 1 -- 172.21.15.116:0/2045103879 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285866056 0 0) 0x7f1c9c1603b0 con 0x7f1c9c1598f0 2024-05-14T11:18:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9b400640 1 -- 172.21.15.116:0/2045103879 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c160690 con 0x7f1c9c06f660 2024-05-14T11:18:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.568+0000 7f1c9be00640 1 --2- 172.21.15.116:0/2045103879 >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 0x7f1c9c0714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1c9b400640 1 --2- 172.21.15.116:0/2045103879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c071a00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c880097c0 tx=0x7f1c8801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea63f002e4d41cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1c9aa00640 1 -- 172.21.15.116:0/2045103879 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880b1070 con 0x7f1c9c06f660 2024-05-14T11:18:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1c9aa00640 1 -- 172.21.15.116:0/2045103879 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c880ac600 con 0x7f1c9c06f660 2024-05-14T11:18:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1c9aa00640 1 -- 172.21.15.116:0/2045103879 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880ac9d0 con 0x7f1c9c06f660 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- 172.21.15.116:0/2045103879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 msgr2=0x7f1c9c071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 --2- 172.21.15.116:0/2045103879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c071a00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c880097c0 tx=0x7f1c8801c3a0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- 172.21.15.116:0/2045103879 shutdown_connections 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 --2- 172.21.15.116:0/2045103879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c071a00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 --2- 172.21.15.116:0/2045103879 >> v2:172.21.15.149:3300/0 conn(0x7f1c9c06f250 0x7f1c9c0714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- 172.21.15.116:0/2045103879 >> 172.21.15.116:0/2045103879 conn(0x7f1c9c073ca0 msgr2=0x7f1c9c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- 172.21.15.116:0/2045103879 shutdown_connections 2024-05-14T11:18:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- 172.21.15.116:0/2045103879 wait complete. 2024-05-14T11:18:00.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 Processor -- start 2024-05-14T11:18:00.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.572+0000 7f1ca3200640 1 -- start start 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c06f250 0x7f1c9c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c9c1598f0 0x7f1c9c142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0ce470 con 0x7f1c9c06f660 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c164750 con 0x7f1c9c06f250 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0ce5f0 con 0x7f1c9c1598f0 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58272/0 (socket says 172.21.15.116:58272) 2024-05-14T11:18:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 -- 172.21.15.116:0/1091069498 learned_addr learned my addr 172.21.15.116:0/1091069498 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9be00640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c06f250 0x7f1c9c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca1200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c9c1598f0 0x7f1c9c142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 -- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c9c1598f0 msgr2=0x7f1c9c142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c9c1598f0 0x7f1c9c142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 -- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c06f250 msgr2=0x7f1c9c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c06f250 0x7f1c9c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 -- 172.21.15.116:0/1091069498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c1588e0 con 0x7f1c9c06f660 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c9b400640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1c88019fd0 tx=0x7f1c8801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880b1040 con 0x7f1c9c06f660 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c880b7450 con 0x7f1c9c06f660 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880b7820 con 0x7f1c9c06f660 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c158bc0 con 0x7f1c9c06f660 2024-05-14T11:18:00.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c0b0250 con 0x7f1c9c06f660 2024-05-14T11:18:00.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.576+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c06a9c0 con 0x7f1c9c06f660 2024-05-14T11:18:00.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.580+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c880be030 con 0x7f1c9c06f660 2024-05-14T11:18:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.580+0000 7f1c93e00640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 0x7f1c7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.580+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f1c88128b20 con 0x7f1c9c06f660 2024-05-14T11:18:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.580+0000 7f1c9be00640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 0x7f1c7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.580+0000 7f1c9be00640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 0x7f1c7c071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0710c0 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.584+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1c88016050 con 0x7f1c9c06f660 2024-05-14T11:18:00.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.744+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c9c158670 con 0x7f1c7c06f400 2024-05-14T11:18:00.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.748+0000 7f1c93e00640 1 -- 172.21.15.116:0/1091069498 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240065 (secure 0 0 0) 0x7f1c9c158670 con 0x7f1c7c06f400 2024-05-14T11:18:00.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:00.756 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 msgr2=0x7f1c7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 0x7f1c7c071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0710c0 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 msgr2=0x7f1c9c0b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1c88019fd0 tx=0x7f1c8801abe0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 shutdown_connections 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c7c06f400 0x7f1c7c071910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c9c1598f0 0x7f1c9c142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c9c06f660 0x7f1c9c0b3850 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 --2- 172.21.15.116:0/1091069498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c06f250 0x7f1c9c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 >> 172.21.15.116:0/1091069498 conn(0x7f1c9c073ca0 msgr2=0x7f1c9c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 shutdown_connections 2024-05-14T11:18:00.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.752+0000 7f1ca3200640 1 -- 172.21.15.116:0/1091069498 wait complete. 2024-05-14T11:18:00.775 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-05-14T11:18:00.384388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229896,"kb_used_data":5368,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744824,"statfs":{"total":773094113280,"available":772858699776,"internally_reserved":0,"allocated":5496832,"data_stored":3052268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767949"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620076+0000","last_change":"2024-05-14T11:17:54.620076+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620076+0000","last_undegraded":"2024-05-14T11:17:54.620076+0000","last_fullsized":"2024-05-14T11:17:54.620076+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620498+0000","last_change":"2024-05-14T11:17:54.620498+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620498+0000","last_undegraded":"2024-05-14T11:17:54.620498+0000","last_fullsized":"2024-05-14T11:17:54.620498+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620675+0000","last_change":"2024-05-14T11:17:54.620675+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620675+0000","last_undegraded":"2024-05-14T11:17:54.620675+0000","last_fullsized":"2024-05-14T11:17:54.620675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619089+0000","last_change":"2024-05-14T11:17:54.619089+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619089+0000","last_undegraded":"2024-05-14T11:17:54.619089+0000","last_fullsized":"2024-05-14T11:17:54.619089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624273+0000","last_change":"2024-05-14T11:17:54.624273+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624273+0000","last_undegraded":"2024-05-14T11:17:54.624273+0000","last_fullsized":"2024-05-14T11:17:54.624273+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626617+0000","last_change":"2024-05-14T11:17:54.626617+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626617+0000","last_undegraded":"2024-05-14T11:17:54.626617+0000","last_fullsized":"2024-05-14T11:17:54.626617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619115+0000","last_change":"2024-05-14T11:17:54.619115+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619115+0000","last_undegraded":"2024-05-14T11:17:54.619115+0000","last_fullsized":"2024-05-14T11:17:54.619115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623167+0000","last_change":"2024-05-14T11:17:54.623167+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623167+0000","last_undegraded":"2024-05-14T11:17:54.623167+0000","last_fullsized":"2024-05-14T11:17:54.623167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624482+0000","last_change":"2024-05-14T11:17:54.624482+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624482+0000","last_undegraded":"2024-05-14T11:17:54.624482+0000","last_fullsized":"2024-05-14T11:17:54.624482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626095+0000","last_change":"2024-05-14T11:17:54.626095+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626095+0000","last_undegraded":"2024-05-14T11:17:54.626095+0000","last_fullsized":"2024-05-14T11:17:54.626095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625084+0000","last_change":"2024-05-14T11:17:54.625084+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625084+0000","last_undegraded":"2024-05-14T11:17:54.625084+0000","last_fullsized":"2024-05-14T11:17:54.625084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624901+0000","last_change":"2024-05-14T11:17:54.624901+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624901+0000","last_undegraded":"2024-05-14T11:17:54.624901+0000","last_fullsized":"2024-05-14T11:17:54.624901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623165+0000","last_change":"2024-05-14T11:17:54.623165+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623165+0000","last_undegraded":"2024-05-14T11:17:54.623165+0000","last_fullsized":"2024-05-14T11:17:54.623165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619452+0000","last_change":"2024-05-14T11:17:54.619452+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619452+0000","last_undegraded":"2024-05-14T11:17:54.619452+0000","last_fullsized":"2024-05-14T11:17:54.619452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624432+0000","last_change":"2024-05-14T11:17:54.624432+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624432+0000","last_undegraded":"2024-05-14T11:17:54.624432+0000","last_fullsized":"2024-05-14T11:17:54.624432+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624438+0000","last_change":"2024-05-14T11:17:54.624438+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624438+0000","last_undegraded":"2024-05-14T11:17:54.624438+0000","last_fullsized":"2024-05-14T11:17:54.624438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623126+0000","last_change":"2024-05-14T11:17:54.623126+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623126+0000","last_undegraded":"2024-05-14T11:17:54.623126+0000","last_fullsized":"2024-05-14T11:17:54.623126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.629246+0000","last_change":"2024-05-14T11:17:54.629246+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.629246+0000","last_undegraded":"2024-05-14T11:17:54.629246+0000","last_fullsized":"2024-05-14T11:17:54.629246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619171+0000","last_change":"2024-05-14T11:17:54.619171+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619171+0000","last_undegraded":"2024-05-14T11:17:54.619171+0000","last_fullsized":"2024-05-14T11:17:54.619171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619109+0000","last_change":"2024-05-14T11:17:54.619109+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619109+0000","last_undegraded":"2024-05-14T11:17:54.619109+0000","last_fullsized":"2024-05-14T11:17:54.619109+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620479+0000","last_change":"2024-05-14T11:17:54.620479+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620479+0000","last_undegraded":"2024-05-14T11:17:54.620479+0000","last_fullsized":"2024-05-14T11:17:54.620479+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626224+0000","last_change":"2024-05-14T11:17:54.626224+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626224+0000","last_undegraded":"2024-05-14T11:17:54.626224+0000","last_fullsized":"2024-05-14T11:17:54.626224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623134+0000","last_change":"2024-05-14T11:17:54.623134+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623134+0000","last_undegraded":"2024-05-14T11:17:54.623134+0000","last_fullsized":"2024-05-14T11:17:54.623134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.628832+0000","last_change":"2024-05-14T11:17:54.628832+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.628832+0000","last_undegraded":"2024-05-14T11:17:54.628832+0000","last_fullsized":"2024-05-14T11:17:54.628832+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619639+0000","last_change":"2024-05-14T11:17:54.619639+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619639+0000","last_undegraded":"2024-05-14T11:17:54.619639+0000","last_fullsized":"2024-05-14T11:17:54.619639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624446+0000","last_change":"2024-05-14T11:17:54.624446+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624446+0000","last_undegraded":"2024-05-14T11:17:54.624446+0000","last_fullsized":"2024-05-14T11:17:54.624446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623233+0000","last_change":"2024-05-14T11:17:54.623233+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623233+0000","last_undegraded":"2024-05-14T11:17:54.623233+0000","last_fullsized":"2024-05-14T11:17:54.623233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624809+0000","last_change":"2024-05-14T11:17:54.624809+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624809+0000","last_undegraded":"2024-05-14T11:17:54.624809+0000","last_fullsized":"2024-05-14T11:17:54.624809+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625684+0000","last_change":"2024-05-14T11:17:54.625684+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625684+0000","last_undegraded":"2024-05-14T11:17:54.625684+0000","last_fullsized":"2024-05-14T11:17:54.625684+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626011+0000","last_change":"2024-05-14T11:17:54.626011+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626011+0000","last_undegraded":"2024-05-14T11:17:54.626011+0000","last_fullsized":"2024-05-14T11:17:54.626011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619093+0000","last_change":"2024-05-14T11:17:54.619093+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619093+0000","last_undegraded":"2024-05-14T11:17:54.619093+0000","last_fullsized":"2024-05-14T11:17:54.619093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623497+0000","last_change":"2024-05-14T11:17:54.623497+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623497+0000","last_undegraded":"2024-05-14T11:17:54.623497+0000","last_fullsized":"2024-05-14T11:17:54.623497+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-14T11:17:54.625629+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:54.625629+0000","last_peered":"2024-05-14T11:17:54.625629+0000","last_clean":"2024-05-14T11:17:54.625629+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:54.625629+0000","last_undegraded":"2024-05-14T11:17:54.625629+0000","last_fullsized":"2024-05-14T11:17:54.625629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":827414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:18:00.779 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:18:00.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.964+0000 7fd5b6c00640 1 Processor -- start 2024-05-14T11:18:00.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.964+0000 7fd5b6c00640 1 -- start start 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5b6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd5b00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5b6c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b00d1fc0 con 0x7fd5b0156fa0 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5b6c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b0160510 con 0x7fd5b00af380 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5b6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00706f0 con 0x7fd5b00d1670 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5afe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd5b0156fa0 0x7fd5b006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37446/0 (socket says 172.21.15.116:37446) 2024-05-14T11:18:00.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5afe00640 1 -- 172.21.15.116:0/851073397 learned_addr learned my addr 172.21.15.116:0/851073397 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd5b00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217003641 0 0) 0x7fd5a00014a0 con 0x7fd5b0156fa0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd594003530 con 0x7fd5b0156fa0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046131997 0 0) 0x7fd594003530 con 0x7fd5b0156fa0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 msgr2=0x7fd5b00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd5b00d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 >> v1:172.21.15.149:6789/0 conn(0x7fd5b00af380 legacy=0x7fd5b00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01607f0 con 0x7fd5b0156fa0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fd5a0001c50 con 0x7fd5b0156fa0 2024-05-14T11:18:00.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 >> v1:172.21.15.116:6789/0 conn(0x7fd5b0156fa0 legacy=0x7fd5b006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd594003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd594004010 0x7fd594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd594004d50 0x7fd594005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00706f0 con 0x7fd594004010 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd59400bc10 con 0x7fd594004d50 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd59400e020 con 0x7fd5b00d1670 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd594004010 0x7fd594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5afe00640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd594003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5b4c00640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd594004d50 0x7fd594005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 -- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 msgr2=0x7fd594003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5b00d1670 0x7fd594003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 -- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd594004d50 msgr2=0x7fd594005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd594004d50 0x7fd594005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.968+0000 7fd5a7400640 1 -- 172.21.15.116:0/851073397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd594003530 con 0x7fd594004010 2024-05-14T11:18:00.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5a7400640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd594004010 0x7fd594004810 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ae0 tx=0x7fd59801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a81511215de6483 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980b2070 con 0x7fd594004010 2024-05-14T11:18:00.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5980a5680 con 0x7fd594004010 2024-05-14T11:18:00.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5af400640 1 -- 172.21.15.116:0/851073397 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980096b0 con 0x7fd594004010 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 -- 172.21.15.116:0/851073397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd594004010 msgr2=0x7fd594004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/851073397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd594004010 0x7fd594004810 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ae0 tx=0x7fd59801ca90 comp rx=0 tx=0).stop 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b4c00640 1 -- 172.21.15.116:0/851073397 reap_dead start 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 -- 172.21.15.116:0/851073397 shutdown_connections 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 -- 172.21.15.116:0/851073397 >> 172.21.15.116:0/851073397 conn(0x7fd5b00b4620 msgr2=0x7fd5b00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 -- 172.21.15.116:0/851073397 shutdown_connections 2024-05-14T11:18:00.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.972+0000 7fd5b6c00640 1 -- 172.21.15.116:0/851073397 wait complete. 2024-05-14T11:18:00.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 Processor -- start 2024-05-14T11:18:00.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- start start 2024-05-14T11:18:00.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd594004010 0x7fd5b00739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b00b2250 0x7fd5b00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0161120 con 0x7fd5b014e490 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0160db0 con 0x7fd5b00b2250 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b00d1a80 con 0x7fd594004010 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58288/0 (socket says 172.21.15.116:58288) 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 -- 172.21.15.116:0/86216919 learned_addr learned my addr 172.21.15.116:0/86216919 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5afe00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd594004010 0x7fd5b00739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b4c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b00b2250 0x7fd5b00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 -- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd594004010 msgr2=0x7fd5b00739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd594004010 0x7fd5b00739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 -- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b00b2250 msgr2=0x7fd5b00b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b00b2250 0x7fd5b00b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 -- 172.21.15.116:0/86216919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00b2ba0 con 0x7fd5b014e490 2024-05-14T11:18:00.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5a7400640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ae0 tx=0x7fd594005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980bd070 con 0x7fd5b014e490 2024-05-14T11:18:00.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd598004030 con 0x7fd5b014e490 2024-05-14T11:18:00.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980b2040 con 0x7fd5b014e490 2024-05-14T11:18:00.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b0073f20 con 0x7fd5b014e490 2024-05-14T11:18:00.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.976+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01510b0 con 0x7fd5b014e490 2024-05-14T11:18:00.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b006a9c0 con 0x7fd5b014e490 2024-05-14T11:18:00.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5980c0070 con 0x7fd5b014e490 2024-05-14T11:18:00.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5acc00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 0x7fd58c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:00.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd598129080 con 0x7fd5b014e490 2024-05-14T11:18:00.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5afe00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 0x7fd58c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:00.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.980+0000 7fd5afe00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 0x7fd58c071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00b0d60 tx=0x7fd5a000a250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:00.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:00.984+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd5b00b2ba0 con 0x7fd5b014e490 2024-05-14T11:18:01.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.140+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5b006fdc0 con 0x7fd58c06f430 2024-05-14T11:18:01.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.148+0000 7fd5acc00640 1 -- 172.21.15.116:0/86216919 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240065 (secure 0 0 0) 0x7fd5b006fdc0 con 0x7fd58c06f430 2024-05-14T11:18:01.154 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:01.156 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:18:01.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 msgr2=0x7fd58c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 0x7fd58c071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00b0d60 tx=0x7fd5a000a250 comp rx=0 tx=0).stop 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 msgr2=0x7fd5b0148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ae0 tx=0x7fd594005840 comp rx=0 tx=0).stop 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 shutdown_connections 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd58c06f430 0x7fd58c071940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b00b2250 0x7fd5b00b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b014e490 0x7fd5b0148850 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 --2- 172.21.15.116:0/86216919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd594004010 0x7fd5b00739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 >> 172.21.15.116:0/86216919 conn(0x7fd5b00b4620 msgr2=0x7fd5b014f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:01.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 shutdown_connections 2024-05-14T11:18:01.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.152+0000 7fd5b6c00640 1 -- 172.21.15.116:0/86216919 wait complete. 2024-05-14T11:18:01.173 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-05-14T11:18:00.384388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229896,"kb_used_data":5368,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744824,"statfs":{"total":773094113280,"available":772858699776,"internally_reserved":0,"allocated":5496832,"data_stored":3052268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767949"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620076+0000","last_change":"2024-05-14T11:17:54.620076+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620076+0000","last_undegraded":"2024-05-14T11:17:54.620076+0000","last_fullsized":"2024-05-14T11:17:54.620076+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620498+0000","last_change":"2024-05-14T11:17:54.620498+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620498+0000","last_undegraded":"2024-05-14T11:17:54.620498+0000","last_fullsized":"2024-05-14T11:17:54.620498+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620675+0000","last_change":"2024-05-14T11:17:54.620675+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620675+0000","last_undegraded":"2024-05-14T11:17:54.620675+0000","last_fullsized":"2024-05-14T11:17:54.620675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619089+0000","last_change":"2024-05-14T11:17:54.619089+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619089+0000","last_undegraded":"2024-05-14T11:17:54.619089+0000","last_fullsized":"2024-05-14T11:17:54.619089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624273+0000","last_change":"2024-05-14T11:17:54.624273+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624273+0000","last_undegraded":"2024-05-14T11:17:54.624273+0000","last_fullsized":"2024-05-14T11:17:54.624273+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626617+0000","last_change":"2024-05-14T11:17:54.626617+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626617+0000","last_undegraded":"2024-05-14T11:17:54.626617+0000","last_fullsized":"2024-05-14T11:17:54.626617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619115+0000","last_change":"2024-05-14T11:17:54.619115+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619115+0000","last_undegraded":"2024-05-14T11:17:54.619115+0000","last_fullsized":"2024-05-14T11:17:54.619115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623167+0000","last_change":"2024-05-14T11:17:54.623167+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623167+0000","last_undegraded":"2024-05-14T11:17:54.623167+0000","last_fullsized":"2024-05-14T11:17:54.623167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624482+0000","last_change":"2024-05-14T11:17:54.624482+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624482+0000","last_undegraded":"2024-05-14T11:17:54.624482+0000","last_fullsized":"2024-05-14T11:17:54.624482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626095+0000","last_change":"2024-05-14T11:17:54.626095+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626095+0000","last_undegraded":"2024-05-14T11:17:54.626095+0000","last_fullsized":"2024-05-14T11:17:54.626095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625084+0000","last_change":"2024-05-14T11:17:54.625084+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625084+0000","last_undegraded":"2024-05-14T11:17:54.625084+0000","last_fullsized":"2024-05-14T11:17:54.625084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624901+0000","last_change":"2024-05-14T11:17:54.624901+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624901+0000","last_undegraded":"2024-05-14T11:17:54.624901+0000","last_fullsized":"2024-05-14T11:17:54.624901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623165+0000","last_change":"2024-05-14T11:17:54.623165+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623165+0000","last_undegraded":"2024-05-14T11:17:54.623165+0000","last_fullsized":"2024-05-14T11:17:54.623165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619452+0000","last_change":"2024-05-14T11:17:54.619452+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619452+0000","last_undegraded":"2024-05-14T11:17:54.619452+0000","last_fullsized":"2024-05-14T11:17:54.619452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624432+0000","last_change":"2024-05-14T11:17:54.624432+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624432+0000","last_undegraded":"2024-05-14T11:17:54.624432+0000","last_fullsized":"2024-05-14T11:17:54.624432+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624438+0000","last_change":"2024-05-14T11:17:54.624438+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624438+0000","last_undegraded":"2024-05-14T11:17:54.624438+0000","last_fullsized":"2024-05-14T11:17:54.624438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623126+0000","last_change":"2024-05-14T11:17:54.623126+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623126+0000","last_undegraded":"2024-05-14T11:17:54.623126+0000","last_fullsized":"2024-05-14T11:17:54.623126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.629246+0000","last_change":"2024-05-14T11:17:54.629246+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.629246+0000","last_undegraded":"2024-05-14T11:17:54.629246+0000","last_fullsized":"2024-05-14T11:17:54.629246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619171+0000","last_change":"2024-05-14T11:17:54.619171+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619171+0000","last_undegraded":"2024-05-14T11:17:54.619171+0000","last_fullsized":"2024-05-14T11:17:54.619171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619109+0000","last_change":"2024-05-14T11:17:54.619109+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619109+0000","last_undegraded":"2024-05-14T11:17:54.619109+0000","last_fullsized":"2024-05-14T11:17:54.619109+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620479+0000","last_change":"2024-05-14T11:17:54.620479+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620479+0000","last_undegraded":"2024-05-14T11:17:54.620479+0000","last_fullsized":"2024-05-14T11:17:54.620479+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626224+0000","last_change":"2024-05-14T11:17:54.626224+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626224+0000","last_undegraded":"2024-05-14T11:17:54.626224+0000","last_fullsized":"2024-05-14T11:17:54.626224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623134+0000","last_change":"2024-05-14T11:17:54.623134+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623134+0000","last_undegraded":"2024-05-14T11:17:54.623134+0000","last_fullsized":"2024-05-14T11:17:54.623134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.628832+0000","last_change":"2024-05-14T11:17:54.628832+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.628832+0000","last_undegraded":"2024-05-14T11:17:54.628832+0000","last_fullsized":"2024-05-14T11:17:54.628832+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619639+0000","last_change":"2024-05-14T11:17:54.619639+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619639+0000","last_undegraded":"2024-05-14T11:17:54.619639+0000","last_fullsized":"2024-05-14T11:17:54.619639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624446+0000","last_change":"2024-05-14T11:17:54.624446+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624446+0000","last_undegraded":"2024-05-14T11:17:54.624446+0000","last_fullsized":"2024-05-14T11:17:54.624446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623233+0000","last_change":"2024-05-14T11:17:54.623233+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623233+0000","last_undegraded":"2024-05-14T11:17:54.623233+0000","last_fullsized":"2024-05-14T11:17:54.623233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624809+0000","last_change":"2024-05-14T11:17:54.624809+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624809+0000","last_undegraded":"2024-05-14T11:17:54.624809+0000","last_fullsized":"2024-05-14T11:17:54.624809+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625684+0000","last_change":"2024-05-14T11:17:54.625684+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625684+0000","last_undegraded":"2024-05-14T11:17:54.625684+0000","last_fullsized":"2024-05-14T11:17:54.625684+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626011+0000","last_change":"2024-05-14T11:17:54.626011+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626011+0000","last_undegraded":"2024-05-14T11:17:54.626011+0000","last_fullsized":"2024-05-14T11:17:54.626011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619093+0000","last_change":"2024-05-14T11:17:54.619093+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619093+0000","last_undegraded":"2024-05-14T11:17:54.619093+0000","last_fullsized":"2024-05-14T11:17:54.619093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623497+0000","last_change":"2024-05-14T11:17:54.623497+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623497+0000","last_undegraded":"2024-05-14T11:17:54.623497+0000","last_fullsized":"2024-05-14T11:17:54.623497+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-14T11:17:54.625629+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:54.625629+0000","last_peered":"2024-05-14T11:17:54.625629+0000","last_clean":"2024-05-14T11:17:54.625629+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:54.625629+0000","last_undegraded":"2024-05-14T11:17:54.625629+0000","last_fullsized":"2024-05-14T11:17:54.625629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":827414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:18:01.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:18:01.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 Processor -- start 2024-05-14T11:18:01.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 -- start start 2024-05-14T11:18:01.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f21c01602c0 0x7f21c01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c00c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c0160d50 con 0x7f21c00cc910 2024-05-14T11:18:01.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f21c0161310 con 0x7f21c01602c0 2024-05-14T11:18:01.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f21c0161030 con 0x7f21c00c7b00 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f21c01602c0 0x7f21c01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c6600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c00c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f21c01602c0 0x7f21c01606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:53364/0 (socket says 172.21.15.116:53364) 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 -- 172.21.15.116:0/1789460918 learned_addr learned my addr 172.21.15.116:0/1789460918 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5200640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f21c00cc910 0x7f21c00ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54090/0 (socket says 172.21.15.116:54090) 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 -- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 msgr2=0x7f21c00c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 --2- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c00c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 -- 172.21.15.116:0/1789460918 >> v1:172.21.15.149:6789/0 conn(0x7f21c00cc910 legacy=0x7f21c00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.360+0000 7f21c5c00640 1 -- 172.21.15.116:0/1789460918 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c00c7300 con 0x7f21c01602c0 2024-05-14T11:18:01.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c5c00640 1 --2- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c01602c0 0x7f21c01606d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b8761233bea96b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21bfe00640 1 -- 172.21.15.116:0/1789460918 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40ab650 con 0x7f21c01602c0 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21bfe00640 1 -- 172.21.15.116:0/1789460918 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b40b0070 con 0x7f21c01602c0 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21bfe00640 1 -- 172.21.15.116:0/1789460918 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40aba10 con 0x7f21c01602c0 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 -- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c01602c0 msgr2=0x7f21c01606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 --2- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c01602c0 0x7f21c01606d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009870 tx=0x7f21b401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 -- 172.21.15.116:0/1789460918 shutdown_connections 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 --2- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c00c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 --2- 172.21.15.116:0/1789460918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c01602c0 0x7f21c01606d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 -- 172.21.15.116:0/1789460918 >> 172.21.15.116:0/1789460918 conn(0x7f21c00d28c0 msgr2=0x7f21c00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 -- 172.21.15.116:0/1789460918 shutdown_connections 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.364+0000 7f21c7000640 1 -- 172.21.15.116:0/1789460918 wait complete. 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 Processor -- start 2024-05-14T11:18:01.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 -- start start 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c01fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00cc910 0x7f21c0142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0163c30 con 0x7f21c01602c0 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0163ab0 con 0x7f21c00cc910 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f21c016ca80 con 0x7f21c00c7b00 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c6600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c6600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58304/0 (socket says 172.21.15.116:58304) 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c6600640 1 -- 172.21.15.116:0/2505533814 learned_addr learned my addr 172.21.15.116:0/2505533814 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c5200640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00cc910 0x7f21c0142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c5c00640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c01fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c6600640 1 -- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 msgr2=0x7f21c01fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.368+0000 7f21c6600640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c01fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c6600640 1 -- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00cc910 msgr2=0x7f21c0142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c6600640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00cc910 0x7f21c0142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c6600640 1 -- 172.21.15.116:0/2505533814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c01fd680 con 0x7f21c01602c0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c6600640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009d30 tx=0x7f21b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8018020 con 0x7f21c01602c0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b8014480 con 0x7f21c01602c0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c0144490 con 0x7f21c01602c0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8014870 con 0x7f21c01602c0 2024-05-14T11:18:01.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c00b73e0 con 0x7f21c01602c0 2024-05-14T11:18:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c00c93c0 con 0x7f21c01602c0 2024-05-14T11:18:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f21b801b020 con 0x7f21c01602c0 2024-05-14T11:18:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 0x7f21a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f21b8090190 con 0x7f21c01602c0 2024-05-14T11:18:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.372+0000 7f21c5c00640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 0x7f21a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.376+0000 7f21c5c00640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 0x7f21a0071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21b40af000 tx=0x7f21b40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.376+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f21b8022030 con 0x7f21c01602c0 2024-05-14T11:18:01.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.532+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21c00b2820 con 0x7f21a006f400 2024-05-14T11:18:01.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.540+0000 7f21bd600640 1 -- 172.21.15.116:0/2505533814 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240065 (secure 0 0 0) 0x7f21c00b2820 con 0x7f21a006f400 2024-05-14T11:18:01.545 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:01.546 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 msgr2=0x7f21a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 0x7f21a0071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21b40af000 tx=0x7f21b40a7000 comp rx=0 tx=0).stop 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 msgr2=0x7f21c0148970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009d30 tx=0x7f21b800c490 comp rx=0 tx=0).stop 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 shutdown_connections 2024-05-14T11:18:01.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f21a006f400 0x7f21a0071910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00cc910 0x7f21c0142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01602c0 0x7f21c0148970 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 --2- 172.21.15.116:0/2505533814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00c7b00 0x7f21c01fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 >> 172.21.15.116:0/2505533814 conn(0x7f21c00d28c0 msgr2=0x7f21c00b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 shutdown_connections 2024-05-14T11:18:01.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.544+0000 7f21c7000640 1 -- 172.21.15.116:0/2505533814 wait complete. 2024-05-14T11:18:01.566 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-05-14T11:18:00.384388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229896,"kb_used_data":5368,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744824,"statfs":{"total":773094113280,"available":772858699776,"internally_reserved":0,"allocated":5496832,"data_stored":3052268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767949"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620076+0000","last_change":"2024-05-14T11:17:54.620076+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620076+0000","last_undegraded":"2024-05-14T11:17:54.620076+0000","last_fullsized":"2024-05-14T11:17:54.620076+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620498+0000","last_change":"2024-05-14T11:17:54.620498+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620498+0000","last_undegraded":"2024-05-14T11:17:54.620498+0000","last_fullsized":"2024-05-14T11:17:54.620498+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620675+0000","last_change":"2024-05-14T11:17:54.620675+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620675+0000","last_undegraded":"2024-05-14T11:17:54.620675+0000","last_fullsized":"2024-05-14T11:17:54.620675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619089+0000","last_change":"2024-05-14T11:17:54.619089+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619089+0000","last_undegraded":"2024-05-14T11:17:54.619089+0000","last_fullsized":"2024-05-14T11:17:54.619089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624273+0000","last_change":"2024-05-14T11:17:54.624273+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624273+0000","last_undegraded":"2024-05-14T11:17:54.624273+0000","last_fullsized":"2024-05-14T11:17:54.624273+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626617+0000","last_change":"2024-05-14T11:17:54.626617+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626617+0000","last_undegraded":"2024-05-14T11:17:54.626617+0000","last_fullsized":"2024-05-14T11:17:54.626617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619115+0000","last_change":"2024-05-14T11:17:54.619115+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619115+0000","last_undegraded":"2024-05-14T11:17:54.619115+0000","last_fullsized":"2024-05-14T11:17:54.619115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623167+0000","last_change":"2024-05-14T11:17:54.623167+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623167+0000","last_undegraded":"2024-05-14T11:17:54.623167+0000","last_fullsized":"2024-05-14T11:17:54.623167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624482+0000","last_change":"2024-05-14T11:17:54.624482+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624482+0000","last_undegraded":"2024-05-14T11:17:54.624482+0000","last_fullsized":"2024-05-14T11:17:54.624482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626095+0000","last_change":"2024-05-14T11:17:54.626095+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626095+0000","last_undegraded":"2024-05-14T11:17:54.626095+0000","last_fullsized":"2024-05-14T11:17:54.626095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625084+0000","last_change":"2024-05-14T11:17:54.625084+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625084+0000","last_undegraded":"2024-05-14T11:17:54.625084+0000","last_fullsized":"2024-05-14T11:17:54.625084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624901+0000","last_change":"2024-05-14T11:17:54.624901+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624901+0000","last_undegraded":"2024-05-14T11:17:54.624901+0000","last_fullsized":"2024-05-14T11:17:54.624901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623165+0000","last_change":"2024-05-14T11:17:54.623165+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623165+0000","last_undegraded":"2024-05-14T11:17:54.623165+0000","last_fullsized":"2024-05-14T11:17:54.623165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619452+0000","last_change":"2024-05-14T11:17:54.619452+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619452+0000","last_undegraded":"2024-05-14T11:17:54.619452+0000","last_fullsized":"2024-05-14T11:17:54.619452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624432+0000","last_change":"2024-05-14T11:17:54.624432+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624432+0000","last_undegraded":"2024-05-14T11:17:54.624432+0000","last_fullsized":"2024-05-14T11:17:54.624432+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624438+0000","last_change":"2024-05-14T11:17:54.624438+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624438+0000","last_undegraded":"2024-05-14T11:17:54.624438+0000","last_fullsized":"2024-05-14T11:17:54.624438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623126+0000","last_change":"2024-05-14T11:17:54.623126+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623126+0000","last_undegraded":"2024-05-14T11:17:54.623126+0000","last_fullsized":"2024-05-14T11:17:54.623126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.629246+0000","last_change":"2024-05-14T11:17:54.629246+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.629246+0000","last_undegraded":"2024-05-14T11:17:54.629246+0000","last_fullsized":"2024-05-14T11:17:54.629246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619171+0000","last_change":"2024-05-14T11:17:54.619171+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619171+0000","last_undegraded":"2024-05-14T11:17:54.619171+0000","last_fullsized":"2024-05-14T11:17:54.619171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619109+0000","last_change":"2024-05-14T11:17:54.619109+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619109+0000","last_undegraded":"2024-05-14T11:17:54.619109+0000","last_fullsized":"2024-05-14T11:17:54.619109+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620479+0000","last_change":"2024-05-14T11:17:54.620479+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620479+0000","last_undegraded":"2024-05-14T11:17:54.620479+0000","last_fullsized":"2024-05-14T11:17:54.620479+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626224+0000","last_change":"2024-05-14T11:17:54.626224+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626224+0000","last_undegraded":"2024-05-14T11:17:54.626224+0000","last_fullsized":"2024-05-14T11:17:54.626224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623134+0000","last_change":"2024-05-14T11:17:54.623134+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623134+0000","last_undegraded":"2024-05-14T11:17:54.623134+0000","last_fullsized":"2024-05-14T11:17:54.623134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.628832+0000","last_change":"2024-05-14T11:17:54.628832+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.628832+0000","last_undegraded":"2024-05-14T11:17:54.628832+0000","last_fullsized":"2024-05-14T11:17:54.628832+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619639+0000","last_change":"2024-05-14T11:17:54.619639+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619639+0000","last_undegraded":"2024-05-14T11:17:54.619639+0000","last_fullsized":"2024-05-14T11:17:54.619639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624446+0000","last_change":"2024-05-14T11:17:54.624446+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624446+0000","last_undegraded":"2024-05-14T11:17:54.624446+0000","last_fullsized":"2024-05-14T11:17:54.624446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623233+0000","last_change":"2024-05-14T11:17:54.623233+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623233+0000","last_undegraded":"2024-05-14T11:17:54.623233+0000","last_fullsized":"2024-05-14T11:17:54.623233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624809+0000","last_change":"2024-05-14T11:17:54.624809+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624809+0000","last_undegraded":"2024-05-14T11:17:54.624809+0000","last_fullsized":"2024-05-14T11:17:54.624809+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625684+0000","last_change":"2024-05-14T11:17:54.625684+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625684+0000","last_undegraded":"2024-05-14T11:17:54.625684+0000","last_fullsized":"2024-05-14T11:17:54.625684+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626011+0000","last_change":"2024-05-14T11:17:54.626011+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626011+0000","last_undegraded":"2024-05-14T11:17:54.626011+0000","last_fullsized":"2024-05-14T11:17:54.626011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619093+0000","last_change":"2024-05-14T11:17:54.619093+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619093+0000","last_undegraded":"2024-05-14T11:17:54.619093+0000","last_fullsized":"2024-05-14T11:17:54.619093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623497+0000","last_change":"2024-05-14T11:17:54.623497+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623497+0000","last_undegraded":"2024-05-14T11:17:54.623497+0000","last_fullsized":"2024-05-14T11:17:54.623497+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-14T11:17:54.625629+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:54.625629+0000","last_peered":"2024-05-14T11:17:54.625629+0000","last_clean":"2024-05-14T11:17:54.625629+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:54.625629+0000","last_undegraded":"2024-05-14T11:17:54.625629+0000","last_fullsized":"2024-05-14T11:17:54.625629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":827414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:18:01.569 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3f is not active+clean 2024-05-14T11:18:01.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.569 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3e is not active+clean 2024-05-14T11:18:01.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3d is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3c is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3b is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3a is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.39 is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:PG 8.38 is not active+clean 2024-05-14T11:18:01.570 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.571 INFO:tasks.ceph.ceph_manager.ceph:PG 8.37 is not active+clean 2024-05-14T11:18:01.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.571 INFO:tasks.ceph.ceph_manager.ceph:PG 8.36 is not active+clean 2024-05-14T11:18:01.571 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.573 INFO:tasks.ceph.ceph_manager.ceph:PG 8.35 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:PG 8.34 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:PG 8.33 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:PG 8.32 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:PG 8.31 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:PG 8.30 is not active+clean 2024-05-14T11:18:01.574 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2f is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2e is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2d is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2c is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2b is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2a is not active+clean 2024-05-14T11:18:01.575 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.29 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.28 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.27 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.26 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.25 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:PG 8.24 is not active+clean 2024-05-14T11:18:01.576 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 8.23 is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 8.22 is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 8.21 is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 8.20 is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 8.10 is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-05-14T11:18:01.577 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.620076+0000', 'last_change': '2024-05-14T11:17:54.620076+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.620076+0000', 'last_undegraded': '2024-05-14T11:17:54.620076+0000', 'last_fullsized': '2024-05-14T11:17:54.620076+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:PG 8.11 is not active+clean 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.620498+0000', 'last_change': '2024-05-14T11:17:54.620498+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.620498+0000', 'last_undegraded': '2024-05-14T11:17:54.620498+0000', 'last_fullsized': '2024-05-14T11:17:54.620498+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1d is not active+clean 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.620675+0000', 'last_change': '2024-05-14T11:17:54.620675+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.620675+0000', 'last_undegraded': '2024-05-14T11:17:54.620675+0000', 'last_fullsized': '2024-05-14T11:17:54.620675+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:PG 8.13 is not active+clean 2024-05-14T11:18:01.578 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619089+0000', 'last_change': '2024-05-14T11:17:54.619089+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619089+0000', 'last_undegraded': '2024-05-14T11:17:54.619089+0000', 'last_fullsized': '2024-05-14T11:17:54.619089+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1b is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624273+0000', 'last_change': '2024-05-14T11:17:54.624273+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624273+0000', 'last_undegraded': '2024-05-14T11:17:54.624273+0000', 'last_fullsized': '2024-05-14T11:17:54.624273+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 8.15 is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.626617+0000', 'last_change': '2024-05-14T11:17:54.626617+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.626617+0000', 'last_undegraded': '2024-05-14T11:17:54.626617+0000', 'last_fullsized': '2024-05-14T11:17:54.626617+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:PG 8.16 is not active+clean 2024-05-14T11:18:01.579 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619115+0000', 'last_change': '2024-05-14T11:17:54.619115+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619115+0000', 'last_undegraded': '2024-05-14T11:17:54.619115+0000', 'last_fullsized': '2024-05-14T11:17:54.619115+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 8.17 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 7.18 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623167+0000', 'last_change': '2024-05-14T11:17:54.623167+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623167+0000', 'last_undegraded': '2024-05-14T11:17:54.623167+0000', 'last_fullsized': '2024-05-14T11:17:54.623167+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624482+0000', 'last_change': '2024-05-14T11:17:54.624482+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624482+0000', 'last_undegraded': '2024-05-14T11:17:54.624482+0000', 'last_fullsized': '2024-05-14T11:17:54.624482+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:PG 8.19 is not active+clean 2024-05-14T11:18:01.580 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.626095+0000', 'last_change': '2024-05-14T11:17:54.626095+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.626095+0000', 'last_undegraded': '2024-05-14T11:17:54.626095+0000', 'last_fullsized': '2024-05-14T11:17:54.626095+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.625084+0000', 'last_change': '2024-05-14T11:17:54.625084+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.625084+0000', 'last_undegraded': '2024-05-14T11:17:54.625084+0000', 'last_fullsized': '2024-05-14T11:17:54.625084+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624901+0000', 'last_change': '2024-05-14T11:17:54.624901+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624901+0000', 'last_undegraded': '2024-05-14T11:17:54.624901+0000', 'last_fullsized': '2024-05-14T11:17:54.624901+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:18:01.581 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1c is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623165+0000', 'last_change': '2024-05-14T11:17:54.623165+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623165+0000', 'last_undegraded': '2024-05-14T11:17:54.623165+0000', 'last_fullsized': '2024-05-14T11:17:54.623165+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619452+0000', 'last_change': '2024-05-14T11:17:54.619452+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619452+0000', 'last_undegraded': '2024-05-14T11:17:54.619452+0000', 'last_fullsized': '2024-05-14T11:17:54.619452+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-05-14T11:18:01.582 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624432+0000', 'last_change': '2024-05-14T11:17:54.624432+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624432+0000', 'last_undegraded': '2024-05-14T11:17:54.624432+0000', 'last_fullsized': '2024-05-14T11:17:54.624432+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1f is not active+clean 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624438+0000', 'last_change': '2024-05-14T11:17:54.624438+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624438+0000', 'last_undegraded': '2024-05-14T11:17:54.624438+0000', 'last_fullsized': '2024-05-14T11:17:54.624438+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:PG 8.0 is not active+clean 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623126+0000', 'last_change': '2024-05-14T11:17:54.623126+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623126+0000', 'last_undegraded': '2024-05-14T11:17:54.623126+0000', 'last_fullsized': '2024-05-14T11:17:54.623126+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.583 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.629246+0000', 'last_change': '2024-05-14T11:17:54.629246+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.629246+0000', 'last_undegraded': '2024-05-14T11:17:54.629246+0000', 'last_fullsized': '2024-05-14T11:17:54.629246+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619171+0000', 'last_change': '2024-05-14T11:17:54.619171+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619171+0000', 'last_undegraded': '2024-05-14T11:17:54.619171+0000', 'last_fullsized': '2024-05-14T11:17:54.619171+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-05-14T11:18:01.584 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619109+0000', 'last_change': '2024-05-14T11:17:54.619109+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619109+0000', 'last_undegraded': '2024-05-14T11:17:54.619109+0000', 'last_fullsized': '2024-05-14T11:17:54.619109+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 8.4 is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.620479+0000', 'last_change': '2024-05-14T11:17:54.620479+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.620479+0000', 'last_undegraded': '2024-05-14T11:17:54.620479+0000', 'last_fullsized': '2024-05-14T11:17:54.620479+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 8.5 is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.626224+0000', 'last_change': '2024-05-14T11:17:54.626224+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.626224+0000', 'last_undegraded': '2024-05-14T11:17:54.626224+0000', 'last_fullsized': '2024-05-14T11:17:54.626224+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-05-14T11:18:01.585 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623134+0000', 'last_change': '2024-05-14T11:17:54.623134+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623134+0000', 'last_undegraded': '2024-05-14T11:17:54.623134+0000', 'last_fullsized': '2024-05-14T11:17:54.623134+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.628832+0000', 'last_change': '2024-05-14T11:17:54.628832+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.628832+0000', 'last_undegraded': '2024-05-14T11:17:54.628832+0000', 'last_fullsized': '2024-05-14T11:17:54.628832+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:PG 8.8 is not active+clean 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-05-14T11:18:01.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619639+0000', 'last_change': '2024-05-14T11:17:54.619639+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619639+0000', 'last_undegraded': '2024-05-14T11:17:54.619639+0000', 'last_fullsized': '2024-05-14T11:17:54.619639+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:18:01.587 INFO:tasks.ceph.ceph_manager.ceph:PG 8.a is not active+clean 2024-05-14T11:18:01.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.587 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-05-14T11:18:01.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624446+0000', 'last_change': '2024-05-14T11:17:54.624446+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624446+0000', 'last_undegraded': '2024-05-14T11:17:54.624446+0000', 'last_fullsized': '2024-05-14T11:17:54.624446+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:18:01.587 INFO:tasks.ceph.ceph_manager.ceph:PG 8.b is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:PG 7.4 is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623233+0000', 'last_change': '2024-05-14T11:17:54.623233+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623233+0000', 'last_undegraded': '2024-05-14T11:17:54.623233+0000', 'last_fullsized': '2024-05-14T11:17:54.623233+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.624809+0000', 'last_change': '2024-05-14T11:17:54.624809+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.624809+0000', 'last_undegraded': '2024-05-14T11:17:54.624809+0000', 'last_fullsized': '2024-05-14T11:17:54.624809+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:PG 8.d is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-05-14T11:18:01.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.625684+0000', 'last_change': '2024-05-14T11:17:54.625684+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.625684+0000', 'last_undegraded': '2024-05-14T11:17:54.625684+0000', 'last_fullsized': '2024-05-14T11:17:54.625684+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.626011+0000', 'last_change': '2024-05-14T11:17:54.626011+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.626011+0000', 'last_undegraded': '2024-05-14T11:17:54.626011+0000', 'last_fullsized': '2024-05-14T11:17:54.626011+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.619093+0000', 'last_change': '2024-05-14T11:17:54.619093+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.619093+0000', 'last_undegraded': '2024-05-14T11:17:54.619093+0000', 'last_fullsized': '2024-05-14T11:17:54.619093+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:17:55.615640+0000', 'last_change': '2024-05-14T11:17:55.615640+0000', 'last_active': '2024-05-14T11:17:55.615640+0000', 'last_peered': '2024-05-14T11:17:55.615640+0000', 'last_clean': '2024-05-14T11:17:55.615640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:55.615640+0000', 'last_undegraded': '2024-05-14T11:17:55.615640+0000', 'last_fullsized': '2024-05-14T11:17:55.615640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:55.615640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-05-14T11:18:01.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:17:54.623497+0000', 'last_change': '2024-05-14T11:17:54.623497+0000', 'last_active': '2024-05-14T11:17:54.606578+0000', 'last_peered': '2024-05-14T11:17:54.606578+0000', 'last_clean': '2024-05-14T11:17:54.606578+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:17:54.623497+0000', 'last_undegraded': '2024-05-14T11:17:54.623497+0000', 'last_fullsized': '2024-05-14T11:17:54.623497+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'last_clean_scrub_stamp': '2024-05-14T11:17:54.606578+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:18:01.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-14T11:18:01.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 Processor -- start 2024-05-14T11:18:01.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 -- start start 2024-05-14T11:18:01.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fac48072470 0x7fac480714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48153c90 0x7fac48071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac480ccaa0 con 0x7fac480d1810 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fac480b1f00 con 0x7fac48072470 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.752+0000 7fac4f600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fac4815e070 con 0x7fac48153c90 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fac48072470 0x7fac480714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fac48072470 0x7fac480714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:53376/0 (socket says 172.21.15.116:53376) 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4d600640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fac480d1810 0x7fac480d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54106/0 (socket says 172.21.15.116:54106) 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 -- 172.21.15.116:0/4157749114 learned_addr learned my addr 172.21.15.116:0/4157749114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac47e00640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48153c90 0x7fac48071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 -- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48153c90 msgr2=0x7fac48071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:01.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48153c90 0x7fac48071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 -- 172.21.15.116:0/4157749114 >> v1:172.21.15.149:6789/0 conn(0x7fac480d1810 legacy=0x7fac480d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 -- 172.21.15.116:0/4157749114 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac480d2330 con 0x7fac48072470 2024-05-14T11:18:01.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac4cc00640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48072470 0x7fac480714c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac3000bb10 tx=0x7fac3001ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95ef0248a56916 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac47400640 1 -- 172.21.15.116:0/4157749114 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac300b4070 con 0x7fac48072470 2024-05-14T11:18:01.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac47400640 1 -- 172.21.15.116:0/4157749114 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac300a7b10 con 0x7fac48072470 2024-05-14T11:18:01.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.756+0000 7fac47400640 1 -- 172.21.15.116:0/4157749114 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac300af440 con 0x7fac48072470 2024-05-14T11:18:01.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48072470 msgr2=0x7fac480714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48072470 0x7fac480714c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac3000bb10 tx=0x7fac3001ec30 comp rx=0 tx=0).stop 2024-05-14T11:18:01.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- 172.21.15.116:0/4157749114 shutdown_connections 2024-05-14T11:18:01.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48153c90 0x7fac48071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 --2- 172.21.15.116:0/4157749114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48072470 0x7fac480714c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- 172.21.15.116:0/4157749114 >> 172.21.15.116:0/4157749114 conn(0x7fac48073ca0 msgr2=0x7fac480740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:01.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- 172.21.15.116:0/4157749114 shutdown_connections 2024-05-14T11:18:01.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- 172.21.15.116:0/4157749114 wait complete. 2024-05-14T11:18:01.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 Processor -- start 2024-05-14T11:18:01.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.760+0000 7fac4f600640 1 -- start start 2024-05-14T11:18:01.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 0x7fac48140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 0x7fac480cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fac480c8ec0 con 0x7fac480d1810 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac4815db50 con 0x7fac48153c90 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fac480c9040 con 0x7fac48072470 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58310/0 (socket says 172.21.15.116:58310) 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 -- 172.21.15.116:0/3962625644 learned_addr learned my addr 172.21.15.116:0/3962625644 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4d600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 0x7fac480cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4cc00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 0x7fac48140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 -- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 msgr2=0x7fac48140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 0x7fac48140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 -- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 msgr2=0x7fac480cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 0x7fac480cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 -- 172.21.15.116:0/3962625644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac48140550 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4d600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 0x7fac480cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4cc00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 0x7fac48140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac47e00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac3800e8a0 tx=0x7fac3800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38017070 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3800c940 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac480cc3a0 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800ccf0 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4814afb0 con 0x7fac480d1810 2024-05-14T11:18:01.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.764+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4806a9c0 con 0x7fac480d1810 2024-05-14T11:18:01.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.768+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac3801a070 con 0x7fac480d1810 2024-05-14T11:18:01.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.768+0000 7fac44c00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 0x7fac24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:01.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.768+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fac3808f600 con 0x7fac480d1810 2024-05-14T11:18:01.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.768+0000 7fac4cc00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 0x7fac24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:01.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.768+0000 7fac4cc00640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 0x7fac24071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac300b1000 tx=0x7fac300a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:01.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.772+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fac3805c3e0 con 0x7fac480d1810 2024-05-14T11:18:01.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.984+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fac480cc130 con 0x7fac480d1810 2024-05-14T11:18:01.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac44c00640 1 -- 172.21.15.116:0/3962625644 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1657 (secure 0 0 0) 0x7fac3805bce0 con 0x7fac480d1810 2024-05-14T11:18:01.989 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:01.989 INFO:teuthology.orchestra.run.smithi116.stdout:{"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1715685359,"num_in_osds":8,"osd_in_since":1715685350,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":235413504,"bytes_avail":772858699776,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5319}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-14T11:17:40.380187+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-14T11:18:01.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 msgr2=0x7fac24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 0x7fac24071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac300b1000 tx=0x7fac300a9070 comp rx=0 tx=0).stop 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 msgr2=0x7fac4814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fac3800e8a0 tx=0x7fac3800eca0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 shutdown_connections 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fac2406f400 0x7fac24071910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac48153c90 0x7fac480cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480d1810 0x7fac4814b7b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 --2- 172.21.15.116:0/3962625644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fac48072470 0x7fac48140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:01.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 >> 172.21.15.116:0/3962625644 conn(0x7fac48073ca0 msgr2=0x7fac480ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:01.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 shutdown_connections 2024-05-14T11:18:01.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:01.988+0000 7fac4f600640 1 -- 172.21.15.116:0/3962625644 wait complete. 2024-05-14T11:18:02.010 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-14T11:18:02.010 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:18:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.152+0000 7f5c99200640 1 Processor -- start 2024-05-14T11:18:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.152+0000 7f5c99200640 1 -- start start 2024-05-14T11:18:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c940aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 0x7f5c940d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c94161600 0x7f5c94161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5c940ae100 con 0x7f5c94161600 2024-05-14T11:18:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5c940ae870 con 0x7f5c940d1670 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c99200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9406ff20 con 0x7f5c9415db30 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c94161600 0x7f5c94161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c94161600 0x7f5c94161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58316/0 (socket says 172.21.15.116:58316) 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 -- 172.21.15.116:0/64565898 learned_addr learned my addr 172.21.15.116:0/64565898 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c91e00640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c940aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c89400640 1 --2- 172.21.15.116:0/64565898 >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 0x7f5c940d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 -- 172.21.15.116:0/64565898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 msgr2=0x7f5c940aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c940aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 -- 172.21.15.116:0/64565898 >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 msgr2=0x7f5c940d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 --2- 172.21.15.116:0/64565898 >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 0x7f5c940d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c92800640 1 -- 172.21.15.116:0/64565898 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c940d1fc0 con 0x7f5c94161600 2024-05-14T11:18:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.156+0000 7f5c89400640 1 --2- 172.21.15.116:0/64565898 >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 0x7f5c940d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c92800640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94161a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5c940d2280 tx=0x7f5c8c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5c1f50d6a7df28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c91400640 1 -- 172.21.15.116:0/64565898 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0a56d0 con 0x7f5c94161600 2024-05-14T11:18:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c91400640 1 -- 172.21.15.116:0/64565898 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0a5870 con 0x7f5c94161600 2024-05-14T11:18:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c91400640 1 -- 172.21.15.116:0/64565898 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0a5c40 con 0x7f5c94161600 2024-05-14T11:18:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 -- 172.21.15.116:0/64565898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 msgr2=0x7f5c94161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94161a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5c940d2280 tx=0x7f5c8c01c960 comp rx=0 tx=0).stop 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 -- 172.21.15.116:0/64565898 shutdown_connections 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94161a10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 --2- 172.21.15.116:0/64565898 >> v2:172.21.15.149:3300/0 conn(0x7f5c940d1670 0x7f5c940d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 --2- 172.21.15.116:0/64565898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c940aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 -- 172.21.15.116:0/64565898 >> 172.21.15.116:0/64565898 conn(0x7f5c940b4620 msgr2=0x7f5c940b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 -- 172.21.15.116:0/64565898 shutdown_connections 2024-05-14T11:18:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 -- 172.21.15.116:0/64565898 wait complete. 2024-05-14T11:18:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.160+0000 7f5c99200640 1 Processor -- start 2024-05-14T11:18:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- start start 2024-05-14T11:18:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c940d1670 0x7f5c94099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c941499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94162360 con 0x7f5c94161600 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c940aefe0 con 0x7f5c940d1670 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9415df40 con 0x7f5c9415db30 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58324/0 (socket says 172.21.15.116:58324) 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 -- 172.21.15.116:0/2873832341 learned_addr learned my addr 172.21.15.116:0/2873832341 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c91e00640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c940d1670 0x7f5c94099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c89400640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c941499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 -- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 msgr2=0x7f5c941499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c941499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 -- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c940d1670 msgr2=0x7f5c94099bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c940d1670 0x7f5c94099bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 -- 172.21.15.116:0/2873832341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c94160490 con 0x7f5c94161600 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c92800640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0af040 tx=0x7f5c8c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c009510 con 0x7f5c94161600 2024-05-14T11:18:02.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0be070 con 0x7f5c94161600 2024-05-14T11:18:02.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9409a100 con 0x7f5c94161600 2024-05-14T11:18:02.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0b88b0 con 0x7f5c94161600 2024-05-14T11:18:02.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.164+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941609e0 con 0x7f5c94161600 2024-05-14T11:18:02.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9406a9c0 con 0x7f5c94161600 2024-05-14T11:18:02.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c8c0b8a50 con 0x7f5c94161600 2024-05-14T11:18:02.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c8aa00640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 0x7f5c70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f5c8c12a530 con 0x7f5c94161600 2024-05-14T11:18:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c91e00640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 0x7f5c70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.168+0000 7f5c91e00640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 0x7f5c70071910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007940 tx=0x7f5c84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.172+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5c940d1fc0 con 0x7f5c94161600 2024-05-14T11:18:02.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.324+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c940aecd0 con 0x7f5c7006f400 2024-05-14T11:18:02.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.332+0000 7f5c8aa00640 1 -- 172.21.15.116:0/2873832341 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240065 (secure 0 0 0) 0x7f5c940aecd0 con 0x7f5c7006f400 2024-05-14T11:18:02.335 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:02.337 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 msgr2=0x7f5c70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 0x7f5c70071910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007940 tx=0x7f5c84008040 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 msgr2=0x7f5c94157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0af040 tx=0x7f5c8c019700 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 shutdown_connections 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c7006f400 0x7f5c70071910 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c94161600 0x7f5c94157c40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c9415db30 0x7f5c941499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 --2- 172.21.15.116:0/2873832341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c940d1670 0x7f5c94099bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 >> 172.21.15.116:0/2873832341 conn(0x7f5c940b4620 msgr2=0x7f5c940b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 shutdown_connections 2024-05-14T11:18:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:02.336+0000 7f5c99200640 1 -- 172.21.15.116:0/2873832341 wait complete. 2024-05-14T11:18:02.357 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-05-14T11:18:00.384388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229896,"kb_used_data":5368,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744824,"statfs":{"total":773094113280,"available":772858699776,"internally_reserved":0,"allocated":5496832,"data_stored":3052268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767949"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620076+0000","last_change":"2024-05-14T11:17:54.620076+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620076+0000","last_undegraded":"2024-05-14T11:17:54.620076+0000","last_fullsized":"2024-05-14T11:17:54.620076+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620498+0000","last_change":"2024-05-14T11:17:54.620498+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620498+0000","last_undegraded":"2024-05-14T11:17:54.620498+0000","last_fullsized":"2024-05-14T11:17:54.620498+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620675+0000","last_change":"2024-05-14T11:17:54.620675+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620675+0000","last_undegraded":"2024-05-14T11:17:54.620675+0000","last_fullsized":"2024-05-14T11:17:54.620675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619089+0000","last_change":"2024-05-14T11:17:54.619089+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619089+0000","last_undegraded":"2024-05-14T11:17:54.619089+0000","last_fullsized":"2024-05-14T11:17:54.619089+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624273+0000","last_change":"2024-05-14T11:17:54.624273+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624273+0000","last_undegraded":"2024-05-14T11:17:54.624273+0000","last_fullsized":"2024-05-14T11:17:54.624273+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626617+0000","last_change":"2024-05-14T11:17:54.626617+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626617+0000","last_undegraded":"2024-05-14T11:17:54.626617+0000","last_fullsized":"2024-05-14T11:17:54.626617+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619115+0000","last_change":"2024-05-14T11:17:54.619115+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619115+0000","last_undegraded":"2024-05-14T11:17:54.619115+0000","last_fullsized":"2024-05-14T11:17:54.619115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623167+0000","last_change":"2024-05-14T11:17:54.623167+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623167+0000","last_undegraded":"2024-05-14T11:17:54.623167+0000","last_fullsized":"2024-05-14T11:17:54.623167+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624482+0000","last_change":"2024-05-14T11:17:54.624482+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624482+0000","last_undegraded":"2024-05-14T11:17:54.624482+0000","last_fullsized":"2024-05-14T11:17:54.624482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626095+0000","last_change":"2024-05-14T11:17:54.626095+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626095+0000","last_undegraded":"2024-05-14T11:17:54.626095+0000","last_fullsized":"2024-05-14T11:17:54.626095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625084+0000","last_change":"2024-05-14T11:17:54.625084+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625084+0000","last_undegraded":"2024-05-14T11:17:54.625084+0000","last_fullsized":"2024-05-14T11:17:54.625084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624901+0000","last_change":"2024-05-14T11:17:54.624901+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624901+0000","last_undegraded":"2024-05-14T11:17:54.624901+0000","last_fullsized":"2024-05-14T11:17:54.624901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623165+0000","last_change":"2024-05-14T11:17:54.623165+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623165+0000","last_undegraded":"2024-05-14T11:17:54.623165+0000","last_fullsized":"2024-05-14T11:17:54.623165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619452+0000","last_change":"2024-05-14T11:17:54.619452+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619452+0000","last_undegraded":"2024-05-14T11:17:54.619452+0000","last_fullsized":"2024-05-14T11:17:54.619452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624432+0000","last_change":"2024-05-14T11:17:54.624432+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624432+0000","last_undegraded":"2024-05-14T11:17:54.624432+0000","last_fullsized":"2024-05-14T11:17:54.624432+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624438+0000","last_change":"2024-05-14T11:17:54.624438+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624438+0000","last_undegraded":"2024-05-14T11:17:54.624438+0000","last_fullsized":"2024-05-14T11:17:54.624438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623126+0000","last_change":"2024-05-14T11:17:54.623126+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623126+0000","last_undegraded":"2024-05-14T11:17:54.623126+0000","last_fullsized":"2024-05-14T11:17:54.623126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.629246+0000","last_change":"2024-05-14T11:17:54.629246+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.629246+0000","last_undegraded":"2024-05-14T11:17:54.629246+0000","last_fullsized":"2024-05-14T11:17:54.629246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619171+0000","last_change":"2024-05-14T11:17:54.619171+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619171+0000","last_undegraded":"2024-05-14T11:17:54.619171+0000","last_fullsized":"2024-05-14T11:17:54.619171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619109+0000","last_change":"2024-05-14T11:17:54.619109+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619109+0000","last_undegraded":"2024-05-14T11:17:54.619109+0000","last_fullsized":"2024-05-14T11:17:54.619109+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.620479+0000","last_change":"2024-05-14T11:17:54.620479+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.620479+0000","last_undegraded":"2024-05-14T11:17:54.620479+0000","last_fullsized":"2024-05-14T11:17:54.620479+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626224+0000","last_change":"2024-05-14T11:17:54.626224+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626224+0000","last_undegraded":"2024-05-14T11:17:54.626224+0000","last_fullsized":"2024-05-14T11:17:54.626224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623134+0000","last_change":"2024-05-14T11:17:54.623134+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623134+0000","last_undegraded":"2024-05-14T11:17:54.623134+0000","last_fullsized":"2024-05-14T11:17:54.623134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.628832+0000","last_change":"2024-05-14T11:17:54.628832+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.628832+0000","last_undegraded":"2024-05-14T11:17:54.628832+0000","last_fullsized":"2024-05-14T11:17:54.628832+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619639+0000","last_change":"2024-05-14T11:17:54.619639+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619639+0000","last_undegraded":"2024-05-14T11:17:54.619639+0000","last_fullsized":"2024-05-14T11:17:54.619639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624446+0000","last_change":"2024-05-14T11:17:54.624446+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624446+0000","last_undegraded":"2024-05-14T11:17:54.624446+0000","last_fullsized":"2024-05-14T11:17:54.624446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623233+0000","last_change":"2024-05-14T11:17:54.623233+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623233+0000","last_undegraded":"2024-05-14T11:17:54.623233+0000","last_fullsized":"2024-05-14T11:17:54.623233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.624809+0000","last_change":"2024-05-14T11:17:54.624809+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.624809+0000","last_undegraded":"2024-05-14T11:17:54.624809+0000","last_fullsized":"2024-05-14T11:17:54.624809+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.625684+0000","last_change":"2024-05-14T11:17:54.625684+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.625684+0000","last_undegraded":"2024-05-14T11:17:54.625684+0000","last_fullsized":"2024-05-14T11:17:54.625684+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.626011+0000","last_change":"2024-05-14T11:17:54.626011+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.626011+0000","last_undegraded":"2024-05-14T11:17:54.626011+0000","last_fullsized":"2024-05-14T11:17:54.626011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.619093+0000","last_change":"2024-05-14T11:17:54.619093+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.619093+0000","last_undegraded":"2024-05-14T11:17:54.619093+0000","last_fullsized":"2024-05-14T11:17:54.619093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:17:55.615640+0000","last_change":"2024-05-14T11:17:55.615640+0000","last_active":"2024-05-14T11:17:55.615640+0000","last_peered":"2024-05-14T11:17:55.615640+0000","last_clean":"2024-05-14T11:17:55.615640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:55.615640+0000","last_undegraded":"2024-05-14T11:17:55.615640+0000","last_fullsized":"2024-05-14T11:17:55.615640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-14T11:17:54.623497+0000","last_change":"2024-05-14T11:17:54.623497+0000","last_active":"2024-05-14T11:17:54.606578+0000","last_peered":"2024-05-14T11:17:54.606578+0000","last_clean":"2024-05-14T11:17:54.606578+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:17:54.623497+0000","last_undegraded":"2024-05-14T11:17:54.623497+0000","last_fullsized":"2024-05-14T11:17:54.623497+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-14T11:17:54.625629+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:54.625629+0000","last_peered":"2024-05-14T11:17:54.625629+0000","last_clean":"2024-05-14T11:17:54.625629+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:54.625629+0000","last_undegraded":"2024-05-14T11:17:54.625629+0000","last_fullsized":"2024-05-14T11:17:54.625629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":827414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":230837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":237046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":821205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:18:05.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:18:05.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 Processor -- start 2024-05-14T11:18:05.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 -- start start 2024-05-14T11:18:05.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4cb406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb406cc40 con 0x7f4cb4072a30 2024-05-14T11:18:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb406cf10 con 0x7f4cb406f730 2024-05-14T11:18:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cbc400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb406db20 con 0x7f4cb40b0770 2024-05-14T11:18:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cba400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4cb4072a30 0x7f4cb4072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37456/0 (socket says 172.21.15.116:37456) 2024-05-14T11:18:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cba400640 1 -- 172.21.15.116:0/421540885 learned_addr learned my addr 172.21.15.116:0/421540885 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021044574 0 0) 0x7f4cb406cc40 con 0x7f4cb4072a30 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c9c003530 con 0x7f4cb4072a30 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.520+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4cb406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1043810423 0 0) 0x7f4c9c003530 con 0x7f4cb4072a30 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 msgr2=0x7f4cb406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4cb406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 >> v1:172.21.15.149:6789/0 conn(0x7f4cb406f730 legacy=0x7f4cb406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb406d1f0 con 0x7f4cb4072a30 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f4cac001a80 con 0x7f4cb4072a30 2024-05-14T11:18:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 >> v1:172.21.15.116:6789/0 conn(0x7f4cb4072a30 legacy=0x7f4cb4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4c9c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4c9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c9c004e70 0x7f4c9c0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb406db20 con 0x7f4c9c003a40 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c00df00 con 0x7f4c9c004e70 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c00bcc0 con 0x7f4cb40b0770 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4c9c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cba400640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c9c004e70 0x7f4c9c0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9000640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4c9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 -- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 msgr2=0x7f4c9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb40b0770 0x7f4c9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 -- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c9c004e70 msgr2=0x7f4c9c0052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c9c004e70 0x7f4c9c0052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 -- 172.21.15.116:0/421540885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c003530 con 0x7f4c9c003a40 2024-05-14T11:18:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.524+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4c9c003eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800ba10 tx=0x7f4ca801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d267c1d3ea46f96 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800b610 con 0x7f4c9c003a40 2024-05-14T11:18:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca80b1070 con 0x7f4c9c003a40 2024-05-14T11:18:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cb3e00640 1 -- 172.21.15.116:0/421540885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80ac850 con 0x7f4c9c003a40 2024-05-14T11:18:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- 172.21.15.116:0/421540885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 msgr2=0x7f4c9c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 --2- 172.21.15.116:0/421540885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4c9c003eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800ba10 tx=0x7f4ca801c670 comp rx=0 tx=0).stop 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cba400640 1 -- 172.21.15.116:0/421540885 reap_dead start 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- 172.21.15.116:0/421540885 shutdown_connections 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- 172.21.15.116:0/421540885 >> 172.21.15.116:0/421540885 conn(0x7f4cb40b4620 msgr2=0x7f4cb40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- 172.21.15.116:0/421540885 shutdown_connections 2024-05-14T11:18:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- 172.21.15.116:0/421540885 wait complete. 2024-05-14T11:18:05.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 Processor -- start 2024-05-14T11:18:05.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.528+0000 7f4cbc400640 1 -- start start 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb41496a0 0x7f4cb4148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb4158b90 0x7f4cb4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4073ae0 con 0x7f4c9c003a40 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4073960 con 0x7f4cb41496a0 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb4167fa0 con 0x7f4cb4158b90 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58334/0 (socket says 172.21.15.116:58334) 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 -- 172.21.15.116:0/3592205909 learned_addr learned my addr 172.21.15.116:0/3592205909 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9000640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb41496a0 0x7f4cb4148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cba400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb4158b90 0x7f4cb4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 -- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb4158b90 msgr2=0x7f4cb4158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb4158b90 0x7f4cb4158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 -- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb41496a0 msgr2=0x7f4cb4148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb41496a0 0x7f4cb4148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 -- 172.21.15.116:0/3592205909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb41594e0 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb9a00640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4ca801c4a0 tx=0x7f4c9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80bb070 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca80ac890 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80acc80 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41500f0 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb4150a70 con 0x7f4c9c003a40 2024-05-14T11:18:05.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.532+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb406a9c0 con 0x7f4c9c003a40 2024-05-14T11:18:05.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.536+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ca80c6060 con 0x7f4c9c003a40 2024-05-14T11:18:05.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.536+0000 7f4cb1600640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 0x7f4c94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.536+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f4ca8129610 con 0x7f4c9c003a40 2024-05-14T11:18:05.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.536+0000 7f4cb9000640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 0x7f4c94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.536+0000 7f4cb9000640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 0x7f4c94071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4073020 tx=0x7f4ca0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.540+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4cb41594e0 con 0x7f4c9c003a40 2024-05-14T11:18:05.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.696+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cb406f510 con 0x7f4c9406f430 2024-05-14T11:18:05.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.700+0000 7f4cb1600640 1 -- 172.21.15.116:0/3592205909 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f4cb406f510 con 0x7f4c9406f430 2024-05-14T11:18:05.706 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 msgr2=0x7f4c94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 0x7f4c94071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4073020 tx=0x7f4ca0009040 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 msgr2=0x7f4cb414f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4ca801c4a0 tx=0x7f4c9c005840 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 shutdown_connections 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c9406f430 0x7f4c94071940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb41496a0 0x7f4cb4148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4cb4158b90 0x7f4cb4158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 --2- 172.21.15.116:0/3592205909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9c003a40 0x7f4cb414f780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 >> 172.21.15.116:0/3592205909 conn(0x7f4cb40b4620 msgr2=0x7f4cb40b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 shutdown_connections 2024-05-14T11:18:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.704+0000 7f4cbc400640 1 -- 172.21.15.116:0/3592205909 wait complete. 2024-05-14T11:18:05.727 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-05-14T11:18:04.385343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230272,"kb_used_data":5768,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744448,"statfs":{"total":773094113280,"available":772858314752,"internally_reserved":0,"allocated":5906432,"data_stored":3246132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001803"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830876+0000","last_change":"2024-05-14T11:17:56.654994+0000","last_active":"2024-05-14T11:17:58.830876+0000","last_peered":"2024-05-14T11:17:58.830876+0000","last_clean":"2024-05-14T11:17:58.830876+0000","last_became_active":"2024-05-14T11:17:56.654656+0000","last_became_peered":"2024-05-14T11:17:56.654656+0000","last_unstale":"2024-05-14T11:17:58.830876+0000","last_undegraded":"2024-05-14T11:17:58.830876+0000","last_fullsized":"2024-05-14T11:17:58.830876+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:38.104044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655200+0000","last_change":"2024-05-14T11:17:56.659564+0000","last_active":"2024-05-14T11:17:57.655200+0000","last_peered":"2024-05-14T11:17:57.655200+0000","last_clean":"2024-05-14T11:17:57.655200+0000","last_became_active":"2024-05-14T11:17:56.659241+0000","last_became_peered":"2024-05-14T11:17:56.659241+0000","last_unstale":"2024-05-14T11:17:57.655200+0000","last_undegraded":"2024-05-14T11:17:57.655200+0000","last_fullsized":"2024-05-14T11:17:57.655200+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:30:16.516688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644352+0000","last_change":"2024-05-14T11:17:56.671768+0000","last_active":"2024-05-14T11:17:57.644352+0000","last_peered":"2024-05-14T11:17:57.644352+0000","last_clean":"2024-05-14T11:17:57.644352+0000","last_became_active":"2024-05-14T11:17:56.667992+0000","last_became_peered":"2024-05-14T11:17:56.667992+0000","last_unstale":"2024-05-14T11:17:57.644352+0000","last_undegraded":"2024-05-14T11:17:57.644352+0000","last_fullsized":"2024-05-14T11:17:57.644352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:56.556404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655112+0000","last_change":"2024-05-14T11:17:56.661768+0000","last_active":"2024-05-14T11:17:57.655112+0000","last_peered":"2024-05-14T11:17:57.655112+0000","last_clean":"2024-05-14T11:17:57.655112+0000","last_became_active":"2024-05-14T11:17:56.661404+0000","last_became_peered":"2024-05-14T11:17:56.661404+0000","last_unstale":"2024-05-14T11:17:57.655112+0000","last_undegraded":"2024-05-14T11:17:57.655112+0000","last_fullsized":"2024-05-14T11:17:57.655112+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:20:17.473724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.646322+0000","last_change":"2024-05-14T11:17:56.643038+0000","last_active":"2024-05-14T11:17:57.646322+0000","last_peered":"2024-05-14T11:17:57.646322+0000","last_clean":"2024-05-14T11:17:57.646322+0000","last_became_active":"2024-05-14T11:17:56.642688+0000","last_became_peered":"2024-05-14T11:17:56.642688+0000","last_unstale":"2024-05-14T11:17:57.646322+0000","last_undegraded":"2024-05-14T11:17:57.646322+0000","last_fullsized":"2024-05-14T11:17:57.646322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:06.019238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830662+0000","last_change":"2024-05-14T11:17:56.644640+0000","last_active":"2024-05-14T11:17:58.830662+0000","last_peered":"2024-05-14T11:17:58.830662+0000","last_clean":"2024-05-14T11:17:58.830662+0000","last_became_active":"2024-05-14T11:17:56.644344+0000","last_became_peered":"2024-05-14T11:17:56.644344+0000","last_unstale":"2024-05-14T11:17:58.830662+0000","last_undegraded":"2024-05-14T11:17:58.830662+0000","last_fullsized":"2024-05-14T11:17:58.830662+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:20:24.432368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643447+0000","last_change":"2024-05-14T11:17:56.648966+0000","last_active":"2024-05-14T11:17:57.643447+0000","last_peered":"2024-05-14T11:17:57.643447+0000","last_clean":"2024-05-14T11:17:57.643447+0000","last_became_active":"2024-05-14T11:17:56.648606+0000","last_became_peered":"2024-05-14T11:17:56.648606+0000","last_unstale":"2024-05-14T11:17:57.643447+0000","last_undegraded":"2024-05-14T11:17:57.643447+0000","last_fullsized":"2024-05-14T11:17:57.643447+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:09:34.282947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830203+0000","last_change":"2024-05-14T11:17:56.660974+0000","last_active":"2024-05-14T11:17:58.830203+0000","last_peered":"2024-05-14T11:17:58.830203+0000","last_clean":"2024-05-14T11:17:58.830203+0000","last_became_active":"2024-05-14T11:17:56.660669+0000","last_became_peered":"2024-05-14T11:17:56.660669+0000","last_unstale":"2024-05-14T11:17:58.830203+0000","last_undegraded":"2024-05-14T11:17:58.830203+0000","last_fullsized":"2024-05-14T11:17:58.830203+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:33:59.678226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830135+0000","last_change":"2024-05-14T11:17:56.656279+0000","last_active":"2024-05-14T11:17:58.830135+0000","last_peered":"2024-05-14T11:17:58.830135+0000","last_clean":"2024-05-14T11:17:58.830135+0000","last_became_active":"2024-05-14T11:17:56.655829+0000","last_became_peered":"2024-05-14T11:17:56.655829+0000","last_unstale":"2024-05-14T11:17:58.830135+0000","last_undegraded":"2024-05-14T11:17:58.830135+0000","last_fullsized":"2024-05-14T11:17:58.830135+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:01.424594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830767+0000","last_change":"2024-05-14T11:17:56.638172+0000","last_active":"2024-05-14T11:17:58.830767+0000","last_peered":"2024-05-14T11:17:58.830767+0000","last_clean":"2024-05-14T11:17:58.830767+0000","last_became_active":"2024-05-14T11:17:56.637757+0000","last_became_peered":"2024-05-14T11:17:56.637757+0000","last_unstale":"2024-05-14T11:17:58.830767+0000","last_undegraded":"2024-05-14T11:17:58.830767+0000","last_fullsized":"2024-05-14T11:17:58.830767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:17.712019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829900+0000","last_change":"2024-05-14T11:17:56.642769+0000","last_active":"2024-05-14T11:17:58.829900+0000","last_peered":"2024-05-14T11:17:58.829900+0000","last_clean":"2024-05-14T11:17:58.829900+0000","last_became_active":"2024-05-14T11:17:56.641836+0000","last_became_peered":"2024-05-14T11:17:56.641836+0000","last_unstale":"2024-05-14T11:17:58.829900+0000","last_undegraded":"2024-05-14T11:17:58.829900+0000","last_fullsized":"2024-05-14T11:17:58.829900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:52.333411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829336+0000","last_change":"2024-05-14T11:17:56.650444+0000","last_active":"2024-05-14T11:17:58.829336+0000","last_peered":"2024-05-14T11:17:58.829336+0000","last_clean":"2024-05-14T11:17:58.829336+0000","last_became_active":"2024-05-14T11:17:56.649036+0000","last_became_peered":"2024-05-14T11:17:56.649036+0000","last_unstale":"2024-05-14T11:17:58.829336+0000","last_undegraded":"2024-05-14T11:17:58.829336+0000","last_fullsized":"2024-05-14T11:17:58.829336+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:39:04.660221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830025+0000","last_change":"2024-05-14T11:17:56.636012+0000","last_active":"2024-05-14T11:17:58.830025+0000","last_peered":"2024-05-14T11:17:58.830025+0000","last_clean":"2024-05-14T11:17:58.830025+0000","last_became_active":"2024-05-14T11:17:56.635656+0000","last_became_peered":"2024-05-14T11:17:56.635656+0000","last_unstale":"2024-05-14T11:17:58.830025+0000","last_undegraded":"2024-05-14T11:17:58.830025+0000","last_fullsized":"2024-05-14T11:17:58.830025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:58.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643608+0000","last_change":"2024-05-14T11:17:56.648932+0000","last_active":"2024-05-14T11:17:57.643608+0000","last_peered":"2024-05-14T11:17:57.643608+0000","last_clean":"2024-05-14T11:17:57.643608+0000","last_became_active":"2024-05-14T11:17:56.648462+0000","last_became_peered":"2024-05-14T11:17:56.648462+0000","last_unstale":"2024-05-14T11:17:57.643608+0000","last_undegraded":"2024-05-14T11:17:57.643608+0000","last_fullsized":"2024-05-14T11:17:57.643608+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:39:04.660221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.654896+0000","last_change":"2024-05-14T11:17:56.651469+0000","last_active":"2024-05-14T11:17:57.654896+0000","last_peered":"2024-05-14T11:17:57.654896+0000","last_clean":"2024-05-14T11:17:57.654896+0000","last_became_active":"2024-05-14T11:17:56.651250+0000","last_became_peered":"2024-05-14T11:17:56.651250+0000","last_unstale":"2024-05-14T11:17:57.654896+0000","last_undegraded":"2024-05-14T11:17:57.654896+0000","last_fullsized":"2024-05-14T11:17:57.654896+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:35.799237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829550+0000","last_change":"2024-05-14T11:17:56.635018+0000","last_active":"2024-05-14T11:17:58.829550+0000","last_peered":"2024-05-14T11:17:58.829550+0000","last_clean":"2024-05-14T11:17:58.829550+0000","last_became_active":"2024-05-14T11:17:56.634752+0000","last_became_peered":"2024-05-14T11:17:56.634752+0000","last_unstale":"2024-05-14T11:17:58.829550+0000","last_undegraded":"2024-05-14T11:17:58.829550+0000","last_fullsized":"2024-05-14T11:17:58.829550+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:48.362314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644329+0000","last_change":"2024-05-14T11:17:56.649528+0000","last_active":"2024-05-14T11:17:57.644329+0000","last_peered":"2024-05-14T11:17:57.644329+0000","last_clean":"2024-05-14T11:17:57.644329+0000","last_became_active":"2024-05-14T11:17:56.649129+0000","last_became_peered":"2024-05-14T11:17:56.649129+0000","last_unstale":"2024-05-14T11:17:57.644329+0000","last_undegraded":"2024-05-14T11:17:57.644329+0000","last_fullsized":"2024-05-14T11:17:57.644329+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:31.149313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655237+0000","last_change":"2024-05-14T11:17:56.659846+0000","last_active":"2024-05-14T11:17:57.655237+0000","last_peered":"2024-05-14T11:17:57.655237+0000","last_clean":"2024-05-14T11:17:57.655237+0000","last_became_active":"2024-05-14T11:17:56.659522+0000","last_became_peered":"2024-05-14T11:17:56.659522+0000","last_unstale":"2024-05-14T11:17:57.655237+0000","last_undegraded":"2024-05-14T11:17:57.655237+0000","last_fullsized":"2024-05-14T11:17:57.655237+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:30:10.999992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644187+0000","last_change":"2024-05-14T11:17:56.656055+0000","last_active":"2024-05-14T11:17:57.644187+0000","last_peered":"2024-05-14T11:17:57.644187+0000","last_clean":"2024-05-14T11:17:57.644187+0000","last_became_active":"2024-05-14T11:17:56.655757+0000","last_became_peered":"2024-05-14T11:17:56.655757+0000","last_unstale":"2024-05-14T11:17:57.644187+0000","last_undegraded":"2024-05-14T11:17:57.644187+0000","last_fullsized":"2024-05-14T11:17:57.644187+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:01.424594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645089+0000","last_change":"2024-05-14T11:17:56.641313+0000","last_active":"2024-05-14T11:17:57.645089+0000","last_peered":"2024-05-14T11:17:57.645089+0000","last_clean":"2024-05-14T11:17:57.645089+0000","last_became_active":"2024-05-14T11:17:56.640977+0000","last_became_peered":"2024-05-14T11:17:56.640977+0000","last_unstale":"2024-05-14T11:17:57.645089+0000","last_undegraded":"2024-05-14T11:17:57.645089+0000","last_fullsized":"2024-05-14T11:17:57.645089+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:48.362314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645081+0000","last_change":"2024-05-14T11:17:56.644251+0000","last_active":"2024-05-14T11:17:57.645081+0000","last_peered":"2024-05-14T11:17:57.645081+0000","last_clean":"2024-05-14T11:17:57.645081+0000","last_became_active":"2024-05-14T11:17:56.641431+0000","last_became_peered":"2024-05-14T11:17:56.641431+0000","last_unstale":"2024-05-14T11:17:57.645081+0000","last_undegraded":"2024-05-14T11:17:57.645081+0000","last_fullsized":"2024-05-14T11:17:57.645081+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:52.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829684+0000","last_change":"2024-05-14T11:17:56.647048+0000","last_active":"2024-05-14T11:17:58.829684+0000","last_peered":"2024-05-14T11:17:58.829684+0000","last_clean":"2024-05-14T11:17:58.829684+0000","last_became_active":"2024-05-14T11:17:56.646638+0000","last_became_peered":"2024-05-14T11:17:56.646638+0000","last_unstale":"2024-05-14T11:17:58.829684+0000","last_undegraded":"2024-05-14T11:17:58.829684+0000","last_fullsized":"2024-05-14T11:17:58.829684+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:57:55.213354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829303+0000","last_change":"2024-05-14T11:17:56.645638+0000","last_active":"2024-05-14T11:17:58.829303+0000","last_peered":"2024-05-14T11:17:58.829303+0000","last_clean":"2024-05-14T11:17:58.829303+0000","last_became_active":"2024-05-14T11:17:56.643271+0000","last_became_peered":"2024-05-14T11:17:56.643271+0000","last_unstale":"2024-05-14T11:17:58.829303+0000","last_undegraded":"2024-05-14T11:17:58.829303+0000","last_fullsized":"2024-05-14T11:17:58.829303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:13:36.131392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643393+0000","last_change":"2024-05-14T11:17:56.655044+0000","last_active":"2024-05-14T11:17:57.643393+0000","last_peered":"2024-05-14T11:17:57.643393+0000","last_clean":"2024-05-14T11:17:57.643393+0000","last_became_active":"2024-05-14T11:17:56.654700+0000","last_became_peered":"2024-05-14T11:17:56.654700+0000","last_unstale":"2024-05-14T11:17:57.643393+0000","last_undegraded":"2024-05-14T11:17:57.643393+0000","last_fullsized":"2024-05-14T11:17:57.643393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:03.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644510+0000","last_change":"2024-05-14T11:17:56.678873+0000","last_active":"2024-05-14T11:17:57.644510+0000","last_peered":"2024-05-14T11:17:57.644510+0000","last_clean":"2024-05-14T11:17:57.644510+0000","last_became_active":"2024-05-14T11:17:56.673470+0000","last_became_peered":"2024-05-14T11:17:56.673470+0000","last_unstale":"2024-05-14T11:17:57.644510+0000","last_undegraded":"2024-05-14T11:17:57.644510+0000","last_fullsized":"2024-05-14T11:17:57.644510+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:46:59.637848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829194+0000","last_change":"2024-05-14T11:17:56.666702+0000","last_active":"2024-05-14T11:17:58.829194+0000","last_peered":"2024-05-14T11:17:58.829194+0000","last_clean":"2024-05-14T11:17:58.829194+0000","last_became_active":"2024-05-14T11:17:56.666451+0000","last_became_peered":"2024-05-14T11:17:56.666451+0000","last_unstale":"2024-05-14T11:17:58.829194+0000","last_undegraded":"2024-05-14T11:17:58.829194+0000","last_fullsized":"2024-05-14T11:17:58.829194+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:09:34.282947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830441+0000","last_change":"2024-05-14T11:17:56.650554+0000","last_active":"2024-05-14T11:17:58.830441+0000","last_peered":"2024-05-14T11:17:58.830441+0000","last_clean":"2024-05-14T11:17:58.830441+0000","last_became_active":"2024-05-14T11:17:56.650236+0000","last_became_peered":"2024-05-14T11:17:56.650236+0000","last_unstale":"2024-05-14T11:17:58.830441+0000","last_undegraded":"2024-05-14T11:17:58.830441+0000","last_fullsized":"2024-05-14T11:17:58.830441+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:40.716449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645611+0000","last_change":"2024-05-14T11:17:56.644922+0000","last_active":"2024-05-14T11:17:57.645611+0000","last_peered":"2024-05-14T11:17:57.645611+0000","last_clean":"2024-05-14T11:17:57.645611+0000","last_became_active":"2024-05-14T11:17:56.644620+0000","last_became_peered":"2024-05-14T11:17:56.644620+0000","last_unstale":"2024-05-14T11:17:57.645611+0000","last_undegraded":"2024-05-14T11:17:57.645611+0000","last_fullsized":"2024-05-14T11:17:57.645611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:58.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643699+0000","last_change":"2024-05-14T11:17:56.653616+0000","last_active":"2024-05-14T11:17:57.643699+0000","last_peered":"2024-05-14T11:17:57.643699+0000","last_clean":"2024-05-14T11:17:57.643699+0000","last_became_active":"2024-05-14T11:17:56.653302+0000","last_became_peered":"2024-05-14T11:17:56.653302+0000","last_unstale":"2024-05-14T11:17:57.643699+0000","last_undegraded":"2024-05-14T11:17:57.643699+0000","last_fullsized":"2024-05-14T11:17:57.643699+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:44.569765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.654953+0000","last_change":"2024-05-14T11:17:56.657349+0000","last_active":"2024-05-14T11:17:57.654953+0000","last_peered":"2024-05-14T11:17:57.654953+0000","last_clean":"2024-05-14T11:17:57.654953+0000","last_became_active":"2024-05-14T11:17:56.657057+0000","last_became_peered":"2024-05-14T11:17:56.657057+0000","last_unstale":"2024-05-14T11:17:57.654953+0000","last_undegraded":"2024-05-14T11:17:57.654953+0000","last_fullsized":"2024-05-14T11:17:57.654953+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:06.379073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643571+0000","last_change":"2024-05-14T11:17:56.648712+0000","last_active":"2024-05-14T11:17:57.643571+0000","last_peered":"2024-05-14T11:17:57.643571+0000","last_clean":"2024-05-14T11:17:57.643571+0000","last_became_active":"2024-05-14T11:17:56.648400+0000","last_became_peered":"2024-05-14T11:17:56.648400+0000","last_unstale":"2024-05-14T11:17:57.643571+0000","last_undegraded":"2024-05-14T11:17:57.643571+0000","last_fullsized":"2024-05-14T11:17:57.643571+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:01.697696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830390+0000","last_change":"2024-05-14T11:17:56.661742+0000","last_active":"2024-05-14T11:17:58.830390+0000","last_peered":"2024-05-14T11:17:58.830390+0000","last_clean":"2024-05-14T11:17:58.830390+0000","last_became_active":"2024-05-14T11:17:56.660880+0000","last_became_peered":"2024-05-14T11:17:56.660880+0000","last_unstale":"2024-05-14T11:17:58.830390+0000","last_undegraded":"2024-05-14T11:17:58.830390+0000","last_fullsized":"2024-05-14T11:17:58.830390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:02:58.227751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.654709+0000","last_change":"2024-05-14T11:17:56.647225+0000","last_active":"2024-05-14T11:17:57.654709+0000","last_peered":"2024-05-14T11:17:57.654709+0000","last_clean":"2024-05-14T11:17:57.654709+0000","last_became_active":"2024-05-14T11:17:56.646927+0000","last_became_peered":"2024-05-14T11:17:56.646927+0000","last_unstale":"2024-05-14T11:17:57.654709+0000","last_undegraded":"2024-05-14T11:17:57.654709+0000","last_fullsized":"2024-05-14T11:17:57.654709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:34.821815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645946+0000","last_change":"2024-05-14T11:17:55.641818+0000","last_active":"2024-05-14T11:17:57.645946+0000","last_peered":"2024-05-14T11:17:57.645946+0000","last_clean":"2024-05-14T11:17:57.645946+0000","last_became_active":"2024-05-14T11:17:55.640288+0000","last_became_peered":"2024-05-14T11:17:55.640288+0000","last_unstale":"2024-05-14T11:17:57.645946+0000","last_undegraded":"2024-05-14T11:17:57.645946+0000","last_fullsized":"2024-05-14T11:17:57.645946+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:48.132847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830613+0000","last_change":"2024-05-14T11:17:56.647171+0000","last_active":"2024-05-14T11:17:58.830613+0000","last_peered":"2024-05-14T11:17:58.830613+0000","last_clean":"2024-05-14T11:17:58.830613+0000","last_became_active":"2024-05-14T11:17:56.646912+0000","last_became_peered":"2024-05-14T11:17:56.646912+0000","last_unstale":"2024-05-14T11:17:58.830613+0000","last_undegraded":"2024-05-14T11:17:58.830613+0000","last_fullsized":"2024-05-14T11:17:58.830613+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:06.379073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830808+0000","last_change":"2024-05-14T11:17:55.655279+0000","last_active":"2024-05-14T11:17:58.830808+0000","last_peered":"2024-05-14T11:17:58.830808+0000","last_clean":"2024-05-14T11:17:58.830808+0000","last_became_active":"2024-05-14T11:17:55.651780+0000","last_became_peered":"2024-05-14T11:17:55.651780+0000","last_unstale":"2024-05-14T11:17:58.830808+0000","last_undegraded":"2024-05-14T11:17:58.830808+0000","last_fullsized":"2024-05-14T11:17:58.830808+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:04:02.856664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645025+0000","last_change":"2024-05-14T11:17:56.650466+0000","last_active":"2024-05-14T11:17:57.645025+0000","last_peered":"2024-05-14T11:17:57.645025+0000","last_clean":"2024-05-14T11:17:57.645025+0000","last_became_active":"2024-05-14T11:17:56.650186+0000","last_became_peered":"2024-05-14T11:17:56.650186+0000","last_unstale":"2024-05-14T11:17:57.645025+0000","last_undegraded":"2024-05-14T11:17:57.645025+0000","last_fullsized":"2024-05-14T11:17:57.645025+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:13:25.340359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830502+0000","last_change":"2024-05-14T11:17:55.646665+0000","last_active":"2024-05-14T11:17:58.830502+0000","last_peered":"2024-05-14T11:17:58.830502+0000","last_clean":"2024-05-14T11:17:58.830502+0000","last_became_active":"2024-05-14T11:17:55.646308+0000","last_became_peered":"2024-05-14T11:17:55.646308+0000","last_unstale":"2024-05-14T11:17:58.830502+0000","last_undegraded":"2024-05-14T11:17:58.830502+0000","last_fullsized":"2024-05-14T11:17:58.830502+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:27:48.132847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829096+0000","last_change":"2024-05-14T11:17:56.661532+0000","last_active":"2024-05-14T11:17:58.829096+0000","last_peered":"2024-05-14T11:17:58.829096+0000","last_clean":"2024-05-14T11:17:58.829096+0000","last_became_active":"2024-05-14T11:17:56.661069+0000","last_became_peered":"2024-05-14T11:17:56.661069+0000","last_unstale":"2024-05-14T11:17:58.829096+0000","last_undegraded":"2024-05-14T11:17:58.829096+0000","last_fullsized":"2024-05-14T11:17:58.829096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:41:03.591737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830524+0000","last_change":"2024-05-14T11:17:55.649086+0000","last_active":"2024-05-14T11:17:58.830524+0000","last_peered":"2024-05-14T11:17:58.830524+0000","last_clean":"2024-05-14T11:17:58.830524+0000","last_became_active":"2024-05-14T11:17:55.648763+0000","last_became_peered":"2024-05-14T11:17:55.648763+0000","last_unstale":"2024-05-14T11:17:58.830524+0000","last_undegraded":"2024-05-14T11:17:58.830524+0000","last_fullsized":"2024-05-14T11:17:58.830524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:51.484776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.646094+0000","last_change":"2024-05-14T11:17:56.643809+0000","last_active":"2024-05-14T11:17:57.646094+0000","last_peered":"2024-05-14T11:17:57.646094+0000","last_clean":"2024-05-14T11:17:57.646094+0000","last_became_active":"2024-05-14T11:17:56.643475+0000","last_became_peered":"2024-05-14T11:17:56.643475+0000","last_unstale":"2024-05-14T11:17:57.646094+0000","last_undegraded":"2024-05-14T11:17:57.646094+0000","last_fullsized":"2024-05-14T11:17:57.646094+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:48.362314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643646+0000","last_change":"2024-05-14T11:17:55.652269+0000","last_active":"2024-05-14T11:17:57.643646+0000","last_peered":"2024-05-14T11:17:57.643646+0000","last_clean":"2024-05-14T11:17:57.643646+0000","last_became_active":"2024-05-14T11:17:55.652000+0000","last_became_peered":"2024-05-14T11:17:55.652000+0000","last_unstale":"2024-05-14T11:17:57.643646+0000","last_undegraded":"2024-05-14T11:17:57.643646+0000","last_fullsized":"2024-05-14T11:17:57.643646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:34:53.260344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655196+0000","last_change":"2024-05-14T11:17:56.657286+0000","last_active":"2024-05-14T11:17:57.655196+0000","last_peered":"2024-05-14T11:17:57.655196+0000","last_clean":"2024-05-14T11:17:57.655196+0000","last_became_active":"2024-05-14T11:17:56.656147+0000","last_became_peered":"2024-05-14T11:17:56.656147+0000","last_unstale":"2024-05-14T11:17:57.655196+0000","last_undegraded":"2024-05-14T11:17:57.655196+0000","last_fullsized":"2024-05-14T11:17:57.655196+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:48:40.670244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645530+0000","last_change":"2024-05-14T11:17:55.641379+0000","last_active":"2024-05-14T11:17:57.645530+0000","last_peered":"2024-05-14T11:17:57.645530+0000","last_clean":"2024-05-14T11:17:57.645530+0000","last_became_active":"2024-05-14T11:17:55.640845+0000","last_became_peered":"2024-05-14T11:17:55.640845+0000","last_unstale":"2024-05-14T11:17:57.645530+0000","last_undegraded":"2024-05-14T11:17:57.645530+0000","last_fullsized":"2024-05-14T11:17:57.645530+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:15:55.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829510+0000","last_change":"2024-05-14T11:17:56.666497+0000","last_active":"2024-05-14T11:17:58.829510+0000","last_peered":"2024-05-14T11:17:58.829510+0000","last_clean":"2024-05-14T11:17:58.829510+0000","last_became_active":"2024-05-14T11:17:56.666177+0000","last_became_peered":"2024-05-14T11:17:56.666177+0000","last_unstale":"2024-05-14T11:17:58.829510+0000","last_undegraded":"2024-05-14T11:17:58.829510+0000","last_fullsized":"2024-05-14T11:17:58.829510+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:21.012340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644840+0000","last_change":"2024-05-14T11:17:55.637463+0000","last_active":"2024-05-14T11:17:57.644840+0000","last_peered":"2024-05-14T11:17:57.644840+0000","last_clean":"2024-05-14T11:17:57.644840+0000","last_became_active":"2024-05-14T11:17:55.636673+0000","last_became_peered":"2024-05-14T11:17:55.636673+0000","last_unstale":"2024-05-14T11:17:57.644840+0000","last_undegraded":"2024-05-14T11:17:57.644840+0000","last_fullsized":"2024-05-14T11:17:57.644840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:28.897750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655105+0000","last_change":"2024-05-14T11:17:56.674328+0000","last_active":"2024-05-14T11:17:57.655105+0000","last_peered":"2024-05-14T11:17:57.655105+0000","last_clean":"2024-05-14T11:17:57.655105+0000","last_became_active":"2024-05-14T11:17:56.673973+0000","last_became_peered":"2024-05-14T11:17:56.673973+0000","last_unstale":"2024-05-14T11:17:57.655105+0000","last_undegraded":"2024-05-14T11:17:57.655105+0000","last_fullsized":"2024-05-14T11:17:57.655105+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:08.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643285+0000","last_change":"2024-05-14T11:17:55.643650+0000","last_active":"2024-05-14T11:17:57.643285+0000","last_peered":"2024-05-14T11:17:57.643285+0000","last_clean":"2024-05-14T11:17:57.643285+0000","last_became_active":"2024-05-14T11:17:55.642148+0000","last_became_peered":"2024-05-14T11:17:55.642148+0000","last_unstale":"2024-05-14T11:17:57.643285+0000","last_undegraded":"2024-05-14T11:17:57.643285+0000","last_fullsized":"2024-05-14T11:17:57.643285+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:47.353253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645343+0000","last_change":"2024-05-14T11:17:56.648432+0000","last_active":"2024-05-14T11:17:57.645343+0000","last_peered":"2024-05-14T11:17:57.645343+0000","last_clean":"2024-05-14T11:17:57.645343+0000","last_became_active":"2024-05-14T11:17:56.647809+0000","last_became_peered":"2024-05-14T11:17:56.647809+0000","last_unstale":"2024-05-14T11:17:57.645343+0000","last_undegraded":"2024-05-14T11:17:57.645343+0000","last_fullsized":"2024-05-14T11:17:57.645343+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:13:25.340359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655007+0000","last_change":"2024-05-14T11:17:55.641063+0000","last_active":"2024-05-14T11:17:57.655007+0000","last_peered":"2024-05-14T11:17:57.655007+0000","last_clean":"2024-05-14T11:17:57.655007+0000","last_became_active":"2024-05-14T11:17:55.640786+0000","last_became_peered":"2024-05-14T11:17:55.640786+0000","last_unstale":"2024-05-14T11:17:57.655007+0000","last_undegraded":"2024-05-14T11:17:57.655007+0000","last_fullsized":"2024-05-14T11:17:57.655007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:51.484776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830356+0000","last_change":"2024-05-14T11:17:56.651514+0000","last_active":"2024-05-14T11:17:58.830356+0000","last_peered":"2024-05-14T11:17:58.830356+0000","last_clean":"2024-05-14T11:17:58.830356+0000","last_became_active":"2024-05-14T11:17:56.651099+0000","last_became_peered":"2024-05-14T11:17:56.651099+0000","last_unstale":"2024-05-14T11:17:58.830356+0000","last_undegraded":"2024-05-14T11:17:58.830356+0000","last_fullsized":"2024-05-14T11:17:58.830356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:38.104044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645313+0000","last_change":"2024-05-14T11:17:55.653762+0000","last_active":"2024-05-14T11:17:57.645313+0000","last_peered":"2024-05-14T11:17:57.645313+0000","last_clean":"2024-05-14T11:17:57.645313+0000","last_became_active":"2024-05-14T11:17:55.653334+0000","last_became_peered":"2024-05-14T11:17:55.653334+0000","last_unstale":"2024-05-14T11:17:57.645313+0000","last_undegraded":"2024-05-14T11:17:57.645313+0000","last_fullsized":"2024-05-14T11:17:57.645313+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:45.304282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643778+0000","last_change":"2024-05-14T11:17:56.668993+0000","last_active":"2024-05-14T11:17:57.643778+0000","last_peered":"2024-05-14T11:17:57.643778+0000","last_clean":"2024-05-14T11:17:57.643778+0000","last_became_active":"2024-05-14T11:17:56.654001+0000","last_became_peered":"2024-05-14T11:17:56.654001+0000","last_unstale":"2024-05-14T11:17:57.643778+0000","last_undegraded":"2024-05-14T11:17:57.643778+0000","last_fullsized":"2024-05-14T11:17:57.643778+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:21.012340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655136+0000","last_change":"2024-05-14T11:17:55.656128+0000","last_active":"2024-05-14T11:17:57.655136+0000","last_peered":"2024-05-14T11:17:57.655136+0000","last_clean":"2024-05-14T11:17:57.655136+0000","last_became_active":"2024-05-14T11:17:55.655701+0000","last_became_peered":"2024-05-14T11:17:55.655701+0000","last_unstale":"2024-05-14T11:17:57.655136+0000","last_undegraded":"2024-05-14T11:17:57.655136+0000","last_fullsized":"2024-05-14T11:17:57.655136+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:47.353253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.646447+0000","last_change":"2024-05-14T11:17:56.657617+0000","last_active":"2024-05-14T11:17:57.646447+0000","last_peered":"2024-05-14T11:17:57.646447+0000","last_clean":"2024-05-14T11:17:57.646447+0000","last_became_active":"2024-05-14T11:17:56.657320+0000","last_became_peered":"2024-05-14T11:17:56.657320+0000","last_unstale":"2024-05-14T11:17:57.646447+0000","last_undegraded":"2024-05-14T11:17:57.646447+0000","last_fullsized":"2024-05-14T11:17:57.646447+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:58.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829395+0000","last_change":"2024-05-14T11:17:55.648244+0000","last_active":"2024-05-14T11:17:58.829395+0000","last_peered":"2024-05-14T11:17:58.829395+0000","last_clean":"2024-05-14T11:17:58.829395+0000","last_became_active":"2024-05-14T11:17:55.647942+0000","last_became_peered":"2024-05-14T11:17:55.647942+0000","last_unstale":"2024-05-14T11:17:58.829395+0000","last_undegraded":"2024-05-14T11:17:58.829395+0000","last_fullsized":"2024-05-14T11:17:58.829395+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:54.022835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830650+0000","last_change":"2024-05-14T11:17:56.648777+0000","last_active":"2024-05-14T11:17:58.830650+0000","last_peered":"2024-05-14T11:17:58.830650+0000","last_clean":"2024-05-14T11:17:58.830650+0000","last_became_active":"2024-05-14T11:17:56.648463+0000","last_became_peered":"2024-05-14T11:17:56.648463+0000","last_unstale":"2024-05-14T11:17:58.830650+0000","last_undegraded":"2024-05-14T11:17:58.830650+0000","last_fullsized":"2024-05-14T11:17:58.830650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:08.266622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643763+0000","last_change":"2024-05-14T11:17:55.649938+0000","last_active":"2024-05-14T11:17:57.643763+0000","last_peered":"2024-05-14T11:17:57.643763+0000","last_clean":"2024-05-14T11:17:57.643763+0000","last_became_active":"2024-05-14T11:17:55.649675+0000","last_became_peered":"2024-05-14T11:17:55.649675+0000","last_unstale":"2024-05-14T11:17:57.643763+0000","last_undegraded":"2024-05-14T11:17:57.643763+0000","last_fullsized":"2024-05-14T11:17:57.643763+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:13:24.331297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644365+0000","last_change":"2024-05-14T11:17:56.653022+0000","last_active":"2024-05-14T11:17:57.644365+0000","last_peered":"2024-05-14T11:17:57.644365+0000","last_clean":"2024-05-14T11:17:57.644365+0000","last_became_active":"2024-05-14T11:17:56.649642+0000","last_became_peered":"2024-05-14T11:17:56.649642+0000","last_unstale":"2024-05-14T11:17:57.644365+0000","last_undegraded":"2024-05-14T11:17:57.644365+0000","last_fullsized":"2024-05-14T11:17:57.644365+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:43:46.613774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645095+0000","last_change":"2024-05-14T11:17:55.644208+0000","last_active":"2024-05-14T11:17:57.645095+0000","last_peered":"2024-05-14T11:17:57.645095+0000","last_clean":"2024-05-14T11:17:57.645095+0000","last_became_active":"2024-05-14T11:17:55.643884+0000","last_became_peered":"2024-05-14T11:17:55.643884+0000","last_unstale":"2024-05-14T11:17:57.645095+0000","last_undegraded":"2024-05-14T11:17:57.645095+0000","last_fullsized":"2024-05-14T11:17:57.645095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:57:12.372838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829350+0000","last_change":"2024-05-14T11:17:56.656090+0000","last_active":"2024-05-14T11:17:58.829350+0000","last_peered":"2024-05-14T11:17:58.829350+0000","last_clean":"2024-05-14T11:17:58.829350+0000","last_became_active":"2024-05-14T11:17:56.655686+0000","last_became_peered":"2024-05-14T11:17:56.655686+0000","last_unstale":"2024-05-14T11:17:58.829350+0000","last_undegraded":"2024-05-14T11:17:58.829350+0000","last_fullsized":"2024-05-14T11:17:58.829350+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:44.569765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.654771+0000","last_change":"2024-05-14T11:17:55.641687+0000","last_active":"2024-05-14T11:17:57.654771+0000","last_peered":"2024-05-14T11:17:57.654771+0000","last_clean":"2024-05-14T11:17:57.654771+0000","last_became_active":"2024-05-14T11:17:55.641241+0000","last_became_peered":"2024-05-14T11:17:55.641241+0000","last_unstale":"2024-05-14T11:17:57.654771+0000","last_undegraded":"2024-05-14T11:17:57.654771+0000","last_fullsized":"2024-05-14T11:17:57.654771+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:13:24.331297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644312+0000","last_change":"2024-05-14T11:17:56.649810+0000","last_active":"2024-05-14T11:17:57.644312+0000","last_peered":"2024-05-14T11:17:57.644312+0000","last_clean":"2024-05-14T11:17:57.644312+0000","last_became_active":"2024-05-14T11:17:56.648833+0000","last_became_peered":"2024-05-14T11:17:56.648833+0000","last_unstale":"2024-05-14T11:17:57.644312+0000","last_undegraded":"2024-05-14T11:17:57.644312+0000","last_fullsized":"2024-05-14T11:17:57.644312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:53.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":22,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830475+0000","last_change":"2024-05-14T11:17:55.644962+0000","last_active":"2024-05-14T11:17:58.830475+0000","last_peered":"2024-05-14T11:17:58.830475+0000","last_clean":"2024-05-14T11:17:58.830475+0000","last_became_active":"2024-05-14T11:17:55.644623+0000","last_became_peered":"2024-05-14T11:17:55.644623+0000","last_unstale":"2024-05-14T11:17:58.830475+0000","last_undegraded":"2024-05-14T11:17:58.830475+0000","last_fullsized":"2024-05-14T11:17:58.830475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:37.094983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830420+0000","last_change":"2024-05-14T11:17:56.659720+0000","last_active":"2024-05-14T11:17:58.830420+0000","last_peered":"2024-05-14T11:17:58.830420+0000","last_clean":"2024-05-14T11:17:58.830420+0000","last_became_active":"2024-05-14T11:17:56.658574+0000","last_became_peered":"2024-05-14T11:17:56.658574+0000","last_unstale":"2024-05-14T11:17:58.830420+0000","last_undegraded":"2024-05-14T11:17:58.830420+0000","last_fullsized":"2024-05-14T11:17:58.830420+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:14:53.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644509+0000","last_change":"2024-05-14T11:17:55.652253+0000","last_active":"2024-05-14T11:17:57.644509+0000","last_peered":"2024-05-14T11:17:57.644509+0000","last_clean":"2024-05-14T11:17:57.644509+0000","last_became_active":"2024-05-14T11:17:55.651965+0000","last_became_peered":"2024-05-14T11:17:55.651965+0000","last_unstale":"2024-05-14T11:17:57.644509+0000","last_undegraded":"2024-05-14T11:17:57.644509+0000","last_fullsized":"2024-05-14T11:17:57.644509+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:13.633334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829840+0000","last_change":"2024-05-14T11:17:56.640716+0000","last_active":"2024-05-14T11:17:58.829840+0000","last_peered":"2024-05-14T11:17:58.829840+0000","last_clean":"2024-05-14T11:17:58.829840+0000","last_became_active":"2024-05-14T11:17:56.640404+0000","last_became_peered":"2024-05-14T11:17:56.640404+0000","last_unstale":"2024-05-14T11:17:58.829840+0000","last_undegraded":"2024-05-14T11:17:58.829840+0000","last_fullsized":"2024-05-14T11:17:58.829840+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:06.019238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830440+0000","last_change":"2024-05-14T11:17:55.649239+0000","last_active":"2024-05-14T11:17:58.830440+0000","last_peered":"2024-05-14T11:17:58.830440+0000","last_clean":"2024-05-14T11:17:58.830440+0000","last_became_active":"2024-05-14T11:17:55.648868+0000","last_became_peered":"2024-05-14T11:17:55.648868+0000","last_unstale":"2024-05-14T11:17:58.830440+0000","last_undegraded":"2024-05-14T11:17:58.830440+0000","last_fullsized":"2024-05-14T11:17:58.830440+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:55:28.931631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645518+0000","last_change":"2024-05-14T11:17:56.646932+0000","last_active":"2024-05-14T11:17:57.645518+0000","last_peered":"2024-05-14T11:17:57.645518+0000","last_clean":"2024-05-14T11:17:57.645518+0000","last_became_active":"2024-05-14T11:17:56.646621+0000","last_became_peered":"2024-05-14T11:17:56.646621+0000","last_unstale":"2024-05-14T11:17:57.645518+0000","last_undegraded":"2024-05-14T11:17:57.645518+0000","last_fullsized":"2024-05-14T11:17:57.645518+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:52.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645775+0000","last_change":"2024-05-14T11:17:55.652704+0000","last_active":"2024-05-14T11:17:57.645775+0000","last_peered":"2024-05-14T11:17:57.645775+0000","last_clean":"2024-05-14T11:17:57.645775+0000","last_became_active":"2024-05-14T11:17:55.652348+0000","last_became_peered":"2024-05-14T11:17:55.652348+0000","last_unstale":"2024-05-14T11:17:57.645775+0000","last_undegraded":"2024-05-14T11:17:57.645775+0000","last_fullsized":"2024-05-14T11:17:57.645775+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:45.304282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830193+0000","last_change":"2024-05-14T11:17:56.647995+0000","last_active":"2024-05-14T11:17:58.830193+0000","last_peered":"2024-05-14T11:17:58.830193+0000","last_clean":"2024-05-14T11:17:58.830193+0000","last_became_active":"2024-05-14T11:17:56.647240+0000","last_became_peered":"2024-05-14T11:17:56.647240+0000","last_unstale":"2024-05-14T11:17:58.830193+0000","last_undegraded":"2024-05-14T11:17:58.830193+0000","last_fullsized":"2024-05-14T11:17:58.830193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:14.642396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645629+0000","last_change":"2024-05-14T11:17:55.638409+0000","last_active":"2024-05-14T11:17:57.645629+0000","last_peered":"2024-05-14T11:17:57.645629+0000","last_clean":"2024-05-14T11:17:57.645629+0000","last_became_active":"2024-05-14T11:17:55.636963+0000","last_became_peered":"2024-05-14T11:17:55.636963+0000","last_unstale":"2024-05-14T11:17:57.645629+0000","last_undegraded":"2024-05-14T11:17:57.645629+0000","last_fullsized":"2024-05-14T11:17:57.645629+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:04:02.856664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830368+0000","last_change":"2024-05-14T11:17:56.654826+0000","last_active":"2024-05-14T11:17:58.830368+0000","last_peered":"2024-05-14T11:17:58.830368+0000","last_clean":"2024-05-14T11:17:58.830368+0000","last_became_active":"2024-05-14T11:17:56.654480+0000","last_became_peered":"2024-05-14T11:17:56.654480+0000","last_unstale":"2024-05-14T11:17:58.830368+0000","last_undegraded":"2024-05-14T11:17:58.830368+0000","last_fullsized":"2024-05-14T11:17:58.830368+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:55.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830250+0000","last_change":"2024-05-14T11:17:55.646773+0000","last_active":"2024-05-14T11:17:58.830250+0000","last_peered":"2024-05-14T11:17:58.830250+0000","last_clean":"2024-05-14T11:17:58.830250+0000","last_became_active":"2024-05-14T11:17:55.646401+0000","last_became_peered":"2024-05-14T11:17:55.646401+0000","last_unstale":"2024-05-14T11:17:58.830250+0000","last_undegraded":"2024-05-14T11:17:58.830250+0000","last_fullsized":"2024-05-14T11:17:58.830250+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:17.528091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645821+0000","last_change":"2024-05-14T11:17:56.640569+0000","last_active":"2024-05-14T11:17:57.645821+0000","last_peered":"2024-05-14T11:17:57.645821+0000","last_clean":"2024-05-14T11:17:57.645821+0000","last_became_active":"2024-05-14T11:17:56.640195+0000","last_became_peered":"2024-05-14T11:17:56.640195+0000","last_unstale":"2024-05-14T11:17:57.645821+0000","last_undegraded":"2024-05-14T11:17:57.645821+0000","last_fullsized":"2024-05-14T11:17:57.645821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:14.642396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829734+0000","last_change":"2024-05-14T11:17:55.635513+0000","last_active":"2024-05-14T11:17:58.829734+0000","last_peered":"2024-05-14T11:17:58.829734+0000","last_clean":"2024-05-14T11:17:58.829734+0000","last_became_active":"2024-05-14T11:17:55.635170+0000","last_became_peered":"2024-05-14T11:17:55.635170+0000","last_unstale":"2024-05-14T11:17:58.829734+0000","last_undegraded":"2024-05-14T11:17:58.829734+0000","last_fullsized":"2024-05-14T11:17:58.829734+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:15:39.707387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644215+0000","last_change":"2024-05-14T11:17:56.653617+0000","last_active":"2024-05-14T11:17:57.644215+0000","last_peered":"2024-05-14T11:17:57.644215+0000","last_clean":"2024-05-14T11:17:57.644215+0000","last_became_active":"2024-05-14T11:17:56.653260+0000","last_became_peered":"2024-05-14T11:17:56.653260+0000","last_unstale":"2024-05-14T11:17:57.644215+0000","last_undegraded":"2024-05-14T11:17:57.644215+0000","last_fullsized":"2024-05-14T11:17:57.644215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:33:59.678226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.643496+0000","last_change":"2024-05-14T11:17:55.650975+0000","last_active":"2024-05-14T11:17:57.643496+0000","last_peered":"2024-05-14T11:17:57.643496+0000","last_clean":"2024-05-14T11:17:57.643496+0000","last_became_active":"2024-05-14T11:17:55.650697+0000","last_became_peered":"2024-05-14T11:17:55.650697+0000","last_unstale":"2024-05-14T11:17:57.643496+0000","last_undegraded":"2024-05-14T11:17:57.643496+0000","last_fullsized":"2024-05-14T11:17:57.643496+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:57.688003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830874+0000","last_change":"2024-05-14T11:17:56.658975+0000","last_active":"2024-05-14T11:17:58.830874+0000","last_peered":"2024-05-14T11:17:58.830874+0000","last_clean":"2024-05-14T11:17:58.830874+0000","last_became_active":"2024-05-14T11:17:56.657329+0000","last_became_peered":"2024-05-14T11:17:56.657329+0000","last_unstale":"2024-05-14T11:17:58.830874+0000","last_undegraded":"2024-05-14T11:17:58.830874+0000","last_fullsized":"2024-05-14T11:17:58.830874+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:29:55.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829018+0000","last_change":"2024-05-14T11:17:55.639873+0000","last_active":"2024-05-14T11:17:58.829018+0000","last_peered":"2024-05-14T11:17:58.829018+0000","last_clean":"2024-05-14T11:17:58.829018+0000","last_became_active":"2024-05-14T11:17:55.639536+0000","last_became_peered":"2024-05-14T11:17:55.639536+0000","last_unstale":"2024-05-14T11:17:58.829018+0000","last_undegraded":"2024-05-14T11:17:58.829018+0000","last_fullsized":"2024-05-14T11:17:58.829018+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:48:20.341882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830551+0000","last_change":"2024-05-14T11:17:56.662014+0000","last_active":"2024-05-14T11:17:58.830551+0000","last_peered":"2024-05-14T11:17:58.830551+0000","last_clean":"2024-05-14T11:17:58.830551+0000","last_became_active":"2024-05-14T11:17:56.660903+0000","last_became_peered":"2024-05-14T11:17:56.660903+0000","last_unstale":"2024-05-14T11:17:58.830551+0000","last_undegraded":"2024-05-14T11:17:58.830551+0000","last_fullsized":"2024-05-14T11:17:58.830551+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:34.821815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.646044+0000","last_change":"2024-05-14T11:17:55.647271+0000","last_active":"2024-05-14T11:17:57.646044+0000","last_peered":"2024-05-14T11:17:57.646044+0000","last_clean":"2024-05-14T11:17:57.646044+0000","last_became_active":"2024-05-14T11:17:55.646926+0000","last_became_peered":"2024-05-14T11:17:55.646926+0000","last_unstale":"2024-05-14T11:17:57.646044+0000","last_undegraded":"2024-05-14T11:17:57.646044+0000","last_fullsized":"2024-05-14T11:17:57.646044+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:10:17.528091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830737+0000","last_change":"2024-05-14T11:17:56.645324+0000","last_active":"2024-05-14T11:17:58.830737+0000","last_peered":"2024-05-14T11:17:58.830737+0000","last_clean":"2024-05-14T11:17:58.830737+0000","last_became_active":"2024-05-14T11:17:56.645023+0000","last_became_peered":"2024-05-14T11:17:56.645023+0000","last_unstale":"2024-05-14T11:17:58.830737+0000","last_undegraded":"2024-05-14T11:17:58.830737+0000","last_fullsized":"2024-05-14T11:17:58.830737+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:48.967429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.655256+0000","last_change":"2024-05-14T11:17:55.647986+0000","last_active":"2024-05-14T11:17:57.655256+0000","last_peered":"2024-05-14T11:17:57.655256+0000","last_clean":"2024-05-14T11:17:57.655256+0000","last_became_active":"2024-05-14T11:17:55.647640+0000","last_became_peered":"2024-05-14T11:17:55.647640+0000","last_unstale":"2024-05-14T11:17:57.655256+0000","last_undegraded":"2024-05-14T11:17:57.655256+0000","last_fullsized":"2024-05-14T11:17:57.655256+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:05.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829245+0000","last_change":"2024-05-14T11:17:56.646749+0000","last_active":"2024-05-14T11:17:58.829245+0000","last_peered":"2024-05-14T11:17:58.829245+0000","last_clean":"2024-05-14T11:17:58.829245+0000","last_became_active":"2024-05-14T11:17:56.646364+0000","last_became_peered":"2024-05-14T11:17:56.646364+0000","last_unstale":"2024-05-14T11:17:58.829245+0000","last_undegraded":"2024-05-14T11:17:58.829245+0000","last_fullsized":"2024-05-14T11:17:58.829245+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:04:01.697696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644114+0000","last_change":"2024-05-14T11:17:55.647962+0000","last_active":"2024-05-14T11:17:57.644114+0000","last_peered":"2024-05-14T11:17:57.644114+0000","last_clean":"2024-05-14T11:17:57.644114+0000","last_became_active":"2024-05-14T11:17:55.645389+0000","last_became_peered":"2024-05-14T11:17:55.645389+0000","last_unstale":"2024-05-14T11:17:57.644114+0000","last_undegraded":"2024-05-14T11:17:57.644114+0000","last_fullsized":"2024-05-14T11:17:57.644114+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:51.484776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644073+0000","last_change":"2024-05-14T11:17:56.673802+0000","last_active":"2024-05-14T11:17:57.644073+0000","last_peered":"2024-05-14T11:17:57.644073+0000","last_clean":"2024-05-14T11:17:57.644073+0000","last_became_active":"2024-05-14T11:17:56.673480+0000","last_became_peered":"2024-05-14T11:17:56.673480+0000","last_unstale":"2024-05-14T11:17:57.644073+0000","last_undegraded":"2024-05-14T11:17:57.644073+0000","last_fullsized":"2024-05-14T11:17:57.644073+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:57:55.213354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829152+0000","last_change":"2024-05-14T11:17:55.652430+0000","last_active":"2024-05-14T11:17:58.829152+0000","last_peered":"2024-05-14T11:17:58.829152+0000","last_clean":"2024-05-14T11:17:58.829152+0000","last_became_active":"2024-05-14T11:17:55.652111+0000","last_became_peered":"2024-05-14T11:17:55.652111+0000","last_unstale":"2024-05-14T11:17:58.829152+0000","last_undegraded":"2024-05-14T11:17:58.829152+0000","last_fullsized":"2024-05-14T11:17:58.829152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:13.633334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830578+0000","last_change":"2024-05-14T11:17:56.655817+0000","last_active":"2024-05-14T11:17:58.830578+0000","last_peered":"2024-05-14T11:17:58.830578+0000","last_clean":"2024-05-14T11:17:58.830578+0000","last_became_active":"2024-05-14T11:17:56.653660+0000","last_became_peered":"2024-05-14T11:17:56.653660+0000","last_unstale":"2024-05-14T11:17:58.830578+0000","last_undegraded":"2024-05-14T11:17:58.830578+0000","last_fullsized":"2024-05-14T11:17:58.830578+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:56:35.799237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.829824+0000","last_change":"2024-05-14T11:17:55.653463+0000","last_active":"2024-05-14T11:17:58.829824+0000","last_peered":"2024-05-14T11:17:58.829824+0000","last_clean":"2024-05-14T11:17:58.829824+0000","last_became_active":"2024-05-14T11:17:55.652994+0000","last_became_peered":"2024-05-14T11:17:55.652994+0000","last_unstale":"2024-05-14T11:17:58.829824+0000","last_undegraded":"2024-05-14T11:17:58.829824+0000","last_fullsized":"2024-05-14T11:17:58.829824+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:20:23.423306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644458+0000","last_change":"2024-05-14T11:17:56.677950+0000","last_active":"2024-05-14T11:17:57.644458+0000","last_peered":"2024-05-14T11:17:57.644458+0000","last_clean":"2024-05-14T11:17:57.644458+0000","last_became_active":"2024-05-14T11:17:56.676271+0000","last_became_peered":"2024-05-14T11:17:56.676271+0000","last_unstale":"2024-05-14T11:17:57.644458+0000","last_undegraded":"2024-05-14T11:17:57.644458+0000","last_fullsized":"2024-05-14T11:17:57.644458+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:41.552911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645035+0000","last_change":"2024-05-14T11:17:55.641080+0000","last_active":"2024-05-14T11:17:57.645035+0000","last_peered":"2024-05-14T11:17:57.645035+0000","last_clean":"2024-05-14T11:17:57.645035+0000","last_became_active":"2024-05-14T11:17:55.640707+0000","last_became_peered":"2024-05-14T11:17:55.640707+0000","last_unstale":"2024-05-14T11:17:57.645035+0000","last_undegraded":"2024-05-14T11:17:57.645035+0000","last_fullsized":"2024-05-14T11:17:57.645035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:57:12.372838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830453+0000","last_change":"2024-05-14T11:17:56.662610+0000","last_active":"2024-05-14T11:17:58.830453+0000","last_peered":"2024-05-14T11:17:58.830453+0000","last_clean":"2024-05-14T11:17:58.830453+0000","last_became_active":"2024-05-14T11:17:56.662266+0000","last_became_peered":"2024-05-14T11:17:56.662266+0000","last_unstale":"2024-05-14T11:17:58.830453+0000","last_undegraded":"2024-05-14T11:17:58.830453+0000","last_fullsized":"2024-05-14T11:17:58.830453+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:53:31.149313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:58.830536+0000","last_change":"2024-05-14T11:17:55.655950+0000","last_active":"2024-05-14T11:17:58.830536+0000","last_peered":"2024-05-14T11:17:58.830536+0000","last_clean":"2024-05-14T11:17:58.830536+0000","last_became_active":"2024-05-14T11:17:55.655668+0000","last_became_peered":"2024-05-14T11:17:55.655668+0000","last_unstale":"2024-05-14T11:17:58.830536+0000","last_undegraded":"2024-05-14T11:17:58.830536+0000","last_fullsized":"2024-05-14T11:17:58.830536+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:13:24.331297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.645304+0000","last_change":"2024-05-14T11:17:56.647726+0000","last_active":"2024-05-14T11:17:57.645304+0000","last_peered":"2024-05-14T11:17:57.645304+0000","last_clean":"2024-05-14T11:17:57.645304+0000","last_became_active":"2024-05-14T11:17:56.647406+0000","last_became_peered":"2024-05-14T11:17:56.647406+0000","last_unstale":"2024-05-14T11:17:57.645304+0000","last_undegraded":"2024-05-14T11:17:57.645304+0000","last_fullsized":"2024-05-14T11:17:57.645304+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:55.615640+0000","last_clean_scrub_stamp":"2024-05-14T11:17:55.615640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:06.019238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.644325+0000","last_change":"2024-05-14T11:17:55.643745+0000","last_active":"2024-05-14T11:17:57.644325+0000","last_peered":"2024-05-14T11:17:57.644325+0000","last_clean":"2024-05-14T11:17:57.644325+0000","last_became_active":"2024-05-14T11:17:55.643447+0000","last_became_peered":"2024-05-14T11:17:55.643447+0000","last_unstale":"2024-05-14T11:17:57.644325+0000","last_undegraded":"2024-05-14T11:17:57.644325+0000","last_fullsized":"2024-05-14T11:17:57.644325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:17:54.606578+0000","last_clean_scrub_stamp":"2024-05-14T11:17:54.606578+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:05.010176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":161,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-14T11:17:57.654612+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:17:57.654612+0000","last_peered":"2024-05-14T11:17:57.654612+0000","last_clean":"2024-05-14T11:17:57.654612+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:17:57.654612+0000","last_undegraded":"2024-05-14T11:17:57.654612+0000","last_fullsized":"2024-05-14T11:17:57.654612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":848064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":258224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":257572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":258154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":259206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":259184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":257732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":847996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:18:05.731 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:18:05.731 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:18:05.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.916+0000 7f5c90800640 1 Processor -- start 2024-05-14T11:18:05.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.916+0000 7f5c90800640 1 -- start start 2024-05-14T11:18:05.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c90800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c8806f450 0x7f5c8806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c90800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5c880d1730 0x7f5c880d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c90800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c8806f860 con 0x7f5c880cce60 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c90800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8806db80 con 0x7f5c8806f450 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c90800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5c880d2080 con 0x7f5c880d1730 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c8806f450 0x7f5c8806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c8806f450 0x7f5c8806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58348/0 (socket says 172.21.15.116:58348) 2024-05-14T11:18:05.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 -- 172.21.15.116:0/1388495115 learned_addr learned my addr 172.21.15.116:0/1388495115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8d400640 1 --2- 172.21.15.116:0/1388495115 >> v2:172.21.15.149:3300/0 conn(0x7f5c880d1730 0x7f5c880d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 -- 172.21.15.116:0/1388495115 >> v2:172.21.15.149:3300/0 conn(0x7f5c880d1730 msgr2=0x7f5c880d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 --2- 172.21.15.116:0/1388495115 >> v2:172.21.15.149:3300/0 conn(0x7f5c880d1730 0x7f5c880d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 -- 172.21.15.116:0/1388495115 >> v1:172.21.15.116:6789/0 conn(0x7f5c880cce60 legacy=0x7f5c880cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8ca00640 1 -- 172.21.15.116:0/1388495115 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163298721 0 0) 0x7f5c840014a0 con 0x7f5c880cce60 2024-05-14T11:18:05.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.920+0000 7f5c8de00640 1 -- 172.21.15.116:0/1388495115 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8806fae0 con 0x7f5c8806f450 2024-05-14T11:18:05.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c8de00640 1 --2- 172.21.15.116:0/1388495115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c8806d640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5c880d2430 tx=0x7f5c7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ec1ba9e89cc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c8ca00640 1 -- 172.21.15.116:0/1388495115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880ce190 con 0x7f5c8806f450 2024-05-14T11:18:05.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c8ca00640 1 -- 172.21.15.116:0/1388495115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0b0070 con 0x7f5c8806f450 2024-05-14T11:18:05.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c8ca00640 1 -- 172.21.15.116:0/1388495115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0ab830 con 0x7f5c8806f450 2024-05-14T11:18:05.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 -- 172.21.15.116:0/1388495115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 msgr2=0x7f5c8806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 --2- 172.21.15.116:0/1388495115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c8806d640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5c880d2430 tx=0x7f5c7c01c520 comp rx=0 tx=0).stop 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 -- 172.21.15.116:0/1388495115 shutdown_connections 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 --2- 172.21.15.116:0/1388495115 >> v2:172.21.15.149:3300/0 conn(0x7f5c880d1730 0x7f5c880d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 --2- 172.21.15.116:0/1388495115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c8806d640 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 -- 172.21.15.116:0/1388495115 >> 172.21.15.116:0/1388495115 conn(0x7f5c880b4720 msgr2=0x7f5c880b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 -- 172.21.15.116:0/1388495115 shutdown_connections 2024-05-14T11:18:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.924+0000 7f5c90800640 1 -- 172.21.15.116:0/1388495115 wait complete. 2024-05-14T11:18:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 Processor -- start 2024-05-14T11:18:05.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- start start 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 0x7f5c88072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 0x7f5c88073440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8816a610 con 0x7f5c8806f450 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c88070660 con 0x7f5c880cce60 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8816a790 con 0x7f5c880d1730 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58354/0 (socket says 172.21.15.116:58354) 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 -- 172.21.15.116:0/2686866764 learned_addr learned my addr 172.21.15.116:0/2686866764 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8d400640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 0x7f5c88072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 -- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 msgr2=0x7f5c88073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8e800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 0x7f5c88073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 0x7f5c88073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 -- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 msgr2=0x7f5c88072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 0x7f5c88072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 -- 172.21.15.116:0/2686866764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c880cfc10 con 0x7f5c8806f450 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8d400640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 0x7f5c88072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8e800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 0x7f5c88073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c8de00640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0ad040 tx=0x7f5c7c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0be5f0 con 0x7f5c8806f450 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0be7b0 con 0x7f5c8806f450 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0beba0 con 0x7f5c8806f450 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8815cb50 con 0x7f5c8806f450 2024-05-14T11:18:05.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.928+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c88159cb0 con 0x7f5c8806f450 2024-05-14T11:18:05.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8806a9c0 con 0x7f5c8806f450 2024-05-14T11:18:05.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c7c0b43f0 con 0x7f5c8806f450 2024-05-14T11:18:05.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c82000640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 0x7f5c6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:05.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f5c7c12a1f0 con 0x7f5c8806f450 2024-05-14T11:18:05.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c8d400640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 0x7f5c6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:05.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.932+0000 7f5c8d400640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 0x7f5c6c071910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c8806ec60 tx=0x7f5c78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:05.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:05.936+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5c8815cb50 con 0x7f5c8806f450 2024-05-14T11:18:06.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.148+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5c88069de0 con 0x7f5c8806f450 2024-05-14T11:18:06.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.152+0000 7f5c82000640 1 -- 172.21.15.116:0/2686866764 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f5c7c0c4060 con 0x7f5c8806f450 2024-05-14T11:18:06.155 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:18:06.155 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:18:06.156 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:18:06.156 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:18:06.156 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:18:06.156 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 msgr2=0x7f5c6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 0x7f5c6c071910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c8806ec60 tx=0x7f5c78008040 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 msgr2=0x7f5c880729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0ad040 tx=0x7f5c7c01ad60 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 shutdown_connections 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c6c06f400 0x7f5c6c071910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c880cce60 0x7f5c88072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5c880d1730 0x7f5c88073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 --2- 172.21.15.116:0/2686866764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c8806f450 0x7f5c880729c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 >> 172.21.15.116:0/2686866764 conn(0x7f5c880b4720 msgr2=0x7f5c8815ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 shutdown_connections 2024-05-14T11:18:06.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.156+0000 7f5c90800640 1 -- 172.21.15.116:0/2686866764 wait complete. 2024-05-14T11:18:06.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:06.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.320+0000 7fc2a1000640 1 Processor -- start 2024-05-14T11:18:06.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.320+0000 7fc2a1000640 1 -- start start 2024-05-14T11:18:06.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc2a1000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 0x7fc29c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc2a1000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc29c06c700 0x7fc29c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc2a1000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c06ff20 con 0x7fc29c15db30 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc2a1000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c0ae100 con 0x7fc29c06c700 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc2a1000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c0ae870 con 0x7fc29c0d1670 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc29c06c700 0x7fc29c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc299e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc29c15db30 0x7fc29c0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37464/0 (socket says 172.21.15.116:37464) 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc299e00640 1 -- 172.21.15.116:0/1148741037 learned_addr learned my addr 172.21.15.116:0/1148741037 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:06.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc29c06c700 0x7fc29c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58364/0 (socket says 172.21.15.116:58364) 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 -- 172.21.15.116:0/1148741037 >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 msgr2=0x7fc29c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc299400640 1 --2- 172.21.15.116:0/1148741037 >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 0x7fc29c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 --2- 172.21.15.116:0/1148741037 >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 0x7fc29c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc298a00640 1 -- 172.21.15.116:0/1148741037 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013359719 0 0) 0x7fc29c06ff20 con 0x7fc29c15db30 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 -- 172.21.15.116:0/1148741037 >> v1:172.21.15.116:6789/0 conn(0x7fc29c15db30 legacy=0x7fc29c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc29a800640 1 -- 172.21.15.116:0/1148741037 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c0d1fc0 con 0x7fc29c06c700 2024-05-14T11:18:06.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.324+0000 7fc299400640 1 --2- 172.21.15.116:0/1148741037 >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 0x7fc29c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc29a800640 1 --2- 172.21.15.116:0/1148741037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c06c700 0x7fc29c06cb10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0d2370 tx=0x7fc29401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff98616fdbec52b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc298a00640 1 -- 172.21.15.116:0/1148741037 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940b0070 con 0x7fc29c06c700 2024-05-14T11:18:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc298a00640 1 -- 172.21.15.116:0/1148741037 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2940ab600 con 0x7fc29c06c700 2024-05-14T11:18:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc298a00640 1 -- 172.21.15.116:0/1148741037 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940ab9d0 con 0x7fc29c06c700 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 -- 172.21.15.116:0/1148741037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c06c700 msgr2=0x7fc29c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1148741037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c06c700 0x7fc29c06cb10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0d2370 tx=0x7fc29401c320 comp rx=0 tx=0).stop 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 -- 172.21.15.116:0/1148741037 shutdown_connections 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1148741037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c06c700 0x7fc29c06cb10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1148741037 >> v2:172.21.15.149:3300/0 conn(0x7fc29c0d1670 0x7fc29c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 -- 172.21.15.116:0/1148741037 >> 172.21.15.116:0/1148741037 conn(0x7fc29c0b4620 msgr2=0x7fc29c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 -- 172.21.15.116:0/1148741037 shutdown_connections 2024-05-14T11:18:06.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.328+0000 7fc2a1000640 1 -- 172.21.15.116:0/1148741037 wait complete. 2024-05-14T11:18:06.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 Processor -- start 2024-05-14T11:18:06.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- start start 2024-05-14T11:18:06.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 0x7fc29c09dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29c15db30 0x7fc29c14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c16a770 con 0x7fc29c0d1670 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c15df40 con 0x7fc29c15db30 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c16a8f0 con 0x7fc29c06c700 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58372/0 (socket says 172.21.15.116:58372) 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 -- 172.21.15.116:0/1543924768 learned_addr learned my addr 172.21.15.116:0/1543924768 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc29a800640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29c15db30 0x7fc29c14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299e00640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 0x7fc29c09dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 -- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 msgr2=0x7fc29c09dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 0x7fc29c09dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 -- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29c15db30 msgr2=0x7fc29c14d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29c15db30 0x7fc29c14d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 -- 172.21.15.116:0/1543924768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c159530 con 0x7fc29c0d1670 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299e00640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 0x7fc29c09dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc299400640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c7f0 tx=0x7fc28800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288015070 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc288007820 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c09e280 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288007c10 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.332+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c09b600 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c06a9c0 con 0x7fc29c0d1670 2024-05-14T11:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc288018070 con 0x7fc29c0d1670 2024-05-14T11:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc292000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 0x7fc27c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fc28808f4d0 con 0x7fc29c0d1670 2024-05-14T11:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc299e00640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 0x7fc27c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.336+0000 7fc299e00640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 0x7fc27c071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc28c007fd0 tx=0x7fc28c005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:06.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.340+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc28805bbe0 con 0x7fc29c0d1670 2024-05-14T11:18:06.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.532+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc29c06f290 con 0x7fc29c0d1670 2024-05-14T11:18:06.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc292000640 1 -- 172.21.15.116:0/1543924768 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7fc28805fee0 con 0x7fc29c0d1670 2024-05-14T11:18:06.537 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:06.537 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:06.537 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 msgr2=0x7fc27c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 0x7fc27c071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc28c007fd0 tx=0x7fc28c005b80 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 msgr2=0x7fc29c14d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c7f0 tx=0x7fc28800ccb0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 shutdown_connections 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc27c06f400 0x7fc27c071910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc29c15db30 0x7fc29c14d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc29c0d1670 0x7fc29c14d2f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 --2- 172.21.15.116:0/1543924768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc29c06c700 0x7fc29c09dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:06.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 >> 172.21.15.116:0/1543924768 conn(0x7fc29c0b4620 msgr2=0x7fc29c0b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:06.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 shutdown_connections 2024-05-14T11:18:06.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:06.536+0000 7fc2a1000640 1 -- 172.21.15.116:0/1543924768 wait complete. 2024-05-14T11:18:06.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:17:56.025284+0000', 'modified': '2024-05-14T11:17:59.049985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5319': {'gid': 5319, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6837/2735194331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 2735194331}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 2735194331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5319, 'qdb_cluster': [5319]} 2024-05-14T11:18:06.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:18:06.961 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:18:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 Processor -- start 2024-05-14T11:18:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 -- start start 2024-05-14T11:18:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 0x7f1d640d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640d1fc0 con 0x7f1d64156fa0 2024-05-14T11:18:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1d640706f0 con 0x7f1d640d1670 2024-05-14T11:18:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.108+0000 7f1d69e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1d641573b0 con 0x7f1d640af380 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d5a000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 0x7f1d640d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d5a000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 0x7f1d640d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:53444/0 (socket says 172.21.15.116:53444) 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d62a00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f1d64156fa0 0x7f1d6406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54122/0 (socket says 172.21.15.116:54122) 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:47442/0 (socket says 172.21.15.116:47442) 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d5a000640 1 -- 172.21.15.116:0/3198429232 learned_addr learned my addr 172.21.15.116:0/3198429232 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 -- 172.21.15.116:0/3198429232 >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 msgr2=0x7f1d640d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 --2- 172.21.15.116:0/3198429232 >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 0x7f1d640d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 -- 172.21.15.116:0/3198429232 >> v1:172.21.15.149:6789/0 conn(0x7f1d64156fa0 legacy=0x7f1d6406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 -- 172.21.15.116:0/3198429232 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64160510 con 0x7f1d640af380 2024-05-14T11:18:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.112+0000 7f1d63400640 1 --2- 172.21.15.116:0/3198429232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009d90 tx=0x7f1d5c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd73f419b9a9282 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d62000640 1 -- 172.21.15.116:0/3198429232 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0b2070 con 0x7f1d640af380 2024-05-14T11:18:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d62000640 1 -- 172.21.15.116:0/3198429232 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c0a5920 con 0x7f1d640af380 2024-05-14T11:18:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d62000640 1 -- 172.21.15.116:0/3198429232 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0a5cf0 con 0x7f1d640af380 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- 172.21.15.116:0/3198429232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 msgr2=0x7f1d640af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 --2- 172.21.15.116:0/3198429232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009d90 tx=0x7f1d5c01c970 comp rx=0 tx=0).stop 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- 172.21.15.116:0/3198429232 shutdown_connections 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 --2- 172.21.15.116:0/3198429232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d640af790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 --2- 172.21.15.116:0/3198429232 >> v2:172.21.15.149:3300/0 conn(0x7f1d640d1670 0x7f1d640d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- 172.21.15.116:0/3198429232 >> 172.21.15.116:0/3198429232 conn(0x7f1d640b4620 msgr2=0x7f1d640b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- 172.21.15.116:0/3198429232 shutdown_connections 2024-05-14T11:18:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- 172.21.15.116:0/3198429232 wait complete. 2024-05-14T11:18:07.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 Processor -- start 2024-05-14T11:18:07.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.116+0000 7f1d69e00640 1 -- start start 2024-05-14T11:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d64141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d640d1670 0x7f1d6415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b0660 con 0x7f1d64156fa0 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b04e0 con 0x7f1d640d1670 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1d64169810 con 0x7f1d640af380 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58374/0 (socket says 172.21.15.116:58374) 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 -- 172.21.15.116:0/4293802309 learned_addr learned my addr 172.21.15.116:0/4293802309 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d5a000640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d640d1670 0x7f1d6415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d64141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 -- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 msgr2=0x7f1d64141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d64141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 -- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d640d1670 msgr2=0x7f1d6415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d640d1670 0x7f1d6415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 -- 172.21.15.116:0/4293802309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64141d90 con 0x7f1d64156fa0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d63400640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0b0040 tx=0x7f1d5c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d5be00640 1 -- 172.21.15.116:0/4293802309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0b2050 con 0x7f1d64156fa0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d5be00640 1 -- 172.21.15.116:0/4293802309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c0be070 con 0x7f1d64156fa0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d64155530 con 0x7f1d64156fa0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d5be00640 1 -- 172.21.15.116:0/4293802309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0ada70 con 0x7f1d64156fa0 2024-05-14T11:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.120+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d64155af0 con 0x7f1d64156fa0 2024-05-14T11:18:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d5be00640 1 -- 172.21.15.116:0/4293802309 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d5c0a57a0 con 0x7f1d64156fa0 2024-05-14T11:18:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d5be00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 0x7f1d40071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d5be00640 1 -- 172.21.15.116:0/4293802309 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f1d5c12a4b0 con 0x7f1d64156fa0 2024-05-14T11:18:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 0x7f1d40071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d71469000 1 Processor -- start 2024-05-14T11:18:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d71469000 1 -- start start 2024-05-14T11:18:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.124+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 0x7f1d40071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000f70 tx=0x7f1d54007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x56437656f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x56437656fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56437640c620 con 0x56437641df80 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564376327740 con 0x5643764076b0 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x564376375090 con 0x5643760357c0 2024-05-14T11:18:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58390/0 (socket says 172.21.15.116:58390) 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 -- 172.21.15.116:0/3718030990 learned_addr learned my addr 172.21.15.116:0/3718030990 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d62a00640 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x56437656f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d63400640 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x56437656fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 -- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 msgr2=0x56437656f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x56437656f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 -- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 msgr2=0x56437656fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x56437656fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 -- 172.21.15.116:0/3718030990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643765090a0 con 0x56437641df80 2024-05-14T11:18:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d5a000640 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009b50 tx=0x7f1d4c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acbc821268c254 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d53400640 1 -- 172.21.15.116:0/3718030990 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c009e50 con 0x56437641df80 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d53400640 1 -- 172.21.15.116:0/3718030990 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c004020 con 0x56437641df80 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d53400640 1 -- 172.21.15.116:0/3718030990 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0043d0 con 0x56437641df80 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- 172.21.15.116:0/3718030990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 msgr2=0x56437656f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009b50 tx=0x7f1d4c011690 comp rx=0 tx=0).stop 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- 172.21.15.116:0/3718030990 shutdown_connections 2024-05-14T11:18:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x56437656fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56437641df80 0x56437656f860 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 --2- 172.21.15.116:0/3718030990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x56437656f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- 172.21.15.116:0/3718030990 >> 172.21.15.116:0/3718030990 conn(0x5643764f4340 msgr2=0x564376523d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- 172.21.15.116:0/3718030990 shutdown_connections 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.128+0000 7f1d71469000 1 -- 172.21.15.116:0/3718030990 wait complete. 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- start start 2024-05-14T11:18:07.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 Processor -- start 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x564376575ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 0x564376553720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564376371050 con 0x5643760357c0 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56437632dec0 con 0x5643764076b0 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x564376314c90 con 0x56437641df80 2024-05-14T11:18:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 10 client.0 _ll_register_callbacks cb 0x564376510e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58406/0 (socket says 172.21.15.116:58406) 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 -- 172.21.15.116:0/351379887 learned_addr learned my addr 172.21.15.116:0/351379887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d63400640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 0x564376553720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d5a000640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x564376575ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 -- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 msgr2=0x564376553720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 0x564376553720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 -- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 msgr2=0x564376575ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x564376575ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 -- 172.21.15.116:0/351379887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437641f6f0 con 0x5643760357c0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d63400640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 0x564376553720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d62a00640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d540136a0 tx=0x7f1d54000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfa146322bc4372 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d47e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54020580 con 0x5643760357c0 2024-05-14T11:18:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f1d54020720 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d4c004020 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d5402e050 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 10 client.5457 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.132+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x56437641b6a0 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c004020 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f1d54031020 con 0x5643760357c0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 10 client.5457 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 10 client.5457 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 20 client.5457 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x56437601a7d0 0x564376554490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 4 client.5457 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56437630dbe0 con 0x56437601a7d0 2024-05-14T11:18:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d5a000640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x56437601a7d0 0x564376554490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d5a000640 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x56437601a7d0 0x564376554490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d223a4f4fd80df server_cookie=df7a595c351b9ff6 in_seq=0 out_seq=0 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 10 client.5457 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 1 -- 172.21.15.116:0/351379887 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56437630dbe0 con 0x56437601a7d0 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 10 client.5457 handle_command_reply: tid=0 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 client.5457 shutdown 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 msgr2=0x564376575980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1d540136a0 tx=0x7f1d54000bd0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 shutdown_connections 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x56437601a7d0 0x564376554490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643764076b0 0x564376575ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56437641df80 0x564376553720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 --2- 172.21.15.116:0/351379887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x564376575980 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d47e00640 0 client.5457 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 >> 172.21.15.116:0/351379887 conn(0x5643764f4340 msgr2=0x564376524490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 shutdown_connections 2024-05-14T11:18:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 1 -- 172.21.15.116:0/351379887 wait complete. 2024-05-14T11:18:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.136+0000 7f1d71469000 20 client.5457 trim_cache size 0 max 16384 2024-05-14T11:18:07.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 Processor -- start 2024-05-14T11:18:07.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 -- start start 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643764076b0 0x5643765dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x5643765ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d642050f0 con 0x5643760357c0 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640748e0 con 0x56437641df80 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1d64204aa0 con 0x5643764076b0 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58414/0 (socket says 172.21.15.116:58414) 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 -- 172.21.15.116:0/473410356 learned_addr learned my addr 172.21.15.116:0/473410356 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d63400640 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x5643765ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d5a000640 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643764076b0 0x5643765dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 -- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643764076b0 msgr2=0x5643765dda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643764076b0 0x5643765dda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 -- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 msgr2=0x5643765ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x5643765ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 -- 172.21.15.116:0/473410356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56437634eaf0 con 0x5643760357c0 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d62a00640 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1d54001600 tx=0x7f1d54021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ce86376463fe53 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d52a00640 1 -- 172.21.15.116:0/473410356 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54020d20 con 0x5643760357c0 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d52a00640 1 -- 172.21.15.116:0/473410356 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54026070 con 0x5643760357c0 2024-05-14T11:18:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.156+0000 7f1d52a00640 1 -- 172.21.15.116:0/473410356 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d540207c0 con 0x5643760357c0 2024-05-14T11:18:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- 172.21.15.116:0/473410356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 msgr2=0x5643765dd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1d54001600 tx=0x7f1d54021000 comp rx=0 tx=0).stop 2024-05-14T11:18:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- 172.21.15.116:0/473410356 shutdown_connections 2024-05-14T11:18:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x5643765ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643764076b0 0x5643765dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- 172.21.15.116:0/473410356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643760357c0 0x5643765dd4c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- 172.21.15.116:0/473410356 >> 172.21.15.116:0/473410356 conn(0x5643764f4340 msgr2=0x56437657b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- 172.21.15.116:0/473410356 shutdown_connections 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- 172.21.15.116:0/473410356 wait complete. 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- start start 2024-05-14T11:18:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 Processor -- start 2024-05-14T11:18:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x564376573960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x56437659e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564376373f80 con 0x5643764076b0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643763796c0 con 0x56437641df80 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56437637ec30 con 0x5643760357c0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d71469000 10 client.0 _ll_register_callbacks cb 0x7f1d54007690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58420/0 (socket says 172.21.15.116:58420) 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 -- 172.21.15.116:0/4289321298 learned_addr learned my addr 172.21.15.116:0/4289321298 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d63400640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x56437659e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x564376573960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 -- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 msgr2=0x564376573960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x564376573960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 -- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 msgr2=0x56437659e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x56437659e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.160+0000 7f1d5a000640 1 -- 172.21.15.116:0/4289321298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437634de40 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d5a000640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0090b0 tx=0x7f1d4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e9ee00686b760a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c02b5f0 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f1d4c02b790 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54026070 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d4c004500 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 10 client.5463 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5643763f54e0 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54026070 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f1d4c003620 con 0x5643764076b0 2024-05-14T11:18:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 10 client.5463 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 10 client.5463 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 20 client.5463 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x564376008830 0x56437659e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 4 client.5463 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "status"}) -- 0x564376382ca0 con 0x564376008830 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x564376008830 0x56437659e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d62a00640 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x564376008830 0x56437659e9c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f361573b29efe63 server_cookie=523762aa1babf4 in_seq=0 out_seq=0 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 10 client.5463 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 1 -- 172.21.15.116:0/4289321298 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564376382ca0 con 0x564376008830 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 10 client.5463 handle_command_reply: tid=0 2024-05-14T11:18:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 client.5463 shutdown 2024-05-14T11:18:07.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 msgr2=0x564376573ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0090b0 tx=0x7f1d4c014000 comp rx=0 tx=0).stop 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 shutdown_connections 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x564376008830 0x56437659e9c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56437641df80 0x56437659e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643764076b0 0x564376573ea0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 --2- 172.21.15.116:0/4289321298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5643760357c0 0x564376573960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d47e00640 0 client.5463 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 >> 172.21.15.116:0/4289321298 conn(0x5643764f4340 msgr2=0x56437657b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 shutdown_connections 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 1 -- 172.21.15.116:0/4289321298 wait complete. 2024-05-14T11:18:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d71469000 20 client.5463 trim_cache size 0 max 16384 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5319, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 10.531188755000001, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 26, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 49, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 49, 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 15.363571521000001 2024-05-14T11:18:07.173 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 msgr2=0x7f1d40071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 0x7f1d40071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000f70 tx=0x7f1d54007e50 comp rx=0 tx=0).stop 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 msgr2=0x7f1d6415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0b0040 tx=0x7f1d5c019710 comp rx=0 tx=0).stop 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 shutdown_connections 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1d4006f400 0x7f1d40071910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d640d1670 0x7f1d6415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d64156fa0 0x7f1d6415be90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 --2- 172.21.15.116:0/4293802309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1d640af380 0x7f1d64141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 >> 172.21.15.116:0/4293802309 conn(0x7f1d640b4620 msgr2=0x7f1d640b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 shutdown_connections 2024-05-14T11:18:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.164+0000 7f1d69e00640 1 -- 172.21.15.116:0/4293802309 wait complete. 2024-05-14T11:18:07.187 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:18:07.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 Processor -- start 2024-05-14T11:18:07.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 -- start start 2024-05-14T11:18:07.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f36600d1670 0x7f36600d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366006ff20 con 0x7f366015db30 2024-05-14T11:18:07.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36600d1fc0 con 0x7f366006c700 2024-05-14T11:18:07.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3668400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f36600ae100 con 0x7f36600d1670 2024-05-14T11:18:07.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3665a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f366015db30 0x7f36600aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37468/0 (socket says 172.21.15.116:37468) 2024-05-14T11:18:07.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3665a00640 1 -- 172.21.15.116:0/379444593 learned_addr learned my addr 172.21.15.116:0/379444593 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:07.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3665000640 1 --2- 172.21.15.116:0/379444593 >> v2:172.21.15.149:3300/0 conn(0x7f36600d1670 0x7f36600d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223733368 0 0) 0x7f366006ff20 con 0x7f366015db30 2024-05-14T11:18:07.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3648003530 con 0x7f366015db30 2024-05-14T11:18:07.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729733407 0 0) 0x7f36580014a0 con 0x7f366006c700 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f366006ff20 con 0x7f366006c700 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 490130672 0 0) 0x7f3648003530 con 0x7f366015db30 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 >> v2:172.21.15.149:3300/0 conn(0x7f36600d1670 msgr2=0x7f36600d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 --2- 172.21.15.116:0/379444593 >> v2:172.21.15.149:3300/0 conn(0x7f36600d1670 0x7f36600d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 >> v1:172.21.15.149:6789/0 conn(0x7f366006c700 legacy=0x7f366006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3660070470 con 0x7f366015db30 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f3654001700 con 0x7f366015db30 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 >> v1:172.21.15.116:6789/0 conn(0x7f366015db30 legacy=0x7f36600aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3648003c40 0x7f36480040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648004810 0x7f3648004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 0x7f364800baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36600ae100 con 0x7f36480053f0 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f364800c380 con 0x7f3648004810 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f364800c580 con 0x7f3648003c40 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3666400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 0x7f364800baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3665000640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648004810 0x7f3648004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3665a00640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3648003c40 0x7f36480040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3666400640 1 -- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3648003c40 msgr2=0x7f36480040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.332+0000 7f3666400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3648003c40 0x7f36480040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3666400640 1 -- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648004810 msgr2=0x7f3648004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3666400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3648004810 0x7f3648004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3666400640 1 -- 172.21.15.116:0/379444593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3648003530 con 0x7f36480053f0 2024-05-14T11:18:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3666400640 1 -- 172.21.15.116:0/379444593 reap_dead start 2024-05-14T11:18:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3666400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 0x7f364800baa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f365800c0e0 tx=0x7f365801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15b51585f2b1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658001e20 con 0x7f36480053f0 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3658003b80 con 0x7f36480053f0 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f365fe00640 1 -- 172.21.15.116:0/379444593 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580050d0 con 0x7f36480053f0 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 -- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 msgr2=0x7f364800baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 0x7f364800baa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f365800c0e0 tx=0x7f365801c8e0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 -- 172.21.15.116:0/379444593 shutdown_connections 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 --2- 172.21.15.116:0/379444593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36480053f0 0x7f364800baa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 -- 172.21.15.116:0/379444593 >> 172.21.15.116:0/379444593 conn(0x7f36600b4620 msgr2=0x7f36600b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.336+0000 7f3668400640 1 -- 172.21.15.116:0/379444593 shutdown_connections 2024-05-14T11:18:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 -- 172.21.15.116:0/379444593 wait complete. 2024-05-14T11:18:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 Processor -- start 2024-05-14T11:18:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 -- start start 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36480053f0 0x7f366014a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3660072e70 0x7f36601529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36600d1a80 con 0x7f36600746f0 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f366015df40 con 0x7f36480053f0 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3668400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f36600d1c00 con 0x7f3660072e70 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3666400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3666400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58438/0 (socket says 172.21.15.116:58438) 2024-05-14T11:18:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.340+0000 7f3666400640 1 -- 172.21.15.116:0/3039325706 learned_addr learned my addr 172.21.15.116:0/3039325706 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 -- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3660072e70 msgr2=0x7f36601529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3660072e70 0x7f36601529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 -- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36480053f0 msgr2=0x7f366014a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36480053f0 0x7f366014a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 -- 172.21.15.116:0/3039325706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36601695a0 con 0x7f36600746f0 2024-05-14T11:18:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3666400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f365801d180 tx=0x7f3648005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658001e20 con 0x7f36600746f0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3658003980 con 0x7f36600746f0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3660073650 con 0x7f36600746f0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658007da0 con 0x7f36600746f0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3660073c10 con 0x7f36600746f0 2024-05-14T11:18:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36600733c0 con 0x7f36600746f0 2024-05-14T11:18:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.344+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f36580b9060 con 0x7f36600746f0 2024-05-14T11:18:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.348+0000 7f365d600640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 0x7f3640071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.348+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f3658127d60 con 0x7f36600746f0 2024-05-14T11:18:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.348+0000 7f3665a00640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 0x7f3640071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.348+0000 7f3665a00640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 0x7f3640071910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3654001b00 tx=0x7f3654001a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:07.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.352+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f36600d1fc0 con 0x7f36600746f0 2024-05-14T11:18:07.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.544+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f366006a9c0 con 0x7f36600746f0 2024-05-14T11:18:07.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.544+0000 7f365d600640 1 -- 172.21.15.116:0/3039325706 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f36580f4a20 con 0x7f36600746f0 2024-05-14T11:18:07.548 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 msgr2=0x7f3640071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 0x7f3640071910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3654001b00 tx=0x7f3654001a70 comp rx=0 tx=0).stop 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 msgr2=0x7f3660152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f365801d180 tx=0x7f3648005840 comp rx=0 tx=0).stop 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 shutdown_connections 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f364006f400 0x7f3640071910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36600746f0 0x7f3660152f20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3660072e70 0x7f36601529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 --2- 172.21.15.116:0/3039325706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36480053f0 0x7f366014a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 >> 172.21.15.116:0/3039325706 conn(0x7f36600b4620 msgr2=0x7f3660157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 shutdown_connections 2024-05-14T11:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07.548+0000 7f3668400640 1 -- 172.21.15.116:0/3039325706 wait complete. 2024-05-14T11:18:07.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:18:07.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:18:07.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:18:07.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:07.570 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:18:07.580 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42315sec preferred_lft 42315sec 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:18:07.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:18:07.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:07.583 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:18:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:18:07.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:07.640 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:18:07.695 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:18:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:18:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:18:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:18:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:18:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:18:07.696 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:18:07.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:18:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:18:07.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:07.830 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:18:07.885 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:18:07.885 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:18:07.885 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:18:07.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:07.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:18:07.948 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:18:07.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:18:07.948 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:08.000 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:18:08.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:18:08.000 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:08.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:18:08.116 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:18:08.165 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:18:08.165 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:18:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:18:08.167 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:18:08.168 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:18:08.168 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:18:08.168 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:18:08.168 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:18:08.168 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:18:08.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.169 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:18:08.220 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:18:08.220 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:18:08.267 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:18:08.288 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:18:08.342 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:18:08.342 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:18:08.343 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:18:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:18:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:18:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:18:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:18:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:18:08.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.346 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:18:08.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:18:08.368+0000 7f2ef0eba580 -1 init, newargv = 0x7f2ed8001260 newargc=15 2024-05-14T11:18:08.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[38527]: starting ceph client 2024-05-14T11:18:08.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[38527]: starting fuse 2024-05-14T11:18:08.381 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:18:08.381 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:18:08.382 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:18:08.383 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:18:08.384 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:18:08.384 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:18:08.384 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:18:08.384 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:18:08.504 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.38527.asok 2024-05-14T11:18:08.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:18:08.515 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38527.asok 2024-05-14T11:18:08.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.515 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38527.asok status 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:18:08.696 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "38527", 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5475, 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 5475 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 51972309 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:08.697 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 51972309 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.5475 192.168.0.1:0/51972309", 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/51972309", 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 26, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 49, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:18:08.698 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.711 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:18:08.712 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:08.721 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:18:08.721 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:08.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.721 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:08.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.784 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-14T11:18:08.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:08.843 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-14T11:18:08.900 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:18:09.007 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:18:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:18:09.016 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:09.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 Processor -- start 2024-05-14T11:18:09.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 -- start start 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f84480b4760 0x7f84480ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84480b2fb0 con 0x7f84480cdf80 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84480b3290 con 0x7f84480d1c20 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844ec00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f84480b3570 con 0x7f84480b4760 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8447e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84480cdf80 0x7f84480d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44338/0 (socket says 172.21.15.116:44338) 2024-05-14T11:18:09.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8447e00640 1 -- 172.21.15.116:0/4075739916 learned_addr learned my addr 172.21.15.116:0/4075739916 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:09.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844cc00640 1 --2- 172.21.15.116:0/4075739916 >> v2:172.21.15.149:3300/0 conn(0x7f84480b4760 0x7f84480ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254219243 0 0) 0x7f84480b2fb0 con 0x7f84480cdf80 2024-05-14T11:18:09.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f842c003530 con 0x7f84480cdf80 2024-05-14T11:18:09.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2168044585 0 0) 0x7f842c003530 con 0x7f84480cdf80 2024-05-14T11:18:09.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 >> v2:172.21.15.149:3300/0 conn(0x7f84480b4760 msgr2=0x7f84480ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 --2- 172.21.15.116:0/4075739916 >> v2:172.21.15.149:3300/0 conn(0x7f84480b4760 0x7f84480ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 >> v1:172.21.15.149:6789/0 conn(0x7f84480d1c20 legacy=0x7f84480b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84480b3850 con 0x7f84480cdf80 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f8438001a80 con 0x7f84480cdf80 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f844cc00640 1 --2- 172.21.15.116:0/4075739916 >> v2:172.21.15.149:3300/0 conn(0x7f84480b4760 0x7f84480ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.204+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 >> v1:172.21.15.116:6789/0 conn(0x7f84480cdf80 legacy=0x7f84480d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f842c003a60 0x7f842c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f842c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f842c005260 0x7f842c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f84480b3570 con 0x7f842c004810 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f842c00c1f0 con 0x7f842c003a60 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f842c00c3f0 con 0x7f842c005260 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f842c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447e00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f842c003a60 0x7f842c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f844cc00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f842c005260 0x7f842c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 -- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f842c005260 msgr2=0x7f842c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f842c005260 0x7f842c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 -- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f842c003a60 msgr2=0x7f842c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f842c003a60 0x7f842c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 -- 172.21.15.116:0/4075739916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c003530 con 0x7f842c004810 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447e00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f842c003a60 0x7f842c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:09.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f844cc00640 1 -- 172.21.15.116:0/4075739916 reap_dead start 2024-05-14T11:18:09.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8447400640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f842c004c20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f84480d35c0 tx=0x7f843401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722c9f76d681bac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84340013e0 con 0x7f842c004810 2024-05-14T11:18:09.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84340052d0 con 0x7f842c004810 2024-05-14T11:18:09.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.208+0000 7f8446a00640 1 -- 172.21.15.116:0/4075739916 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434003840 con 0x7f842c004810 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 msgr2=0x7f842c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f842c004c20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f84480d35c0 tx=0x7f843401c390 comp rx=0 tx=0).stop 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- 172.21.15.116:0/4075739916 shutdown_connections 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 --2- 172.21.15.116:0/4075739916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f842c004c20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- 172.21.15.116:0/4075739916 >> 172.21.15.116:0/4075739916 conn(0x7f84480d0770 msgr2=0x7f84480d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- 172.21.15.116:0/4075739916 shutdown_connections 2024-05-14T11:18:09.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- 172.21.15.116:0/4075739916 wait complete. 2024-05-14T11:18:09.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 Processor -- start 2024-05-14T11:18:09.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.212+0000 7f844ec00640 1 -- start start 2024-05-14T11:18:09.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84480ca7f0 0x7f84480cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f844815b360 0x7f844815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f84480ce510 con 0x7f842c004810 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84480ce390 con 0x7f84480ca7f0 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f84480ce690 con 0x7f844815b360 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47684/0 (socket says 172.21.15.116:47684) 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 -- 172.21.15.116:0/2888523993 learned_addr learned my addr 172.21.15.116:0/2888523993 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844cc00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f844815b360 0x7f844815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447400640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84480ca7f0 0x7f84480cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 -- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f844815b360 msgr2=0x7f844815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f844815b360 0x7f844815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 -- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84480ca7f0 msgr2=0x7f84480cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84480ca7f0 0x7f84480cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 -- 172.21.15.116:0/2888523993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844815bcb0 con 0x7f842c004810 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f8447e00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f84380015a0 tx=0x7f84380126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843800a4e0 con 0x7f842c004810 2024-05-14T11:18:09.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8438016440 con 0x7f842c004810 2024-05-14T11:18:09.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380167f0 con 0x7f842c004810 2024-05-14T11:18:09.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844814f4b0 con 0x7f842c004810 2024-05-14T11:18:09.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84480b6420 con 0x7f842c004810 2024-05-14T11:18:09.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.216+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844814f680 con 0x7f842c004810 2024-05-14T11:18:09.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.220+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f843801e020 con 0x7f842c004810 2024-05-14T11:18:09.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.220+0000 7f843fe00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 0x7f8428071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.220+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f8438092de0 con 0x7f842c004810 2024-05-14T11:18:09.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.220+0000 7f8447400640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 0x7f8428071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.220+0000 7f8447400640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 0x7f8428071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84340ac000 tx=0x7f84340011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.228+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f844815bcb0 con 0x7f842c004810 2024-05-14T11:18:09.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.416+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f844806a9c0 con 0x7f842c004810 2024-05-14T11:18:09.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.416+0000 7f843fe00640 1 -- 172.21.15.116:0/2888523993 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f843805f570 con 0x7f842c004810 2024-05-14T11:18:09.420 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:09.420 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:09.420 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 msgr2=0x7f8428071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 0x7f8428071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84340ac000 tx=0x7f84340011a0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 msgr2=0x7f84480cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f84380015a0 tx=0x7f84380126f0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 shutdown_connections 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f842806f400 0x7f8428071910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84480ca7f0 0x7f84480cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f844815b360 0x7f844815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 --2- 172.21.15.116:0/2888523993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842c004810 0x7f84480cce90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 >> 172.21.15.116:0/2888523993 conn(0x7f84480d0770 msgr2=0x7f84481588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 shutdown_connections 2024-05-14T11:18:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.420+0000 7f844ec00640 1 -- 172.21.15.116:0/2888523993 wait complete. 2024-05-14T11:18:09.441 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-14T11:18:09.619 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:18:09.619 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:18:09.620 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-14T11:18:09.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 Processor -- start 2024-05-14T11:18:09.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- start start 2024-05-14T11:18:09.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d1ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d123020 0x55930d1aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55930d1212d0 con 0x55930d10aa20 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55930d123970 con 0x55930d1221f0 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55930d121570 con 0x55930d123020 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47696/0 (socket says 172.21.15.116:47696) 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 -- 172.21.15.116:0/121719370 learned_addr learned my addr 172.21.15.116:0/121719370 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f3c00640 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d1ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f5000640 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d123020 0x55930d1aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 -- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d123020 msgr2=0x55930d1aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d123020 0x55930d1aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 -- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 msgr2=0x55930d1ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d1ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 -- 172.21.15.116:0/121719370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930d0191c0 con 0x55930d10aa20 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f4600640 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009850 tx=0x7f49e4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c64a6999672e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f2800640 1 -- 172.21.15.116:0/121719370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4019070 con 0x55930d10aa20 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f2800640 1 -- 172.21.15.116:0/121719370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4008da0 con 0x55930d10aa20 2024-05-14T11:18:09.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49f2800640 1 -- 172.21.15.116:0/121719370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40146a0 con 0x55930d10aa20 2024-05-14T11:18:09.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- 172.21.15.116:0/121719370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 msgr2=0x55930d1b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009850 tx=0x7f49e4005a10 comp rx=0 tx=0).stop 2024-05-14T11:18:09.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- 172.21.15.116:0/121719370 shutdown_connections 2024-05-14T11:18:09.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d1ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d123020 0x55930d1aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/121719370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d10aa20 0x55930d1b22d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- 172.21.15.116:0/121719370 >> 172.21.15.116:0/121719370 conn(0x55930d0c5e10 msgr2=0x55930d0dcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- 172.21.15.116:0/121719370 shutdown_connections 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.676+0000 7f49fb4b8080 1 -- 172.21.15.116:0/121719370 wait complete. 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 Processor -- start 2024-05-14T11:18:09.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 -- start start 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d10aa20 0x55930d1a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d11cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 0x55930d11f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55930d11a310 con 0x55930d123020 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55930d1b1790 con 0x55930d1221f0 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55930d1aa2b0 con 0x55930d10aa20 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 0x55930d11f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f4600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d10aa20 0x55930d1a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 0x55930d11f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47708/0 (socket says 172.21.15.116:47708) 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 -- 172.21.15.116:0/2641776925 learned_addr learned my addr 172.21.15.116:0/2641776925 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:09.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f3c00640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d11cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 -- 172.21.15.116:0/2641776925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d10aa20 msgr2=0x55930d1a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55930d10aa20 0x55930d1a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 -- 172.21.15.116:0/2641776925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 msgr2=0x55930d11cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55930d1221f0 0x55930d11cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 -- 172.21.15.116:0/2641776925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930d11d390 con 0x55930d123020 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f5000640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 0x55930d11f070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00f5f0 tx=0x7f49ec00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f1400640 1 -- 172.21.15.116:0/2641776925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec019020 con 0x55930d123020 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f1400640 1 -- 172.21.15.116:0/2641776925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec015690 con 0x55930d123020 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55930d11d630 con 0x55930d123020 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.680+0000 7f49f1400640 1 -- 172.21.15.116:0/2641776925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec015a60 con 0x55930d123020 2024-05-14T11:18:09.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930d11db70 con 0x55930d123020 2024-05-14T11:18:09.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49f1400640 1 -- 172.21.15.116:0/2641776925 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f49ec01c020 con 0x55930d123020 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49f1400640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f49d406dde0 0x7f49d40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49f1400640 1 -- 172.21.15.116:0/2641776925 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f49ec08e530 con 0x55930d123020 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49f4600640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f49d406dde0 0x7f49d40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55930d1a70d0 0x55930d1a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) -- 0x55930d1a7a20 con 0x55930d1a70d0 2024-05-14T11:18:09.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.684+0000 7f49f3c00640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55930d1a70d0 0x55930d1a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49f4600640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f49d406dde0 0x7f49d40702f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49e40097f0 tx=0x7f49e4011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49f3c00640 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55930d1a70d0 0x55930d1a74e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49f3c00640 1 -- 172.21.15.116:0/2641776925 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f49e00a7ba0 con 0x55930d1a70d0 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55930d1a70d0 msgr2=0x55930d1a74e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55930d1a70d0 0x55930d1a74e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f49d406dde0 msgr2=0x7f49d40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f49d406dde0 0x7f49d40702f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49e40097f0 tx=0x7f49e4011000 comp rx=0 tx=0).stop 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 msgr2=0x55930d11f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 --2- 172.21.15.116:0/2641776925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55930d123020 0x55930d11f070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00f5f0 tx=0x7f49ec00fab0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49f5000640 1 -- 172.21.15.116:0/2641776925 reap_dead start 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 shutdown_connections 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 >> 172.21.15.116:0/2641776925 conn(0x55930d0c5e10 msgr2=0x55930d1ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 shutdown_connections 2024-05-14T11:18:09.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.688+0000 7f49fb4b8080 1 -- 172.21.15.116:0/2641776925 wait complete. 2024-05-14T11:18:09.694 INFO:teuthology.orchestra.run.smithi116.stdout:Yalphaparent_a 2024-05-14T11:18:09.695 DEBUG:teuthology.orchestra.run.smithi116:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-14T11:18:09.756 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: "ancestors": [ 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1099511627776, 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "alpha", 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 2 2024-05-14T11:18:09.769 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1, 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "parent_a", 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 4 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 8, 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: "old_pools": [] 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:09.770 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:09.771 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-14T11:18:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 Processor -- start 2024-05-14T11:18:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 -- start start 2024-05-14T11:18:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c4f64e0 0x56335c509150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c580750 0x56335c4f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56335c510750 con 0x56335c580340 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56335c469aa0 con 0x56335c580750 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f237d680080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56335c4c2da0 con 0x56335c4f64e0 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f2375e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f2375e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47722/0 (socket says 172.21.15.116:47722) 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.868+0000 7f2375e00640 1 -- 172.21.15.116:0/4009280974 learned_addr learned my addr 172.21.15.116:0/4009280974 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2377200640 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c580750 0x56335c4f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2376800640 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c4f64e0 0x56335c509150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 -- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c4f64e0 msgr2=0x56335c509150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c4f64e0 0x56335c509150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 -- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c580750 msgr2=0x56335c4f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c580750 0x56335c4f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 -- 172.21.15.116:0/4009280974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335c507090 con 0x56335c580340 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2375e00640 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f236400a7e0 tx=0x7f236400aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5557ddc6a47d83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2374a00640 1 -- 172.21.15.116:0/4009280974 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364018070 con 0x56335c580340 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2374a00640 1 -- 172.21.15.116:0/4009280974 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2364008c00 con 0x56335c580340 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f2374a00640 1 -- 172.21.15.116:0/4009280974 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364013430 con 0x56335c580340 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- 172.21.15.116:0/4009280974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 msgr2=0x56335c594fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f236400a7e0 tx=0x7f236400aca0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- 172.21.15.116:0/4009280974 shutdown_connections 2024-05-14T11:18:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c580750 0x56335c4f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580340 0x56335c594fb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 --2- 172.21.15.116:0/4009280974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c4f64e0 0x56335c509150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- 172.21.15.116:0/4009280974 >> 172.21.15.116:0/4009280974 conn(0x56335c4aee10 msgr2=0x56335c5077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- 172.21.15.116:0/4009280974 shutdown_connections 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- 172.21.15.116:0/4009280974 wait complete. 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 Processor -- start 2024-05-14T11:18:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.872+0000 7f237d680080 1 -- start start 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c4f64e0 0x56335c4e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c580340 0x56335c636830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 0x56335c633c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56335c5979b0 con 0x56335c580750 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56335c507330 con 0x56335c4f64e0 2024-05-14T11:18:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56335c597c50 con 0x56335c580340 2024-05-14T11:18:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 0x56335c633c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2376800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c4f64e0 0x56335c4e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 0x56335c633c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47730/0 (socket says 172.21.15.116:47730) 2024-05-14T11:18:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 -- 172.21.15.116:0/2281535525 learned_addr learned my addr 172.21.15.116:0/2281535525 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2375e00640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c580340 0x56335c636830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 -- 172.21.15.116:0/2281535525 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c580340 msgr2=0x56335c636830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56335c580340 0x56335c636830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 -- 172.21.15.116:0/2281535525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c4f64e0 msgr2=0x56335c4e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56335c4f64e0 0x56335c4e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 -- 172.21.15.116:0/2281535525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335c637510 con 0x56335c580750 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f2377200640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 0x56335c633c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f23700ad040 tx=0x7f2370017e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f236f400640 1 -- 172.21.15.116:0/2281535525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700ba020 con 0x56335c580750 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f236f400640 1 -- 172.21.15.116:0/2281535525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23700b5470 con 0x56335c580750 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56335c4f25d0 con 0x56335c580750 2024-05-14T11:18:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f236f400640 1 -- 172.21.15.116:0/2281535525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700af040 con 0x56335c580750 2024-05-14T11:18:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.876+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335c57c320 con 0x56335c580750 2024-05-14T11:18:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f236f400640 1 -- 172.21.15.116:0/2281535525 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23700bd020 con 0x56335c580750 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f236f400640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f235806dde0 0x7f23580702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f236f400640 1 -- 172.21.15.116:0/2281535525 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f2370128e40 con 0x56335c580750 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2376800640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f235806dde0 0x7f23580702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56335c636d70 0x56335c634690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49) -- 0x56335c597160 con 0x56335c636d70 2024-05-14T11:18:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2375e00640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56335c636d70 0x56335c634690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:09.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2376800640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f235806dde0 0x7f23580702f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2368001230 tx=0x7f23680016c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2375e00640 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56335c636d70 0x56335c634690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2375e00640 1 -- 172.21.15.116:0/2281535525 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v49'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f23640137c0 con 0x56335c636d70 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56335c636d70 msgr2=0x56335c634690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56335c636d70 0x56335c634690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f235806dde0 msgr2=0x7f23580702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f235806dde0 0x7f23580702f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2368001230 tx=0x7f23680016c0 comp rx=0 tx=0).stop 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 msgr2=0x56335c633c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 --2- 172.21.15.116:0/2281535525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56335c580750 0x56335c633c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f23700ad040 tx=0x7f2370017e60 comp rx=0 tx=0).stop 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f2377200640 1 -- 172.21.15.116:0/2281535525 reap_dead start 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 shutdown_connections 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 >> 172.21.15.116:0/2281535525 conn(0x56335c4aee10 msgr2=0x56335c4f3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 shutdown_connections 2024-05-14T11:18:09.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:09.880+0000 7f237d680080 1 -- 172.21.15.116:0/2281535525 wait complete. 2024-05-14T11:18:09.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-14T11:18:09.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:09.892 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-14T11:18:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.044+0000 7f7d97400640 1 Processor -- start 2024-05-14T11:18:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.044+0000 7f7d97400640 1 -- start start 2024-05-14T11:18:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d97400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d900d2ca0 0x7f7d90161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d97400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7d900d25f0 0x7f7d900b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d97400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d900b27a0 con 0x7f7d90161b50 2024-05-14T11:18:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d97400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7d900b3340 con 0x7f7d900d2ca0 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d97400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7d90161f60 con 0x7f7d900d25f0 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d900d2ca0 0x7f7d90161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d900d2ca0 0x7f7d90161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47736/0 (socket says 172.21.15.116:47736) 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 -- 172.21.15.116:0/4211584933 learned_addr learned my addr 172.21.15.116:0/4211584933 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d95400640 1 --2- 172.21.15.116:0/4211584933 >> v2:172.21.15.149:3300/0 conn(0x7f7d900d25f0 0x7f7d900b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 -- 172.21.15.116:0/4211584933 >> v2:172.21.15.149:3300/0 conn(0x7f7d900d25f0 msgr2=0x7f7d900b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 --2- 172.21.15.116:0/4211584933 >> v2:172.21.15.149:3300/0 conn(0x7f7d900d25f0 0x7f7d900b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 -- 172.21.15.116:0/4211584933 >> v1:172.21.15.149:6789/0 conn(0x7f7d90161b50 legacy=0x7f7d900b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.048+0000 7f7d94a00640 1 -- 172.21.15.116:0/4211584933 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900b2a10 con 0x7f7d900d2ca0 2024-05-14T11:18:10.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d94a00640 1 --2- 172.21.15.116:0/4211584933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d2ca0 0x7f7d90161610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d900b4cb0 tx=0x7f7d8401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f6da335303c9ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d8f400640 1 -- 172.21.15.116:0/4211584933 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b5070 con 0x7f7d900d2ca0 2024-05-14T11:18:10.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d8f400640 1 -- 172.21.15.116:0/4211584933 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840a89d0 con 0x7f7d900d2ca0 2024-05-14T11:18:10.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d8f400640 1 -- 172.21.15.116:0/4211584933 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840a8da0 con 0x7f7d900d2ca0 2024-05-14T11:18:10.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 -- 172.21.15.116:0/4211584933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d2ca0 msgr2=0x7f7d90161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 --2- 172.21.15.116:0/4211584933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d2ca0 0x7f7d90161610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d900b4cb0 tx=0x7f7d8401ed10 comp rx=0 tx=0).stop 2024-05-14T11:18:10.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 -- 172.21.15.116:0/4211584933 shutdown_connections 2024-05-14T11:18:10.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 --2- 172.21.15.116:0/4211584933 >> v2:172.21.15.149:3300/0 conn(0x7f7d900d25f0 0x7f7d900b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 --2- 172.21.15.116:0/4211584933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d2ca0 0x7f7d90161610 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 -- 172.21.15.116:0/4211584933 >> 172.21.15.116:0/4211584933 conn(0x7f7d900b56d0 msgr2=0x7f7d900d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 -- 172.21.15.116:0/4211584933 shutdown_connections 2024-05-14T11:18:10.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.052+0000 7f7d97400640 1 -- 172.21.15.116:0/4211584933 wait complete. 2024-05-14T11:18:10.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 Processor -- start 2024-05-14T11:18:10.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- start start 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 0x7f7d900c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 0x7f7d9014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900b64d0 con 0x7f7d900d25f0 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900b2fd0 con 0x7f7d90161b50 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900d30b0 con 0x7f7d900d2ca0 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47750/0 (socket says 172.21.15.116:47750) 2024-05-14T11:18:10.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 -- 172.21.15.116:0/2542510188 learned_addr learned my addr 172.21.15.116:0/2542510188 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d95400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 0x7f7d9014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 -- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 msgr2=0x7f7d900c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 0x7f7d900c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 0x7f7d900c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 -- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 msgr2=0x7f7d9014bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 0x7f7d9014bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 -- 172.21.15.116:0/2542510188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9014aee0 con 0x7f7d900d25f0 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d95400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 0x7f7d9014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 0x7f7d900c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:10.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b3040 tx=0x7f7d8401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d8cc00640 1 -- 172.21.15.116:0/2542510188 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840c0070 con 0x7f7d900d25f0 2024-05-14T11:18:10.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d8cc00640 1 -- 172.21.15.116:0/2542510188 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840b0820 con 0x7f7d900d25f0 2024-05-14T11:18:10.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9014b1c0 con 0x7f7d900d25f0 2024-05-14T11:18:10.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d8cc00640 1 -- 172.21.15.116:0/2542510188 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b5040 con 0x7f7d900d25f0 2024-05-14T11:18:10.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.056+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9014b780 con 0x7f7d900d25f0 2024-05-14T11:18:10.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d8cc00640 1 -- 172.21.15.116:0/2542510188 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d840c3070 con 0x7f7d900d25f0 2024-05-14T11:18:10.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d8cc00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 0x7f7d70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d8cc00640 1 -- 172.21.15.116:0/2542510188 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f7d8412c310 con 0x7f7d900d25f0 2024-05-14T11:18:10.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 0x7f7d70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d9e9a3000 1 Processor -- start 2024-05-14T11:18:10.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.060+0000 7f7d9e9a3000 1 -- start start 2024-05-14T11:18:10.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 0x7f7d70071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d900c9590 tx=0x7f7d7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7344e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7345380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f73458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5588f71e2960 con 0x5588f6dcb630 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5588f714a200 con 0x5588f71f42c0 2024-05-14T11:18:10.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5588f7102da0 con 0x5588f71dd9f0 2024-05-14T11:18:10.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d95400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f73458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d95400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f73458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47762/0 (socket says 172.21.15.116:47762) 2024-05-14T11:18:10.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7345380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d95400640 1 -- 172.21.15.116:0/3200053892 learned_addr learned my addr 172.21.15.116:0/3200053892 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d94a00640 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7344e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 -- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 msgr2=0x5588f7344e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7344e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 -- 172.21.15.116:0/3200053892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 msgr2=0x5588f73458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f73458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 -- 172.21.15.116:0/3200053892 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f7107900 con 0x5588f71dd9f0 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7345380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006fd0 tx=0x7f7d7c00f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1380eace9def8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d81200640 1 -- 172.21.15.116:0/3200053892 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004220 con 0x5588f71dd9f0 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d81200640 1 -- 172.21.15.116:0/3200053892 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c0043c0 con 0x5588f71dd9f0 2024-05-14T11:18:10.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d81200640 1 -- 172.21.15.116:0/3200053892 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c013bb0 con 0x5588f71dd9f0 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 msgr2=0x5588f7345380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.064+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7345380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006fd0 tx=0x7f7d7c00f000 comp rx=0 tx=0).stop 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/3200053892 shutdown_connections 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f73458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7345380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/3200053892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7344e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/3200053892 >> 172.21.15.116:0/3200053892 conn(0x5588f7201760 msgr2=0x5588f72f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/3200053892 shutdown_connections 2024-05-14T11:18:10.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/3200053892 wait complete. 2024-05-14T11:18:10.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 Processor -- start 2024-05-14T11:18:10.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 -- start start 2024-05-14T11:18:10.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- start start 2024-05-14T11:18:10.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 Processor -- start 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9d101407a0 0x7f9d1006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d100d0390 0x7f9d10143230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d1015ee20 con 0x7f9d10070310 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9d100d0dc0 con 0x7f9d100d0390 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1006e9b0 con 0x7f9d101407a0 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d100d0390 0x7f9d10143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0e000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d10070310 0x7f9d100cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44356/0 (socket says 172.21.15.116:44356) 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0e000640 1 -- 172.21.15.116:0/2326347100 learned_addr learned my addr 172.21.15.116:0/2326347100 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0ea00640 1 --2- 172.21.15.116:0/2326347100 >> v2:172.21.15.149:3300/0 conn(0x7f9d101407a0 0x7f9d1006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0f400640 1 -- 172.21.15.116:0/2326347100 >> v2:172.21.15.149:3300/0 conn(0x7f9d101407a0 msgr2=0x7f9d1006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0f400640 1 --2- 172.21.15.116:0/2326347100 >> v2:172.21.15.149:3300/0 conn(0x7f9d101407a0 0x7f9d1006fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0f400640 1 -- 172.21.15.116:0/2326347100 >> v1:172.21.15.116:6789/0 conn(0x7f9d10070310 legacy=0x7f9d100cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f9d0f400640 1 -- 172.21.15.116:0/2326347100 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1015e060 con 0x7f9d100d0390 2024-05-14T11:18:10.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7307270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f73077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5588f7124df0 con 0x5588f6dcb630 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5588f7106180 con 0x5588f71f42c0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5588f7149510 con 0x5588f71dd9f0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d9e9a3000 10 client.0 _ll_register_callbacks cb 0x5588f71fd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47780/0 (socket says 172.21.15.116:47780) 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 -- 172.21.15.116:0/2230948176 learned_addr learned my addr 172.21.15.116:0/2230948176 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7307270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d95400640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f73077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 -- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 msgr2=0x5588f7307270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7307270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 -- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 msgr2=0x5588f73077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.068+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f73077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d94a00640 1 -- 172.21.15.116:0/2230948176 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f70f4960 con 0x5588f6dcb630 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009630 tx=0x7f7d84000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76ce9944106525c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840f8dc0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7c0043c0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f7d841012a0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d8412a050 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 10 client.5520 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5588f7102ff0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c0043c0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f7d840f7bc0 con 0x5588f6dcb630 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 10 client.5520 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 10 client.5520 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 20 client.5520 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6defb80 0x5588f734be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 4 client.5520 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f70e2ef0 con 0x5588f6defb80 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6defb80 0x5588f734be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6defb80 0x5588f734be00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936581978cde2d server_cookie=dc8bb2293c902f36 in_seq=0 out_seq=0 2024-05-14T11:18:10.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 10 client.5520 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0f400640 1 --2- 172.21.15.116:0/2326347100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d100d0390 0x7f9d10143230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9d100723a0 tx=0x7f9d0801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ca28bdcd4b60fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0d600640 1 -- 172.21.15.116:0/2326347100 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080b2070 con 0x7f9d100d0390 2024-05-14T11:18:10.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0d600640 1 -- 172.21.15.116:0/2326347100 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d080a57f0 con 0x7f9d100d0390 2024-05-14T11:18:10.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0d600640 1 -- 172.21.15.116:0/2326347100 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080a5bc0 con 0x7f9d100d0390 2024-05-14T11:18:10.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 1 -- 172.21.15.116:0/2230948176 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5588f70e2ef0 con 0x5588f6defb80 2024-05-14T11:18:10.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 10 client.5520 handle_command_reply: tid=0 2024-05-14T11:18:10.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 client.5520 shutdown 2024-05-14T11:18:10.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 msgr2=0x5588f7300fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009630 tx=0x7f7d84000bd0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 shutdown_connections 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6defb80 0x5588f734be00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f73077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f7307270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2230948176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f6dcb630 0x5588f7300fa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 >> 172.21.15.116:0/2230948176 conn(0x5588f7201760 msgr2=0x5588f72fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d75600640 0 client.5520 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 shutdown_connections 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 -- 172.21.15.116:0/2326347100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d100d0390 msgr2=0x7f9d10143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 --2- 172.21.15.116:0/2326347100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d100d0390 0x7f9d10143230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9d100723a0 tx=0x7f9d0801c930 comp rx=0 tx=0).stop 2024-05-14T11:18:10.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2230948176 wait complete. 2024-05-14T11:18:10.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f7d9e9a3000 20 client.5520 trim_cache size 0 max 16384 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 -- 172.21.15.116:0/2326347100 shutdown_connections 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 --2- 172.21.15.116:0/2326347100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d100d0390 0x7f9d10143230 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 --2- 172.21.15.116:0/2326347100 >> v2:172.21.15.149:3300/0 conn(0x7f9d101407a0 0x7f9d1006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 -- 172.21.15.116:0/2326347100 >> 172.21.15.116:0/2326347100 conn(0x7f9d100710f0 msgr2=0x7f9d10071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 -- 172.21.15.116:0/2326347100 shutdown_connections 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.072+0000 7f9d0fe00640 1 -- 172.21.15.116:0/2326347100 wait complete. 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.076+0000 7f9d0fe00640 1 Processor -- start 2024-05-14T11:18:10.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.076+0000 7f9d0fe00640 1 -- start start 2024-05-14T11:18:10.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d10070310 0x7f9d10138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d100d0390 0x7f9d10157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d101407a0 0x7f9d1015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10140bb0 con 0x7f9d10070310 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1015e990 con 0x7f9d101407a0 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9d10070720 con 0x7f9d100d0390 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d10070310 0x7f9d10138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d10070310 0x7f9d10138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47788/0 (socket says 172.21.15.116:47788) 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 -- 172.21.15.116:0/861439446 learned_addr learned my addr 172.21.15.116:0/861439446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0e000640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d100d0390 0x7f9d10157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0f400640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d101407a0 0x7f9d1015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 -- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d100d0390 msgr2=0x7f9d10157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9d100d0390 0x7f9d10157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 -- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d101407a0 msgr2=0x7f9d1015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d101407a0 0x7f9d1015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 -- 172.21.15.116:0/861439446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d101371c0 con 0x7f9d10070310 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0ea00640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d10070310 0x7f9d10138260 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000e870 tx=0x7f9d0000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00017070 con 0x7f9d10070310 2024-05-14T11:18:10.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0000c940 con 0x7f9d10070310 2024-05-14T11:18:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000ccf0 con 0x7f9d10070310 2024-05-14T11:18:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 -- 172.21.15.116:0/861439446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d101374a0 con 0x7f9d10070310 2024-05-14T11:18:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.080+0000 7f9d0fe00640 1 -- 172.21.15.116:0/861439446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d10137a60 con 0x7f9d10070310 2024-05-14T11:18:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d0fe00640 1 -- 172.21.15.116:0/861439446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9d1006a9c0 con 0x7f9d10070310 2024-05-14T11:18:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d0001a070 con 0x7f9d10070310 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d06a00640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9cf006f400 0x7f9cf0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f9d0008fa40 con 0x7f9d10070310 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d0e000640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9cf006f400 0x7f9cf0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f9d000902d0 con 0x7f9d10070310 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.084+0000 7f9d0fe00640 1 -- 172.21.15.116:0/861439446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f9d100d0ba0 con 0x7f9d10070310 2024-05-14T11:18:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.088+0000 7f9d0e000640 1 --2- 172.21.15.116:0/861439446 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9cf006f400 0x7f9cf0071910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d1013c570 tx=0x7f9cfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 Processor -- start 2024-05-14T11:18:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 -- start start 2024-05-14T11:18:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f6dcb630 0x5588f735bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f735c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900c74c0 con 0x5588f71f42c0 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900b04d0 con 0x5588f6dcb630 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.096+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900b0b20 con 0x5588f71dd9f0 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47790/0 (socket says 172.21.15.116:47790) 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 -- 172.21.15.116:0/1655384503 learned_addr learned my addr 172.21.15.116:0/1655384503 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d94a00640 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f6dcb630 0x5588f735bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f735c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 -- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 msgr2=0x5588f735c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f735c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 -- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f6dcb630 msgr2=0x5588f735bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f6dcb630 0x5588f735bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 -- 172.21.15.116:0/1655384503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f71cb970 con 0x5588f71f42c0 2024-05-14T11:18:10.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d95400640 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d88015650 tx=0x7f7d88015b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93e9679131b3bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d77e00640 1 -- 172.21.15.116:0/1655384503 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88007a90 con 0x5588f71f42c0 2024-05-14T11:18:10.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d77e00640 1 -- 172.21.15.116:0/1655384503 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88007c30 con 0x5588f71f42c0 2024-05-14T11:18:10.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d77e00640 1 -- 172.21.15.116:0/1655384503 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d880206c0 con 0x5588f71f42c0 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/1655384503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 msgr2=0x5588f734de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d88015650 tx=0x7f7d88015b10 comp rx=0 tx=0).stop 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/1655384503 shutdown_connections 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71f42c0 0x5588f734de10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f71dd9f0 0x5588f735c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/1655384503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f6dcb630 0x5588f735bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/1655384503 >> 172.21.15.116:0/1655384503 conn(0x5588f7201760 msgr2=0x5588f73072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/1655384503 shutdown_connections 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/1655384503 wait complete. 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 -- start start 2024-05-14T11:18:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.100+0000 7f7d9e9a3000 1 Processor -- start 2024-05-14T11:18:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 0x5588f73518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7374d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5588f7157600 con 0x5588f71dd9f0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5588f714e830 con 0x5588f71f42c0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5588f70ff140 con 0x5588f6dcb630 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 10 client.0 _ll_register_callbacks cb 0x7f7d840f8730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47794/0 (socket says 172.21.15.116:47794) 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 -- 172.21.15.116:0/2507198841 learned_addr learned my addr 172.21.15.116:0/2507198841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d95400640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7374d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 0x5588f73518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 -- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 msgr2=0x5588f73518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 0x5588f73518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 -- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 msgr2=0x5588f7374d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7374d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 -- 172.21.15.116:0/2507198841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f70e69e0 con 0x5588f71dd9f0 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 0x5588f73518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d8fe00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c008010 tx=0x7f7d7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c447a449684317 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c013e10 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f7d7c004020 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d88007c30 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c03d050 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 10 client.5529 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5588f6e03470 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88007c30 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f7d7c040020 con 0x5588f71dd9f0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 10 client.5529 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 10 client.5529 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 20 client.5529 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6dddbe0 0x5588f73b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 4 client.5529 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5588f7157e10 con 0x5588f6dddbe0 2024-05-14T11:18:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6dddbe0 0x5588f73b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d75600640 10 client.5529 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.104+0000 7f7d94a00640 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6dddbe0 0x5588f73b8070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a050368de31cdf08 server_cookie=85b6196c55a7f1ea in_seq=0 out_seq=0 2024-05-14T11:18:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d75600640 1 -- 172.21.15.116:0/2507198841 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5588f7157e10 con 0x5588f6dddbe0 2024-05-14T11:18:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d75600640 10 client.5529 handle_command_reply: tid=0 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 client.5529 shutdown 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 msgr2=0x5588f7374820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c008010 tx=0x7f7d7c013000 comp rx=0 tx=0).stop 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 shutdown_connections 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5588f6dddbe0 0x5588f73b8070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588f71f42c0 0x5588f7374d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.111 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f2eec800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5475=0-4194304@1} caps={5475=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5475 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55fb3cb39180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588f71dd9f0 0x5588f7374820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 --2- 172.21.15.116:0/2507198841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5588f6dcb630 0x5588f73518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 >> 172.21.15.116:0/2507198841 conn(0x5588f7201760 msgr2=0x5588f735aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d75600640 0 client.5529 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 shutdown_connections 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 1 -- 172.21.15.116:0/2507198841 wait complete. 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d9e9a3000 20 client.5529 trim_cache size 0 max 16384 2024-05-14T11:18:10.112 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "49a76a62-d8ad-4336-acf9-0ba29c9b6ac8", 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 msgr2=0x7f7d70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 0x7f7d70071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d900c9590 tx=0x7f7d7c008040 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 msgr2=0x7f7d9014cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b3040 tx=0x7f7d8401bab0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 shutdown_connections 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d7006f400 0x7f7d70071910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d90161b50 0x7f7d9014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d900d2ca0 0x7f7d900c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 --2- 172.21.15.116:0/2542510188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900d25f0 0x7f7d9014cd60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 >> 172.21.15.116:0/2542510188 conn(0x7f7d900b56d0 msgr2=0x7f7d900b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 shutdown_connections 2024-05-14T11:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.108+0000 7f7d97400640 1 -- 172.21.15.116:0/2542510188 wait complete. 2024-05-14T11:18:10.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:10.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-14T11:18:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.276+0000 7ff948e00640 1 Processor -- start 2024-05-14T11:18:10.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 -- start start 2024-05-14T11:18:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 0x7ff94406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff9440d1670 0x7ff9440d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9440d1fc0 con 0x7ff94406e9e0 2024-05-14T11:18:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff9441573b0 con 0x7ff9440d1670 2024-05-14T11:18:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff948e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff944161120 con 0x7ff944156fa0 2024-05-14T11:18:10.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff9440d1670 0x7ff9440d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 0x7ff94406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff942800640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7ff94406e9e0 0x7ff94406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:53532/0 (socket says 172.21.15.116:53532) 2024-05-14T11:18:10.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff942800640 1 -- 172.21.15.116:0/3688426441 learned_addr learned my addr 172.21.15.116:0/3688426441 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff940a00640 1 -- 172.21.15.116:0/3688426441 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290708548 0 0) 0x7ff93c0014a0 con 0x7ff94406e9e0 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff940a00640 1 -- 172.21.15.116:0/3688426441 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff928003530 con 0x7ff94406e9e0 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941e00640 1 -- 172.21.15.116:0/3688426441 >> v2:172.21.15.149:3300/0 conn(0x7ff9440d1670 msgr2=0x7ff9440d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941e00640 1 --2- 172.21.15.116:0/3688426441 >> v2:172.21.15.149:3300/0 conn(0x7ff9440d1670 0x7ff9440d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941e00640 1 -- 172.21.15.116:0/3688426441 >> v1:172.21.15.149:6789/0 conn(0x7ff94406e9e0 legacy=0x7ff94406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.280+0000 7ff941e00640 1 -- 172.21.15.116:0/3688426441 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff944160510 con 0x7ff944156fa0 2024-05-14T11:18:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff941e00640 1 --2- 172.21.15.116:0/3688426441 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 0x7ff94406ffe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0097c0 tx=0x7ff92c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82cb11abb8c0b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff940a00640 1 -- 172.21.15.116:0/3688426441 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0a7070 con 0x7ff944156fa0 2024-05-14T11:18:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff940a00640 1 -- 172.21.15.116:0/3688426441 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c0b0070 con 0x7ff944156fa0 2024-05-14T11:18:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff940a00640 1 -- 172.21.15.116:0/3688426441 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0ab870 con 0x7ff944156fa0 2024-05-14T11:18:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 -- 172.21.15.116:0/3688426441 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 msgr2=0x7ff94406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 --2- 172.21.15.116:0/3688426441 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 0x7ff94406ffe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0097c0 tx=0x7ff92c01c470 comp rx=0 tx=0).stop 2024-05-14T11:18:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 -- 172.21.15.116:0/3688426441 shutdown_connections 2024-05-14T11:18:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 --2- 172.21.15.116:0/3688426441 >> v2:172.21.15.149:3300/0 conn(0x7ff9440d1670 0x7ff9440d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 --2- 172.21.15.116:0/3688426441 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff944156fa0 0x7ff94406ffe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 -- 172.21.15.116:0/3688426441 >> 172.21.15.116:0/3688426441 conn(0x7ff9440b4620 msgr2=0x7ff9440b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 -- 172.21.15.116:0/3688426441 shutdown_connections 2024-05-14T11:18:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.284+0000 7ff948e00640 1 -- 172.21.15.116:0/3688426441 wait complete. 2024-05-14T11:18:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 Processor -- start 2024-05-14T11:18:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 -- start start 2024-05-14T11:18:10.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94406e9e0 0x7ff944154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9440d1670 0x7ff944153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff944070520 con 0x7ff944156fa0 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440b07a0 con 0x7ff9440d1670 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff948e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff9440706a0 con 0x7ff94406e9e0 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47806/0 (socket says 172.21.15.116:47806) 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 -- 172.21.15.116:0/2742810887 learned_addr learned my addr 172.21.15.116:0/2742810887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff941400640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9440d1670 0x7ff944153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff941e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94406e9e0 0x7ff944154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 -- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94406e9e0 msgr2=0x7ff944154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94406e9e0 0x7ff944154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 -- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9440d1670 msgr2=0x7ff944153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9440d1670 0x7ff944153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 -- 172.21.15.116:0/2742810887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff944155880 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff942800640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ca70 tx=0x7ff93c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff93a000640 1 -- 172.21.15.116:0/2742810887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00ab70 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff93a000640 1 -- 172.21.15.116:0/2742810887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c00ad10 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.288+0000 7ff93a000640 1 -- 172.21.15.116:0/2742810887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c012620 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff944155b60 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff944151460 con 0x7ff944156fa0 2024-05-14T11:18:10.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff93a000640 1 -- 172.21.15.116:0/2742810887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff93c0127c0 con 0x7ff944156fa0 2024-05-14T11:18:10.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff93a000640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 0x7ff920071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff93a000640 1 -- 172.21.15.116:0/2742810887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff93c08f8a0 con 0x7ff944156fa0 2024-05-14T11:18:10.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff941e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 0x7ff920071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff9504cb000 1 Processor -- start 2024-05-14T11:18:10.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.292+0000 7ff9504cb000 1 -- start start 2024-05-14T11:18:10.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff941e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 0x7ff920071940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0af000 tx=0x7ff92c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c769d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c769d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5648c74a2100 con 0x5648c71233e0 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648c745aca0 con 0x5648c7523f00 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5648c749e0c0 con 0x5648c7536120 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47812/0 (socket says 172.21.15.116:47812) 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 -- 172.21.15.116:0/753853212 learned_addr learned my addr 172.21.15.116:0/753853212 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff941e00640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c769d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff941400640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c769d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 -- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 msgr2=0x5648c769d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c769d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 -- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 msgr2=0x5648c769d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c769d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 -- 172.21.15.116:0/753853212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648c745f800 con 0x5648c71233e0 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff941400640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c769d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff942800640 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005bc0 tx=0x7ff93c05b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e0f72d4f872a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff932a00640 1 -- 172.21.15.116:0/753853212 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0602a0 con 0x5648c71233e0 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff932a00640 1 -- 172.21.15.116:0/753853212 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c064cc0 con 0x5648c71233e0 2024-05-14T11:18:10.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff932a00640 1 -- 172.21.15.116:0/753853212 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c064a30 con 0x5648c71233e0 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- 172.21.15.116:0/753853212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 msgr2=0x5648c769ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005bc0 tx=0x7ff93c05b710 comp rx=0 tx=0).stop 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- 172.21.15.116:0/753853212 shutdown_connections 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c769d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c769ddd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 --2- 172.21.15.116:0/753853212 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c769d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- 172.21.15.116:0/753853212 >> 172.21.15.116:0/753853212 conn(0x5648c7163400 msgr2=0x5648c7680310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- 172.21.15.116:0/753853212 shutdown_connections 2024-05-14T11:18:10.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.296+0000 7ff9504cb000 1 -- 172.21.15.116:0/753853212 wait complete. 2024-05-14T11:18:10.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- start start 2024-05-14T11:18:10.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 Processor -- start 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c76a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c7656fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5648c756d2a0 con 0x5648c71233e0 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648c755df40 con 0x5648c7523f00 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5648c745e080 con 0x5648c7536120 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 10 client.0 _ll_register_callbacks cb 0x5648c763ec90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47828/0 (socket says 172.21.15.116:47828) 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 -- 172.21.15.116:0/3355542376 learned_addr learned my addr 172.21.15.116:0/3355542376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941400640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c76a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff942800640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c7656fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 -- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 msgr2=0x5648c7656fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c7656fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 -- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 msgr2=0x5648c76a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c76a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 -- 172.21.15.116:0/3355542376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648c744c860 con 0x5648c71233e0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941400640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c76a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff942800640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c7656fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff941e00640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01a0a0 tx=0x7ff92c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca25bd3b3ec143e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0ab470 con 0x5648c71233e0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff92c0ccc80 con 0x5648c71233e0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c064cc0 con 0x5648c71233e0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff92c0cce80 con 0x5648c71233e0 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 10 client.5538 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:10.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5648c745aef0 con 0x5648c71233e0 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c064cc0 con 0x5648c71233e0 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7ff92c0c9070 con 0x5648c71233e0 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 10 client.5538 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 10 client.5538 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 20 client.5538 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7147930 0x5648c7657cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 4 client.5538 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.300+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648c743adf0 con 0x5648c7147930 2024-05-14T11:18:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff941400640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7147930 0x5648c7657cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff927400640 10 client.5538 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff941400640 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7147930 0x5648c7657cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6f4aba016c8b66 server_cookie=d42f6b7919b7661d in_seq=0 out_seq=0 2024-05-14T11:18:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff927400640 1 -- 172.21.15.116:0/3355542376 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648c743adf0 con 0x5648c7147930 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff927400640 10 client.5538 handle_command_reply: tid=0 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 client.5538 shutdown 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 msgr2=0x5648c76a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01a0a0 tx=0x7ff92c01acb0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 shutdown_connections 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7147930 0x5648c7657cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c76a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c7656fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3355542376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c76a3dd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff927400640 0 client.5538 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 >> 172.21.15.116:0/3355542376 conn(0x5648c7163400 msgr2=0x5648c7680200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 shutdown_connections 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 1 -- 172.21.15.116:0/3355542376 wait complete. 2024-05-14T11:18:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.304+0000 7ff9504cb000 20 client.5538 trim_cache size 0 max 16384 2024-05-14T11:18:10.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.320+0000 7ff9504cb000 1 Processor -- start 2024-05-14T11:18:10.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.320+0000 7ff9504cb000 1 -- start start 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c770c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c770c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff944205860 con 0x5648c71233e0 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff94414f910 con 0x5648c7523f00 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff944205210 con 0x5648c7536120 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47832/0 (socket says 172.21.15.116:47832) 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 -- 172.21.15.116:0/3909594463 learned_addr learned my addr 172.21.15.116:0/3909594463 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941400640 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c770c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff942800640 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c770c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 -- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 msgr2=0x5648c770c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c770c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 -- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 msgr2=0x5648c770c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c770c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 -- 172.21.15.116:0/3909594463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648c7655070 con 0x5648c71233e0 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff941e00640 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01cd10 tx=0x7ff92c0c4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c676eae4389b924 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff932000640 1 -- 172.21.15.116:0/3909594463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0d4460 con 0x5648c71233e0 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff932000640 1 -- 172.21.15.116:0/3909594463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c0d4620 con 0x5648c71233e0 2024-05-14T11:18:10.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff932000640 1 -- 172.21.15.116:0/3909594463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0d4a10 con 0x5648c71233e0 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- 172.21.15.116:0/3909594463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 msgr2=0x5648c770bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff92c01cd10 tx=0x7ff92c0c4040 comp rx=0 tx=0).stop 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- 172.21.15.116:0/3909594463 shutdown_connections 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c7523f00 0x5648c770c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7536120 0x5648c770c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3909594463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c71233e0 0x5648c770bd30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- 172.21.15.116:0/3909594463 >> 172.21.15.116:0/3909594463 conn(0x5648c7163400 msgr2=0x5648c7655b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- 172.21.15.116:0/3909594463 shutdown_connections 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.324+0000 7ff9504cb000 1 -- 172.21.15.116:0/3909594463 wait complete. 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- start start 2024-05-14T11:18:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 Processor -- start 2024-05-14T11:18:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c71233e0 0x5648c76a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 0x5648c76cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5648c74a6730 con 0x5648c7536120 2024-05-14T11:18:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648c74a1410 con 0x5648c71233e0 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5648c74af500 con 0x5648c7523f00 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 10 client.0 _ll_register_callbacks cb 0x7ff934009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47846/0 (socket says 172.21.15.116:47846) 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 -- 172.21.15.116:0/3340141032 learned_addr learned my addr 172.21.15.116:0/3340141032 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff941e00640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c71233e0 0x5648c76a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff941400640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 0x5648c76cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 -- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 msgr2=0x5648c76cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 0x5648c76cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 -- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c71233e0 msgr2=0x5648c76a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c71233e0 0x5648c76a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 -- 172.21.15.116:0/3340141032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648c7523340 con 0x5648c7536120 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff941400640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 0x5648c76cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff942800640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff93c05bf30 tx=0x7ff93c05b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82edf3c09c8648fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c057e60 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff93c05b410 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c0d4620 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff93c064cc0 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 10 client.5544 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5648c743e8e0 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c0d4620 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7ff93c004b70 con 0x5648c7536120 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 10 client.5544 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 10 client.5544 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 20 client.5544 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7135990 0x5648c76cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 4 client.5544 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:18:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648c74afd10 con 0x5648c7135990 2024-05-14T11:18:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.328+0000 7ff941e00640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7135990 0x5648c76cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff941e00640 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7135990 0x5648c76cd350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c33eb7e388639d6 server_cookie=6fd0d0bdad2f0a37 in_seq=0 out_seq=0 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff927400640 10 client.5544 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff927400640 1 -- 172.21.15.116:0/3340141032 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5648c74afd10 con 0x5648c7135990 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff927400640 10 client.5544 handle_command_reply: tid=0 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 client.5544 shutdown 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 msgr2=0x5648c76cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff93c05bf30 tx=0x7ff93c05b840 comp rx=0 tx=0).stop 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 shutdown_connections 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x5648c7135990 0x5648c76cd350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648c7536120 0x5648c76cce10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5648c7523f00 0x5648c76cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 --2- 172.21.15.116:0/3340141032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648c71233e0 0x5648c76a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff927400640 0 client.5544 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 >> 172.21.15.116:0/3340141032 conn(0x5648c7163400 msgr2=0x5648c7656550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 shutdown_connections 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 1 -- 172.21.15.116:0/3340141032 wait complete. 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff9504cb000 20 client.5544 trim_cache size 0 max 16384 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "49a76a62-d8ad-4336-acf9-0ba29c9b6ac8": { 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "49a76a62-d8ad-4336-acf9-0ba29c9b6ac8", 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,repair" 2024-05-14T11:18:10.341 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:10.342 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:10.342 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 msgr2=0x7ff920071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 0x7ff920071940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0af000 tx=0x7ff92c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 msgr2=0x7ff9441539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00ca70 tx=0x7ff93c00cf30 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 shutdown_connections 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff92006f430 0x7ff920071940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9440d1670 0x7ff944153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff944156fa0 0x7ff9441539d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 --2- 172.21.15.116:0/2742810887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff94406e9e0 0x7ff944154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 >> 172.21.15.116:0/2742810887 conn(0x7ff9440b4620 msgr2=0x7ff94415dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 shutdown_connections 2024-05-14T11:18:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:10.332+0000 7ff948e00640 1 -- 172.21.15.116:0/2742810887 wait complete. 2024-05-14T11:18:10.353 INFO:tasks.cephfs.filesystem:scrub status for tag:49a76a62-d8ad-4336-acf9-0ba29c9b6ac8 - {'path': '/', 'tag': '49a76a62-d8ad-4336-acf9-0ba29c9b6ac8', 'options': 'recursive,repair'} 2024-05-14T11:18:11.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:11.224+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-14T11:18:10.110990+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f9d0008def0 con 0x7f9d10070310 2024-05-14T11:18:11.226 INFO:teuthology.orchestra.run.smithi116.stdout: cluster: 2024-05-14T11:18:11.226 INFO:teuthology.orchestra.run.smithi116.stdout: id: 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:18:11.226 INFO:teuthology.orchestra.run.smithi116.stdout: health: HEALTH_OK 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: services: 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: mgr: y(active, since 2m), standbys: x 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: mds: 1/1 daemons up, 3 standby 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: data: 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: volumes: 1/1 healthy 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: pools: 3 pools, 97 pgs 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: objects: 24 objects, 579 KiB 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: usage: 225 MiB used, 720 GiB / 720 GiB avail 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: pgs: 97 active+clean 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:11.227 INFO:teuthology.orchestra.run.smithi116.stdout: io: 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout: client: 1.3 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.110990+0000 mds.c [INF] scrub queued for path: / 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.110997+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.111004+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.112122+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.112134+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-14T11:18:11.228 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:10.112488+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-14T11:18:11.352 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:18:11.355 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:18:13.450 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:18:11.809688+0000 mds.c [INF] scrub summary: idle 2024-05-14T11:18:13.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:13.447+0000 7f9d06a00640 1 -- 172.21.15.116:0/861439446 <== mon.0 v2:172.21.15.116:3300/0 8 ==== log(1 entries from seq 7 at 2024-05-14T11:18:11.809688+0000) ==== 189+0+0 (secure 0 0 0) 0x7f9d00090000 con 0x7f9d10070310 2024-05-14T11:18:40.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:40.355 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-14T11:18:40.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.515+0000 7ff65dc00640 1 Processor -- start 2024-05-14T11:18:40.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.515+0000 7ff65dc00640 1 -- start start 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff6580cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65dc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 0x7ff65806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65dc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff658160720 con 0x7ff6580d15f0 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65dc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff658161330 con 0x7ff65806eac0 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65dc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff658160a00 con 0x7ff6580cf870 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff64fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff6580d15f0 0x7ff6580d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49464/0 (socket says 172.21.15.116:49464) 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff64fe00640 1 -- 172.21.15.116:0/3509673972 learned_addr learned my addr 172.21.15.116:0/3509673972 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65d200640 1 --2- 172.21.15.116:0/3509673972 >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 0x7ff65806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657e00640 1 --2- 172.21.15.116:0/3509673972 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff6580cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657400640 1 -- 172.21.15.116:0/3509673972 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782386791 0 0) 0x7ff658160720 con 0x7ff6580d15f0 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657400640 1 -- 172.21.15.116:0/3509673972 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff63c003530 con 0x7ff6580d15f0 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657e00640 1 -- 172.21.15.116:0/3509673972 >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 msgr2=0x7ff65806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657e00640 1 --2- 172.21.15.116:0/3509673972 >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 0x7ff65806eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657e00640 1 -- 172.21.15.116:0/3509673972 >> v1:172.21.15.116:6789/0 conn(0x7ff6580d15f0 legacy=0x7ff6580d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff657e00640 1 -- 172.21.15.116:0/3509673972 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff658160ce0 con 0x7ff6580cf870 2024-05-14T11:18:40.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.519+0000 7ff65d200640 1 --2- 172.21.15.116:0/3509673972 >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 0x7ff65806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff657e00640 1 --2- 172.21.15.116:0/3509673972 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff6580cfc80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff6581614b0 tx=0x7ff64801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9079d2c41a90ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff657400640 1 -- 172.21.15.116:0/3509673972 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480aa5f0 con 0x7ff6580cf870 2024-05-14T11:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff657400640 1 -- 172.21.15.116:0/3509673972 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6480af070 con 0x7ff6580cf870 2024-05-14T11:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff657400640 1 -- 172.21.15.116:0/3509673972 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480aa9b0 con 0x7ff6580cf870 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- 172.21.15.116:0/3509673972 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 msgr2=0x7ff6580cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 --2- 172.21.15.116:0/3509673972 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff6580cfc80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff6581614b0 tx=0x7ff64801c470 comp rx=0 tx=0).stop 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- 172.21.15.116:0/3509673972 shutdown_connections 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 --2- 172.21.15.116:0/3509673972 >> v2:172.21.15.149:3300/0 conn(0x7ff65806eac0 0x7ff65806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 --2- 172.21.15.116:0/3509673972 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff6580cfc80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- 172.21.15.116:0/3509673972 >> 172.21.15.116:0/3509673972 conn(0x7ff6580b4510 msgr2=0x7ff6580b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- 172.21.15.116:0/3509673972 shutdown_connections 2024-05-14T11:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- 172.21.15.116:0/3509673972 wait complete. 2024-05-14T11:18:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 Processor -- start 2024-05-14T11:18:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.523+0000 7ff65dc00640 1 -- start start 2024-05-14T11:18:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff658155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff658169870 con 0x7ff6580d15f0 2024-05-14T11:18:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6581696f0 con 0x7ff65806eac0 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff6581699f0 con 0x7ff6580cf870 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff657e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43664/0 (socket says 172.21.15.116:43664) 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 -- 172.21.15.116:0/693415762 learned_addr learned my addr 172.21.15.116:0/693415762 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff657e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:37796/0 (socket says 172.21.15.116:37796) 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff64fe00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff658155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 -- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 msgr2=0x7ff658155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff658155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 -- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 msgr2=0x7ff6580afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 -- 172.21.15.116:0/693415762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6580b2a80 con 0x7ff6580d15f0 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff657e00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff64fe00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff658155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65d200640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff650009e10 tx=0x7ff65000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff654c00640 1 -- 172.21.15.116:0/693415762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650018020 con 0x7ff6580d15f0 2024-05-14T11:18:40.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff654c00640 1 -- 172.21.15.116:0/693415762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff650014480 con 0x7ff6580d15f0 2024-05-14T11:18:40.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff658157240 con 0x7ff6580d15f0 2024-05-14T11:18:40.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff654c00640 1 -- 172.21.15.116:0/693415762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650014850 con 0x7ff6580d15f0 2024-05-14T11:18:40.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.527+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff658157800 con 0x7ff6580d15f0 2024-05-14T11:18:40.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff654c00640 1 -- 172.21.15.116:0/693415762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff65001b020 con 0x7ff6580d15f0 2024-05-14T11:18:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff654c00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 0x7ff634071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff654c00640 1 -- 172.21.15.116:0/693415762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff6500901a0 con 0x7ff6580d15f0 2024-05-14T11:18:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff657e00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 0x7ff634071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff66692a000 1 Processor -- start 2024-05-14T11:18:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff66692a000 1 -- start start 2024-05-14T11:18:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.531+0000 7ff657e00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 0x7ff634071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6480ae000 tx=0x7ff6480a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea62a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea62a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563ea60a4100 con 0x563ea5d253e0 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ea605cca0 con 0x563ea6138120 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x563ea60a00c0 con 0x563ea6125f00 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43678/0 (socket says 172.21.15.116:43678) 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 -- 172.21.15.116:0/782848028 learned_addr learned my addr 172.21.15.116:0/782848028 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff657e00640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea62a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff64fe00640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea62a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 -- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 msgr2=0x563ea62a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea62a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 -- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 msgr2=0x563ea62a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea62a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 -- 172.21.15.116:0/782848028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea6061800 con 0x563ea5d253e0 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff657e00640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea62a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff65d200640 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff650058d50 tx=0x7ff65005c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76371a2e5e91203 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff64cc00640 1 -- 172.21.15.116:0/782848028 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65008c070 con 0x563ea5d253e0 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff64cc00640 1 -- 172.21.15.116:0/782848028 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65005c760 con 0x563ea5d253e0 2024-05-14T11:18:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff64cc00640 1 -- 172.21.15.116:0/782848028 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650075070 con 0x563ea5d253e0 2024-05-14T11:18:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- 172.21.15.116:0/782848028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 msgr2=0x563ea62a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff650058d50 tx=0x7ff65005c170 comp rx=0 tx=0).stop 2024-05-14T11:18:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- 172.21.15.116:0/782848028 shutdown_connections 2024-05-14T11:18:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62a4230 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea62a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 --2- 172.21.15.116:0/782848028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea62a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- 172.21.15.116:0/782848028 >> 172.21.15.116:0/782848028 conn(0x563ea5d65400 msgr2=0x563ea6282310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- 172.21.15.116:0/782848028 shutdown_connections 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- 172.21.15.116:0/782848028 wait complete. 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 -- start start 2024-05-14T11:18:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.535+0000 7ff66692a000 1 Processor -- start 2024-05-14T11:18:40.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea5d253e0 0x563ea62aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6138120 0x563ea6261ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563ea616f2a0 con 0x563ea6125f00 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ea615ff40 con 0x563ea5d253e0 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x563ea6060080 con 0x563ea6138120 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 10 client.0 _ll_register_callbacks cb 0x563ea6240c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43682/0 (socket says 172.21.15.116:43682) 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 -- 172.21.15.116:0/1821521149 learned_addr learned my addr 172.21.15.116:0/1821521149 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff657e00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea5d253e0 0x563ea62aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff65d200640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6138120 0x563ea6261ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 -- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6138120 msgr2=0x563ea6261ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6138120 0x563ea6261ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 -- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea5d253e0 msgr2=0x563ea62aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea5d253e0 0x563ea62aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 -- 172.21.15.116:0/1821521149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea604e860 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff64fe00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff640001ea0 tx=0x7ff6400116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90a684d30654e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640005ad0 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff640005c70 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65005c760 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6400070d0 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 10 client.5556 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563ea605cef0 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65005c760 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7ff640025020 con 0x563ea6125f00 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 10 client.5556 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 10 client.5556 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 20 client.5556 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:40.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d49930 0x563ea62629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 4 client.5556 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:18:40.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ea603cdf0 con 0x563ea5d49930 2024-05-14T11:18:40.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.539+0000 7ff657e00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d49930 0x563ea62629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff657e00640 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d49930 0x563ea62629e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb35a44a90972f4 server_cookie=a1ff02d4522c00a3 in_seq=0 out_seq=0 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff645600640 10 client.5556 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff645600640 1 -- 172.21.15.116:0/1821521149 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ea603cdf0 con 0x563ea5d49930 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff645600640 10 client.5556 handle_command_reply: tid=0 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 client.5556 shutdown 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 msgr2=0x563ea62ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff640001ea0 tx=0x7ff6400116e0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 shutdown_connections 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d49930 0x563ea62629e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6138120 0x563ea6261ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6125f00 0x563ea62ab150 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 --2- 172.21.15.116:0/1821521149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea5d253e0 0x563ea62aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff645600640 0 client.5556 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 >> 172.21.15.116:0/1821521149 conn(0x563ea5d65400 msgr2=0x563ea6282310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 shutdown_connections 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 1 -- 172.21.15.116:0/1821521149 wait complete. 2024-05-14T11:18:40.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.543+0000 7ff66692a000 20 client.5556 trim_cache size 0 max 16384 2024-05-14T11:18:40.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.559+0000 7ff66692a000 1 Processor -- start 2024-05-14T11:18:40.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.559+0000 7ff66692a000 1 -- start start 2024-05-14T11:18:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea627f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea627f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff658073cc0 con 0x563ea5d253e0 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6580aeb00 con 0x563ea6138120 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff6580af150 con 0x563ea6125f00 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43684/0 (socket says 172.21.15.116:43684) 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 -- 172.21.15.116:0/352190513 learned_addr learned my addr 172.21.15.116:0/352190513 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 -- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 msgr2=0x563ea627f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea627f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 -- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 msgr2=0x563ea627f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea627f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 -- 172.21.15.116:0/352190513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea6125340 con 0x563ea5d253e0 2024-05-14T11:18:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff657e00640 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff648009160 tx=0x7ff648009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b240ecdec24cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff647e00640 1 -- 172.21.15.116:0/352190513 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480cf070 con 0x563ea5d253e0 2024-05-14T11:18:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff647e00640 1 -- 172.21.15.116:0/352190513 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6480ca680 con 0x563ea5d253e0 2024-05-14T11:18:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff647e00640 1 -- 172.21.15.116:0/352190513 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480ba070 con 0x563ea5d253e0 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- 172.21.15.116:0/352190513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 msgr2=0x563ea62bafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff648009160 tx=0x7ff648009190 comp rx=0 tx=0).stop 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- 172.21.15.116:0/352190513 shutdown_connections 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6138120 0x563ea627f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea6125f00 0x563ea627f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 --2- 172.21.15.116:0/352190513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea5d253e0 0x563ea62bafc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- 172.21.15.116:0/352190513 >> 172.21.15.116:0/352190513 conn(0x563ea5d65400 msgr2=0x563ea627eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- 172.21.15.116:0/352190513 shutdown_connections 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- 172.21.15.116:0/352190513 wait complete. 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 -- start start 2024-05-14T11:18:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.563+0000 7ff66692a000 1 Processor -- start 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea5d253e0 0x563ea6316750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6125f00 0x563ea6316c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563ea60a8730 con 0x563ea6138120 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ea60a3410 con 0x563ea6125f00 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x563ea60b1500 con 0x563ea5d253e0 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 10 client.0 _ll_register_callbacks cb 0x7ff640007cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43688/0 (socket says 172.21.15.116:43688) 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 -- 172.21.15.116:0/486469158 learned_addr learned my addr 172.21.15.116:0/486469158 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff657e00640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea5d253e0 0x563ea6316750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff64fe00640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6125f00 0x563ea6316c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 -- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea5d253e0 msgr2=0x563ea6316750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea5d253e0 0x563ea6316750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 -- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6125f00 msgr2=0x563ea6316c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6125f00 0x563ea6316c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 -- 172.21.15.116:0/486469158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea60408e0 con 0x563ea6138120 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff65d200640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff65000f010 tx=0x7ff65008e370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126245921a0226b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65008c040 con 0x563ea6138120 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7ff6500149f0 con 0x563ea6138120 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6480ca680 con 0x563ea6138120 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff650083050 con 0x563ea6138120 2024-05-14T11:18:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 10 client.5562 fetch_fsmap learned FSMap version 26 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x563ea5d5d220 con 0x563ea6138120 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6480ca680 con 0x563ea6138120 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7ff650086020 con 0x563ea6138120 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 10 client.5562 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 10 client.5562 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 20 client.5562 populate_metadata read hostname 'smithi116' 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d37990 0x563ea62ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 4 client.5562 mds_command: new command op to 5319 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 --> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ea60b1d10 con 0x563ea5d37990 2024-05-14T11:18:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff657e00640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d37990 0x563ea62ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff657e00640 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d37990 0x563ea62ad3d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f534fab07d9c962d server_cookie=102e3f10d7a4402a in_seq=0 out_seq=0 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 10 client.5562 ms_handle_connect on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 1 -- 172.21.15.116:0/486469158 <== mds.0 v2:172.21.15.116:6836/2735194331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ea60b1d10 con 0x563ea5d37990 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff645600640 10 client.5562 handle_command_reply: tid=0 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 client.5562 shutdown 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 msgr2=0x563ea62aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.567+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff65000f010 tx=0x7ff65008e370 comp rx=0 tx=0).stop 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 shutdown_connections 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:6836/2735194331,v1:172.21.15.116:6837/2735194331] conn(0x563ea5d37990 0x563ea62ad3d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea6125f00 0x563ea6316c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563ea6138120 0x563ea62aca40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 --2- 172.21.15.116:0/486469158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x563ea5d253e0 0x563ea6316750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff645600640 0 client.5562 ms_handle_reset on v2:172.21.15.116:6836/2735194331 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 >> 172.21.15.116:0/486469158 conn(0x563ea5d65400 msgr2=0x563ea62a9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 shutdown_connections 2024-05-14T11:18:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 1 -- 172.21.15.116:0/486469158 wait complete. 2024-05-14T11:18:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff66692a000 20 client.5562 trim_cache size 0 max 16384 2024-05-14T11:18:40.576 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:40.576 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:18:40.576 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:18:40.576 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 msgr2=0x7ff634071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 0x7ff634071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6480ae000 tx=0x7ff6480a7000 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 msgr2=0x7ff658156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff650009e10 tx=0x7ff65000c490 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 shutdown_connections 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff63406f400 0x7ff634071910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6580d15f0 0x7ff658156170 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6580cf870 0x7ff658155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 --2- 172.21.15.116:0/693415762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff65806eac0 0x7ff6580afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 >> 172.21.15.116:0/693415762 conn(0x7ff6580b4510 msgr2=0x7ff65815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 shutdown_connections 2024-05-14T11:18:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.571+0000 7ff65dc00640 1 -- 172.21.15.116:0/693415762 wait complete. 2024-05-14T11:18:40.592 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:18:40.693 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.835+0000 7f4360c00640 1 Processor -- start 2024-05-14T11:18:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.835+0000 7f4360c00640 1 -- start start 2024-05-14T11:18:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f435c15db30 0x7f435c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f435c161600 0x7f435c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f435c0ae100 con 0x7f435c161600 2024-05-14T11:18:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f435c0ae870 con 0x7f435c15db30 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4360c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f435c06ff20 con 0x7f435c0d1670 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f435c161600 0x7f435c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f435c161600 0x7f435c161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43700/0 (socket says 172.21.15.116:43700) 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 -- 172.21.15.116:0/48542086 learned_addr learned my addr 172.21.15.116:0/48542086 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4351400640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4359e00640 1 --2- 172.21.15.116:0/48542086 >> v2:172.21.15.149:3300/0 conn(0x7f435c15db30 0x7f435c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 -- 172.21.15.116:0/48542086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 msgr2=0x7f435c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 -- 172.21.15.116:0/48542086 >> v2:172.21.15.149:3300/0 conn(0x7f435c15db30 msgr2=0x7f435c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 --2- 172.21.15.116:0/48542086 >> v2:172.21.15.149:3300/0 conn(0x7f435c15db30 0x7f435c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f435a800640 1 -- 172.21.15.116:0/48542086 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c0d1fc0 con 0x7f435c161600 2024-05-14T11:18:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.839+0000 7f4351400640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f435a800640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c161a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f435c071f20 tx=0x7f435401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6669dba9c63c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4359400640 1 -- 172.21.15.116:0/48542086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540b0070 con 0x7f435c161600 2024-05-14T11:18:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4359400640 1 -- 172.21.15.116:0/48542086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43540ab600 con 0x7f435c161600 2024-05-14T11:18:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4359400640 1 -- 172.21.15.116:0/48542086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540ab9d0 con 0x7f435c161600 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- 172.21.15.116:0/48542086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 msgr2=0x7f435c161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c161a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f435c071f20 tx=0x7f435401c490 comp rx=0 tx=0).stop 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- 172.21.15.116:0/48542086 shutdown_connections 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c161a10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 --2- 172.21.15.116:0/48542086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 --2- 172.21.15.116:0/48542086 >> v2:172.21.15.149:3300/0 conn(0x7f435c15db30 0x7f435c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- 172.21.15.116:0/48542086 >> 172.21.15.116:0/48542086 conn(0x7f435c0b4620 msgr2=0x7f435c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- 172.21.15.116:0/48542086 shutdown_connections 2024-05-14T11:18:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- 172.21.15.116:0/48542086 wait complete. 2024-05-14T11:18:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 Processor -- start 2024-05-14T11:18:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.843+0000 7f4360c00640 1 -- start start 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f435c15db30 0x7f435c14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f435c162360 con 0x7f435c161600 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f435c0aefe0 con 0x7f435c15db30 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f435c15df40 con 0x7f435c0d1670 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43706/0 (socket says 172.21.15.116:43706) 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 -- 172.21.15.116:0/4195269301 learned_addr learned my addr 172.21.15.116:0/4195269301 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4351400640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f435c15db30 0x7f435c14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4359e00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 -- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 msgr2=0x7f435c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 -- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f435c15db30 msgr2=0x7f435c14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f435c15db30 0x7f435c14b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 -- 172.21.15.116:0/4195269301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c15eca0 con 0x7f435c161600 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f435a800640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f435401ab40 tx=0x7f4354019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540b9070 con 0x7f435c161600 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43540b4420 con 0x7f435c161600 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540b0040 con 0x7f435c161600 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c15ef80 con 0x7f435c161600 2024-05-14T11:18:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c0b3eb0 con 0x7f435c161600 2024-05-14T11:18:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.847+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c06a9c0 con 0x7f435c161600 2024-05-14T11:18:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.851+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43540bc070 con 0x7f435c161600 2024-05-14T11:18:40.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.851+0000 7f4352a00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 0x7f4338071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:40.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.851+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f43541284f0 con 0x7f435c161600 2024-05-14T11:18:40.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.851+0000 7f4359e00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 0x7f4338071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:40.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.851+0000 7f4359e00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 0x7f4338071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f434c003fd0 tx=0x7f434c003ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:40.855+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f435c0d1fc0 con 0x7f435c161600 2024-05-14T11:18:41.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.043+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f435c0cd310 con 0x7f435c161600 2024-05-14T11:18:41.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.043+0000 7f4352a00640 1 -- 172.21.15.116:0/4195269301 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f4354016110 con 0x7f435c161600 2024-05-14T11:18:41.048 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:41.048 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:41.049 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 msgr2=0x7f4338071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 0x7f4338071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f434c003fd0 tx=0x7f434c003ce0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 msgr2=0x7f435c14f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f435401ab40 tx=0x7f4354019230 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 shutdown_connections 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f433806f400 0x7f4338071910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f435c15db30 0x7f435c14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f435c161600 0x7f435c14f8f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 --2- 172.21.15.116:0/4195269301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f435c0d1670 0x7f435c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 >> 172.21.15.116:0/4195269301 conn(0x7f435c0b4620 msgr2=0x7f435c166b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 shutdown_connections 2024-05-14T11:18:41.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.047+0000 7f4360c00640 1 -- 172.21.15.116:0/4195269301 wait complete. 2024-05-14T11:18:41.070 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-14T11:18:41.246 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:18:41.246 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:18:41.246 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-14T11:18:41.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.299+0000 7f1578ce2080 1 Processor -- start 2024-05-14T11:18:41.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.299+0000 7f1578ce2080 1 -- start start 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412501f0 0x559a41370ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 0x559a41371000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559a4124f2d0 con 0x559a41238a20 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559a411aeaa0 con 0x559a412dff10 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x559a4124f570 con 0x559a412501f0 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43712/0 (socket says 172.21.15.116:43712) 2024-05-14T11:18:41.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 -- 172.21.15.116:0/3268332020 learned_addr learned my addr 172.21.15.116:0/3268332020 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 -- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412501f0 msgr2=0x559a41370ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412501f0 0x559a41370ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 -- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 msgr2=0x559a41371000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1572800640 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 0x559a41371000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 0x559a41371000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 -- 172.21.15.116:0/3268332020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a41233720 con 0x559a41238a20 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1572800640 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 0x559a41371000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1571e00640 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15640097a0 tx=0x7f1564005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3014d930912ea3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1570a00640 1 -- 172.21.15.116:0/3268332020 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564008da0 con 0x559a41238a20 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1570a00640 1 -- 172.21.15.116:0/3268332020 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1564018070 con 0x559a41238a20 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1570a00640 1 -- 172.21.15.116:0/3268332020 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564013640 con 0x559a41238a20 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- 172.21.15.116:0/3268332020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 msgr2=0x559a412df9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15640097a0 tx=0x7f1564005a10 comp rx=0 tx=0).stop 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- 172.21.15.116:0/3268332020 shutdown_connections 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412dff10 0x559a41371000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412501f0 0x559a41370ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 --2- 172.21.15.116:0/3268332020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412df9d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- 172.21.15.116:0/3268332020 >> 172.21.15.116:0/3268332020 conn(0x559a411f3e10 msgr2=0x559a41209540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- 172.21.15.116:0/3268332020 shutdown_connections 2024-05-14T11:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- 172.21.15.116:0/3268332020 wait complete. 2024-05-14T11:18:41.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 Processor -- start 2024-05-14T11:18:41.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.303+0000 7f1578ce2080 1 -- start start 2024-05-14T11:18:41.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412501f0 0x559a41377020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412dff10 0x559a412359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559a41233c60 con 0x559a41238a20 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559a412339c0 con 0x559a412501f0 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x559a41233f00 con 0x559a412dff10 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43722/0 (socket says 172.21.15.116:43722) 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 -- 172.21.15.116:0/1028240620 learned_addr learned my addr 172.21.15.116:0/1028240620 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f156be00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412501f0 0x559a41377020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1572800640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412dff10 0x559a412359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 -- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412dff10 msgr2=0x559a412359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x559a412dff10 0x559a412359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 -- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412501f0 msgr2=0x559a41377020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a412501f0 0x559a41377020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 -- 172.21.15.116:0/1028240620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a413779d0 con 0x559a41238a20 2024-05-14T11:18:41.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1571e00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412c3360 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1564005a50 tx=0x7f1564003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f156aa00640 1 -- 172.21.15.116:0/1028240620 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564008ea0 con 0x559a41238a20 2024-05-14T11:18:41.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f156aa00640 1 -- 172.21.15.116:0/1028240620 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1564021070 con 0x559a41238a20 2024-05-14T11:18:41.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a41377c70 con 0x559a41238a20 2024-05-14T11:18:41.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f156aa00640 1 -- 172.21.15.116:0/1028240620 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400be80 con 0x559a41238a20 2024-05-14T11:18:41.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.307+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a413781b0 con 0x559a41238a20 2024-05-14T11:18:41.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f156aa00640 1 -- 172.21.15.116:0/1028240620 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f156401d400 con 0x559a41238a20 2024-05-14T11:18:41.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f156aa00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f155006de10 0x7f1550070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f156aa00640 1 -- 172.21.15.116:0/1028240620 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f15640905a0 con 0x559a41238a20 2024-05-14T11:18:41.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f156be00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f155006de10 0x7f1550070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x559a41248890 0x559a41248ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) -- 0x559a41377560 con 0x559a41248890 2024-05-14T11:18:41.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1572800640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x559a41248890 0x559a41248ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f156be00640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f155006de10 0x7f1550070320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f155c0018f0 tx=0x7f155c001880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1572800640 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x559a41248890 0x559a41248ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1572800640 1 -- 172.21.15.116:0/1028240620 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f156c0aaa10 con 0x559a41248890 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x559a41248890 msgr2=0x559a41248ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x559a41248890 0x559a41248ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f155006de10 msgr2=0x7f1550070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f155006de10 0x7f1550070320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f155c0018f0 tx=0x7f155c001880 comp rx=0 tx=0).stop 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 msgr2=0x559a412c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 --2- 172.21.15.116:0/1028240620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559a41238a20 0x559a412c3360 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1564005a50 tx=0x7f1564003030 comp rx=0 tx=0).stop 2024-05-14T11:18:41.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1572800640 1 -- 172.21.15.116:0/1028240620 reap_dead start 2024-05-14T11:18:41.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 shutdown_connections 2024-05-14T11:18:41.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 >> 172.21.15.116:0/1028240620 conn(0x559a411f3e10 msgr2=0x559a4120a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:41.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 shutdown_connections 2024-05-14T11:18:41.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.311+0000 7f1578ce2080 1 -- 172.21.15.116:0/1028240620 wait complete. 2024-05-14T11:18:41.320 INFO:teuthology.orchestra.run.smithi116.stdout:Yalphaparent_a 2024-05-14T11:18:41.321 DEBUG:teuthology.orchestra.run.smithi116:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: "ancestors": [ 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1099511627776, 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "alpha", 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 4 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:41.369 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1, 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "parent_a", 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 7 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 8, 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: "old_pools": [] 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:41.370 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:41.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:18:41.376 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:18:41.505 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:18:41.509 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:18:41.511 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:18:41.569 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:18:41.569 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:18:41.569 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:18:41.569 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:41.577 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:18:41.578 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:41.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:18:41.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 Processor -- start 2024-05-14T11:18:41.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 -- start start 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc06e5b0 0x7f95dc15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f95dc071fb0 0x7f95dc06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95dc0709e0 con 0x7f95dc06fdb0 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f95dc15f1e0 con 0x7f95dc071fb0 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc070700 con 0x7f95dc06e5b0 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95d3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f95dc06fdb0 0x7f95dc0718a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49484/0 (socket says 172.21.15.116:49484) 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95d3400640 1 -- 172.21.15.116:0/993627621 learned_addr learned my addr 172.21.15.116:0/993627621 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95e1200640 1 --2- 172.21.15.116:0/993627621 >> v2:172.21.15.116:3300/0 conn(0x7f95dc071fb0 0x7f95dc06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.819+0000 7f95dbe00640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc06e5b0 0x7f95dc15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95e1200640 1 -- 172.21.15.116:0/993627621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc06e5b0 msgr2=0x7f95dc15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95e1200640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc06e5b0 0x7f95dc15eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95e1200640 1 -- 172.21.15.116:0/993627621 >> v1:172.21.15.116:6789/0 conn(0x7f95dc06fdb0 legacy=0x7f95dc0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95e1200640 1 -- 172.21.15.116:0/993627621 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc15e1c0 con 0x7f95dc071fb0 2024-05-14T11:18:41.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95e1200640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc071fb0 0x7f95dc06cf90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009d90 tx=0x7f95d401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cb1d9a531136 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95db400640 1 -- 172.21.15.116:0/993627621 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40b2070 con 0x7f95dc071fb0 2024-05-14T11:18:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95db400640 1 -- 172.21.15.116:0/993627621 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d40a58b0 con 0x7f95dc071fb0 2024-05-14T11:18:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.823+0000 7f95db400640 1 -- 172.21.15.116:0/993627621 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40a5c80 con 0x7f95dc071fb0 2024-05-14T11:18:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- 172.21.15.116:0/993627621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc071fb0 msgr2=0x7f95dc06cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc071fb0 0x7f95dc06cf90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009d90 tx=0x7f95d401c8e0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- 172.21.15.116:0/993627621 shutdown_connections 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc071fb0 0x7f95dc06cf90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 --2- 172.21.15.116:0/993627621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc06e5b0 0x7f95dc15eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- 172.21.15.116:0/993627621 >> 172.21.15.116:0/993627621 conn(0x7f95dc0d2fe0 msgr2=0x7f95dc071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- 172.21.15.116:0/993627621 shutdown_connections 2024-05-14T11:18:41.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- 172.21.15.116:0/993627621 wait complete. 2024-05-14T11:18:41.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 Processor -- start 2024-05-14T11:18:41.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.827+0000 7f95e1c00640 1 -- start start 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95dc06e5b0 0x7f95dc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 0x7f95dc167cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc16ad40 con 0x7f95dc06fdb0 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc16abc0 con 0x7f95dc06e5b0 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc16aec0 con 0x7f95dc146820 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43734/0 (socket says 172.21.15.116:43734) 2024-05-14T11:18:41.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 -- 172.21.15.116:0/4191853820 learned_addr learned my addr 172.21.15.116:0/4191853820 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95dbe00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95dc06e5b0 0x7f95dc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1200640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 0x7f95dc167cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 -- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 msgr2=0x7f95dc167cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 0x7f95dc167cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 -- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95dc06e5b0 msgr2=0x7f95dc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95dc06e5b0 0x7f95dc13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 -- 172.21.15.116:0/4191853820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc0ab110 con 0x7f95dc06fdb0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1200640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 0x7f95dc167cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d3400640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c460 tx=0x7f95c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4017070 con 0x7f95dc06fdb0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4005d10 con 0x7f95dc06fdb0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c40072b0 con 0x7f95dc06fdb0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc145780 con 0x7f95dc06fdb0 2024-05-14T11:18:41.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc145d40 con 0x7f95dc06fdb0 2024-05-14T11:18:41.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95dc1624e0 con 0x7f95dc06fdb0 2024-05-14T11:18:41.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.831+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95c401a070 con 0x7f95dc06fdb0 2024-05-14T11:18:41.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.835+0000 7f95d8c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:41.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.835+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f95c408e6b0 con 0x7f95dc06fdb0 2024-05-14T11:18:41.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.835+0000 7f95dbe00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:41.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.835+0000 7f95dbe00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 0x7f95b8071940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b930 tx=0x7f95cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:41.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:41.839+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f95c405b410 con 0x7f95dc06fdb0 2024-05-14T11:18:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.003+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95dc0d0c10 con 0x7f95dc06fdb0 2024-05-14T11:18:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95d8c00640 1 -- 172.21.15.116:0/4191853820 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f95c405ad10 con 0x7f95dc06fdb0 2024-05-14T11:18:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 msgr2=0x7f95b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 0x7f95b8071940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b930 tx=0x7f95cc008040 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 msgr2=0x7f95dc0aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c460 tx=0x7f95c400c920 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 shutdown_connections 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95dc146820 0x7f95dc167cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc06fdb0 0x7f95dc0aabd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 --2- 172.21.15.116:0/4191853820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95dc06e5b0 0x7f95dc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 >> 172.21.15.116:0/4191853820 conn(0x7f95dc0d2fe0 msgr2=0x7f95dc15f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 shutdown_connections 2024-05-14T11:18:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.007+0000 7f95e1c00640 1 -- 172.21.15.116:0/4191853820 wait complete. 2024-05-14T11:18:42.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:18:42.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:42.030 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:18:42.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[38527]: fuse finished with error 0 and tester_r 0 2024-05-14T11:18:42.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:18:42.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:18:42.050 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:18:42.191 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:18:42.192 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:18:42.192 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:42.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:18:42.201 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:18:42.201 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:18:42.201 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:42.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:18:42.210 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:18:42.210 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:18:42.210 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:42.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:18:42.250 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 Processor -- start 2024-05-14T11:18:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 -- start start 2024-05-14T11:18:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 0x7f124c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124c0d1f30 con 0x7f124c06ead0 2024-05-14T11:18:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124c15ce20 con 0x7f124c0cbaf0 2024-05-14T11:18:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1251c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f124c06f5f0 con 0x7f124c0d15e0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1243e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f124c06ead0 0x7f124c06eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49486/0 (socket says 172.21.15.116:49486) 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.395+0000 7f1243e00640 1 -- 172.21.15.116:0/2628174944 learned_addr learned my addr 172.21.15.116:0/2628174944 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124be00640 1 --2- 172.21.15.116:0/2628174944 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 0x7f124c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867312201 0 0) 0x7f1234001120 con 0x7f124c06ead0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1230003530 con 0x7f124c06ead0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91626151 0 0) 0x7f124c15ce20 con 0x7f124c0cbaf0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1234001120 con 0x7f124c0cbaf0 2024-05-14T11:18:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124be00640 1 -- 172.21.15.116:0/2628174944 >> v1:172.21.15.149:6789/0 conn(0x7f124c0cbaf0 legacy=0x7f124c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124be00640 1 -- 172.21.15.116:0/2628174944 >> v1:172.21.15.116:6789/0 conn(0x7f124c06ead0 legacy=0x7f124c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124be00640 1 -- 172.21.15.116:0/2628174944 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c06f850 con 0x7f124c0d15e0 2024-05-14T11:18:42.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.399+0000 7f124be00640 1 --2- 172.21.15.116:0/2628174944 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 0x7f124c0d19f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123c0097f0 tx=0x7f123c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37df36245f3b10a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a6040 con 0x7f124c0d15e0 2024-05-14T11:18:42.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c0af070 con 0x7f124c0d15e0 2024-05-14T11:18:42.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f124b400640 1 -- 172.21.15.116:0/2628174944 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0aa840 con 0x7f124c0d15e0 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- 172.21.15.116:0/2628174944 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 msgr2=0x7f124c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 --2- 172.21.15.116:0/2628174944 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 0x7f124c0d19f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123c0097f0 tx=0x7f123c01c4e0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- 172.21.15.116:0/2628174944 shutdown_connections 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 --2- 172.21.15.116:0/2628174944 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c0d15e0 0x7f124c0d19f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- 172.21.15.116:0/2628174944 >> 172.21.15.116:0/2628174944 conn(0x7f124c0b4530 msgr2=0x7f124c0b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- 172.21.15.116:0/2628174944 shutdown_connections 2024-05-14T11:18:42.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- 172.21.15.116:0/2628174944 wait complete. 2024-05-14T11:18:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 Processor -- start 2024-05-14T11:18:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.403+0000 7f1251c00640 1 -- start start 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c06ead0 0x7f124c099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 0x7f124c0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f124c16a580 con 0x7f124c0cbaf0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f124c070170 con 0x7f124c0d15e0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f124c16a700 con 0x7f124c06ead0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43746/0 (socket says 172.21.15.116:43746) 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 -- 172.21.15.116:0/2173153768 learned_addr learned my addr 172.21.15.116:0/2173153768 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251200640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 0x7f124c0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f124be00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c06ead0 0x7f124c099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 -- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c06ead0 msgr2=0x7f124c099d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c06ead0 0x7f124c099d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 -- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 msgr2=0x7f124c0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 0x7f124c0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 -- 172.21.15.116:0/2173153768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c09a2c0 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251200640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 0x7f124c0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1243e00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f123400c700 tx=0x7f123400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234017070 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1234005ac0 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234007070 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c15c3c0 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c0c7050 con 0x7f124c0cbaf0 2024-05-14T11:18:42.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.407+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c050880 con 0x7f124c0cbaf0 2024-05-14T11:18:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.411+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f123401a070 con 0x7f124c0cbaf0 2024-05-14T11:18:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.411+0000 7f1248c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 0x7f1228071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.411+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f123408e730 con 0x7f124c0cbaf0 2024-05-14T11:18:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.411+0000 7f124be00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 0x7f1228071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.411+0000 7f124be00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 0x7f1228071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f123c0ae000 tx=0x7f123c0b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.415+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f124c0d1f30 con 0x7f124c0cbaf0 2024-05-14T11:18:42.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.611+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f124c0b04e0 con 0x7f124c0cbaf0 2024-05-14T11:18:42.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.611+0000 7f1248c00640 1 -- 172.21.15.116:0/2173153768 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f123405af80 con 0x7f124c0cbaf0 2024-05-14T11:18:42.616 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:42.616 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:42.616 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 msgr2=0x7f1228071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 0x7f1228071910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f123c0ae000 tx=0x7f123c0b5040 comp rx=0 tx=0).stop 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 msgr2=0x7f124c0c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f123400c700 tx=0x7f123400cbc0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 shutdown_connections 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f122806f400 0x7f1228071910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f124c0d15e0 0x7f124c0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f124c0cbaf0 0x7f124c0c4d10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 --2- 172.21.15.116:0/2173153768 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f124c06ead0 0x7f124c099d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 >> 172.21.15.116:0/2173153768 conn(0x7f124c0b4530 msgr2=0x7f124c0b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:42.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 shutdown_connections 2024-05-14T11:18:42.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.615+0000 7f1251c00640 1 -- 172.21.15.116:0/2173153768 wait complete. 2024-05-14T11:18:42.637 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:42.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.779+0000 7f7244600640 1 Processor -- start 2024-05-14T11:18:42.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.779+0000 7f7244600640 1 -- start start 2024-05-14T11:18:42.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7244600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7244600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f723c072a30 0x7f723c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7244600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c06cc40 con 0x7f723c06f730 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7244600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f723c06db20 con 0x7f723c072a30 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7244600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f723c06cf10 con 0x7f723c0b0770 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f723c072a30 0x7f723c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f723c072a30 0x7f723c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43754/0 (socket says 172.21.15.116:43754) 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 -- 172.21.15.116:0/2071096833 learned_addr learned my addr 172.21.15.116:0/2071096833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:42.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7241c00640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 -- 172.21.15.116:0/2071096833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 msgr2=0x7f723c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:42.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 -- 172.21.15.116:0/2071096833 >> v1:172.21.15.149:6789/0 conn(0x7f723c06f730 legacy=0x7f723c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.783+0000 7f7242600640 1 -- 172.21.15.116:0/2071096833 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c06d1f0 con 0x7f723c072a30 2024-05-14T11:18:42.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7242600640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c072e40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f723400d150 tx=0x7f723401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3ddd72fc3638aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f723be00640 1 -- 172.21.15.116:0/2071096833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340ad070 con 0x7f723c072a30 2024-05-14T11:18:42.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f723be00640 1 -- 172.21.15.116:0/2071096833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72340a7bc0 con 0x7f723c072a30 2024-05-14T11:18:42.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f723be00640 1 -- 172.21.15.116:0/2071096833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340b1400 con 0x7f723c072a30 2024-05-14T11:18:42.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 -- 172.21.15.116:0/2071096833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 msgr2=0x7f723c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c072e40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f723400d150 tx=0x7f723401eb70 comp rx=0 tx=0).stop 2024-05-14T11:18:42.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 -- 172.21.15.116:0/2071096833 shutdown_connections 2024-05-14T11:18:42.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c072e40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 --2- 172.21.15.116:0/2071096833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 -- 172.21.15.116:0/2071096833 >> 172.21.15.116:0/2071096833 conn(0x7f723c0b4620 msgr2=0x7f723c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:42.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 -- 172.21.15.116:0/2071096833 shutdown_connections 2024-05-14T11:18:42.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.787+0000 7f7244600640 1 -- 172.21.15.116:0/2071096833 wait complete. 2024-05-14T11:18:42.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 Processor -- start 2024-05-14T11:18:42.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- start start 2024-05-14T11:18:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 0x7f723c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f723c073500 con 0x7f723c072a30 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f723c073380 con 0x7f723c06f730 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f723c073680 con 0x7f723c0b0770 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43756/0 (socket says 172.21.15.116:43756) 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 -- 172.21.15.116:0/611747828 learned_addr learned my addr 172.21.15.116:0/611747828 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7242600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241c00640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 0x7f723c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 -- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 msgr2=0x7f723c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 -- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 msgr2=0x7f723c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 0x7f723c1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 -- 172.21.15.116:0/611747828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c15d850 con 0x7f723c072a30 2024-05-14T11:18:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241c00640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 0x7f723c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7241200640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f722800c3c0 tx=0x7f722800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228017070 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72280052d0 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228003840 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c154600 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.791+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c15dac0 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c06a9c0 con 0x7f723c072a30 2024-05-14T11:18:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f722801a070 con 0x7f723c072a30 2024-05-14T11:18:42.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7239600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 0x7f721c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:42.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f722808e7a0 con 0x7f723c072a30 2024-05-14T11:18:42.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7241c00640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 0x7f721c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.795+0000 7f7241c00640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 0x7f721c071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f723c0d1240 tx=0x7f7230008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:42.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.799+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7228005480 con 0x7f723c072a30 2024-05-14T11:18:42.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.991+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f723c1547d0 con 0x7f723c072a30 2024-05-14T11:18:42.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.991+0000 7f7239600640 1 -- 172.21.15.116:0/611747828 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f722805b460 con 0x7f723c072a30 2024-05-14T11:18:42.995 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:42.995 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:42.995 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 msgr2=0x7f721c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 0x7f721c071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f723c0d1240 tx=0x7f7230008040 comp rx=0 tx=0).stop 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 msgr2=0x7f723c1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f722800c3c0 tx=0x7f722800c880 comp rx=0 tx=0).stop 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 shutdown_connections 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f721c06f400 0x7f721c071910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f723c0b0770 0x7f723c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f723c072a30 0x7f723c1540c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 --2- 172.21.15.116:0/611747828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f723c06f730 0x7f723c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:42.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 >> 172.21.15.116:0/611747828 conn(0x7f723c0b4620 msgr2=0x7f723c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:42.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 shutdown_connections 2024-05-14T11:18:42.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:42.995+0000 7f7244600640 1 -- 172.21.15.116:0/611747828 wait complete. 2024-05-14T11:18:43.016 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:43.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 Processor -- start 2024-05-14T11:18:43.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 -- start start 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2c80af420 0x7fe2c8074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c80d1d80 0x7fe2c806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c815d340 con 0x7fe2c8074b70 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c815d620 con 0x7fe2c80af420 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.159+0000 7fe2cf200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c815d900 con 0x7fe2c80d1d80 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2c80af420 0x7fe2c8074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2c80af420 0x7fe2c8074630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43768/0 (socket says 172.21.15.116:43768) 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 -- 172.21.15.116:0/1138292078 learned_addr learned my addr 172.21.15.116:0/1138292078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2cd200640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c80d1d80 0x7fe2c806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 -- 172.21.15.116:0/1138292078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c80d1d80 msgr2=0x7fe2c806edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c80d1d80 0x7fe2c806edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 -- 172.21.15.116:0/1138292078 >> v1:172.21.15.149:6789/0 conn(0x7fe2c8074b70 legacy=0x7fe2c80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 -- 172.21.15.116:0/1138292078 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c815dbe0 con 0x7fe2c80af420 2024-05-14T11:18:43.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7e00640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c8074630 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009e50 tx=0x7fe2b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fb4f7e6c562e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7400640 1 -- 172.21.15.116:0/1138292078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80a58f0 con 0x7fe2c80af420 2024-05-14T11:18:43.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7400640 1 -- 172.21.15.116:0/1138292078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b80a5a90 con 0x7fe2c80af420 2024-05-14T11:18:43.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.163+0000 7fe2c7400640 1 -- 172.21.15.116:0/1138292078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80a5e60 con 0x7fe2c80af420 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- 172.21.15.116:0/1138292078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 msgr2=0x7fe2c8074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c8074630 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009e50 tx=0x7fe2b801ca30 comp rx=0 tx=0).stop 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- 172.21.15.116:0/1138292078 shutdown_connections 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c80d1d80 0x7fe2c806edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1138292078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c8074630 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- 172.21.15.116:0/1138292078 >> 172.21.15.116:0/1138292078 conn(0x7fe2c80b4620 msgr2=0x7fe2c80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- 172.21.15.116:0/1138292078 shutdown_connections 2024-05-14T11:18:43.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- 172.21.15.116:0/1138292078 wait complete. 2024-05-14T11:18:43.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 Processor -- start 2024-05-14T11:18:43.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.167+0000 7fe2cf200640 1 -- start start 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c8074b70 0x7fe2c80d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 0x7fe2c815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c806c880 con 0x7fe2c80af420 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c806c700 con 0x7fe2c815e470 2024-05-14T11:18:43.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c806ca00 con 0x7fe2c8074b70 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43772/0 (socket says 172.21.15.116:43772) 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 -- 172.21.15.116:0/1218941827 learned_addr learned my addr 172.21.15.116:0/1218941827 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cd200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 0x7fe2c815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2c7e00640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c8074b70 0x7fe2c80d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 -- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c8074b70 msgr2=0x7fe2c80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c8074b70 0x7fe2c80d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 -- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 msgr2=0x7fe2c815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 0x7fe2c815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 -- 172.21.15.116:0/1218941827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c815edc0 con 0x7fe2c80af420 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cd200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 0x7fe2c815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2bf400640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c420 tx=0x7fe2b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0017070 con 0x7fe2c80af420 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b00052d0 con 0x7fe2c80af420 2024-05-14T11:18:43.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c80d07e0 con 0x7fe2c80af420 2024-05-14T11:18:43.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00054c0 con 0x7fe2c80af420 2024-05-14T11:18:43.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c80ce410 con 0x7fe2c80af420 2024-05-14T11:18:43.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c806a9c0 con 0x7fe2c80af420 2024-05-14T11:18:43.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.171+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2b001a070 con 0x7fe2c80af420 2024-05-14T11:18:43.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.175+0000 7fe2c4c00640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 0x7fe2a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.175+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fe2b008e730 con 0x7fe2c80af420 2024-05-14T11:18:43.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.175+0000 7fe2c7e00640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 0x7fe2a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.175+0000 7fe2c7e00640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 0x7fe2a4071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80ae000 tx=0x7fe2b80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.179+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe2c815edc0 con 0x7fe2c80af420 2024-05-14T11:18:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.335+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2c8075dc0 con 0x7fe2c80af420 2024-05-14T11:18:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2c4c00640 1 -- 172.21.15.116:0/1218941827 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16990 (secure 0 0 0) 0x7fe2b005ae30 con 0x7fe2c80af420 2024-05-14T11:18:43.341 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":49,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:57.635905+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:53.726817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:54.993539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:43.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 msgr2=0x7fe2a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 0x7fe2a4071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80ae000 tx=0x7fe2b80a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 msgr2=0x7fe2c814a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c420 tx=0x7fe2b000c8e0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 shutdown_connections 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2a406f430 0x7fe2a4071940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2c815e470 0x7fe2c815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c80af420 0x7fe2c814a790 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 --2- 172.21.15.116:0/1218941827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe2c8074b70 0x7fe2c80d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 >> 172.21.15.116:0/1218941827 conn(0x7fe2c80b4620 msgr2=0x7fe2c81530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 shutdown_connections 2024-05-14T11:18:43.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.339+0000 7fe2cf200640 1 -- 172.21.15.116:0/1218941827 wait complete. 2024-05-14T11:18:43.363 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:18:43.364 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:43.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.507+0000 7f7ede600640 1 Processor -- start 2024-05-14T11:18:43.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.507+0000 7f7ede600640 1 -- start start 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ede600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7ed80b0770 0x7f7ed806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ede600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7ed806f730 0x7f7ed806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ede600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed8070080 con 0x7f7ed815fc00 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ede600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed806db20 con 0x7f7ed80b0770 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ede600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed806cc40 con 0x7f7ed806f730 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7ed80b0770 0x7f7ed806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7ed815fc00 0x7f7ed8160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49490/0 (socket says 172.21.15.116:49490) 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7e00640 1 -- 172.21.15.116:0/540308787 learned_addr learned my addr 172.21.15.116:0/540308787 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:43.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ecea00640 1 --2- 172.21.15.116:0/540308787 >> v2:172.21.15.149:3300/0 conn(0x7f7ed806f730 0x7f7ed806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7400640 1 -- 172.21.15.116:0/540308787 >> v2:172.21.15.149:3300/0 conn(0x7f7ed806f730 msgr2=0x7f7ed806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:43.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7400640 1 --2- 172.21.15.116:0/540308787 >> v2:172.21.15.149:3300/0 conn(0x7f7ed806f730 0x7f7ed806fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7400640 1 -- 172.21.15.116:0/540308787 >> v1:172.21.15.116:6789/0 conn(0x7f7ed815fc00 legacy=0x7f7ed8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed7400640 1 -- 172.21.15.116:0/540308787 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed806cf10 con 0x7f7ed80b0770 2024-05-14T11:18:43.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.511+0000 7f7ed6a00640 1 -- 172.21.15.116:0/540308787 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265626309 0 0) 0x7f7ed00014a0 con 0x7f7ed815fc00 2024-05-14T11:18:43.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ed7400640 1 --2- 172.21.15.116:0/540308787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed806c700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806f4e0 tx=0x7f7ec801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca7ec81f69a9ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ed6a00640 1 -- 172.21.15.116:0/540308787 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b0070 con 0x7f7ed80b0770 2024-05-14T11:18:43.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ed6a00640 1 -- 172.21.15.116:0/540308787 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80ab600 con 0x7f7ed80b0770 2024-05-14T11:18:43.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ed6a00640 1 -- 172.21.15.116:0/540308787 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80ab9d0 con 0x7f7ed80b0770 2024-05-14T11:18:43.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 -- 172.21.15.116:0/540308787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 msgr2=0x7f7ed806c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 --2- 172.21.15.116:0/540308787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed806c700 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806f4e0 tx=0x7f7ec801c3c0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 -- 172.21.15.116:0/540308787 shutdown_connections 2024-05-14T11:18:43.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 --2- 172.21.15.116:0/540308787 >> v2:172.21.15.149:3300/0 conn(0x7f7ed806f730 0x7f7ed806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 --2- 172.21.15.116:0/540308787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed806c700 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 -- 172.21.15.116:0/540308787 >> 172.21.15.116:0/540308787 conn(0x7f7ed80b4620 msgr2=0x7f7ed80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:43.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 -- 172.21.15.116:0/540308787 shutdown_connections 2024-05-14T11:18:43.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.515+0000 7f7ede600640 1 -- 172.21.15.116:0/540308787 wait complete. 2024-05-14T11:18:43.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 Processor -- start 2024-05-14T11:18:43.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- start start 2024-05-14T11:18:43.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7ed806f730 0x7f7ed80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 0x7f7ed815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8073a40 con 0x7f7ed80b0770 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed806dfb0 con 0x7f7ed815fc00 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed8166f90 con 0x7f7ed806f730 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43794/0 (socket says 172.21.15.116:43794) 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 -- 172.21.15.116:0/572826093 learned_addr learned my addr 172.21.15.116:0/572826093 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ed7400640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7ed806f730 0x7f7ed80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ed7e00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 0x7f7ed815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 -- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7ed806f730 msgr2=0x7f7ed80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7ed806f730 0x7f7ed80cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 -- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 msgr2=0x7f7ed815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 0x7f7ed815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 -- 172.21.15.116:0/572826093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80d0510 con 0x7f7ed80b0770 2024-05-14T11:18:43.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ed7e00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 0x7f7ed815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecea00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e870 tx=0x7f7ec000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0017070 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec000c940 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000cd10 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed80ce050 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.519+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed80ce610 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed806a9c0 con 0x7f7ed80b0770 2024-05-14T11:18:43.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ec001a070 con 0x7f7ed80b0770 2024-05-14T11:18:43.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ecfe00640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 0x7f7eb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f7ec008f370 con 0x7f7ed80b0770 2024-05-14T11:18:43.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ed7400640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 0x7f7eb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.523+0000 7f7ed7400640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 0x7f7eb4071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80729c0 tx=0x7f7ec80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.527+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7ec005b9e0 con 0x7f7ed80b0770 2024-05-14T11:18:43.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed806f510 con 0x7f7ed80b0770 2024-05-14T11:18:43.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ecfe00640 1 -- 172.21.15.116:0/572826093 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7f7ec005fce0 con 0x7f7ed80b0770 2024-05-14T11:18:43.725 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:43.725 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:43.725 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:43.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 msgr2=0x7f7eb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 0x7f7eb4071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80729c0 tx=0x7f7ec80a6000 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 msgr2=0x7f7ed80cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e870 tx=0x7f7ec000ed30 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 shutdown_connections 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7eb406f400 0x7f7eb4071910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ed815fc00 0x7f7ed815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ed80b0770 0x7f7ed80cf0f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 --2- 172.21.15.116:0/572826093 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7ed806f730 0x7f7ed80cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 >> 172.21.15.116:0/572826093 conn(0x7f7ed80b4620 msgr2=0x7f7ed80b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 shutdown_connections 2024-05-14T11:18:43.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.723+0000 7f7ede600640 1 -- 172.21.15.116:0/572826093 wait complete. 2024-05-14T11:18:43.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:43.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 Processor -- start 2024-05-14T11:18:43.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 -- start start 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc9100d0bf0 0x7fc9100d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9100b5660 0x7fc9100b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9100b1f10 con 0x7fc910163860 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc9100cf6d0 con 0x7fc9100b5660 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc916600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc9100cf9b0 con 0x7fc9100d0bf0 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9100b5660 0x7fc9100b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9100b5660 0x7fc9100b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43804/0 (socket says 172.21.15.116:43804) 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 -- 172.21.15.116:0/1408008642 learned_addr learned my addr 172.21.15.116:0/1408008642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc906a00640 1 --1- 172.21.15.116:0/1408008642 >> v1:172.21.15.116:6789/0 conn(0x7fc910163860 0x7fc910163c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49496/0 (socket says 172.21.15.116:49496) 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90f400640 1 --2- 172.21.15.116:0/1408008642 >> v2:172.21.15.149:3300/0 conn(0x7fc9100d0bf0 0x7fc9100d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 -- 172.21.15.116:0/1408008642 >> v2:172.21.15.149:3300/0 conn(0x7fc9100d0bf0 msgr2=0x7fc9100d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 --2- 172.21.15.116:0/1408008642 >> v2:172.21.15.149:3300/0 conn(0x7fc9100d0bf0 0x7fc9100d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 -- 172.21.15.116:0/1408008642 >> v1:172.21.15.116:6789/0 conn(0x7fc910163860 legacy=0x7fc910163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.891+0000 7fc90fe00640 1 -- 172.21.15.116:0/1408008642 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9100cfc90 con 0x7fc9100b5660 2024-05-14T11:18:43.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc90fe00640 1 --2- 172.21.15.116:0/1408008642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100b5660 0x7fc9100b5a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc9100b72f0 tx=0x7fc90801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb938fd11b434931 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc90ea00640 1 -- 172.21.15.116:0/1408008642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a5620 con 0x7fc9100b5660 2024-05-14T11:18:43.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc90ea00640 1 -- 172.21.15.116:0/1408008642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9080a57c0 con 0x7fc9100b5660 2024-05-14T11:18:43.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc90ea00640 1 -- 172.21.15.116:0/1408008642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a5b90 con 0x7fc9100b5660 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 -- 172.21.15.116:0/1408008642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100b5660 msgr2=0x7fc9100b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 --2- 172.21.15.116:0/1408008642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100b5660 0x7fc9100b5a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc9100b72f0 tx=0x7fc90801c960 comp rx=0 tx=0).stop 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 -- 172.21.15.116:0/1408008642 shutdown_connections 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 --2- 172.21.15.116:0/1408008642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100b5660 0x7fc9100b5a70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 --2- 172.21.15.116:0/1408008642 >> v2:172.21.15.149:3300/0 conn(0x7fc9100d0bf0 0x7fc9100d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 -- 172.21.15.116:0/1408008642 >> 172.21.15.116:0/1408008642 conn(0x7fc910071370 msgr2=0x7fc910071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:43.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 -- 172.21.15.116:0/1408008642 shutdown_connections 2024-05-14T11:18:43.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.895+0000 7fc916600640 1 -- 172.21.15.116:0/1408008642 wait complete. 2024-05-14T11:18:43.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 Processor -- start 2024-05-14T11:18:43.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 -- start start 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9100b5660 0x7fc9101fd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d0bf0 0x7fc910141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc91016cd40 con 0x7fc910163860 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc91016cbc0 con 0x7fc9100b5660 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc916600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc91016cec0 con 0x7fc9100d0bf0 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc90fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc90fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43816/0 (socket says 172.21.15.116:43816) 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.899+0000 7fc90fe00640 1 -- 172.21.15.116:0/76768185 learned_addr learned my addr 172.21.15.116:0/76768185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 -- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d0bf0 msgr2=0x7fc910141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d0bf0 0x7fc910141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 -- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9100b5660 msgr2=0x7fc9101fd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9100b5660 0x7fc9101fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 -- 172.21.15.116:0/76768185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc910156e00 con 0x7fc910163860 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90fe00640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc90801a590 tx=0x7fc908019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908009510 con 0x7fc910163860 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9080be070 con 0x7fc910163860 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101570e0 con 0x7fc910163860 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080b88d0 con 0x7fc910163860 2024-05-14T11:18:43.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101576a0 con 0x7fc910163860 2024-05-14T11:18:43.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91006a9c0 con 0x7fc910163860 2024-05-14T11:18:43.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc9080b8a70 con 0x7fc910163860 2024-05-14T11:18:43.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 0x7fc8ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:43.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fc90812a250 con 0x7fc910163860 2024-05-14T11:18:43.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.903+0000 7fc90f400640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 0x7fc8ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:43.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.907+0000 7fc90f400640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 0x7fc8ec071910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc900006aa0 tx=0x7fc900006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:43.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:43.911+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc9080c8020 con 0x7fc910163860 2024-05-14T11:18:44.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9100b20e0 con 0x7fc910163860 2024-05-14T11:18:44.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc907e00640 1 -- 172.21.15.116:0/76768185 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5193 (secure 0 0 0) 0x7fc9080165e0 con 0x7fc910163860 2024-05-14T11:18:44.102 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:44.102 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:17:56.025284+0000","modified":"2024-05-14T11:17:59.049985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/2735194331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2735194331},{"type":"v1","addr":"172.21.15.116:6837","nonce":2735194331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5319,"qdb_cluster":[5319]},"id":3}]} 2024-05-14T11:18:44.102 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 26 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 msgr2=0x7fc8ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 0x7fc8ec071910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc900006aa0 tx=0x7fc900006a30 comp rx=0 tx=0).stop 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 msgr2=0x7fc910161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc90801a590 tx=0x7fc908019700 comp rx=0 tx=0).stop 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 shutdown_connections 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc8ec06f400 0x7fc8ec071910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc910163860 0x7fc910161bd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc9100d0bf0 0x7fc910141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 --2- 172.21.15.116:0/76768185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9100b5660 0x7fc9101fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 >> 172.21.15.116:0/76768185 conn(0x7fc910071370 msgr2=0x7fc910163c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:44.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 shutdown_connections 2024-05-14T11:18:44.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.099+0000 7fc916600640 1 -- 172.21.15.116:0/76768185 wait complete. 2024-05-14T11:18:44.123 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:44.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 Processor -- start 2024-05-14T11:18:44.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 -- start start 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd530160d50 0x7fd530161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5300b5520 0x7fd5300b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd53006fa80 con 0x7fd53015c990 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd53006f6d0 con 0x7fd530160d50 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd534a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd53006f240 con 0x7fd5300b5520 2024-05-14T11:18:44.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd52de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd53015c990 0x7fd530160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49512/0 (socket says 172.21.15.116:49512) 2024-05-14T11:18:44.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd52de00640 1 -- 172.21.15.116:0/2579864114 learned_addr learned my addr 172.21.15.116:0/2579864114 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:44.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd525400640 1 --2- 172.21.15.116:0/2579864114 >> v2:172.21.15.149:3300/0 conn(0x7fd530160d50 0x7fd530161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd52e800640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5300b5520 0x7fd5300b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd52d400640 1 -- 172.21.15.116:0/2579864114 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255690658 0 0) 0x7fd53006fa80 con 0x7fd53015c990 2024-05-14T11:18:44.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd52d400640 1 -- 172.21.15.116:0/2579864114 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd514003530 con 0x7fd53015c990 2024-05-14T11:18:44.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd525400640 1 -- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5300b5520 msgr2=0x7fd5300b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:44.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd525400640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5300b5520 0x7fd5300b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd525400640 1 -- 172.21.15.116:0/2579864114 >> v1:172.21.15.116:6789/0 conn(0x7fd53015c990 legacy=0x7fd530160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.267+0000 7fd525400640 1 -- 172.21.15.116:0/2579864114 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53006fd40 con 0x7fd530160d50 2024-05-14T11:18:44.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd525400640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd530161160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd518009d90 tx=0x7fd51801c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3c4f31d1f68b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd52d400640 1 -- 172.21.15.116:0/2579864114 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180b2070 con 0x7fd530160d50 2024-05-14T11:18:44.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd52d400640 1 -- 172.21.15.116:0/2579864114 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5180a5960 con 0x7fd530160d50 2024-05-14T11:18:44.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd52d400640 1 -- 172.21.15.116:0/2579864114 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a5d10 con 0x7fd530160d50 2024-05-14T11:18:44.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 -- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 msgr2=0x7fd530161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd530161160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd518009d90 tx=0x7fd51801c880 comp rx=0 tx=0).stop 2024-05-14T11:18:44.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 -- 172.21.15.116:0/2579864114 shutdown_connections 2024-05-14T11:18:44.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5300b5520 0x7fd5300b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 --2- 172.21.15.116:0/2579864114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd530161160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.271+0000 7fd534a00640 1 -- 172.21.15.116:0/2579864114 >> 172.21.15.116:0/2579864114 conn(0x7fd5300d3120 msgr2=0x7fd530071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:44.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- 172.21.15.116:0/2579864114 shutdown_connections 2024-05-14T11:18:44.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- 172.21.15.116:0/2579864114 wait complete. 2024-05-14T11:18:44.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 Processor -- start 2024-05-14T11:18:44.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- start start 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd53015c990 0x7fd53014e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd5301581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd530169950 con 0x7fd5300b5520 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301697d0 con 0x7fd530160d50 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd534a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd530169ad0 con 0x7fd53015c990 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.275+0000 7fd52de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43818/0 (socket says 172.21.15.116:43818) 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 -- 172.21.15.116:0/3842141771 learned_addr learned my addr 172.21.15.116:0/3842141771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd525400640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd53015c990 0x7fd53014e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52e800640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd5301581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 -- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd53015c990 msgr2=0x7fd53014e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd53015c990 0x7fd53014e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 -- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 msgr2=0x7fd5301581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd5301581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 -- 172.21.15.116:0/3842141771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5300b2200 con 0x7fd5300b5520 2024-05-14T11:18:44.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd52de00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c460 tx=0x7fd52000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520017070 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd520005cf0 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520007270 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd530152da0 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd530153360 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53006a9c0 con 0x7fd5300b5520 2024-05-14T11:18:44.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.279+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd52001a070 con 0x7fd5300b5520 2024-05-14T11:18:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.283+0000 7fd526a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 0x7fd50c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.283+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd52008e6c0 con 0x7fd5300b5520 2024-05-14T11:18:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.283+0000 7fd525400640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 0x7fd50c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.283+0000 7fd525400640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 0x7fd50c071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5180af000 tx=0x7fd5180a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.287+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd52005adc0 con 0x7fd5300b5520 2024-05-14T11:18:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.447+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd53015fbe0 con 0x7fd5300b5520 2024-05-14T11:18:44.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd526a00640 1 -- 172.21.15.116:0/3842141771 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16990 (secure 0 0 0) 0x7fd52005f0c0 con 0x7fd5300b5520 2024-05-14T11:18:44.453 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:44.453 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":49,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:17:57.635905+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:17:53.726817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-14T11:17:54.993539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:44.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 msgr2=0x7fd50c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 0x7fd50c071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5180af000 tx=0x7fd5180a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:44.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 msgr2=0x7fd530152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c460 tx=0x7fd52000c920 comp rx=0 tx=0).stop 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 shutdown_connections 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50c06f400 0x7fd50c071910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd530160d50 0x7fd5301581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd53015c990 0x7fd53014e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 --2- 172.21.15.116:0/3842141771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5300b5520 0x7fd530152550 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 >> 172.21.15.116:0/3842141771 conn(0x7fd5300d3120 msgr2=0x7fd5300b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 shutdown_connections 2024-05-14T11:18:44.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.451+0000 7fd534a00640 1 -- 172.21.15.116:0/3842141771 wait complete. 2024-05-14T11:18:44.476 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:18:44.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.619+0000 7fd469800640 1 Processor -- start 2024-05-14T11:18:44.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.619+0000 7fd469800640 1 -- start start 2024-05-14T11:18:44.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.619+0000 7fd469800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 0x7fd4640aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd469800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd4640d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd469800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd464161600 0x7fd464161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd469800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd4640ae100 con 0x7fd464161600 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd469800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd4640ae870 con 0x7fd46415db30 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd469800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd46406ff20 con 0x7fd4640d1670 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd464161600 0x7fd464161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd464161600 0x7fd464161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43826/0 (socket says 172.21.15.116:43826) 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 -- 172.21.15.116:0/1124234548 learned_addr learned my addr 172.21.15.116:0/1124234548 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd45be00640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd4640d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd462a00640 1 --2- 172.21.15.116:0/1124234548 >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 0x7fd4640aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 -- 172.21.15.116:0/1124234548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 msgr2=0x7fd4640d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd4640d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 -- 172.21.15.116:0/1124234548 >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 msgr2=0x7fd4640aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 --2- 172.21.15.116:0/1124234548 >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 0x7fd4640aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 -- 172.21.15.116:0/1124234548 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4640d1fc0 con 0x7fd464161600 2024-05-14T11:18:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd462a00640 1 --2- 172.21.15.116:0/1124234548 >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 0x7fd4640aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd463400640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd464161600 0x7fd464161a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd464071f20 tx=0x7fd45c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4768d52d49a1fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.623+0000 7fd462000640 1 -- 172.21.15.116:0/1124234548 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0a56d0 con 0x7fd464161600 2024-05-14T11:18:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd462000640 1 -- 172.21.15.116:0/1124234548 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0a5870 con 0x7fd464161600 2024-05-14T11:18:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd462000640 1 -- 172.21.15.116:0/1124234548 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0a5c40 con 0x7fd464161600 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- 172.21.15.116:0/1124234548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd464161600 msgr2=0x7fd464161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd464161600 0x7fd464161a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd464071f20 tx=0x7fd45c01c960 comp rx=0 tx=0).stop 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- 172.21.15.116:0/1124234548 shutdown_connections 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd464161600 0x7fd464161a10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 --2- 172.21.15.116:0/1124234548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd4640d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 --2- 172.21.15.116:0/1124234548 >> v2:172.21.15.149:3300/0 conn(0x7fd46415db30 0x7fd4640aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- 172.21.15.116:0/1124234548 >> 172.21.15.116:0/1124234548 conn(0x7fd4640b4620 msgr2=0x7fd4640b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:44.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- 172.21.15.116:0/1124234548 shutdown_connections 2024-05-14T11:18:44.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- 172.21.15.116:0/1124234548 wait complete. 2024-05-14T11:18:44.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 Processor -- start 2024-05-14T11:18:44.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.627+0000 7fd469800640 1 -- start start 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd464160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464161600 0x7fd464150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd464162360 con 0x7fd46415db30 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd4640aefe0 con 0x7fd464161600 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd46415df40 con 0x7fd4640d1670 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43838/0 (socket says 172.21.15.116:43838) 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 -- 172.21.15.116:0/233479204 learned_addr learned my addr 172.21.15.116:0/233479204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd462a00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd464160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd463400640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464161600 0x7fd464150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 -- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 msgr2=0x7fd464160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd464160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 -- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464161600 msgr2=0x7fd464150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464161600 0x7fd464150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 -- 172.21.15.116:0/233479204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd464146b90 con 0x7fd46415db30 2024-05-14T11:18:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45be00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e870 tx=0x7fd44c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c017070 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c00c940 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00ccf0 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46409ae60 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd464148350 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.631+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46406a9c0 con 0x7fd46415db30 2024-05-14T11:18:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.635+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd44c01a070 con 0x7fd46415db30 2024-05-14T11:18:44.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.635+0000 7fd45aa00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 0x7fd440071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.635+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fd44c08f2a0 con 0x7fd46415db30 2024-05-14T11:18:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.635+0000 7fd462a00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 0x7fd440071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.635+0000 7fd462a00640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 0x7fd440071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd454006e90 tx=0x7fd454008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.639+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd464146b90 con 0x7fd46415db30 2024-05-14T11:18:44.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.831+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd46415c520 con 0x7fd46415db30 2024-05-14T11:18:44.835 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:18:44.831+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:18:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.835+0000 7fd45aa00640 1 -- 172.21.15.116:0/233479204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fd44c05b9b0 con 0x7fd46415db30 2024-05-14T11:18:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 msgr2=0x7fd440071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 0x7fd440071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd454006e90 tx=0x7fd454008040 comp rx=0 tx=0).stop 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 msgr2=0x7fd46409a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e870 tx=0x7fd44c00ec70 comp rx=0 tx=0).stop 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 shutdown_connections 2024-05-14T11:18:44.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd44006f400 0x7fd440071910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd464161600 0x7fd464150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd46415db30 0x7fd46409a920 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 --2- 172.21.15.116:0/233479204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd4640d1670 0x7fd464160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 >> 172.21.15.116:0/233479204 conn(0x7fd4640b4620 msgr2=0x7fd46414c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 shutdown_connections 2024-05-14T11:18:44.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:44.839+0000 7fd469800640 1 -- 172.21.15.116:0/233479204 wait complete. 2024-05-14T11:18:44.851 INFO:tasks.ceph.mds.c.smithi116.stderr: -5021> 2024-05-14T11:17:58.608+0000 7f2ef1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:18:44.856 INFO:tasks.ceph.mds.c.smithi116.stderr: -3101> 2024-05-14T11:18:10.108+0000 7f2eec800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5475=0-4194304@1} caps={5475=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5475 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55fb3cb39180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:18:44.864 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:18:44.971 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:18:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 Processor -- start 2024-05-14T11:18:45.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 -- start start 2024-05-14T11:18:45.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a980aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0a980d1670 0x7f0a980d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9806ff20 con 0x7f0a98161600 2024-05-14T11:18:45.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9806d710 con 0x7f0a980d1670 2024-05-14T11:18:45.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0aa0600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980ae100 con 0x7f0a9815db30 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9e600640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0a98161600 0x7f0a98161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49518/0 (socket says 172.21.15.116:49518) 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9e600640 1 -- 172.21.15.116:0/1538063295 learned_addr learned my addr 172.21.15.116:0/1538063295 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768573112 0 0) 0x7f0a9806ff20 con 0x7f0a98161600 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a80003530 con 0x7f0a98161600 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a980aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9d200640 1 --2- 172.21.15.116:0/1538063295 >> v2:172.21.15.149:3300/0 conn(0x7f0a980d1670 0x7f0a980d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3375777279 0 0) 0x7f0a80003530 con 0x7f0a98161600 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 msgr2=0x7f0a980aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a980aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 >> v2:172.21.15.149:3300/0 conn(0x7f0a980d1670 msgr2=0x7f0a980d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 --2- 172.21.15.116:0/1538063295 >> v2:172.21.15.149:3300/0 conn(0x7f0a980d1670 0x7f0a980d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980d1fc0 con 0x7f0a98161600 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f0a90001a80 con 0x7f0a98161600 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 >> v1:172.21.15.116:6789/0 conn(0x7f0a98161600 legacy=0x7f0a98161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a80003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80003fe0 0x7f0a80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a80005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980ae100 con 0x7f0a80004d50 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8000bb90 con 0x7f0a80003fe0 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8000bd90 con 0x7f0a9815db30 2024-05-14T11:18:45.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9d200640 1 --2- 172.21.15.116:0/1538063295 >> v2:172.21.15.149:3300/0 conn(0x7f0a980d1670 0x7f0a980d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.015+0000 7f0a9e600640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a80005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a80003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9d200640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80003fe0 0x7f0a80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 -- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 msgr2=0x7f0a80003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a80003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 -- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80003fe0 msgr2=0x7f0a80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80003fe0 0x7f0a80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 -- 172.21.15.116:0/1538063295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a80003530 con 0x7f0a80004d50 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a9815db30 0x7f0a80003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:45.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9d200640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80003fe0 0x7f0a80004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:45.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a9e600640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a80005160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b0d40 tx=0x7f0a9001f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9e1cf1788558ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000a4c0 con 0x7f0a80004d50 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900b2070 con 0x7f0a80004d50 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.019+0000 7f0a97e00640 1 -- 172.21.15.116:0/1538063295 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900ad8c0 con 0x7f0a80004d50 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- 172.21.15.116:0/1538063295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 msgr2=0x7f0a80005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 --2- 172.21.15.116:0/1538063295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a80005160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b0d40 tx=0x7f0a9001f840 comp rx=0 tx=0).stop 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0a9e600640 1 -- 172.21.15.116:0/1538063295 reap_dead start 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- 172.21.15.116:0/1538063295 shutdown_connections 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- 172.21.15.116:0/1538063295 >> 172.21.15.116:0/1538063295 conn(0x7f0a980b4620 msgr2=0x7f0a980b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- 172.21.15.116:0/1538063295 shutdown_connections 2024-05-14T11:18:45.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- 172.21.15.116:0/1538063295 wait complete. 2024-05-14T11:18:45.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 Processor -- start 2024-05-14T11:18:45.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.023+0000 7f0aa0600640 1 -- start start 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a981539f0 0x7f0a9815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 0x7f0a980cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98070750 con 0x7f0a80004d50 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980ae870 con 0x7f0a9815b6a0 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980ae350 con 0x7f0a981539f0 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43884/0 (socket says 172.21.15.116:43884) 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 -- 172.21.15.116:0/3762305994 learned_addr learned my addr 172.21.15.116:0/3762305994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9e600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 0x7f0a980cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9d200640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a981539f0 0x7f0a9815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 -- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a981539f0 msgr2=0x7f0a9815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a981539f0 0x7f0a9815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 -- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 msgr2=0x7f0a980cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 0x7f0a980cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 -- 172.21.15.116:0/3762305994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9815a0c0 con 0x7f0a80004d50 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9e600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 0x7f0a980cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:45.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a9dc00640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009fd0 tx=0x7f0a8c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004220 con 0x7f0a80004d50 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0043c0 con 0x7f0a80004d50 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00f9e0 con 0x7f0a80004d50 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9815a3a0 con 0x7f0a80004d50 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9815a960 con 0x7f0a80004d50 2024-05-14T11:18:45.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9806a9c0 con 0x7f0a80004d50 2024-05-14T11:18:45.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.027+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a8c00fb80 con 0x7f0a80004d50 2024-05-14T11:18:45.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.031+0000 7f0a95600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 0x7f0a78071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:45.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.031+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7f0a8c08fc80 con 0x7f0a80004d50 2024-05-14T11:18:45.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.031+0000 7f0a9d200640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 0x7f0a78071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:45.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.031+0000 7f0a9d200640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 0x7f0a78071940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a9815d6c0 tx=0x7f0a84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:45.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.035+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0a8c019030 con 0x7f0a80004d50 2024-05-14T11:18:45.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.227+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0a9815cf40 con 0x7f0a80004d50 2024-05-14T11:18:45.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.839+0000 7f0a95600640 1 -- 172.21.15.116:0/3762305994 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f0a8c05c8f0 con 0x7f0a80004d50 2024-05-14T11:18:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 msgr2=0x7f0a78071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 0x7f0a78071940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a9815d6c0 tx=0x7f0a84008040 comp rx=0 tx=0).stop 2024-05-14T11:18:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 msgr2=0x7f0a981f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009fd0 tx=0x7f0a8c011b30 comp rx=0 tx=0).stop 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 shutdown_connections 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0a7806f430 0x7f0a78071940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a9815b6a0 0x7f0a980cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a981539f0 0x7f0a9815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 --2- 172.21.15.116:0/3762305994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a80004d50 0x7f0a981f5d70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 >> 172.21.15.116:0/3762305994 conn(0x7f0a980b4620 msgr2=0x7f0a98074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 shutdown_connections 2024-05-14T11:18:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:45.843+0000 7f0aa0600640 1 -- 172.21.15.116:0/3762305994 wait complete. 2024-05-14T11:18:45.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:18:46.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.015+0000 7feecbc00640 1 Processor -- start 2024-05-14T11:18:46.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.015+0000 7feecbc00640 1 -- start start 2024-05-14T11:18:46.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feecbc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feec40cd620 0x7feec406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feecbc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7feec40cda30 0x7feec406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feecbc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec40d04c0 con 0x7feec4161710 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feecbc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7feec40d0270 con 0x7feec40cd620 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feecbc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7feec40b0c30 con 0x7feec40cda30 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feec40cd620 0x7feec406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feec40cd620 0x7feec406f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43900/0 (socket says 172.21.15.116:43900) 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 -- 172.21.15.116:0/3033309501 learned_addr learned my addr 172.21.15.116:0/3033309501 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec3e00640 1 --2- 172.21.15.116:0/3033309501 >> v2:172.21.15.149:3300/0 conn(0x7feec40cda30 0x7feec406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 -- 172.21.15.116:0/3033309501 >> v2:172.21.15.149:3300/0 conn(0x7feec40cda30 msgr2=0x7feec406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 --2- 172.21.15.116:0/3033309501 >> v2:172.21.15.149:3300/0 conn(0x7feec40cda30 0x7feec406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 -- 172.21.15.116:0/3033309501 >> v1:172.21.15.149:6789/0 conn(0x7feec4161710 legacy=0x7feec4161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.019+0000 7feec9200640 1 -- 172.21.15.116:0/3033309501 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec40d07a0 con 0x7feec40cd620 2024-05-14T11:18:46.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feec9200640 1 --2- 172.21.15.116:0/3033309501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec40cd620 0x7feec406f6b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feec4071a20 tx=0x7feeb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69269dc9a7e96cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:46.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feec3400640 1 -- 172.21.15.116:0/3033309501 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40a5670 con 0x7feec40cd620 2024-05-14T11:18:46.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feec3400640 1 -- 172.21.15.116:0/3033309501 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40a5810 con 0x7feec40cd620 2024-05-14T11:18:46.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feec3400640 1 -- 172.21.15.116:0/3033309501 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40a5be0 con 0x7feec40cd620 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 -- 172.21.15.116:0/3033309501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec40cd620 msgr2=0x7feec406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 --2- 172.21.15.116:0/3033309501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec40cd620 0x7feec406f6b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feec4071a20 tx=0x7feeb401c9f0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 -- 172.21.15.116:0/3033309501 shutdown_connections 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 --2- 172.21.15.116:0/3033309501 >> v2:172.21.15.149:3300/0 conn(0x7feec40cda30 0x7feec406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 --2- 172.21.15.116:0/3033309501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec40cd620 0x7feec406f6b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 -- 172.21.15.116:0/3033309501 >> 172.21.15.116:0/3033309501 conn(0x7feec40cde80 msgr2=0x7feec40ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 -- 172.21.15.116:0/3033309501 shutdown_connections 2024-05-14T11:18:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 -- 172.21.15.116:0/3033309501 wait complete. 2024-05-14T11:18:46.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.023+0000 7feecbc00640 1 Processor -- start 2024-05-14T11:18:46.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- start start 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 0x7feec4154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 0x7feec4148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feec416b650 con 0x7feec4151150 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feec416b4d0 con 0x7feec40cd620 2024-05-14T11:18:46.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feec416b7d0 con 0x7feec4161710 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43916/0 (socket says 172.21.15.116:43916) 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 -- 172.21.15.116:0/1188727756 learned_addr learned my addr 172.21.15.116:0/1188727756 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9200640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 0x7feec4154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec3e00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 0x7feec4148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 -- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 msgr2=0x7feec4148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 0x7feec4148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 -- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 msgr2=0x7feec4154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 0x7feec4154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 -- 172.21.15.116:0/1188727756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec4155430 con 0x7feec4151150 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9200640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 0x7feec4154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec3e00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 0x7feec4148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec9c00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b440 tx=0x7feebc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc015020 con 0x7feec4151150 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc011850 con 0x7feec4151150 2024-05-14T11:18:46.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc011c40 con 0x7feec4151150 2024-05-14T11:18:46.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec4155710 con 0x7feec4151150 2024-05-14T11:18:46.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec414c6d0 con 0x7feec4151150 2024-05-14T11:18:46.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.027+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec406aa30 con 0x7feec4151150 2024-05-14T11:18:46.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.031+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feebc018020 con 0x7feec4151150 2024-05-14T11:18:46.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.031+0000 7feec0c00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 0x7feea4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:46.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.031+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6187+0+0 (secure 0 0 0) 0x7feebc08f120 con 0x7feec4151150 2024-05-14T11:18:46.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.031+0000 7feec9200640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 0x7feea4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:46.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.031+0000 7feec9200640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 0x7feea4071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feeb40ae000 tx=0x7feeb40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:46.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.035+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feebc01f030 con 0x7feec4151150 2024-05-14T11:18:46.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.195+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feec41551c0 con 0x7feec4151150 2024-05-14T11:18:46.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.847+0000 7feec0c00640 1 -- 172.21.15.116:0/1188727756 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7feebc05bd90 con 0x7feec4151150 2024-05-14T11:18:46.851 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:18:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 msgr2=0x7feea4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 0x7feea4071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feeb40ae000 tx=0x7feeb40a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 msgr2=0x7feec414e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b440 tx=0x7feebc00b900 comp rx=0 tx=0).stop 2024-05-14T11:18:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 shutdown_connections 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feea406f400 0x7feea4071910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec4151150 0x7feec414e6a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec4161710 0x7feec4148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 --2- 172.21.15.116:0/1188727756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec40cd620 0x7feec4154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 >> 172.21.15.116:0/1188727756 conn(0x7feec40cde80 msgr2=0x7feec4152040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 shutdown_connections 2024-05-14T11:18:46.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:46.851+0000 7feecbc00640 1 -- 172.21.15.116:0/1188727756 wait complete. 2024-05-14T11:18:46.878 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:18:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.023+0000 7ff24b000640 1 Processor -- start 2024-05-14T11:18:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.023+0000 7ff24b000640 1 -- start start 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff24b000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 0x7ff2440b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff24b000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff24406cf90 0x7ff24406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff24b000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2440b6880 con 0x7ff2440cf280 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff24b000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff24415d0f0 con 0x7ff24406cf90 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff24b000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff24406d8e0 con 0x7ff2440b5f30 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff249000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff2440cf280 0x7ff2440cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49520/0 (socket says 172.21.15.116:49520) 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff249000640 1 -- 172.21.15.116:0/2431793994 learned_addr learned my addr 172.21.15.116:0/2431793994 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243400640 1 --2- 172.21.15.116:0/2431793994 >> v2:172.21.15.149:3300/0 conn(0x7ff24406cf90 0x7ff24406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243e00640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 0x7ff2440b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709944139 0 0) 0x7ff23c0014a0 con 0x7ff2440cf280 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff228003530 con 0x7ff2440cf280 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243400640 1 -- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 msgr2=0x7ff2440b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243400640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 0x7ff2440b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243400640 1 -- 172.21.15.116:0/2431793994 >> v1:172.21.15.116:6789/0 conn(0x7ff2440cf280 legacy=0x7ff2440cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2907152487 0 0) 0x7ff228003530 con 0x7ff2440cf280 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243400640 1 -- 172.21.15.116:0/2431793994 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24415c800 con 0x7ff24406cf90 2024-05-14T11:18:47.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.027+0000 7ff243e00640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 0x7ff2440b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:47.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff243400640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff24406d3a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff2300097c0 tx=0x7ff23001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24163cd64f8fc02c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:47.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2300b0070 con 0x7ff24406cf90 2024-05-14T11:18:47.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2300ab400 con 0x7ff24406cf90 2024-05-14T11:18:47.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff242a00640 1 -- 172.21.15.116:0/2431793994 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c001c50 con 0x7ff24406cf90 2024-05-14T11:18:47.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 -- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 msgr2=0x7ff24406d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff24406d3a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff2300097c0 tx=0x7ff23001c3c0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 -- 172.21.15.116:0/2431793994 shutdown_connections 2024-05-14T11:18:47.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff24406d3a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 --2- 172.21.15.116:0/2431793994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440b5f30 0x7ff2440b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 -- 172.21.15.116:0/2431793994 >> 172.21.15.116:0/2431793994 conn(0x7ff2440cde80 msgr2=0x7ff2440ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:47.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 -- 172.21.15.116:0/2431793994 shutdown_connections 2024-05-14T11:18:47.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.031+0000 7ff24b000640 1 -- 172.21.15.116:0/2431793994 wait complete. 2024-05-14T11:18:47.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 Processor -- start 2024-05-14T11:18:47.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 -- start start 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff244150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440cf280 0x7ff24415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff24415d580 con 0x7ff2440b5f30 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2440b5200 con 0x7ff24406cf90 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff24b000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff244161ed0 con 0x7ff2440cf280 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43918/0 (socket says 172.21.15.116:43918) 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 -- 172.21.15.116:0/1797671573 learned_addr learned my addr 172.21.15.116:0/1797671573 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243e00640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff244150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff249000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440cf280 0x7ff24415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 -- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440cf280 msgr2=0x7ff24415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440cf280 0x7ff24415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 -- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 msgr2=0x7ff244150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff244150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.035+0000 7ff243400640 1 -- 172.21.15.116:0/1797671573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff244150c90 con 0x7ff2440b5f30 2024-05-14T11:18:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff243400640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff230000c10 tx=0x7ff230019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2300b0040 con 0x7ff2440b5f30 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2300abc90 con 0x7ff2440b5f30 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230006e20 con 0x7ff2440b5f30 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff244164f70 con 0x7ff2440b5f30 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff244165530 con 0x7ff2440b5f30 2024-05-14T11:18:47.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24406aa30 con 0x7ff2440b5f30 2024-05-14T11:18:47.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2300bc020 con 0x7ff2440b5f30 2024-05-14T11:18:47.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 0x7ff224071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:47.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5749+0+0 (secure 0 0 0) 0x7ff230128530 con 0x7ff2440b5f30 2024-05-14T11:18:47.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.039+0000 7ff243e00640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 0x7ff224071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:47.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.043+0000 7ff243e00640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 0x7ff224071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff234009580 tx=0x7ff234009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:47.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.047+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2300c3020 con 0x7ff2440b5f30 2024-05-14T11:18:47.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.203+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2440d0790 con 0x7ff2440b5f30 2024-05-14T11:18:47.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.851+0000 7ff23be00640 1 -- 172.21.15.116:0/1797671573 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7ff230015fb0 con 0x7ff2440b5f30 2024-05-14T11:18:47.854 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:18:47.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.851+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 msgr2=0x7ff224071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.851+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 0x7ff224071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff234009580 tx=0x7ff234009190 comp rx=0 tx=0).stop 2024-05-14T11:18:47.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.851+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 msgr2=0x7ff244162db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:47.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.851+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff230000c10 tx=0x7ff230019160 comp rx=0 tx=0).stop 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 shutdown_connections 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff22406f400 0x7ff224071910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2440cf280 0x7ff24415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2440b5f30 0x7ff244162db0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 --2- 172.21.15.116:0/1797671573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24406cf90 0x7ff244150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 >> 172.21.15.116:0/1797671573 conn(0x7ff2440cde80 msgr2=0x7ff2440cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 shutdown_connections 2024-05-14T11:18:47.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:47.855+0000 7ff24b000640 1 -- 172.21.15.116:0/1797671573 wait complete. 2024-05-14T11:18:47.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:18:48.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 Processor -- start 2024-05-14T11:18:48.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 -- start start 2024-05-14T11:18:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0e980d1d80 0x7f0e9806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9815d340 con 0x7f0e980af830 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9815d620 con 0x7f0e980af420 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9fa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9815d900 con 0x7f0e980d1d80 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e8fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0e980af830 0x7f0e980d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50206/0 (socket says 172.21.15.116:50206) 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e8fe00640 1 -- 172.21.15.116:0/2452716225 learned_addr learned my addr 172.21.15.116:0/2452716225 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e9da00640 1 --2- 172.21.15.116:0/2452716225 >> v2:172.21.15.149:3300/0 conn(0x7f0e980d1d80 0x7f0e9806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796433738 0 0) 0x7f0e9815d340 con 0x7f0e980af830 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.031+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e7c003530 con 0x7f0e980af830 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917329298 0 0) 0x7f0e9815d620 con 0x7f0e980af420 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e9815d340 con 0x7f0e980af420 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e9da00640 1 -- 172.21.15.116:0/2452716225 >> v1:172.21.15.149:6789/0 conn(0x7f0e980af420 legacy=0x7f0e98074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e9da00640 1 -- 172.21.15.116:0/2452716225 >> v1:172.21.15.116:6789/0 conn(0x7f0e980af830 legacy=0x7f0e980d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e9da00640 1 -- 172.21.15.116:0/2452716225 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9815dbe0 con 0x7f0e980d1d80 2024-05-14T11:18:48.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e9da00640 1 --2- 172.21.15.116:0/2452716225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980d1d80 0x7f0e9806edc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009d90 tx=0x7f0e9001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45180dfac81c8761 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e900b2070 con 0x7f0e980d1d80 2024-05-14T11:18:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e900a5920 con 0x7f0e980d1d80 2024-05-14T11:18:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.035+0000 7f0e97e00640 1 -- 172.21.15.116:0/2452716225 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e900a5cd0 con 0x7f0e980d1d80 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- 172.21.15.116:0/2452716225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980d1d80 msgr2=0x7f0e9806edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/2452716225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980d1d80 0x7f0e9806edc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009d90 tx=0x7f0e9001c910 comp rx=0 tx=0).stop 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- 172.21.15.116:0/2452716225 shutdown_connections 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/2452716225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980d1d80 0x7f0e9806edc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- 172.21.15.116:0/2452716225 >> 172.21.15.116:0/2452716225 conn(0x7f0e980b4620 msgr2=0x7f0e980b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- 172.21.15.116:0/2452716225 shutdown_connections 2024-05-14T11:18:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- 172.21.15.116:0/2452716225 wait complete. 2024-05-14T11:18:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 Processor -- start 2024-05-14T11:18:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.039+0000 7f0e9fa00640 1 -- start start 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 0x7f0e98142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980b3880 0x7f0e9815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9806ca90 con 0x7f0e980af420 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9806c910 con 0x7f0e980b3880 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9806cc10 con 0x7f0e980af830 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55602/0 (socket says 172.21.15.116:55602) 2024-05-14T11:18:48.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 -- 172.21.15.116:0/332588007 learned_addr learned my addr 172.21.15.116:0/332588007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9da00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980b3880 0x7f0e9815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e8fe00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 0x7f0e98142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 -- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 msgr2=0x7f0e98142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 0x7f0e98142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 -- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980b3880 msgr2=0x7f0e9815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980b3880 0x7f0e9815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 -- 172.21.15.116:0/332588007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980b27e0 con 0x7f0e980af420 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e8fe00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 0x7f0e98142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9d000640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800c3f0 tx=0x7f0e8800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88017070 con 0x7f0e980af420 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e880052d0 con 0x7f0e980af420 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88003840 con 0x7f0e980af420 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e980b2ac0 con 0x7f0e980af420 2024-05-14T11:18:48.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e980b3080 con 0x7f0e980af420 2024-05-14T11:18:48.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9806a9c0 con 0x7f0e980af420 2024-05-14T11:18:48.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.043+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e8801a070 con 0x7f0e980af420 2024-05-14T11:18:48.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.047+0000 7f0e95600640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 0x7f0e74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.047+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f0e8808e400 con 0x7f0e980af420 2024-05-14T11:18:48.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.047+0000 7f0e8fe00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 0x7f0e74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.047+0000 7f0e8fe00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 0x7f0e74071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000c10 tx=0x7f0e80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.051+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0e88005480 con 0x7f0e980af420 2024-05-14T11:18:48.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.219+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0e980705c0 con 0x7f0e980af420 2024-05-14T11:18:48.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.223+0000 7f0e95600640 1 -- 172.21.15.116:0/332588007 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f0e8808ce20 con 0x7f0e980af420 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 msgr2=0x7f0e74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 0x7f0e74071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000c10 tx=0x7f0e80009040 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 msgr2=0x7f0e980d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800c3f0 tx=0x7f0e8800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 shutdown_connections 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e7406f400 0x7f0e74071910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e980b3880 0x7f0e9815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e980af830 0x7f0e98142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 --2- 172.21.15.116:0/332588007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e980af420 0x7f0e980d0690 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 >> 172.21.15.116:0/332588007 conn(0x7f0e980b4620 msgr2=0x7f0e98152260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 shutdown_connections 2024-05-14T11:18:48.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.227+0000 7f0e9fa00640 1 -- 172.21.15.116:0/332588007 wait complete. 2024-05-14T11:18:48.249 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-14T11:18:48.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.391+0000 7f251da00640 1 Processor -- start 2024-05-14T11:18:48.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 -- start start 2024-05-14T11:18:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2518156fa0 0x7f251806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25180d1670 0x7f25180d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25180d1fc0 con 0x7f251806e9e0 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f25181573b0 con 0x7f25180d1670 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f251da00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2518161120 con 0x7f2518156fa0 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25180d1670 0x7f25180d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f2517400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f251806e9e0 0x7f251806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50218/0 (socket says 172.21.15.116:50218) 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f2517400640 1 -- 172.21.15.116:0/1618752080 learned_addr learned my addr 172.21.15.116:0/1618752080 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25180d1670 0x7f25180d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55614/0 (socket says 172.21.15.116:55614) 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f2516a00640 1 --2- 172.21.15.116:0/1618752080 >> v2:172.21.15.149:3300/0 conn(0x7f2518156fa0 0x7f251806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 -- 172.21.15.116:0/1618752080 >> v2:172.21.15.149:3300/0 conn(0x7f2518156fa0 msgr2=0x7f251806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 --2- 172.21.15.116:0/1618752080 >> v2:172.21.15.149:3300/0 conn(0x7f2518156fa0 0x7f251806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 -- 172.21.15.116:0/1618752080 >> v1:172.21.15.116:6789/0 conn(0x7f251806e9e0 legacy=0x7f251806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.395+0000 7f250e000640 1 -- 172.21.15.116:0/1618752080 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2518160510 con 0x7f25180d1670 2024-05-14T11:18:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f250e000640 1 --2- 172.21.15.116:0/1618752080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25180d1670 0x7f25180d1a80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2518071f20 tx=0x7f250001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6138d62bb007d70b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f2516000640 1 -- 172.21.15.116:0/1618752080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000ab650 con 0x7f25180d1670 2024-05-14T11:18:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f2516000640 1 -- 172.21.15.116:0/1618752080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25000b0070 con 0x7f25180d1670 2024-05-14T11:18:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f2516000640 1 -- 172.21.15.116:0/1618752080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000aba10 con 0x7f25180d1670 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 -- 172.21.15.116:0/1618752080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25180d1670 msgr2=0x7f25180d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 --2- 172.21.15.116:0/1618752080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25180d1670 0x7f25180d1a80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2518071f20 tx=0x7f250001c450 comp rx=0 tx=0).stop 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 -- 172.21.15.116:0/1618752080 shutdown_connections 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 --2- 172.21.15.116:0/1618752080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25180d1670 0x7f25180d1a80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 --2- 172.21.15.116:0/1618752080 >> v2:172.21.15.149:3300/0 conn(0x7f2518156fa0 0x7f251806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 -- 172.21.15.116:0/1618752080 >> 172.21.15.116:0/1618752080 conn(0x7f25180b4620 msgr2=0x7f25180b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 -- 172.21.15.116:0/1618752080 shutdown_connections 2024-05-14T11:18:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.399+0000 7f251da00640 1 -- 172.21.15.116:0/1618752080 wait complete. 2024-05-14T11:18:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 Processor -- start 2024-05-14T11:18:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- start start 2024-05-14T11:18:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f251806e9e0 0x7f25181fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25180d1670 0x7f25180cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2518070520 con 0x7f2518156fa0 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25180b07a0 con 0x7f251806e9e0 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f25180706a0 con 0x7f25180d1670 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55616/0 (socket says 172.21.15.116:55616) 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 -- 172.21.15.116:0/1397199377 learned_addr learned my addr 172.21.15.116:0/1397199377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2516a00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f251806e9e0 0x7f25181fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f250e000640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25180d1670 0x7f25180cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 -- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25180d1670 msgr2=0x7f25180cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25180d1670 0x7f25180cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 -- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f251806e9e0 msgr2=0x7f25181fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f251806e9e0 0x7f25181fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 -- 172.21.15.116:0/1397199377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181fcff0 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f2517400640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251000ca80 tx=0x7f251000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510017070 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251000ac30 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510012600 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251814da90 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.403+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251814e050 con 0x7f2518156fa0 2024-05-14T11:18:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251806a9c0 con 0x7f2518156fa0 2024-05-14T11:18:48.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f251001a070 con 0x7f2518156fa0 2024-05-14T11:18:48.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f250f400640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 0x7f24f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:48.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f251008ed70 con 0x7f2518156fa0 2024-05-14T11:18:48.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f2516a00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 0x7f24f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:48.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.407+0000 7f2516a00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 0x7f24f4071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f25180b0d60 tx=0x7f2508008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.411+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f25180d1fc0 con 0x7f2518156fa0 2024-05-14T11:18:48.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.623+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f251815fa80 con 0x7f2518156fa0 2024-05-14T11:18:48.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f250f400640 1 -- 172.21.15.116:0/1397199377 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v155) ==== 149+0+0 (secure 0 0 0) 0x7f251000ae20 con 0x7f2518156fa0 2024-05-14T11:18:48.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 msgr2=0x7f24f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 0x7f24f4071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f25180b0d60 tx=0x7f2508008040 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 msgr2=0x7f25180cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f251000ca80 tx=0x7f251000cf40 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 shutdown_connections 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24f406f430 0x7f24f4071940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2518156fa0 0x7f25180cfb80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f25180d1670 0x7f25180cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 --2- 172.21.15.116:0/1397199377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f251806e9e0 0x7f25181fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 >> 172.21.15.116:0/1397199377 conn(0x7f25180b4620 msgr2=0x7f25180b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 shutdown_connections 2024-05-14T11:18:48.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:48.863+0000 7f251da00640 1 -- 172.21.15.116:0/1397199377 wait complete. 2024-05-14T11:18:48.888 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-14T11:18:48.888 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-14T11:18:48.888 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:18:48.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:49.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 Processor -- start 2024-05-14T11:18:49.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 -- start start 2024-05-14T11:18:49.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 0x7f6c080d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0806ff20 con 0x7f6c08161600 2024-05-14T11:18:49.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0806d710 con 0x7f6c080d1670 2024-05-14T11:18:49.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.031+0000 7f6c10800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6c080ae100 con 0x7f6c0815db30 2024-05-14T11:18:49.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 0x7f6c080d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:49234/0 (socket says 172.21.15.116:49234) 2024-05-14T11:18:49.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 -- 172.21.15.116:0/3601656185 learned_addr learned my addr 172.21.15.116:0/3601656185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:49.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0e800640 1 --1- 172.21.15.116:0/3601656185 >> v1:172.21.15.149:6789/0 conn(0x7f6c08161600 0x7f6c08161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:38938/0 (socket says 172.21.15.116:38938) 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 -- 172.21.15.116:0/3601656185 >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 msgr2=0x7f6c080d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 --2- 172.21.15.116:0/3601656185 >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 0x7f6c080d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 -- 172.21.15.116:0/3601656185 >> v1:172.21.15.149:6789/0 conn(0x7f6c08161600 legacy=0x7f6c08161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0de00640 1 -- 172.21.15.116:0/3601656185 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c080d1fc0 con 0x7f6c0815db30 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0d400640 1 --2- 172.21.15.116:0/3601656185 >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 0x7f6c080d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:49.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.035+0000 7f6c0ca00640 1 -- 172.21.15.116:0/3601656185 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533054064 0 0) 0x7f6c0806ff20 con 0x7f6c08161600 2024-05-14T11:18:49.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c0de00640 1 --2- 172.21.15.116:0/3601656185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009870 tx=0x7f6bfc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8b6a1ccd9cf512 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c0ca00640 1 -- 172.21.15.116:0/3601656185 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0b0070 con 0x7f6c0815db30 2024-05-14T11:18:49.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c0ca00640 1 -- 172.21.15.116:0/3601656185 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc0ab600 con 0x7f6c0815db30 2024-05-14T11:18:49.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c0ca00640 1 -- 172.21.15.116:0/3601656185 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0ab9d0 con 0x7f6c0815db30 2024-05-14T11:18:49.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- 172.21.15.116:0/3601656185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 msgr2=0x7f6c080aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 --2- 172.21.15.116:0/3601656185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009870 tx=0x7f6bfc01c4d0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- 172.21.15.116:0/3601656185 shutdown_connections 2024-05-14T11:18:49.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 --2- 172.21.15.116:0/3601656185 >> v2:172.21.15.149:3300/0 conn(0x7f6c080d1670 0x7f6c080d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 --2- 172.21.15.116:0/3601656185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c0815db30 0x7f6c080aeaa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- 172.21.15.116:0/3601656185 >> 172.21.15.116:0/3601656185 conn(0x7f6c080b4620 msgr2=0x7f6c080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:49.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- 172.21.15.116:0/3601656185 shutdown_connections 2024-05-14T11:18:49.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- 172.21.15.116:0/3601656185 wait complete. 2024-05-14T11:18:49.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 Processor -- start 2024-05-14T11:18:49.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.039+0000 7f6c10800640 1 -- start start 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c080d1670 0x7f6c0809a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c0815db30 0x7f6c0814b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0815df40 con 0x7f6c08161600 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c080aefe0 con 0x7f6c0815db30 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0816af80 con 0x7f6c080d1670 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55628/0 (socket says 172.21.15.116:55628) 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 -- 172.21.15.116:0/3273161699 learned_addr learned my addr 172.21.15.116:0/3273161699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0de00640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c080d1670 0x7f6c0809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0d400640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c0815db30 0x7f6c0814b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 -- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c080d1670 msgr2=0x7f6c0809a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c080d1670 0x7f6c0809a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 -- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c0815db30 msgr2=0x7f6c0814b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c0815db30 0x7f6c0814b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 -- 172.21.15.116:0/3273161699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0809ae80 con 0x7f6c08161600 2024-05-14T11:18:49.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c0e800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400c650 tx=0x7f6c0400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04018070 con 0x7f6c08161600 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c04004d70 con 0x7f6c08161600 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c08157710 con 0x7f6c08161600 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400a880 con 0x7f6c08161600 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c08146a80 con 0x7f6c08161600 2024-05-14T11:18:49.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.043+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c08157470 con 0x7f6c08161600 2024-05-14T11:18:49.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.047+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6c0401b070 con 0x7f6c08161600 2024-05-14T11:18:49.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.047+0000 7f6c02000640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 0x7f6bec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.047+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f6c0408e5a0 con 0x7f6c08161600 2024-05-14T11:18:49.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.047+0000 7f6c0de00640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 0x7f6bec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.047+0000 7f6c0de00640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 0x7f6bec071940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0ad000 tx=0x7f6bfc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.051+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6c08157470 con 0x7f6c08161600 2024-05-14T11:18:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c0806a9c0 con 0x7f6c08161600 2024-05-14T11:18:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c02000640 1 -- 172.21.15.116:0/3273161699 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13343 (secure 0 0 0) 0x7f6c0405b680 con 0x7f6c08161600 2024-05-14T11:18:49.214 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:49.214 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":52,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:47.854249+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6837/2735194331":"2024-05-15T11:18:44.838485+0000","172.21.15.116:6836/2735194331":"2024-05-15T11:18:44.838485+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 msgr2=0x7f6bec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 0x7f6bec071940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0ad000 tx=0x7f6bfc0a6000 comp rx=0 tx=0).stop 2024-05-14T11:18:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 msgr2=0x7f6c0814b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400c650 tx=0x7f6c0400cb10 comp rx=0 tx=0).stop 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 shutdown_connections 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6bec06f430 0x7f6bec071940 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c0815db30 0x7f6c0814b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c08161600 0x7f6c0814b950 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 --2- 172.21.15.116:0/3273161699 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c080d1670 0x7f6c0809a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 >> 172.21.15.116:0/3273161699 conn(0x7f6c080b4620 msgr2=0x7f6c08151f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.211+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 shutdown_connections 2024-05-14T11:18:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.215+0000 7f6c10800640 1 -- 172.21.15.116:0/3273161699 wait complete. 2024-05-14T11:18:49.236 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:18:49.237 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:18:49.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 Processor -- start 2024-05-14T11:18:49.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 -- start start 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 0x7fd1180cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd11806f0a0 con 0x7fd1180cd1c0 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd1180cbb60 con 0x7fd1180d1e40 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.383+0000 7fd120800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd11806f420 con 0x7fd11806c700 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 0x7fd1180cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11de00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fd1180cd1c0 0x7fd1180d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:38944/0 (socket says 172.21.15.116:38944) 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 0x7fd1180cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:38834/0 (socket says 172.21.15.116:38834) 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 --2- 172.21.15.116:0/753509834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:49244/0 (socket says 172.21.15.116:49244) 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11de00640 1 -- 172.21.15.116:0/753509834 learned_addr learned my addr 172.21.15.116:0/753509834 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 -- 172.21.15.116:0/753509834 >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 msgr2=0x7fd1180cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:49.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 --2- 172.21.15.116:0/753509834 >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 0x7fd1180cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 -- 172.21.15.116:0/753509834 >> v1:172.21.15.149:6789/0 conn(0x7fd1180cd1c0 legacy=0x7fd1180d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11ca00640 1 -- 172.21.15.116:0/753509834 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219544602 0 0) 0x7fd11806f0a0 con 0x7fd1180cd1c0 2024-05-14T11:18:49.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 -- 172.21.15.116:0/753509834 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11806f700 con 0x7fd11806c700 2024-05-14T11:18:49.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.387+0000 7fd11e800640 1 --2- 172.21.15.116:0/753509834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd114009d90 tx=0x7fd11401c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1143a34b9a1d29af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd11ca00640 1 -- 172.21.15.116:0/753509834 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b2070 con 0x7fd11806c700 2024-05-14T11:18:49.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd11ca00640 1 -- 172.21.15.116:0/753509834 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1140a5850 con 0x7fd11806c700 2024-05-14T11:18:49.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd11ca00640 1 -- 172.21.15.116:0/753509834 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140a5c20 con 0x7fd11806c700 2024-05-14T11:18:49.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- 172.21.15.116:0/753509834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 msgr2=0x7fd11806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 --2- 172.21.15.116:0/753509834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd114009d90 tx=0x7fd11401c790 comp rx=0 tx=0).stop 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- 172.21.15.116:0/753509834 shutdown_connections 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 --2- 172.21.15.116:0/753509834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd11806c700 0x7fd11806cb10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 --2- 172.21.15.116:0/753509834 >> v2:172.21.15.149:3300/0 conn(0x7fd1180d1e40 0x7fd1180cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- 172.21.15.116:0/753509834 >> 172.21.15.116:0/753509834 conn(0x7fd1180b4720 msgr2=0x7fd1180b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- 172.21.15.116:0/753509834 shutdown_connections 2024-05-14T11:18:49.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- 172.21.15.116:0/753509834 wait complete. 2024-05-14T11:18:49.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 Processor -- start 2024-05-14T11:18:49.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.391+0000 7fd120800640 1 -- start start 2024-05-14T11:18:49.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd11806c700 0x7fd11809a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1180cd1c0 0x7fd1180cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181699b0 con 0x7fd1180c6510 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd11815cf10 con 0x7fd11806c700 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd118169b30 con 0x7fd1180cd1c0 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55634/0 (socket says 172.21.15.116:55634) 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 -- 172.21.15.116:0/3500140740 learned_addr learned my addr 172.21.15.116:0/3500140740 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11d400640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1180cd1c0 0x7fd1180cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11de00640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd11806c700 0x7fd11809a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 -- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1180cd1c0 msgr2=0x7fd1180cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1180cd1c0 0x7fd1180cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 -- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd11806c700 msgr2=0x7fd11809a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd11806c700 0x7fd11809a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 -- 172.21.15.116:0/3500140740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11815b740 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd11e800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd114009d90 tx=0x7fd114019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b2040 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1140a5650 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b8690 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1180c7520 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1180c7ae0 con 0x7fd1180c6510 2024-05-14T11:18:49.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.395+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1180c7cb0 con 0x7fd1180c6510 2024-05-14T11:18:49.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.399+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd1140c1020 con 0x7fd1180c6510 2024-05-14T11:18:49.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.399+0000 7fd112000640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 0x7fd0fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.399+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fd114129720 con 0x7fd1180c6510 2024-05-14T11:18:49.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.399+0000 7fd11de00640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 0x7fd0fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.399+0000 7fd11de00640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 0x7fd0fc071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f70 tx=0x7fd10c007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.403+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd1140c8020 con 0x7fd1180c6510 2024-05-14T11:18:49.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd11806a9c0 con 0x7fd1180c6510 2024-05-14T11:18:49.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd112000640 1 -- 172.21.15.116:0/3500140740 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7fd11806d860 con 0x7fd1180c6510 2024-05-14T11:18:49.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:18:49.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 msgr2=0x7fd0fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 0x7fd0fc071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd10c000f70 tx=0x7fd10c007df0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 msgr2=0x7fd1180c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.559+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd114009d90 tx=0x7fd114019530 comp rx=0 tx=0).stop 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 shutdown_connections 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd0fc06f400 0x7fd0fc071910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1180c6510 0x7fd1180c6920 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1180cd1c0 0x7fd1180cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 --2- 172.21.15.116:0/3500140740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd11806c700 0x7fd11809a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 >> 172.21.15.116:0/3500140740 conn(0x7fd1180b4720 msgr2=0x7fd1180b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 shutdown_connections 2024-05-14T11:18:49.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.563+0000 7fd120800640 1 -- 172.21.15.116:0/3500140740 wait complete. 2024-05-14T11:18:49.584 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-14T11:18:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 Processor -- start 2024-05-14T11:18:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 -- start start 2024-05-14T11:18:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd1fc0d1670 0x7fd1fc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc0d1fc0 con 0x7fd1fc06e9e0 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc1573b0 con 0x7fd1fc156fa0 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd201c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc161120 con 0x7fd1fc0d1670 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd1faa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd1faa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55636/0 (socket says 172.21.15.116:55636) 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.727+0000 7fd1faa00640 1 -- 172.21.15.116:0/4035250216 learned_addr learned my addr 172.21.15.116:0/4035250216 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1f2000640 1 --2- 172.21.15.116:0/4035250216 >> v2:172.21.15.149:3300/0 conn(0x7fd1fc0d1670 0x7fd1fc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1faa00640 1 -- 172.21.15.116:0/4035250216 >> v2:172.21.15.149:3300/0 conn(0x7fd1fc0d1670 msgr2=0x7fd1fc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1faa00640 1 --2- 172.21.15.116:0/4035250216 >> v2:172.21.15.149:3300/0 conn(0x7fd1fc0d1670 0x7fd1fc0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1faa00640 1 -- 172.21.15.116:0/4035250216 >> v1:172.21.15.149:6789/0 conn(0x7fd1fc06e9e0 legacy=0x7fd1fc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1faa00640 1 -- 172.21.15.116:0/4035250216 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc160510 con 0x7fd1fc156fa0 2024-05-14T11:18:49.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1faa00640 1 --2- 172.21.15.116:0/4035250216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc070c70 tx=0x7fd1ec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afa44bc5d85b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1fa000640 1 -- 172.21.15.116:0/4035250216 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0a5880 con 0x7fd1fc156fa0 2024-05-14T11:18:49.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1fa000640 1 -- 172.21.15.116:0/4035250216 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec0a5a20 con 0x7fd1fc156fa0 2024-05-14T11:18:49.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.731+0000 7fd1fa000640 1 -- 172.21.15.116:0/4035250216 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0a5df0 con 0x7fd1fc156fa0 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- 172.21.15.116:0/4035250216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 msgr2=0x7fd1fc06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 --2- 172.21.15.116:0/4035250216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc070c70 tx=0x7fd1ec01c9c0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- 172.21.15.116:0/4035250216 shutdown_connections 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 --2- 172.21.15.116:0/4035250216 >> v2:172.21.15.149:3300/0 conn(0x7fd1fc0d1670 0x7fd1fc0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 --2- 172.21.15.116:0/4035250216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc06ffe0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- 172.21.15.116:0/4035250216 >> 172.21.15.116:0/4035250216 conn(0x7fd1fc0b4620 msgr2=0x7fd1fc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- 172.21.15.116:0/4035250216 shutdown_connections 2024-05-14T11:18:49.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- 172.21.15.116:0/4035250216 wait complete. 2024-05-14T11:18:49.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 Processor -- start 2024-05-14T11:18:49.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.735+0000 7fd201c00640 1 -- start start 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1fc06e9e0 0x7fd1fc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 0x7fd1fc09a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc070520 con 0x7fd1fc156fa0 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b07a0 con 0x7fd1fc0d1670 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0706a0 con 0x7fd1fc06e9e0 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55638/0 (socket says 172.21.15.116:55638) 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 -- 172.21.15.116:0/3335932994 learned_addr learned my addr 172.21.15.116:0/3335932994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1faa00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1fc06e9e0 0x7fd1fc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1f2000640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 0x7fd1fc09a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 -- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1fc06e9e0 msgr2=0x7fd1fc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1fc06e9e0 0x7fd1fc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 -- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 msgr2=0x7fd1fc09a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 0x7fd1fc09a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 -- 172.21.15.116:0/3335932994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0b1250 con 0x7fd1fc156fa0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1f2000640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 0x7fd1fc09a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1fb400640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c3f0 tx=0x7fd1f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4017070 con 0x7fd1fc156fa0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f40052d0 con 0x7fd1fc156fa0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4003820 con 0x7fd1fc156fa0 2024-05-14T11:18:49.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc0b1530 con 0x7fd1fc156fa0 2024-05-14T11:18:49.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc09ac30 con 0x7fd1fc156fa0 2024-05-14T11:18:49.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.739+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc06a9c0 con 0x7fd1fc156fa0 2024-05-14T11:18:49.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.743+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd1f401a070 con 0x7fd1fc156fa0 2024-05-14T11:18:49.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.743+0000 7fd1f3400640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 0x7fd1d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:49.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.743+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fd1f408e310 con 0x7fd1fc156fa0 2024-05-14T11:18:49.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.743+0000 7fd1faa00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 0x7fd1d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:49.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.743+0000 7fd1faa00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 0x7fd1d8071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0ae000 tx=0x7fd1ec0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:49.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.747+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd1fc0d1fc0 con 0x7fd1fc156fa0 2024-05-14T11:18:49.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:49.959+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fd1fc15c950 con 0x7fd1fc156fa0 2024-05-14T11:18:50.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd1f3400640 1 -- 172.21.15.116:0/3335932994 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v157) ==== 172+0+0 (secure 0 0 0) 0x7fd1f4005480 con 0x7fd1fc156fa0 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 msgr2=0x7fd1d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 0x7fd1d8071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0ae000 tx=0x7fd1ec0a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 msgr2=0x7fd1fc15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c3f0 tx=0x7fd1f400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 shutdown_connections 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd1d806f400 0x7fd1d8071910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1fc0d1670 0x7fd1fc09a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1fc156fa0 0x7fd1fc15b950 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:50.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 --2- 172.21.15.116:0/3335932994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1fc06e9e0 0x7fd1fc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:50.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.871+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 >> 172.21.15.116:0/3335932994 conn(0x7fd1fc0b4620 msgr2=0x7fd1fc150560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:50.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.875+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 shutdown_connections 2024-05-14T11:18:50.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:50.875+0000 7fd201c00640 1 -- 172.21.15.116:0/3335932994 wait complete. 2024-05-14T11:18:50.899 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:18:51.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.047+0000 7f976ac00640 1 Processor -- start 2024-05-14T11:18:51.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.047+0000 7f976ac00640 1 -- start start 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f976ac00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640d1e40 0x7f97640cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f976ac00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976406f0a0 con 0x7f97640cd1c0 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f976ac00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97640cbb60 con 0x7f976406c700 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f976ac00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f976406f420 con 0x7f97640d1e40 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f97640cd1c0 0x7f97640d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50220/0 (socket says 172.21.15.116:50220) 2024-05-14T11:18:51.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763e00640 1 -- 172.21.15.116:0/4175204684 learned_addr learned my addr 172.21.15.116:0/4175204684 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:51.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640d1e40 0x7f97640cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866770254 0 0) 0x7f976406f0a0 con 0x7f97640cd1c0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9748003530 con 0x7f97640cd1c0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355018630 0 0) 0x7f9748003530 con 0x7f97640cd1c0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640d1e40 msgr2=0x7f97640cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640d1e40 0x7f97640cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 >> v1:172.21.15.149:6789/0 conn(0x7f976406c700 legacy=0x7f976406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976406f700 con 0x7f97640cd1c0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9754001700 con 0x7f97640cd1c0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 >> v1:172.21.15.116:6789/0 conn(0x7f97640cd1c0 legacy=0x7f97640d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97480039f0 0x7f9748003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9748004810 0x7f9748004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97480051b0 0x7f974800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f976406f420 con 0x7f9748004810 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f974800bf10 con 0x7f97480039f0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f974800c110 con 0x7f97480051b0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9748004810 0x7f9748004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763e00640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97480039f0 0x7f9748003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9768c00640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97480051b0 0x7f974800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 -- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97480051b0 msgr2=0x7f974800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97480051b0 0x7f974800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 -- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97480039f0 msgr2=0x7f9748003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97480039f0 0x7f9748003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f975b400640 1 -- 172.21.15.116:0/4175204684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9748003530 con 0x7f9748004810 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.051+0000 7f9763e00640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97480039f0 0x7f9748003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:51.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f975b400640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9748004810 0x7f9748004c20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f974c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5d2d818d3c3ee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0b2070 con 0x7f9748004810 2024-05-14T11:18:51.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c0a5570 con 0x7f9748004810 2024-05-14T11:18:51.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f9763400640 1 -- 172.21.15.116:0/4175204684 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0096b0 con 0x7f9748004810 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 -- 172.21.15.116:0/4175204684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9748004810 msgr2=0x7f9748004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 --2- 172.21.15.116:0/4175204684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9748004810 0x7f9748004c20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f974c01ca90 comp rx=0 tx=0).stop 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f9768c00640 1 -- 172.21.15.116:0/4175204684 reap_dead start 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 -- 172.21.15.116:0/4175204684 shutdown_connections 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 -- 172.21.15.116:0/4175204684 >> 172.21.15.116:0/4175204684 conn(0x7f97640b4720 msgr2=0x7f97640b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 -- 172.21.15.116:0/4175204684 shutdown_connections 2024-05-14T11:18:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.055+0000 7f976ac00640 1 -- 172.21.15.116:0/4175204684 wait complete. 2024-05-14T11:18:51.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 Processor -- start 2024-05-14T11:18:51.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- start start 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 0x7f976415fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 0x7f97640ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f976406ffa0 con 0x7f97641f45e0 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f976406fcc0 con 0x7f976415f7b0 2024-05-14T11:18:51.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f976415cf10 con 0x7f97640ca210 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55646/0 (socket says 172.21.15.116:55646) 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 -- 172.21.15.116:0/2040968384 learned_addr learned my addr 172.21.15.116:0/2040968384 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f975b400640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 0x7f976415fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9768c00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 0x7f97640ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 -- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 msgr2=0x7f97640ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 0x7f97640ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 -- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 msgr2=0x7f976415fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 0x7f976415fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 -- 172.21.15.116:0/2040968384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97640cab60 con 0x7f97641f45e0 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f975b400640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 0x7f976415fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9768c00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 0x7f97640ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:18:51.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9763e00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f975400e870 tx=0x7f975400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400c870 con 0x7f97641f45e0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9754018070 con 0x7f97641f45e0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976409b380 con 0x7f97641f45e0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400cc10 con 0x7f97641f45e0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.059+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976409b940 con 0x7f97641f45e0 2024-05-14T11:18:51.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976406a9c0 con 0x7f97641f45e0 2024-05-14T11:18:51.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f975401d460 con 0x7f97641f45e0 2024-05-14T11:18:51.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f9760c00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 0x7f9740071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f975408eff0 con 0x7f97641f45e0 2024-05-14T11:18:51.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f975b400640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 0x7f9740071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.063+0000 7f975b400640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 0x7f9740071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f974c0af000 tx=0x7f974c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.067+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f975405c110 con 0x7f97641f45e0 2024-05-14T11:18:51.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.227+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9764069de0 con 0x7f97641f45e0 2024-05-14T11:18:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f9760c00640 1 -- 172.21.15.116:0/2040968384 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f9754064100 con 0x7f97641f45e0 2024-05-14T11:18:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 msgr2=0x7f9740071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 0x7f9740071910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f974c0af000 tx=0x7f974c008040 comp rx=0 tx=0).stop 2024-05-14T11:18:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 msgr2=0x7f97640c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f975400e870 tx=0x7f975400ed30 comp rx=0 tx=0).stop 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 shutdown_connections 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f974006f400 0x7f9740071910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415f7b0 0x7f976415fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97640ca210 0x7f97640ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 --2- 172.21.15.116:0/2040968384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641f45e0 0x7f97640c7200 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 >> 172.21.15.116:0/2040968384 conn(0x7f97640b4720 msgr2=0x7f976406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 shutdown_connections 2024-05-14T11:18:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.231+0000 7f976ac00640 1 -- 172.21.15.116:0/2040968384 wait complete. 2024-05-14T11:18:51.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:18:51.255 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:18:51.326 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:18:51.412 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:18:51.412 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:18:51.412 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:51.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:18:51.421 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:18:51.421 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:51.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:18:51.430 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:18:51.430 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:51.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:18:51.471 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:18:51.471 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:18:51.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:18:51.485 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:18:51.486 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:18:51.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.655+0000 7f4704000640 1 Processor -- start 2024-05-14T11:18:51.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.655+0000 7f4704000640 1 -- start start 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4704000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46fc0b0770 0x7f46fc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4704000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f46fc072a30 0x7f46fc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4704000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc06cc40 con 0x7f46fc06f730 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4704000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc06db20 con 0x7f46fc0b0770 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4704000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc06cf10 con 0x7f46fc072a30 2024-05-14T11:18:51.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46fc0b0770 0x7f46fc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4700c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f46fc06f730 0x7f46fc06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50228/0 (socket says 172.21.15.116:50228) 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46fc0b0770 0x7f46fc06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55660/0 (socket says 172.21.15.116:55660) 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4700c00640 1 -- 172.21.15.116:0/890574698 learned_addr learned my addr 172.21.15.116:0/890574698 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4702000640 1 --2- 172.21.15.116:0/890574698 >> v2:172.21.15.149:3300/0 conn(0x7f46fc072a30 0x7f46fc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 -- 172.21.15.116:0/890574698 >> v2:172.21.15.149:3300/0 conn(0x7f46fc072a30 msgr2=0x7f46fc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 --2- 172.21.15.116:0/890574698 >> v2:172.21.15.149:3300/0 conn(0x7f46fc072a30 0x7f46fc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 -- 172.21.15.116:0/890574698 >> v1:172.21.15.116:6789/0 conn(0x7f46fc06f730 legacy=0x7f46fc06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f4701600640 1 -- 172.21.15.116:0/890574698 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc06d1f0 con 0x7f46fc0b0770 2024-05-14T11:18:51.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.659+0000 7f46fbe00640 1 -- 172.21.15.116:0/890574698 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247384984 0 0) 0x7f46fc06cc40 con 0x7f46fc06f730 2024-05-14T11:18:51.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4701600640 1 --2- 172.21.15.116:0/890574698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc0b0770 0x7f46fc06c700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0729c0 tx=0x7f46f001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e441a3eb11de5f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f46fbe00640 1 -- 172.21.15.116:0/890574698 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00ad070 con 0x7f46fc0b0770 2024-05-14T11:18:51.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f46fbe00640 1 -- 172.21.15.116:0/890574698 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f00a7ac0 con 0x7f46fc0b0770 2024-05-14T11:18:51.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f46fbe00640 1 -- 172.21.15.116:0/890574698 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00b1400 con 0x7f46fc0b0770 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 -- 172.21.15.116:0/890574698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc0b0770 msgr2=0x7f46fc06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 --2- 172.21.15.116:0/890574698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc0b0770 0x7f46fc06c700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0729c0 tx=0x7f46f001ea70 comp rx=0 tx=0).stop 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 -- 172.21.15.116:0/890574698 shutdown_connections 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 --2- 172.21.15.116:0/890574698 >> v2:172.21.15.149:3300/0 conn(0x7f46fc072a30 0x7f46fc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 --2- 172.21.15.116:0/890574698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc0b0770 0x7f46fc06c700 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 -- 172.21.15.116:0/890574698 >> 172.21.15.116:0/890574698 conn(0x7f46fc0b4620 msgr2=0x7f46fc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:51.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 -- 172.21.15.116:0/890574698 shutdown_connections 2024-05-14T11:18:51.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.663+0000 7f4704000640 1 -- 172.21.15.116:0/890574698 wait complete. 2024-05-14T11:18:51.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 Processor -- start 2024-05-14T11:18:51.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- start start 2024-05-14T11:18:51.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc06f730 0x7f46fc14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 0x7f46fc09de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc073500 con 0x7f46fc072a30 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc073380 con 0x7f46fc06f730 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc073680 con 0x7f46fc0b0770 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55672/0 (socket says 172.21.15.116:55672) 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 -- 172.21.15.116:0/187695411 learned_addr learned my addr 172.21.15.116:0/187695411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4701600640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc06f730 0x7f46fc14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4702000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 0x7f46fc09de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 -- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 msgr2=0x7f46fc09de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 0x7f46fc09de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 -- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc06f730 msgr2=0x7f46fc14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc06f730 0x7f46fc14d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 -- 172.21.15.116:0/187695411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc14dc00 con 0x7f46fc072a30 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4702000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 0x7f46fc09de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4700c00640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f46e8005890 tx=0x7f46e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017070 con 0x7f46fc072a30 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8005d10 con 0x7f46fc072a30 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc09e580 con 0x7f46fc072a30 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007310 con 0x7f46fc072a30 2024-05-14T11:18:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.667+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc09eb40 con 0x7f46fc072a30 2024-05-14T11:18:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc06a9c0 con 0x7f46fc072a30 2024-05-14T11:18:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f46e801a070 con 0x7f46fc072a30 2024-05-14T11:18:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f46f9600640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 0x7f46dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f46e808e3f0 con 0x7f46fc072a30 2024-05-14T11:18:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f4701600640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 0x7f46dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.671+0000 7f4701600640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 0x7f46dc071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46f00b2000 tx=0x7f46f00a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:51.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.675+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f46e805ae80 con 0x7f46fc072a30 2024-05-14T11:18:51.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.867+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46fc09c120 con 0x7f46fc072a30 2024-05-14T11:18:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.867+0000 7f46f9600640 1 -- 172.21.15.116:0/187695411 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3878 (secure 0 0 0) 0x7f46e805f180 con 0x7f46fc072a30 2024-05-14T11:18:51.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:51.872 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/2049025678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2049025678},{"type":"v1","addr":"172.21.15.149:6834","nonce":2049025678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3064093845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3064093845},{"type":"v1","addr":"172.21.15.149:6835","nonce":3064093845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3241517751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3241517751},{"type":"v1","addr":"172.21.15.116:6835","nonce":3241517751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/676936532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":676936532},{"type":"v1","addr":"172.21.15.116:6837","nonce":676936532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-14T11:18:51.872 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 msgr2=0x7f46dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 0x7f46dc071910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46f00b2000 tx=0x7f46f00a9070 comp rx=0 tx=0).stop 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 msgr2=0x7f46fc09d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f46e8005890 tx=0x7f46e800c890 comp rx=0 tx=0).stop 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 shutdown_connections 2024-05-14T11:18:51.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f46dc06f400 0x7f46dc071910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f46fc0b0770 0x7f46fc09de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc072a30 0x7f46fc09d8d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 --2- 172.21.15.116:0/187695411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc06f730 0x7f46fc14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 >> 172.21.15.116:0/187695411 conn(0x7f46fc0b4620 msgr2=0x7f46fc0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 shutdown_connections 2024-05-14T11:18:51.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:51.871+0000 7f4704000640 1 -- 172.21.15.116:0/187695411 wait complete. 2024-05-14T11:18:51.894 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:18:51.894 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:18:51.895 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:18:51.895 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:18:51.895 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:18:51.895 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:18:51.895 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:18:51.895 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:18:51.895 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:18:51.896 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:18:51.896 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:18:51.896 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:18:51.896 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:18:51.896 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:18:51.896 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:18:51.896 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:18:51.908 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:18:51.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:18:51.912 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:18:51.912 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:18:51.913 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:18:51.913 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:18:51.917 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:18:51.917 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:18:51.960 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:18:51.960 DEBUG:teuthology.parallel:result is None 2024-05-14T11:18:51.962 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:18:51.962 DEBUG:teuthology.parallel:result is None 2024-05-14T11:18:51.963 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:18:51.963 DEBUG:teuthology.parallel:result is None 2024-05-14T11:18:51.965 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:18:51.965 DEBUG:teuthology.parallel:result is None 2024-05-14T11:18:51.965 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:18:52.064 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:18:52.064 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:18:52.066 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:18:52.067 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:18:52.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 Processor -- start 2024-05-14T11:18:52.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 -- start start 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7febac156fa0 0x7febac06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1670 0x7febac0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac0d1fc0 con 0x7febac06e9e0 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7febac1573b0 con 0x7febac0d1670 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb4000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7febac161120 con 0x7febac156fa0 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1670 0x7febac0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1670 0x7febac0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55710/0 (socket says 172.21.15.116:55710) 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 -- 172.21.15.116:0/3404665803 learned_addr learned my addr 172.21.15.116:0/3404665803 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb1600640 1 --2- 172.21.15.116:0/3404665803 >> v2:172.21.15.149:3300/0 conn(0x7febac156fa0 0x7febac06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 -- 172.21.15.116:0/3404665803 >> v2:172.21.15.149:3300/0 conn(0x7febac156fa0 msgr2=0x7febac06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 --2- 172.21.15.116:0/3404665803 >> v2:172.21.15.149:3300/0 conn(0x7febac156fa0 0x7febac06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 -- 172.21.15.116:0/3404665803 >> v1:172.21.15.149:6789/0 conn(0x7febac06e9e0 legacy=0x7febac06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.119+0000 7febb0c00640 1 -- 172.21.15.116:0/3404665803 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac160510 con 0x7febac0d1670 2024-05-14T11:18:52.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb0c00640 1 --2- 172.21.15.116:0/3404665803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1670 0x7febac0d1a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7febac071f20 tx=0x7feb9801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd85c4257ea9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:52.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febabe00640 1 -- 172.21.15.116:0/3404665803 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980aa650 con 0x7febac0d1670 2024-05-14T11:18:52.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febabe00640 1 -- 172.21.15.116:0/3404665803 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb980af070 con 0x7febac0d1670 2024-05-14T11:18:52.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febabe00640 1 -- 172.21.15.116:0/3404665803 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980aaa10 con 0x7febac0d1670 2024-05-14T11:18:52.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 -- 172.21.15.116:0/3404665803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1670 msgr2=0x7febac0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 --2- 172.21.15.116:0/3404665803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1670 0x7febac0d1a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7febac071f20 tx=0x7feb9801c4f0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 -- 172.21.15.116:0/3404665803 shutdown_connections 2024-05-14T11:18:52.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 --2- 172.21.15.116:0/3404665803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1670 0x7febac0d1a80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 --2- 172.21.15.116:0/3404665803 >> v2:172.21.15.149:3300/0 conn(0x7febac156fa0 0x7febac06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 -- 172.21.15.116:0/3404665803 >> 172.21.15.116:0/3404665803 conn(0x7febac0b4620 msgr2=0x7febac0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:52.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 -- 172.21.15.116:0/3404665803 shutdown_connections 2024-05-14T11:18:52.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.123+0000 7febb4000640 1 -- 172.21.15.116:0/3404665803 wait complete. 2024-05-14T11:18:52.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 Processor -- start 2024-05-14T11:18:52.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 -- start start 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06e9e0 0x7febac14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1670 0x7febac1fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7febac070520 con 0x7febac156fa0 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7febac0b07a0 con 0x7febac06e9e0 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7febac0706a0 con 0x7febac0d1670 2024-05-14T11:18:52.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55720/0 (socket says 172.21.15.116:55720) 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 -- 172.21.15.116:0/2141865724 learned_addr learned my addr 172.21.15.116:0/2141865724 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb0c00640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1670 0x7febac1fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb1600640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06e9e0 0x7febac14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 -- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1670 msgr2=0x7febac1fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1670 0x7febac1fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 -- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06e9e0 msgr2=0x7febac14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06e9e0 0x7febac14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 -- 172.21.15.116:0/2141865724 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac151150 con 0x7febac156fa0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb2000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feba400c3f0 tx=0x7feba400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4017070 con 0x7febac156fa0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba40052d0 con 0x7febac156fa0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4003820 con 0x7febac156fa0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.127+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac151430 con 0x7febac156fa0 2024-05-14T11:18:52.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1519f0 con 0x7febac156fa0 2024-05-14T11:18:52.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac06a9c0 con 0x7febac156fa0 2024-05-14T11:18:52.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feba401a070 con 0x7febac156fa0 2024-05-14T11:18:52.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7feba9600640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 0x7feb8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:52.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7feba408e4f0 con 0x7febac156fa0 2024-05-14T11:18:52.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.131+0000 7febb1600640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 0x7feb8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:52.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.135+0000 7febb1600640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 0x7feb8c071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7febac0b0d60 tx=0x7feba000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:52.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.135+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7febac0d1fc0 con 0x7febac156fa0 2024-05-14T11:18:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.307+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7febac146960 con 0x7febac156fa0 2024-05-14T11:18:52.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.883+0000 7feba9600640 1 -- 172.21.15.116:0/2141865724 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7feba4005480 con 0x7febac156fa0 2024-05-14T11:18:52.888 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 msgr2=0x7feb8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 0x7feb8c071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7febac0b0d60 tx=0x7feba000b040 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 msgr2=0x7febac1fd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feba400c3f0 tx=0x7feba400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 shutdown_connections 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8c06f400 0x7feb8c071910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac156fa0 0x7febac1fd0c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1670 0x7febac1fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 --2- 172.21.15.116:0/2141865724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06e9e0 0x7febac14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 >> 172.21.15.116:0/2141865724 conn(0x7febac0b4620 msgr2=0x7febac15e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 shutdown_connections 2024-05-14T11:18:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:52.887+0000 7febb4000640 1 -- 172.21.15.116:0/2141865724 wait complete. 2024-05-14T11:18:52.914 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:18:53.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.055+0000 7efedd800640 1 Processor -- start 2024-05-14T11:18:53.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.055+0000 7efedd800640 1 -- start start 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efedd800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efed80d1670 0x7efed80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efedd800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed806c700 0x7efed806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efedd800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed806ff20 con 0x7efed815db30 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efedd800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efed80ae100 con 0x7efed80d1670 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efedd800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efed80ae870 con 0x7efed806c700 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efed80d1670 0x7efed80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efed80d1670 0x7efed80d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55732/0 (socket says 172.21.15.116:55732) 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 -- 172.21.15.116:0/969905087 learned_addr learned my addr 172.21.15.116:0/969905087 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:53.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efed7400640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed806c700 0x7efed806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 -- 172.21.15.116:0/969905087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed806c700 msgr2=0x7efed806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed806c700 0x7efed806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 -- 172.21.15.116:0/969905087 >> v1:172.21.15.149:6789/0 conn(0x7efed815db30 legacy=0x7efed80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.059+0000 7efece000640 1 -- 172.21.15.116:0/969905087 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed80d1fc0 con 0x7efed80d1670 2024-05-14T11:18:53.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efece000640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed80d1670 0x7efed80d1a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efec0009870 tx=0x7efec001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5447bef793773717 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efed6000640 1 -- 172.21.15.116:0/969905087 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00b0070 con 0x7efed80d1670 2024-05-14T11:18:53.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efed6000640 1 -- 172.21.15.116:0/969905087 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec00ab600 con 0x7efed80d1670 2024-05-14T11:18:53.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efed6000640 1 -- 172.21.15.116:0/969905087 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00ab9d0 con 0x7efed80d1670 2024-05-14T11:18:53.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 -- 172.21.15.116:0/969905087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed80d1670 msgr2=0x7efed80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed80d1670 0x7efed80d1a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efec0009870 tx=0x7efec001c400 comp rx=0 tx=0).stop 2024-05-14T11:18:53.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 -- 172.21.15.116:0/969905087 shutdown_connections 2024-05-14T11:18:53.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed806c700 0x7efed806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 --2- 172.21.15.116:0/969905087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed80d1670 0x7efed80d1a80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 -- 172.21.15.116:0/969905087 >> 172.21.15.116:0/969905087 conn(0x7efed80b4620 msgr2=0x7efed80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:53.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 -- 172.21.15.116:0/969905087 shutdown_connections 2024-05-14T11:18:53.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.063+0000 7efedd800640 1 -- 172.21.15.116:0/969905087 wait complete. 2024-05-14T11:18:53.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 Processor -- start 2024-05-14T11:18:53.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- start start 2024-05-14T11:18:53.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed80d1670 0x7efed81605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 0x7efed8160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efed816a770 con 0x7efed806c700 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efed815df40 con 0x7efed815db30 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efed816a8f0 con 0x7efed80d1670 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55748/0 (socket says 172.21.15.116:55748) 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 -- 172.21.15.116:0/440628245 learned_addr learned my addr 172.21.15.116:0/440628245 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efece000640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed80d1670 0x7efed81605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed7400640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 0x7efed8160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 -- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed80d1670 msgr2=0x7efed81605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed80d1670 0x7efed81605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 -- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 msgr2=0x7efed8160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 0x7efed8160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 -- 172.21.15.116:0/440628245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed809abb0 con 0x7efed806c700 2024-05-14T11:18:53.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed7400640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 0x7efed8160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efed6a00640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efec800c3c0 tx=0x7efec800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8017070 con 0x7efed806c700 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec80052d0 con 0x7efed806c700 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed815fad0 con 0x7efed806c700 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80054c0 con 0x7efed806c700 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.067+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed815e2f0 con 0x7efed806c700 2024-05-14T11:18:53.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed806a9c0 con 0x7efed806c700 2024-05-14T11:18:53.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efec801a070 con 0x7efed806c700 2024-05-14T11:18:53.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efecf400640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 0x7efeb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7efec808e3a0 con 0x7efed806c700 2024-05-14T11:18:53.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efece000640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 0x7efeb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.071+0000 7efece000640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 0x7efeb4071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efec00ad000 tx=0x7efec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.075+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efec805adf0 con 0x7efed806c700 2024-05-14T11:18:53.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.287+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efed8068860 con 0x7efed806c700 2024-05-14T11:18:53.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.287+0000 7efecf400640 1 -- 172.21.15.116:0/440628245 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7efec8063430 con 0x7efed806c700 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.292 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:18:53.294 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:18:53.295 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.296 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:18:53.297 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.298 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.300 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:18:53.301 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.302 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:18:53.304 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:18:53.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 msgr2=0x7efeb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 0x7efeb4071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efec00ad000 tx=0x7efec0008040 comp rx=0 tx=0).stop 2024-05-14T11:18:53.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 msgr2=0x7efed8099450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efec800c3c0 tx=0x7efec800c880 comp rx=0 tx=0).stop 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 shutdown_connections 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efeb406f400 0x7efeb4071910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed815db30 0x7efed8160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efed80d1670 0x7efed81605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 --2- 172.21.15.116:0/440628245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed806c700 0x7efed8099450 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 >> 172.21.15.116:0/440628245 conn(0x7efed80b4620 msgr2=0x7efed815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 shutdown_connections 2024-05-14T11:18:53.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.291+0000 7efedd800640 1 -- 172.21.15.116:0/440628245 wait complete. 2024-05-14T11:18:53.314 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:18:53.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 Processor -- start 2024-05-14T11:18:53.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 -- start start 2024-05-14T11:18:53.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9ee40b0390 0x7f9ee4167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee4160510 con 0x7f9ee40d1670 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee41607f0 con 0x7f9ee4156fa0 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9eeb200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee40706f0 con 0x7f9ee40b0390 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9edbe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9ee40d1670 0x7f9ee40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50246/0 (socket says 172.21.15.116:50246) 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.459+0000 7f9edbe00640 1 -- 172.21.15.116:0/4201992785 learned_addr learned my addr 172.21.15.116:0/4201992785 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 --2- 172.21.15.116:0/4201992785 >> v2:172.21.15.149:3300/0 conn(0x7f9ee40b0390 0x7f9ee4167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319864096 0 0) 0x7f9ee4160510 con 0x7f9ee40d1670 2024-05-14T11:18:53.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ec8003530 con 0x7f9ee40d1670 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3074505558 0 0) 0x7f9ec8003530 con 0x7f9ee40d1670 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 >> v2:172.21.15.149:3300/0 conn(0x7f9ee40b0390 msgr2=0x7f9ee4167800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 --2- 172.21.15.116:0/4201992785 >> v2:172.21.15.149:3300/0 conn(0x7f9ee40b0390 0x7f9ee4167800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 >> v1:172.21.15.149:6789/0 conn(0x7f9ee4156fa0 legacy=0x7f9ee406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee4160ad0 con 0x7f9ee40d1670 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9ecc001700 con 0x7f9ee40d1670 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 >> v1:172.21.15.116:6789/0 conn(0x7f9ee40d1670 legacy=0x7f9ee40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8003a60 0x7f9ec8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ec8004810 0x7f9ec8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 0x7f9ec800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee40706f0 con 0x7f9ec8005260 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec800c1f0 con 0x7f9ec8003a60 2024-05-14T11:18:53.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec800c3f0 con 0x7f9ec8004810 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 0x7f9ec800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9edbe00640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ec8004810 0x7f9ec8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee3e00640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8003a60 0x7f9ec8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 -- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ec8004810 msgr2=0x7f9ec8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ec8004810 0x7f9ec8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 -- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8003a60 msgr2=0x7f9ec8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8003a60 0x7f9ec8003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.463+0000 7f9ee9200640 1 -- 172.21.15.116:0/4201992785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec8003530 con 0x7f9ec8005260 2024-05-14T11:18:53.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9ee9200640 1 -- 172.21.15.116:0/4201992785 reap_dead start 2024-05-14T11:18:53.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9ee9200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 0x7f9ec800b9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009f90 tx=0x7f9edc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7325199146293f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc009c10 con 0x7f9ec8005260 2024-05-14T11:18:53.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc0a5c90 con 0x7f9ec8005260 2024-05-14T11:18:53.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9ee3400640 1 -- 172.21.15.116:0/4201992785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc003be0 con 0x7f9ec8005260 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 msgr2=0x7f9ec800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 0x7f9ec800b9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009f90 tx=0x7f9edc01cc80 comp rx=0 tx=0).stop 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- 172.21.15.116:0/4201992785 shutdown_connections 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 --2- 172.21.15.116:0/4201992785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ec8005260 0x7f9ec800b9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- 172.21.15.116:0/4201992785 >> 172.21.15.116:0/4201992785 conn(0x7f9ee40b4620 msgr2=0x7f9ee40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- 172.21.15.116:0/4201992785 shutdown_connections 2024-05-14T11:18:53.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- 172.21.15.116:0/4201992785 wait complete. 2024-05-14T11:18:53.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 Processor -- start 2024-05-14T11:18:53.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.467+0000 7f9eeb200640 1 -- start start 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 0x7f9ee41f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4154630 0x7f9ee41f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4161120 con 0x7f9ee409ab80 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee40b07a0 con 0x7f9ec8005260 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee406ffe0 con 0x7f9ee4154630 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 0x7f9ee41f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 0x7f9ee41f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:38976/0 (socket says 172.21.15.116:38976) 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55764/0 (socket says 172.21.15.116:55764) 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee3e00640 1 -- 172.21.15.116:0/2785523133 learned_addr learned my addr 172.21.15.116:0/2785523133 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9edbe00640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4154630 0x7f9ee41f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 -- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4154630 msgr2=0x7f9ee41f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4154630 0x7f9ee41f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 -- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 msgr2=0x7f9ee41f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 0x7f9ee41f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 -- 172.21.15.116:0/2785523133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee4156530 con 0x7f9ee409ab80 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee9200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4072780 tx=0x7f9edc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0bd020 con 0x7f9ee409ab80 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc0a6e50 con 0x7f9ee409ab80 2024-05-14T11:18:53.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee4156810 con 0x7f9ee409ab80 2024-05-14T11:18:53.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc009c10 con 0x7f9ee409ab80 2024-05-14T11:18:53.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41f25a0 con 0x7f9ee409ab80 2024-05-14T11:18:53.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.471+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee406a9c0 con 0x7f9ee409ab80 2024-05-14T11:18:53.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.475+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9edc0c0070 con 0x7f9ee409ab80 2024-05-14T11:18:53.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.475+0000 7f9ee0c00640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 0x7f9ec0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.475+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f9edc129980 con 0x7f9ee409ab80 2024-05-14T11:18:53.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.475+0000 7f9ee3e00640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 0x7f9ec0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.475+0000 7f9ee3e00640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 0x7f9ec0071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4001850 tx=0x7f9ed4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.479+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9edc016990 con 0x7f9ee409ab80 2024-05-14T11:18:53.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9ee4074c80 con 0x7f9ee409ab80 2024-05-14T11:18:53.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9ee0c00640 1 -- 172.21.15.116:0/2785523133 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f9edc0f6340 con 0x7f9ee409ab80 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:18:53.642 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 msgr2=0x7f9ec0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 0x7f9ec0071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4001850 tx=0x7f9ed4009040 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 msgr2=0x7f9ee41f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4072780 tx=0x7f9edc009750 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 shutdown_connections 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ec006f430 0x7f9ec0071940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee409ab80 0x7f9ee41f2ac0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ee4154630 0x7f9ee41f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 --2- 172.21.15.116:0/2785523133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec8005260 0x7f9ee41f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 >> 172.21.15.116:0/2785523133 conn(0x7f9ee40b4620 msgr2=0x7f9ee4150790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 shutdown_connections 2024-05-14T11:18:53.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.639+0000 7f9eeb200640 1 -- 172.21.15.116:0/2785523133 wait complete. 2024-05-14T11:18:53.664 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:18:53.664 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:18:53.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 Processor -- start 2024-05-14T11:18:53.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 -- start start 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60fc0cd620 0x7f60fc06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0cda30 0x7f60fc06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f60fc161710 0x7f60fc161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0d0270 con 0x7f60fc0cd620 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc0b0c30 con 0x7f60fc161710 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f6102000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0d04c0 con 0x7f60fc0cda30 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60fc0cd620 0x7f60fc06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60fc0cd620 0x7f60fc06f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55768/0 (socket says 172.21.15.116:55768) 2024-05-14T11:18:53.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 -- 172.21.15.116:0/387851506 learned_addr learned my addr 172.21.15.116:0/387851506 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fbe00640 1 --2- 172.21.15.116:0/387851506 >> v2:172.21.15.149:3300/0 conn(0x7f60fc161710 0x7f60fc161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60faa00640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0cda30 0x7f60fc06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 -- 172.21.15.116:0/387851506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0cda30 msgr2=0x7f60fc06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0cda30 0x7f60fc06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 -- 172.21.15.116:0/387851506 >> v2:172.21.15.149:3300/0 conn(0x7f60fc161710 msgr2=0x7f60fc161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 --2- 172.21.15.116:0/387851506 >> v2:172.21.15.149:3300/0 conn(0x7f60fc161710 0x7f60fc161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.811+0000 7f60fb400640 1 -- 172.21.15.116:0/387851506 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0d07a0 con 0x7f60fc0cd620 2024-05-14T11:18:53.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f60fb400640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0cd620 0x7f60fc06f6b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1627b0 tx=0x7f60ec01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3af35e3393c15d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f60fa000640 1 -- 172.21.15.116:0/387851506 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0a5670 con 0x7f60fc0cd620 2024-05-14T11:18:53.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f60fa000640 1 -- 172.21.15.116:0/387851506 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec0a5810 con 0x7f60fc0cd620 2024-05-14T11:18:53.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f60fa000640 1 -- 172.21.15.116:0/387851506 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0a5be0 con 0x7f60fc0cd620 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 -- 172.21.15.116:0/387851506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0cd620 msgr2=0x7f60fc06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0cd620 0x7f60fc06f6b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1627b0 tx=0x7f60ec01c9f0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 -- 172.21.15.116:0/387851506 shutdown_connections 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 --2- 172.21.15.116:0/387851506 >> v2:172.21.15.149:3300/0 conn(0x7f60fc161710 0x7f60fc161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0cda30 0x7f60fc06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 --2- 172.21.15.116:0/387851506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0cd620 0x7f60fc06f6b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.815+0000 7f6102000640 1 -- 172.21.15.116:0/387851506 >> 172.21.15.116:0/387851506 conn(0x7f60fc0cde80 msgr2=0x7f60fc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:53.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- 172.21.15.116:0/387851506 shutdown_connections 2024-05-14T11:18:53.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- 172.21.15.116:0/387851506 wait complete. 2024-05-14T11:18:53.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 Processor -- start 2024-05-14T11:18:53.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- start start 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 0x7f60fc14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 0x7f60fc148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06d090 con 0x7f60fc161710 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06cf10 con 0x7f60fc0cd620 2024-05-14T11:18:53.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f6102000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc06d210 con 0x7f60fc148010 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.819+0000 7f60faa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55782/0 (socket says 172.21.15.116:55782) 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 -- 172.21.15.116:0/3529922426 learned_addr learned my addr 172.21.15.116:0/3529922426 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60fbe00640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 0x7f60fc148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60fb400640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 0x7f60fc14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 -- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 msgr2=0x7f60fc148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 0x7f60fc148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 -- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 msgr2=0x7f60fc14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 0x7f60fc14d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 -- 172.21.15.116:0/3529922426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc148960 con 0x7f60fc161710 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60fbe00640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 0x7f60fc148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60fb400640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 0x7f60fc14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60faa00640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d830 tx=0x7f60e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8018030 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e800be10 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc14b8d0 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80098b0 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc14be90 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc14b640 con 0x7f60fc161710 2024-05-14T11:18:53.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.823+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60e801b070 con 0x7f60fc161710 2024-05-14T11:18:53.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.827+0000 7f60f3400640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.827+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5249+0+0 (secure 0 0 0) 0x7f60e808f300 con 0x7f60fc161710 2024-05-14T11:18:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.827+0000 7f60fb400640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.827+0000 7f60fb400640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ae000 tx=0x7f60ec0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:53.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.831+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f60fc148960 con 0x7f60fc161710 2024-05-14T11:18:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:53.991+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f60fc06aa30 con 0x7f60fc161710 2024-05-14T11:18:54.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.891+0000 7f60f3400640 1 -- 172.21.15.116:0/3529922426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f60e805c3b0 con 0x7f60fc161710 2024-05-14T11:18:54.895 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 msgr2=0x7f60dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ae000 tx=0x7f60ec0a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 msgr2=0x7f60fc1f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d830 tx=0x7f60e800dcf0 comp rx=0 tx=0).stop 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 shutdown_connections 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc148010 0x7f60fc148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc161710 0x7f60fc1f8f80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 --2- 172.21.15.116:0/3529922426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0cd620 0x7f60fc14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 >> 172.21.15.116:0/3529922426 conn(0x7f60fc0cde80 msgr2=0x7f60fc0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 shutdown_connections 2024-05-14T11:18:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:54.895+0000 7f6102000640 1 -- 172.21.15.116:0/3529922426 wait complete. 2024-05-14T11:18:54.927 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:18:55.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 Processor -- start 2024-05-14T11:18:55.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 -- start start 2024-05-14T11:18:55.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c806f450 0x7f15c806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15c80d1730 0x7f15c80d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f15c80cce60 0x7f15c80cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f15c806db80 con 0x7f15c80d1730 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f15c806f860 con 0x7f15c80cce60 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15cd000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80d2080 con 0x7f15c806f450 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15c80d1730 0x7f15c80d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15c80d1730 0x7f15c80d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55790/0 (socket says 172.21.15.116:55790) 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 -- 172.21.15.116:0/3784865300 learned_addr learned my addr 172.21.15.116:0/3784865300 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15c5e00640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c806f450 0x7f15c806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15c6800640 1 --2- 172.21.15.116:0/3784865300 >> v2:172.21.15.149:3300/0 conn(0x7f15c80cce60 0x7f15c80cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 -- 172.21.15.116:0/3784865300 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c806f450 msgr2=0x7f15c806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c806f450 0x7f15c806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 -- 172.21.15.116:0/3784865300 >> v2:172.21.15.149:3300/0 conn(0x7f15c80cce60 msgr2=0x7f15c80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 --2- 172.21.15.116:0/3784865300 >> v2:172.21.15.149:3300/0 conn(0x7f15c80cce60 0x7f15c80cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.083+0000 7f15bd400640 1 -- 172.21.15.116:0/3784865300 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c806fae0 con 0x7f15c80d1730 2024-05-14T11:18:55.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15bd400640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80d1b40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15c80727b0 tx=0x7f15b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccea28100e33274e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:55.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15c5400640 1 -- 172.21.15.116:0/3784865300 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b00b1070 con 0x7f15c80d1730 2024-05-14T11:18:55.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15c5400640 1 -- 172.21.15.116:0/3784865300 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b00ac630 con 0x7f15c80d1730 2024-05-14T11:18:55.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15c5400640 1 -- 172.21.15.116:0/3784865300 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b00a8050 con 0x7f15c80d1730 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 -- 172.21.15.116:0/3784865300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 msgr2=0x7f15c80d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80d1b40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15c80727b0 tx=0x7f15b001c520 comp rx=0 tx=0).stop 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 -- 172.21.15.116:0/3784865300 shutdown_connections 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80d1b40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 --2- 172.21.15.116:0/3784865300 >> v2:172.21.15.149:3300/0 conn(0x7f15c80cce60 0x7f15c80cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 --2- 172.21.15.116:0/3784865300 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c806f450 0x7f15c806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 -- 172.21.15.116:0/3784865300 >> 172.21.15.116:0/3784865300 conn(0x7f15c80b4720 msgr2=0x7f15c80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 -- 172.21.15.116:0/3784865300 shutdown_connections 2024-05-14T11:18:55.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.087+0000 7f15cd000640 1 -- 172.21.15.116:0/3784865300 wait complete. 2024-05-14T11:18:55.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 Processor -- start 2024-05-14T11:18:55.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 -- start start 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 0x7f15c8156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c80cce60 0x7f15c80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8169600 con 0x7f15c80d1730 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f15c8070660 con 0x7f15c806f450 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15cd000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f15c8169780 con 0x7f15c80cce60 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15c6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15c6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55800/0 (socket says 172.21.15.116:55800) 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15c6800640 1 -- 172.21.15.116:0/3077874361 learned_addr learned my addr 172.21.15.116:0/3077874361 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.091+0000 7f15c5e00640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 0x7f15c8156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bd400640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c80cce60 0x7f15c80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 -- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c80cce60 msgr2=0x7f15c80c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c80cce60 0x7f15c80c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 -- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 msgr2=0x7f15c8156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 0x7f15c8156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 -- 172.21.15.116:0/3077874361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8157520 con 0x7f15c80d1730 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c5e00640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 0x7f15c8156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15c6800640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009d30 tx=0x7f15c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0018020 con 0x7f15c80d1730 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c0014480 con 0x7f15c80d1730 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0014870 con 0x7f15c80d1730 2024-05-14T11:18:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c80c7ab0 con 0x7f15c80d1730 2024-05-14T11:18:55.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81facb0 con 0x7f15c80d1730 2024-05-14T11:18:55.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c806a9c0 con 0x7f15c80d1730 2024-05-14T11:18:55.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15c001b020 con 0x7f15c80d1730 2024-05-14T11:18:55.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.095+0000 7f15bea00640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 0x7f15a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:55.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.099+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5619+0+0 (secure 0 0 0) 0x7f15c008ffc0 con 0x7f15c80d1730 2024-05-14T11:18:55.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.099+0000 7f15c5e00640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 0x7f15a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:55.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.099+0000 7f15c5e00640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 0x7f15a4071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15c80c83b0 tx=0x7f15b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:55.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.103+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f15c0022030 con 0x7f15c80d1730 2024-05-14T11:18:55.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.259+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f15c806dfd0 con 0x7f15c80d1730 2024-05-14T11:18:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.899+0000 7f15bea00640 1 -- 172.21.15.116:0/3077874361 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f15c0010040 con 0x7f15c80d1730 2024-05-14T11:18:55.904 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 msgr2=0x7f15a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 0x7f15a4071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15c80c83b0 tx=0x7f15b8008040 comp rx=0 tx=0).stop 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 msgr2=0x7f15c80c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009d30 tx=0x7f15c000c490 comp rx=0 tx=0).stop 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 shutdown_connections 2024-05-14T11:18:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f15a406f400 0x7f15a4071910 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15c80c83b0 tx=0x7f15b8008040 comp rx=0 tx=0).stop 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15c80d1730 0x7f15c80c83e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15c80cce60 0x7f15c80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 --2- 172.21.15.116:0/3077874361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15c806f450 0x7f15c8156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 >> 172.21.15.116:0/3077874361 conn(0x7f15c80b4720 msgr2=0x7f15c815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 shutdown_connections 2024-05-14T11:18:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:55.903+0000 7f15cd000640 1 -- 172.21.15.116:0/3077874361 wait complete. 2024-05-14T11:18:55.944 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:18:56.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 Processor -- start 2024-05-14T11:18:56.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 -- start start 2024-05-14T11:18:56.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab406f7e0 0x7f6ab40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 0x7f6ab40d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab4164600 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab406ea40 con 0x7f6ab406fbf0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6abb000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab40ceac0 con 0x7f6ab406f7e0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6ab9000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6ab9000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55804/0 (socket says 172.21.15.116:55804) 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6ab9000640 1 -- 172.21.15.116:0/2426498632 learned_addr learned my addr 172.21.15.116:0/2426498632 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6ab3e00640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab406f7e0 0x7f6ab40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.091+0000 7f6aabe00640 1 --2- 172.21.15.116:0/2426498632 >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 0x7f6ab40d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 -- 172.21.15.116:0/2426498632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab406f7e0 msgr2=0x7f6ab40d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab406f7e0 0x7f6ab40d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 -- 172.21.15.116:0/2426498632 >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 msgr2=0x7f6ab40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 --2- 172.21.15.116:0/2426498632 >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 0x7f6ab40d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 -- 172.21.15.116:0/2426498632 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab406ed00 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6aabe00640 1 --2- 172.21.15.116:0/2426498632 >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 0x7f6ab40d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:56.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab9000640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40cee80 tx=0x7f6aac01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a78c26a726615af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:56.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab3400640 1 -- 172.21.15.116:0/2426498632 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0b4070 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab3400640 1 -- 172.21.15.116:0/2426498632 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac0af400 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.095+0000 7f6ab3400640 1 -- 172.21.15.116:0/2426498632 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0af7d0 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- 172.21.15.116:0/2426498632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab40cf8c0 msgr2=0x7f6ab40cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:56.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40cee80 tx=0x7f6aac01f1e0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- 172.21.15.116:0/2426498632 shutdown_connections 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab40cf8c0 0x7f6ab40cfcd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- 172.21.15.116:0/2426498632 >> v2:172.21.15.149:3300/0 conn(0x7f6ab406fbf0 0x7f6ab40d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- 172.21.15.116:0/2426498632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab406f7e0 0x7f6ab40d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- 172.21.15.116:0/2426498632 >> 172.21.15.116:0/2426498632 conn(0x7f6ab40b4720 msgr2=0x7f6ab40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- 172.21.15.116:0/2426498632 shutdown_connections 2024-05-14T11:18:56.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- 172.21.15.116:0/2426498632 wait complete. 2024-05-14T11:18:56.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 Processor -- start 2024-05-14T11:18:56.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 -- start start 2024-05-14T11:18:56.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 0x7f6ab415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.099+0000 7f6abb000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab40cf8c0 0x7f6ab4073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab40d0210 con 0x7f6ab406fbf0 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406f2c0 con 0x7f6ab406f7e0 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab40d0390 con 0x7f6ab40cf8c0 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55820/0 (socket says 172.21.15.116:55820) 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 -- 172.21.15.116:0/749321701 learned_addr learned my addr 172.21.15.116:0/749321701 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab3e00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 0x7f6ab415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab9000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab40cf8c0 0x7f6ab4073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 -- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab40cf8c0 msgr2=0x7f6ab4073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:56.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab40cf8c0 0x7f6ab4073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 -- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 msgr2=0x7f6ab415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 0x7f6ab415a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 -- 172.21.15.116:0/749321701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab415a710 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab3e00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 0x7f6ab415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6aabe00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d800 tx=0x7f6a9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00b660 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c019070 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00ba40 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab415ad60 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab415b320 con 0x7f6ab406fbf0 2024-05-14T11:18:56.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab406a9c0 con 0x7f6ab406fbf0 2024-05-14T11:18:56.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.103+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6a9c00bbe0 con 0x7f6ab406fbf0 2024-05-14T11:18:56.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.107+0000 7f6ab0c00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 0x7f6a90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:56.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.107+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6001+0+0 (secure 0 0 0) 0x7f6a9c090130 con 0x7f6ab406fbf0 2024-05-14T11:18:56.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.107+0000 7f6ab3e00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 0x7f6a90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:56.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.107+0000 7f6ab3e00640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 0x7f6a90071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009eb0 tx=0x7f6aa4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:56.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.111+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6a9c05c910 con 0x7f6ab406fbf0 2024-05-14T11:18:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.299+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ab4069de0 con 0x7f6ab406fbf0 2024-05-14T11:18:56.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.911+0000 7f6ab0c00640 1 -- 172.21.15.116:0/749321701 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f6a9c060c10 con 0x7f6ab406fbf0 2024-05-14T11:18:56.924 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 9 and data pool 10 2024-05-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.927+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 msgr2=0x7f6a90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.927+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 0x7f6a90071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009eb0 tx=0x7f6aa4009190 comp rx=0 tx=0).stop 2024-05-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.927+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 msgr2=0x7f6ab40c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:56.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.927+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d800 tx=0x7f6a9c00dcc0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 shutdown_connections 2024-05-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9006f400 0x7f6a90071910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab40cf8c0 0x7f6ab4073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab406fbf0 0x7f6ab40c6d40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 --2- 172.21.15.116:0/749321701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab406f7e0 0x7f6ab415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 >> 172.21.15.116:0/749321701 conn(0x7f6ab40b4720 msgr2=0x7f6ab40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 shutdown_connections 2024-05-14T11:18:56.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:56.931+0000 7f6abb000640 1 -- 172.21.15.116:0/749321701 wait complete. 2024-05-14T11:18:56.969 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:57.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.111+0000 7f3fff400640 1 Processor -- start 2024-05-14T11:18:57.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.111+0000 7f3fff400640 1 -- start start 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fff400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff815db30 0x7f3ff80aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fff400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3ff80d1670 0x7f3ff80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fff400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff806ff20 con 0x7f3ff8161600 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fff400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff806d710 con 0x7f3ff815db30 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fff400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff80ae100 con 0x7f3ff80d1670 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff815db30 0x7f3ff80aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff815db30 0x7f3ff80aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55824/0 (socket says 172.21.15.116:55824) 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 -- 172.21.15.116:0/3754659140 learned_addr learned my addr 172.21.15.116:0/3754659140 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffd400640 1 --1- 172.21.15.116:0/3754659140 >> v1:172.21.15.116:6789/0 conn(0x7f3ff8161600 0x7f3ff8161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50252/0 (socket says 172.21.15.116:50252) 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3fefe00640 1 --2- 172.21.15.116:0/3754659140 >> v2:172.21.15.149:3300/0 conn(0x7f3ff80d1670 0x7f3ff80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 -- 172.21.15.116:0/3754659140 >> v2:172.21.15.149:3300/0 conn(0x7f3ff80d1670 msgr2=0x7f3ff80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 --2- 172.21.15.116:0/3754659140 >> v2:172.21.15.149:3300/0 conn(0x7f3ff80d1670 0x7f3ff80d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 -- 172.21.15.116:0/3754659140 >> v1:172.21.15.116:6789/0 conn(0x7f3ff8161600 legacy=0x7f3ff8161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ff7e00640 1 -- 172.21.15.116:0/3754659140 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905259111 0 0) 0x7f3ff806ff20 con 0x7f3ff8161600 2024-05-14T11:18:57.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.115+0000 7f3ffca00640 1 -- 172.21.15.116:0/3754659140 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80d1fc0 con 0x7f3ff815db30 2024-05-14T11:18:57.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3ffca00640 1 --2- 172.21.15.116:0/3754659140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff815db30 0x7f3ff80aeaa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80d2370 tx=0x7f3fe801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663047e03ebe17d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3ff7e00640 1 -- 172.21.15.116:0/3754659140 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80b0070 con 0x7f3ff815db30 2024-05-14T11:18:57.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3ff7e00640 1 -- 172.21.15.116:0/3754659140 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe80ab600 con 0x7f3ff815db30 2024-05-14T11:18:57.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3ff7e00640 1 -- 172.21.15.116:0/3754659140 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80ab9d0 con 0x7f3ff815db30 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 -- 172.21.15.116:0/3754659140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff815db30 msgr2=0x7f3ff80aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 --2- 172.21.15.116:0/3754659140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff815db30 0x7f3ff80aeaa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80d2370 tx=0x7f3fe801c490 comp rx=0 tx=0).stop 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 -- 172.21.15.116:0/3754659140 shutdown_connections 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 --2- 172.21.15.116:0/3754659140 >> v2:172.21.15.149:3300/0 conn(0x7f3ff80d1670 0x7f3ff80d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 --2- 172.21.15.116:0/3754659140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff815db30 0x7f3ff80aeaa0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 -- 172.21.15.116:0/3754659140 >> 172.21.15.116:0/3754659140 conn(0x7f3ff80b4620 msgr2=0x7f3ff80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 -- 172.21.15.116:0/3754659140 shutdown_connections 2024-05-14T11:18:57.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.119+0000 7f3fff400640 1 -- 172.21.15.116:0/3754659140 wait complete. 2024-05-14T11:18:57.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 Processor -- start 2024-05-14T11:18:57.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 -- start start 2024-05-14T11:18:57.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff815db30 0x7f3ff815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 0x7f3ff8157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff815df40 con 0x7f3ff80d1670 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80aefe0 con 0x7f3ff8161600 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff816af80 con 0x7f3ff815db30 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55828/0 (socket says 172.21.15.116:55828) 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 -- 172.21.15.116:0/4081438335 learned_addr learned my addr 172.21.15.116:0/4081438335 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fefe00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff815db30 0x7f3ff815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffd400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 0x7f3ff8157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 -- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff815db30 msgr2=0x7f3ff815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff815db30 0x7f3ff815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 -- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 msgr2=0x7f3ff8157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 0x7f3ff8157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 -- 172.21.15.116:0/4081438335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff81535a0 con 0x7f3ff80d1670 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffd400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 0x7f3ff8157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:57.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ffca00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3fe801a0c0 tx=0x7f3fe801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80ba020 con 0x7f3ff80d1670 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe80b6640 con 0x7f3ff80d1670 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff815ec00 con 0x7f3ff80d1670 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.123+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80b0040 con 0x7f3ff80d1670 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff81532c0 con 0x7f3ff80d1670 2024-05-14T11:18:57.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff806a9c0 con 0x7f3ff80d1670 2024-05-14T11:18:57.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3fe80bd020 con 0x7f3ff80d1670 2024-05-14T11:18:57.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3ff5600640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f3fe81296e0 con 0x7f3ff80d1670 2024-05-14T11:18:57.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.127+0000 7f3fefe00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.131+0000 7f3fefe00640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 0x7f3fd4071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8160e20 tx=0x7f3fe0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.135+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3fe80c4030 con 0x7f3ff80d1670 2024-05-14T11:18:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.291+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ff815d3a0 con 0x7f3ff80d1670 2024-05-14T11:18:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3ff5600640 1 -- 172.21.15.116:0/4081438335 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16945 (secure 0 0 0) 0x7f3ff80d1fc0 con 0x7f3ff80d1670 2024-05-14T11:18:57.297 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:57.297 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:56.305877+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:18:54.005073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-14T11:18:55.275835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 msgr2=0x7f3fd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 0x7f3fd4071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8160e20 tx=0x7f3fe0008040 comp rx=0 tx=0).stop 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 msgr2=0x7f3ff8151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3fe801a0c0 tx=0x7f3fe801acd0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 shutdown_connections 2024-05-14T11:18:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd406f400 0x7f3fd4071910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff8161600 0x7f3ff8157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff815db30 0x7f3ff815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 --2- 172.21.15.116:0/4081438335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff80d1670 0x7f3ff8151600 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 >> 172.21.15.116:0/4081438335 conn(0x7f3ff80b4620 msgr2=0x7f3ff81679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 shutdown_connections 2024-05-14T11:18:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.295+0000 7f3fff400640 1 -- 172.21.15.116:0/4081438335 wait complete. 2024-05-14T11:18:57.319 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:18:57.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 Processor -- start 2024-05-14T11:18:57.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 -- start start 2024-05-14T11:18:57.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff094156fa0 0x7ff09406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff094160510 con 0x7ff0940af380 2024-05-14T11:18:57.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0941607f0 con 0x7ff0940d1670 2024-05-14T11:18:57.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff099200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff0941573b0 con 0x7ff094156fa0 2024-05-14T11:18:57.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff092800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff0940af380 0x7ff0940af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50262/0 (socket says 172.21.15.116:50262) 2024-05-14T11:18:57.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff092800640 1 -- 172.21.15.116:0/3298671479 learned_addr learned my addr 172.21.15.116:0/3298671479 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:57.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff091e00640 1 --2- 172.21.15.116:0/3298671479 >> v2:172.21.15.149:3300/0 conn(0x7ff094156fa0 0x7ff09406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234088830 0 0) 0x7ff094160510 con 0x7ff0940af380 2024-05-14T11:18:57.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.463+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff078005740 con 0x7ff0940af380 2024-05-14T11:18:57.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091e00640 1 -- 172.21.15.116:0/3298671479 >> v1:172.21.15.149:6789/0 conn(0x7ff0940d1670 legacy=0x7ff0940d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965255330 0 0) 0x7ff0941607f0 con 0x7ff0940d1670 2024-05-14T11:18:57.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091e00640 1 -- 172.21.15.116:0/3298671479 >> v1:172.21.15.116:6789/0 conn(0x7ff0940af380 legacy=0x7ff0940af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091e00640 1 -- 172.21.15.116:0/3298671479 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094160ad0 con 0x7ff094156fa0 2024-05-14T11:18:57.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091e00640 1 --2- 172.21.15.116:0/3298671479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff094156fa0 0x7ff09406ffe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff084009d90 tx=0x7ff08401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374fd54cd73acb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b2070 con 0x7ff094156fa0 2024-05-14T11:18:57.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0840a5940 con 0x7ff094156fa0 2024-05-14T11:18:57.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff091400640 1 -- 172.21.15.116:0/3298671479 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840a5d10 con 0x7ff094156fa0 2024-05-14T11:18:57.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff099200640 1 -- 172.21.15.116:0/3298671479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff094156fa0 msgr2=0x7ff09406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.467+0000 7ff099200640 1 --2- 172.21.15.116:0/3298671479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff094156fa0 0x7ff09406ffe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff084009d90 tx=0x7ff08401c970 comp rx=0 tx=0).stop 2024-05-14T11:18:57.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- 172.21.15.116:0/3298671479 shutdown_connections 2024-05-14T11:18:57.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 --2- 172.21.15.116:0/3298671479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff094156fa0 0x7ff09406ffe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- 172.21.15.116:0/3298671479 >> 172.21.15.116:0/3298671479 conn(0x7ff0940b4620 msgr2=0x7ff0940b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:57.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- 172.21.15.116:0/3298671479 shutdown_connections 2024-05-14T11:18:57.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- 172.21.15.116:0/3298671479 wait complete. 2024-05-14T11:18:57.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 Processor -- start 2024-05-14T11:18:57.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- start start 2024-05-14T11:18:57.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 0x7ff094146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0940d1670 0x7ff094155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff09406ca90 con 0x7ff094156fa0 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff09406c910 con 0x7ff0940af380 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.471+0000 7ff099200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff09406cc10 con 0x7ff0940d1670 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55834/0 (socket says 172.21.15.116:55834) 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 -- 172.21.15.116:0/3517788492 learned_addr learned my addr 172.21.15.116:0/3517788492 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff091e00640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 0x7ff094146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff089400640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0940d1670 0x7ff094155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 -- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0940d1670 msgr2=0x7ff094155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0940d1670 0x7ff094155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 -- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 msgr2=0x7ff094146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 0x7ff094146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 -- 172.21.15.116:0/3517788492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09414e800 con 0x7ff094156fa0 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff091e00640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 0x7ff094146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:57.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff092800640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017070 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c005cf0 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0072f0 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09414ec70 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09414f230 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09406a9c0 con 0x7ff094156fa0 2024-05-14T11:18:57.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.475+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff08c01a070 con 0x7ff094156fa0 2024-05-14T11:18:57.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.479+0000 7ff08aa00640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 0x7ff070071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.479+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7ff08c08e7e0 con 0x7ff094156fa0 2024-05-14T11:18:57.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.479+0000 7ff091e00640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 0x7ff070071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.479+0000 7ff091e00640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 0x7ff070071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff0840af000 tx=0x7ff0840a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.483+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff08c05b530 con 0x7ff094156fa0 2024-05-14T11:18:57.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.639+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff094070070 con 0x7ff094156fa0 2024-05-14T11:18:57.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff08aa00640 1 -- 172.21.15.116:0/3517788492 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16945 (secure 0 0 0) 0x7ff08c05ae30 con 0x7ff094156fa0 2024-05-14T11:18:57.645 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:18:57.645 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:56.305877+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:18:54.005073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-14T11:18:55.275835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 msgr2=0x7ff070071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 0x7ff070071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff0840af000 tx=0x7ff0840a7070 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 msgr2=0x7ff094073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 shutdown_connections 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff07006f400 0x7ff070071910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff094156fa0 0x7ff094073f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0940d1670 0x7ff094155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 --2- 172.21.15.116:0/3517788492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0940af380 0x7ff094146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 >> 172.21.15.116:0/3517788492 conn(0x7ff0940b4620 msgr2=0x7ff09415f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:57.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 shutdown_connections 2024-05-14T11:18:57.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.643+0000 7ff099200640 1 -- 172.21.15.116:0/3517788492 wait complete. 2024-05-14T11:18:57.667 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:18:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 Processor -- start 2024-05-14T11:18:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 -- start start 2024-05-14T11:18:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f678415c1d0 0x7f67840d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f67840ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f678415fc50 0x7f6784160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f678406d790 con 0x7f678415fc50 2024-05-14T11:18:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f67840ce690 con 0x7f678415c1d0 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f678406cc10 con 0x7f67840cdd40 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f678415fc50 0x7f6784160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f678415fc50 0x7f6784160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39738/0 (socket says 172.21.15.116:39738) 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 -- 172.21.15.116:0/589576730 learned_addr learned my addr 172.21.15.116:0/589576730 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f6789e00640 1 --2- 172.21.15.116:0/589576730 >> v2:172.21.15.149:3300/0 conn(0x7f678415c1d0 0x7f67840d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f6789400640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f67840ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 -- 172.21.15.116:0/589576730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 msgr2=0x7f67840ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f67840ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 -- 172.21.15.116:0/589576730 >> v2:172.21.15.149:3300/0 conn(0x7f678415c1d0 msgr2=0x7f67840d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 --2- 172.21.15.116:0/589576730 >> v2:172.21.15.149:3300/0 conn(0x7f678415c1d0 0x7f67840d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.815+0000 7f678a800640 1 -- 172.21.15.116:0/589576730 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678406cef0 con 0x7f678415fc50 2024-05-14T11:18:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678a800640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784160060 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f67840b52d0 tx=0x7f678001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e194fbaff1df48c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f6788a00640 1 -- 172.21.15.116:0/589576730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000b2d0 con 0x7f678415fc50 2024-05-14T11:18:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f6788a00640 1 -- 172.21.15.116:0/589576730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678000b470 con 0x7f678415fc50 2024-05-14T11:18:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f6788a00640 1 -- 172.21.15.116:0/589576730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800a7990 con 0x7f678415fc50 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 -- 172.21.15.116:0/589576730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 msgr2=0x7f6784160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784160060 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f67840b52d0 tx=0x7f678001eb60 comp rx=0 tx=0).stop 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 -- 172.21.15.116:0/589576730 shutdown_connections 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784160060 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 --2- 172.21.15.116:0/589576730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f67840ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 --2- 172.21.15.116:0/589576730 >> v2:172.21.15.149:3300/0 conn(0x7f678415c1d0 0x7f67840d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 -- 172.21.15.116:0/589576730 >> 172.21.15.116:0/589576730 conn(0x7f67840b4720 msgr2=0x7f67840b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 -- 172.21.15.116:0/589576730 shutdown_connections 2024-05-14T11:18:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.819+0000 7f678c800640 1 -- 172.21.15.116:0/589576730 wait complete. 2024-05-14T11:18:57.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 Processor -- start 2024-05-14T11:18:57.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 -- start start 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f678415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f678415c1d0 0x7f678415e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67840b0b70 con 0x7f678415fc50 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67840b09f0 con 0x7f678415c1d0 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cbd60 con 0x7f67840cdd40 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39750/0 (socket says 172.21.15.116:39750) 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 -- 172.21.15.116:0/4058089664 learned_addr learned my addr 172.21.15.116:0/4058089664 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f6789400640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f678415c1d0 0x7f678415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f6789e00640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f678415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 -- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 msgr2=0x7f678415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f678415c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 -- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f678415c1d0 msgr2=0x7f678415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f678415c1d0 0x7f678415e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 -- 172.21.15.116:0/4058089664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678415eed0 con 0x7f678415fc50 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.823+0000 7f678a800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f67800b1040 tx=0x7f678001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678000b720 con 0x7f678415fc50 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67800be070 con 0x7f678415fc50 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780009a40 con 0x7f678415fc50 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67840c4f50 con 0x7f678415fc50 2024-05-14T11:18:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67840c5510 con 0x7f678415fc50 2024-05-14T11:18:57.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678406a9c0 con 0x7f678415fc50 2024-05-14T11:18:57.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f67800b6070 con 0x7f678415fc50 2024-05-14T11:18:57.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 0x7f6768071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:57.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6101+0+0 (secure 0 0 0) 0x7f678012a1e0 con 0x7f678415fc50 2024-05-14T11:18:57.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.827+0000 7f6789e00640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 0x7f6768071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:57.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.831+0000 7f6789e00640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 0x7f6768071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6778009a50 tx=0x7f6778008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:57.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:57.831+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f67800187e0 con 0x7f678415fc50 2024-05-14T11:18:57.890 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:18:57.887+0000 7fe6ee200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:18:58.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.031+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6784069de0 con 0x7f678415fc50 2024-05-14T11:18:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f677e000640 1 -- 172.21.15.116:0/4058089664 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f67800f6ea0 con 0x7f678415fc50 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 msgr2=0x7f6768071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 0x7f6768071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6778009a50 tx=0x7f6778008040 comp rx=0 tx=0).stop 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 msgr2=0x7f6784073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f67800b1040 tx=0x7f678001b900 comp rx=0 tx=0).stop 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 shutdown_connections 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f676806f400 0x7f6768071910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f678415c1d0 0x7f678415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f678415fc50 0x7f6784073510 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 --2- 172.21.15.116:0/4058089664 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67840cdd40 0x7f678415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 >> 172.21.15.116:0/4058089664 conn(0x7f67840b4720 msgr2=0x7f6784166810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 shutdown_connections 2024-05-14T11:18:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:58.931+0000 7f678c800640 1 -- 172.21.15.116:0/4058089664 wait complete. 2024-05-14T11:18:58.960 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:18:58.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:18:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 Processor -- start 2024-05-14T11:18:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 -- start start 2024-05-14T11:18:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9500b14d0 0x7fc9500b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc95006cf90 0x7fc95006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc95006d8e0 con 0x7fc95006e660 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc9500b1e20 con 0x7fc9500b14d0 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc955e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc9500d1440 con 0x7fc95006cf90 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc95006e660 0x7fc9500d18d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58974/0 (socket says 172.21.15.116:58974) 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94ea00640 1 -- 172.21.15.116:0/4203596331 learned_addr learned my addr 172.21.15.116:0/4203596331 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94e000640 1 --2- 172.21.15.116:0/4203596331 >> v2:172.21.15.116:3300/0 conn(0x7fc9500b14d0 0x7fc9500b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94f400640 1 --2- 172.21.15.116:0/4203596331 >> v2:172.21.15.149:3300/0 conn(0x7fc95006cf90 0x7fc95006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94e000640 1 -- 172.21.15.116:0/4203596331 >> v2:172.21.15.149:3300/0 conn(0x7fc95006cf90 msgr2=0x7fc95006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94e000640 1 --2- 172.21.15.116:0/4203596331 >> v2:172.21.15.149:3300/0 conn(0x7fc95006cf90 0x7fc95006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94e000640 1 -- 172.21.15.116:0/4203596331 >> v1:172.21.15.116:6789/0 conn(0x7fc95006e660 legacy=0x7fc9500d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.111+0000 7fc94e000640 1 -- 172.21.15.116:0/4203596331 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95015e850 con 0x7fc9500b14d0 2024-05-14T11:18:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc94e000640 1 --2- 172.21.15.116:0/4203596331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc9500b18e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9500b5540 tx=0x7fc93c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a95151d5908593 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc94d600640 1 -- 172.21.15.116:0/4203596331 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0b0070 con 0x7fc9500b14d0 2024-05-14T11:18:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc94d600640 1 -- 172.21.15.116:0/4203596331 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c0ab600 con 0x7fc9500b14d0 2024-05-14T11:18:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc94d600640 1 -- 172.21.15.116:0/4203596331 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0ab9d0 con 0x7fc9500b14d0 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 -- 172.21.15.116:0/4203596331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 msgr2=0x7fc9500b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 --2- 172.21.15.116:0/4203596331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc9500b18e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9500b5540 tx=0x7fc93c01c3f0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 -- 172.21.15.116:0/4203596331 shutdown_connections 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 --2- 172.21.15.116:0/4203596331 >> v2:172.21.15.149:3300/0 conn(0x7fc95006cf90 0x7fc95006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 --2- 172.21.15.116:0/4203596331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc9500b18e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 -- 172.21.15.116:0/4203596331 >> 172.21.15.116:0/4203596331 conn(0x7fc950071370 msgr2=0x7fc950071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 -- 172.21.15.116:0/4203596331 shutdown_connections 2024-05-14T11:18:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.115+0000 7fc955e00640 1 -- 172.21.15.116:0/4203596331 wait complete. 2024-05-14T11:18:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 Processor -- start 2024-05-14T11:18:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 -- start start 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 0x7fc95013d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc95006e660 0x7fc950141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc95006ea70 con 0x7fc9500b14d0 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc95015e4d0 con 0x7fc95006e660 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc955e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc95006ebf0 con 0x7fc95006cf90 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc94f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc94f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39770/0 (socket says 172.21.15.116:39770) 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc94f400640 1 -- 172.21.15.116:0/39238754 learned_addr learned my addr 172.21.15.116:0/39238754 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:18:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc94e000640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc95006e660 0x7fc950141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.119+0000 7fc94ea00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 0x7fc95013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 -- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 msgr2=0x7fc95013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 0x7fc95013d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 -- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc95006e660 msgr2=0x7fc950141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc95006e660 0x7fc950141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 -- 172.21.15.116:0/39238754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9500b2fe0 con 0x7fc9500b14d0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94ea00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 0x7fc95013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc94f400640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d590 tx=0x7fc94800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480125f0 con 0x7fc9500b14d0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc948019070 con 0x7fc9500b14d0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9500b32c0 con 0x7fc9500b14d0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480129f0 con 0x7fc9500b14d0 2024-05-14T11:18:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9500b3880 con 0x7fc9500b14d0 2024-05-14T11:18:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9500b2d50 con 0x7fc9500b14d0 2024-05-14T11:18:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.123+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc948012b90 con 0x7fc9500b14d0 2024-05-14T11:18:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.127+0000 7fc947400640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 0x7fc930071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:18:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.127+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fc94808f840 con 0x7fc9500b14d0 2024-05-14T11:18:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.127+0000 7fc94ea00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 0x7fc930071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:18:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.127+0000 7fc94ea00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 0x7fc930071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc9400012f0 tx=0x7fc940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:18:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.131+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc94805c500 con 0x7fc9500b14d0 2024-05-14T11:18:59.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.319+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc95006a9c0 con 0x7fc9500b14d0 2024-05-14T11:18:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.931+0000 7fc947400640 1 -- 172.21.15.116:0/39238754 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fc94805be00 con 0x7fc9500b14d0 2024-05-14T11:18:59.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 msgr2=0x7fc930071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 0x7fc930071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc9400012f0 tx=0x7fc940009040 comp rx=0 tx=0).stop 2024-05-14T11:18:59.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 msgr2=0x7fc950141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d590 tx=0x7fc94800da50 comp rx=0 tx=0).stop 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 shutdown_connections 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc93006f430 0x7fc930071940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc95006e660 0x7fc950141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9500b14d0 0x7fc950141fe0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 --2- 172.21.15.116:0/39238754 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc95006cf90 0x7fc95013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 >> 172.21.15.116:0/39238754 conn(0x7fc950071370 msgr2=0x7fc95006ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 shutdown_connections 2024-05-14T11:18:59.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:18:59.935+0000 7fc955e00640 1 -- 172.21.15.116:0/39238754 wait complete. 2024-05-14T11:18:59.958 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:18:59.958 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:00.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 Processor -- start 2024-05-14T11:19:00.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 -- start start 2024-05-14T11:19:00.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feb70164220 0x7feb700d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb700d1fc0 con 0x7feb7006ce10 2024-05-14T11:19:00.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb701600a0 con 0x7feb700d1bb0 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb78400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7feb7006f5d0 con 0x7feb70164220 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb75a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feb70164220 0x7feb700d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb76400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feb7006ce10 0x7feb7016aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58990/0 (socket says 172.21.15.116:58990) 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb76400640 1 -- 172.21.15.116:0/161235056 learned_addr learned my addr 172.21.15.116:0/161235056 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb75a00640 1 -- 172.21.15.116:0/161235056 >> v1:172.21.15.149:6789/0 conn(0x7feb700d1bb0 legacy=0x7feb7006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb75a00640 1 -- 172.21.15.116:0/161235056 >> v1:172.21.15.116:6789/0 conn(0x7feb7006ce10 legacy=0x7feb7016aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb75a00640 1 -- 172.21.15.116:0/161235056 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7006f8b0 con 0x7feb70164220 2024-05-14T11:19:00.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.103+0000 7feb6fe00640 1 -- 172.21.15.116:0/161235056 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417080466 0 0) 0x7feb701600a0 con 0x7feb700d1bb0 2024-05-14T11:19:00.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.107+0000 7feb75a00640 1 --2- 172.21.15.116:0/161235056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb70164220 0x7feb700d1670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feb640097c0 tx=0x7feb6401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de74e86820bc1c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.107+0000 7feb6fe00640 1 -- 172.21.15.116:0/161235056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640b1070 con 0x7feb70164220 2024-05-14T11:19:00.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.107+0000 7feb6fe00640 1 -- 172.21.15.116:0/161235056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb640a8050 con 0x7feb70164220 2024-05-14T11:19:00.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.107+0000 7feb6fe00640 1 -- 172.21.15.116:0/161235056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640ac860 con 0x7feb70164220 2024-05-14T11:19:00.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- 172.21.15.116:0/161235056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb70164220 msgr2=0x7feb700d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 --2- 172.21.15.116:0/161235056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb70164220 0x7feb700d1670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feb640097c0 tx=0x7feb6401c3e0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- 172.21.15.116:0/161235056 shutdown_connections 2024-05-14T11:19:00.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 --2- 172.21.15.116:0/161235056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb70164220 0x7feb700d1670 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- 172.21.15.116:0/161235056 >> 172.21.15.116:0/161235056 conn(0x7feb700b4620 msgr2=0x7feb700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:00.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- 172.21.15.116:0/161235056 shutdown_connections 2024-05-14T11:19:00.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- 172.21.15.116:0/161235056 wait complete. 2024-05-14T11:19:00.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 Processor -- start 2024-05-14T11:19:00.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.111+0000 7feb78400640 1 -- start start 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb700d1bb0 0x7feb7009bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb70164220 0x7feb7014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb700af2f0 con 0x7feb7006ce10 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb70073650 con 0x7feb700d1bb0 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb70164630 con 0x7feb70164220 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39796/0 (socket says 172.21.15.116:39796) 2024-05-14T11:19:00.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 -- 172.21.15.116:0/3922054962 learned_addr learned my addr 172.21.15.116:0/3922054962 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75000640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb700d1bb0 0x7feb7009bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb76400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb70164220 0x7feb7014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 -- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb70164220 msgr2=0x7feb7014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb70164220 0x7feb7014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 -- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb700d1bb0 msgr2=0x7feb7009bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb700d1bb0 0x7feb7009bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 -- 172.21.15.116:0/3922054962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7009c920 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb75a00640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feb64018ff0 tx=0x7feb6401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640b1040 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb640be070 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640b6660 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7009cc00 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7009d1c0 con 0x7feb7006ce10 2024-05-14T11:19:00.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7006a9c0 con 0x7feb7006ce10 2024-05-14T11:19:00.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.115+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb640b4070 con 0x7feb7006ce10 2024-05-14T11:19:00.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.119+0000 7feb6d600640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 0x7feb50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.119+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7feb64129720 con 0x7feb7006ce10 2024-05-14T11:19:00.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.119+0000 7feb75000640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 0x7feb50071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.119+0000 7feb75000640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 0x7feb50071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb5c007fd0 tx=0x7feb5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.127+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feb64016090 con 0x7feb7006ce10 2024-05-14T11:19:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.319+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb7006ecf0 con 0x7feb7006ce10 2024-05-14T11:19:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.319+0000 7feb6d600640 1 -- 172.21.15.116:0/3922054962 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5191 (secure 0 0 0) 0x7feb640f5da0 con 0x7feb7006ce10 2024-05-14T11:19:00.323 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:00.323 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:18:59.326933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:00.324 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 34 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 msgr2=0x7feb50071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 0x7feb50071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb5c007fd0 tx=0x7feb5c009040 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 msgr2=0x7feb700af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feb64018ff0 tx=0x7feb6401ac20 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 shutdown_connections 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb5006f400 0x7feb50071910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb700d1bb0 0x7feb7009bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb70164220 0x7feb7014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 --2- 172.21.15.116:0/3922054962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb7006ce10 0x7feb700af470 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 >> 172.21.15.116:0/3922054962 conn(0x7feb700b4620 msgr2=0x7feb7015d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 shutdown_connections 2024-05-14T11:19:00.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.323+0000 7feb78400640 1 -- 172.21.15.116:0/3922054962 wait complete. 2024-05-14T11:19:00.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:19:00.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.499+0000 7fc4d3600640 1 Processor -- start 2024-05-14T11:19:00.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.499+0000 7fc4d3600640 1 -- start start 2024-05-14T11:19:00.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d3600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d3600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc4cc0d1670 0x7fc4cc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d3600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc06ff20 con 0x7fc4cc161600 2024-05-14T11:19:00.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d3600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc06d710 con 0x7fc4cc15db30 2024-05-14T11:19:00.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d3600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc0ae100 con 0x7fc4cc0d1670 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39810/0 (socket says 172.21.15.116:39810) 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 -- 172.21.15.116:0/462881894 learned_addr learned my addr 172.21.15.116:0/462881894 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4cbe00640 1 --2- 172.21.15.116:0/462881894 >> v2:172.21.15.149:3300/0 conn(0x7fc4cc0d1670 0x7fc4cc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 -- 172.21.15.116:0/462881894 >> v2:172.21.15.149:3300/0 conn(0x7fc4cc0d1670 msgr2=0x7fc4cc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 --2- 172.21.15.116:0/462881894 >> v2:172.21.15.149:3300/0 conn(0x7fc4cc0d1670 0x7fc4cc0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 -- 172.21.15.116:0/462881894 >> v1:172.21.15.149:6789/0 conn(0x7fc4cc161600 legacy=0x7fc4cc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.503+0000 7fc4d0c00640 1 -- 172.21.15.116:0/462881894 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0d1fc0 con 0x7fc4cc15db30 2024-05-14T11:19:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d0c00640 1 --2- 172.21.15.116:0/462881894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0d2370 tx=0x7fc4c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69afb3c275c83260 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4cb400640 1 -- 172.21.15.116:0/462881894 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0070 con 0x7fc4cc15db30 2024-05-14T11:19:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4cb400640 1 -- 172.21.15.116:0/462881894 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00ab600 con 0x7fc4cc15db30 2024-05-14T11:19:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4cb400640 1 -- 172.21.15.116:0/462881894 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab9d0 con 0x7fc4cc15db30 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- 172.21.15.116:0/462881894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc15db30 msgr2=0x7fc4cc0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 --2- 172.21.15.116:0/462881894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0d2370 tx=0x7fc4c001c490 comp rx=0 tx=0).stop 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- 172.21.15.116:0/462881894 shutdown_connections 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 --2- 172.21.15.116:0/462881894 >> v2:172.21.15.149:3300/0 conn(0x7fc4cc0d1670 0x7fc4cc0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 --2- 172.21.15.116:0/462881894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc15db30 0x7fc4cc0aeaa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- 172.21.15.116:0/462881894 >> 172.21.15.116:0/462881894 conn(0x7fc4cc0b4620 msgr2=0x7fc4cc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- 172.21.15.116:0/462881894 shutdown_connections 2024-05-14T11:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- 172.21.15.116:0/462881894 wait complete. 2024-05-14T11:19:00.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 Processor -- start 2024-05-14T11:19:00.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.507+0000 7fc4d3600640 1 -- start start 2024-05-14T11:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4cc15db30 0x7fc4cc0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4cc161600 0x7fc4cc158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc15df40 con 0x7fc4cc0d1670 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc0aefe0 con 0x7fc4cc161600 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc16af80 con 0x7fc4cc15db30 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39818/0 (socket says 172.21.15.116:39818) 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 -- 172.21.15.116:0/3505687390 learned_addr learned my addr 172.21.15.116:0/3505687390 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d1600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4cc161600 0x7fc4cc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4cbe00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4cc15db30 0x7fc4cc0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 -- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4cc15db30 msgr2=0x7fc4cc0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4cc15db30 0x7fc4cc0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 -- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4cc161600 msgr2=0x7fc4cc158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4cc161600 0x7fc4cc158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 -- 172.21.15.116:0/3505687390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0b1d40 con 0x7fc4cc0d1670 2024-05-14T11:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d0c00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ba020 con 0x7fc4cc0d1670 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00b6420 con 0x7fc4cc0d1670 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc0b2020 con 0x7fc4cc0d1670 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0040 con 0x7fc4cc0d1670 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc0b25e0 con 0x7fc4cc0d1670 2024-05-14T11:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.511+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc06a9c0 con 0x7fc4cc0d1670 2024-05-14T11:19:00.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.515+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4c00bd020 con 0x7fc4cc0d1670 2024-05-14T11:19:00.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.515+0000 7fc4c8c00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 0x7fc4ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.515+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fc4c0129490 con 0x7fc4cc0d1670 2024-05-14T11:19:00.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.515+0000 7fc4cbe00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 0x7fc4ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.515+0000 7fc4cbe00640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 0x7fc4ac071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0b4b30 tx=0x7fc4b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.519+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc4c00c4030 con 0x7fc4cc0d1670 2024-05-14T11:19:00.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4cc157540 con 0x7fc4cc0d1670 2024-05-14T11:19:00.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4c8c00640 1 -- 172.21.15.116:0/3505687390 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16932 (secure 0 0 0) 0x7fc4cc0d1fc0 con 0x7fc4cc0d1670 2024-05-14T11:19:00.682 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:00.682 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:57.918127+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:18:54.005073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-14T11:18:55.275835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:19:00.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 msgr2=0x7fc4ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 0x7fc4ac071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0b4b30 tx=0x7fc4b8009190 comp rx=0 tx=0).stop 2024-05-14T11:19:00.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 msgr2=0x7fc4cc09a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.679+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001acd0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 shutdown_connections 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc4ac06f400 0x7fc4ac071910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4cc161600 0x7fc4cc158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4cc15db30 0x7fc4cc0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 --2- 172.21.15.116:0/3505687390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4cc0d1670 0x7fc4cc09a820 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 >> 172.21.15.116:0/3505687390 conn(0x7fc4cc0b4620 msgr2=0x7fc4cc073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 shutdown_connections 2024-05-14T11:19:00.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.683+0000 7fc4d3600640 1 -- 172.21.15.116:0/3505687390 wait complete. 2024-05-14T11:19:00.704 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:19:00.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:19:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.847+0000 7f4d4f400640 1 Processor -- start 2024-05-14T11:19:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.847+0000 7f4d4f400640 1 -- start start 2024-05-14T11:19:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 0x7f4d480d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d4806c700 0x7f4d4806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4f400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4806ff20 con 0x7f4d4815db30 2024-05-14T11:19:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4f400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4d480ae100 con 0x7f4d4806c700 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4d480ae870 con 0x7f4d480d1670 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d4806c700 0x7f4d4806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d4806c700 0x7f4d4806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39822/0 (socket says 172.21.15.116:39822) 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 -- 172.21.15.116:0/4235342864 learned_addr learned my addr 172.21.15.116:0/4235342864 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d3fe00640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 0x7f4d480d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 -- 172.21.15.116:0/4235342864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 msgr2=0x7f4d480d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 0x7f4d480d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 -- 172.21.15.116:0/4235342864 >> v1:172.21.15.149:6789/0 conn(0x7f4d4815db30 legacy=0x7f4d480aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d4d400640 1 -- 172.21.15.116:0/4235342864 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d480d1fc0 con 0x7f4d4806c700 2024-05-14T11:19:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.851+0000 7f4d3fe00640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 0x7f4d480d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4d400640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d4806cb10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009e50 tx=0x7f4d4001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b1e4b78f76aa6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d47e00640 1 -- 172.21.15.116:0/4235342864 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d400a5680 con 0x7f4d4806c700 2024-05-14T11:19:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d47e00640 1 -- 172.21.15.116:0/4235342864 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d400a5820 con 0x7f4d4806c700 2024-05-14T11:19:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d47e00640 1 -- 172.21.15.116:0/4235342864 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d400a5bf0 con 0x7f4d4806c700 2024-05-14T11:19:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- 172.21.15.116:0/4235342864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 msgr2=0x7f4d4806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d4806cb10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009e50 tx=0x7f4d4001c8d0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- 172.21.15.116:0/4235342864 shutdown_connections 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d4806cb10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 --2- 172.21.15.116:0/4235342864 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d480d1670 0x7f4d480d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- 172.21.15.116:0/4235342864 >> 172.21.15.116:0/4235342864 conn(0x7f4d480b4620 msgr2=0x7f4d480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- 172.21.15.116:0/4235342864 shutdown_connections 2024-05-14T11:19:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- 172.21.15.116:0/4235342864 wait complete. 2024-05-14T11:19:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 Processor -- start 2024-05-14T11:19:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.855+0000 7f4d4f400640 1 -- start start 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d480d1670 0x7f4d480cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d4815db30 0x7f4d4814a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4816a770 con 0x7f4d4806c700 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4815df40 con 0x7f4d480d1670 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4816a8f0 con 0x7f4d4815db30 2024-05-14T11:19:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39828/0 (socket says 172.21.15.116:39828) 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 -- 172.21.15.116:0/674980361 learned_addr learned my addr 172.21.15.116:0/674980361 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4d400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d4815db30 0x7f4d4814a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d3fe00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d480d1670 0x7f4d480cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 -- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d4815db30 msgr2=0x7f4d4814a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d4815db30 0x7f4d4814a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 -- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d480d1670 msgr2=0x7f4d480cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d480d1670 0x7f4d480cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 -- 172.21.15.116:0/674980361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48074a40 con 0x7f4d4806c700 2024-05-14T11:19:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4ca00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c3f0 tx=0x7f4d3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38017070 con 0x7f4d4806c700 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d380052d0 con 0x7f4d4806c700 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d48074480 con 0x7f4d4806c700 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003820 con 0x7f4d4806c700 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d48150140 con 0x7f4d4806c700 2024-05-14T11:19:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.859+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d48150e80 con 0x7f4d4806c700 2024-05-14T11:19:00.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.863+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4d3801a070 con 0x7f4d4806c700 2024-05-14T11:19:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.863+0000 7f4d45600640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 0x7f4d24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.863+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f4d3808e6d0 con 0x7f4d4806c700 2024-05-14T11:19:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.863+0000 7f4d3fe00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 0x7f4d24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.863+0000 7f4d3fe00640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 0x7f4d24071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d4815e4a0 tx=0x7f4d30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:00.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:00.867+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4d38005480 con 0x7f4d4806c700 2024-05-14T11:19:01.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.027+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d4815e4d0 con 0x7f4d2406f400 2024-05-14T11:19:01.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.035+0000 7f4d45600640 1 -- 172.21.15.116:0/674980361 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239899 (secure 0 0 0) 0x7f4d4815e4d0 con 0x7f4d2406f400 2024-05-14T11:19:01.040 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:01.041 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 msgr2=0x7f4d24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 0x7f4d24071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d4815e4a0 tx=0x7f4d30008040 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 msgr2=0x7f4d48150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c3f0 tx=0x7f4d3800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 shutdown_connections 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4d2406f400 0x7f4d24071910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d480d1670 0x7f4d480cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4d4815db30 0x7f4d4814a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 --2- 172.21.15.116:0/674980361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4806c700 0x7f4d48150940 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 >> 172.21.15.116:0/674980361 conn(0x7f4d480b4620 msgr2=0x7f4d481617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 shutdown_connections 2024-05-14T11:19:01.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.039+0000 7f4d4f400640 1 -- 172.21.15.116:0/674980361 wait complete. 2024-05-14T11:19:01.060 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-05-14T11:19:00.399636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235164,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739556,"statfs":{"total":773094113280,"available":772853305344,"internally_reserved":0,"allocated":6631424,"data_stored":3586155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498636"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903331+0000","last_change":"2024-05-14T11:18:54.903331+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903331+0000","last_undegraded":"2024-05-14T11:18:54.903331+0000","last_fullsized":"2024-05-14T11:18:54.903331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911599+0000","last_change":"2024-05-14T11:18:54.911599+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911599+0000","last_undegraded":"2024-05-14T11:18:54.911599+0000","last_fullsized":"2024-05-14T11:18:54.911599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913847+0000","last_change":"2024-05-14T11:18:54.913847+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913847+0000","last_undegraded":"2024-05-14T11:18:54.913847+0000","last_fullsized":"2024-05-14T11:18:54.913847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911506+0000","last_change":"2024-05-14T11:18:54.911506+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911506+0000","last_undegraded":"2024-05-14T11:18:54.911506+0000","last_fullsized":"2024-05-14T11:18:54.911506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908463+0000","last_change":"2024-05-14T11:18:54.908463+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908463+0000","last_undegraded":"2024-05-14T11:18:54.908463+0000","last_fullsized":"2024-05-14T11:18:54.908463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909689+0000","last_change":"2024-05-14T11:18:54.909689+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909689+0000","last_undegraded":"2024-05-14T11:18:54.909689+0000","last_fullsized":"2024-05-14T11:18:54.909689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909509+0000","last_change":"2024-05-14T11:18:54.909509+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909509+0000","last_undegraded":"2024-05-14T11:18:54.909509+0000","last_fullsized":"2024-05-14T11:18:54.909509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.904529+0000","last_change":"2024-05-14T11:18:54.904529+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.904529+0000","last_undegraded":"2024-05-14T11:18:54.904529+0000","last_fullsized":"2024-05-14T11:18:54.904529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906459+0000","last_change":"2024-05-14T11:18:54.906459+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906459+0000","last_undegraded":"2024-05-14T11:18:54.906459+0000","last_fullsized":"2024-05-14T11:18:54.906459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.907121+0000","last_change":"2024-05-14T11:18:54.907121+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.907121+0000","last_undegraded":"2024-05-14T11:18:54.907121+0000","last_fullsized":"2024-05-14T11:18:54.907121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903310+0000","last_change":"2024-05-14T11:18:54.903310+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903310+0000","last_undegraded":"2024-05-14T11:18:54.903310+0000","last_fullsized":"2024-05-14T11:18:54.903310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906957+0000","last_change":"2024-05-14T11:18:54.906957+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906957+0000","last_undegraded":"2024-05-14T11:18:54.906957+0000","last_fullsized":"2024-05-14T11:18:54.906957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916572+0000","last_change":"2024-05-14T11:18:54.916572+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916572+0000","last_undegraded":"2024-05-14T11:18:54.916572+0000","last_fullsized":"2024-05-14T11:18:54.916572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906397+0000","last_change":"2024-05-14T11:18:54.906397+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906397+0000","last_undegraded":"2024-05-14T11:18:54.906397+0000","last_fullsized":"2024-05-14T11:18:54.906397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916591+0000","last_change":"2024-05-14T11:18:54.916591+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916591+0000","last_undegraded":"2024-05-14T11:18:54.916591+0000","last_fullsized":"2024-05-14T11:18:54.916591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906439+0000","last_change":"2024-05-14T11:18:54.906439+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906439+0000","last_undegraded":"2024-05-14T11:18:54.906439+0000","last_fullsized":"2024-05-14T11:18:54.906439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906403+0000","last_change":"2024-05-14T11:18:54.906403+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906403+0000","last_undegraded":"2024-05-14T11:18:54.906403+0000","last_fullsized":"2024-05-14T11:18:54.906403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908536+0000","last_change":"2024-05-14T11:18:54.908536+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908536+0000","last_undegraded":"2024-05-14T11:18:54.908536+0000","last_fullsized":"2024-05-14T11:18:54.908536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911519+0000","last_change":"2024-05-14T11:18:54.911519+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911519+0000","last_undegraded":"2024-05-14T11:18:54.911519+0000","last_fullsized":"2024-05-14T11:18:54.911519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911393+0000","last_change":"2024-05-14T11:18:54.911393+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911393+0000","last_undegraded":"2024-05-14T11:18:54.911393+0000","last_fullsized":"2024-05-14T11:18:54.911393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916032+0000","last_change":"2024-05-14T11:18:54.916032+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916032+0000","last_undegraded":"2024-05-14T11:18:54.916032+0000","last_fullsized":"2024-05-14T11:18:54.916032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913339+0000","last_change":"2024-05-14T11:18:54.913339+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913339+0000","last_undegraded":"2024-05-14T11:18:54.913339+0000","last_fullsized":"2024-05-14T11:18:54.913339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902685+0000","last_change":"2024-05-14T11:18:54.902685+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902685+0000","last_undegraded":"2024-05-14T11:18:54.902685+0000","last_fullsized":"2024-05-14T11:18:54.902685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906488+0000","last_change":"2024-05-14T11:18:54.906488+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906488+0000","last_undegraded":"2024-05-14T11:18:54.906488+0000","last_fullsized":"2024-05-14T11:18:54.906488+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908508+0000","last_change":"2024-05-14T11:18:54.908508+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908508+0000","last_undegraded":"2024-05-14T11:18:54.908508+0000","last_fullsized":"2024-05-14T11:18:54.908508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908520+0000","last_change":"2024-05-14T11:18:54.908520+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908520+0000","last_undegraded":"2024-05-14T11:18:54.908520+0000","last_fullsized":"2024-05-14T11:18:54.908520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902676+0000","last_change":"2024-05-14T11:18:54.902676+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902676+0000","last_undegraded":"2024-05-14T11:18:54.902676+0000","last_fullsized":"2024-05-14T11:18:54.902676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908510+0000","last_change":"2024-05-14T11:18:54.908510+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908510+0000","last_undegraded":"2024-05-14T11:18:54.908510+0000","last_fullsized":"2024-05-14T11:18:54.908510+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":176,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-14T11:18:54.907696+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:18:54.907696+0000","last_peered":"2024-05-14T11:18:54.907696+0000","last_clean":"2024-05-14T11:18:54.907696+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:18:54.907696+0000","last_undegraded":"2024-05-14T11:18:54.907696+0000","last_fullsized":"2024-05-14T11:18:54.907696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":894926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":888717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:19:01.065 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:19:01.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 Processor -- start 2024-05-14T11:19:01.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 -- start start 2024-05-14T11:19:01.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f95f4164220 0x7f95f40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 0x7f95f406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f406cc40 0x7f95f406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f95f406d590 con 0x7f95f4164220 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f95f40d1fc0 con 0x7f95f40d1bb0 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95fb400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f95f41600a0 con 0x7f95f406cc40 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f95f4164220 0x7f95f40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f95f4164220 0x7f95f40d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39844/0 (socket says 172.21.15.116:39844) 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 -- 172.21.15.116:0/2994970585 learned_addr learned my addr 172.21.15.116:0/2994970585 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f9400640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f406cc40 0x7f95f406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f3e00640 1 --2- 172.21.15.116:0/2994970585 >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 0x7f95f406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 -- 172.21.15.116:0/2994970585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f406cc40 msgr2=0x7f95f406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f406cc40 0x7f95f406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 -- 172.21.15.116:0/2994970585 >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 msgr2=0x7f95f406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:01.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 --2- 172.21.15.116:0/2994970585 >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 0x7f95f406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f8a00640 1 -- 172.21.15.116:0/2994970585 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f406f5d0 con 0x7f95f4164220 2024-05-14T11:19:01.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.247+0000 7f95f3e00640 1 --2- 172.21.15.116:0/2994970585 >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 0x7f95f406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:01.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95f8a00640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f4164220 0x7f95f40d1670 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f95f4072830 tx=0x7f95e401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7387593f27ccc7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95f3400640 1 -- 172.21.15.116:0/2994970585 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ab620 con 0x7f95f4164220 2024-05-14T11:19:01.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95f3400640 1 -- 172.21.15.116:0/2994970585 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e40b0070 con 0x7f95f4164220 2024-05-14T11:19:01.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95f3400640 1 -- 172.21.15.116:0/2994970585 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ab9e0 con 0x7f95f4164220 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 -- 172.21.15.116:0/2994970585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f4164220 msgr2=0x7f95f40d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f4164220 0x7f95f40d1670 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f95f4072830 tx=0x7f95e401c460 comp rx=0 tx=0).stop 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 -- 172.21.15.116:0/2994970585 shutdown_connections 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f406cc40 0x7f95f406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 --2- 172.21.15.116:0/2994970585 >> v2:172.21.15.149:3300/0 conn(0x7f95f40d1bb0 0x7f95f406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 --2- 172.21.15.116:0/2994970585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f4164220 0x7f95f40d1670 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 -- 172.21.15.116:0/2994970585 >> 172.21.15.116:0/2994970585 conn(0x7f95f40b4620 msgr2=0x7f95f40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 -- 172.21.15.116:0/2994970585 shutdown_connections 2024-05-14T11:19:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.251+0000 7f95fb400640 1 -- 172.21.15.116:0/2994970585 wait complete. 2024-05-14T11:19:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 Processor -- start 2024-05-14T11:19:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 -- start start 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f406cc40 0x7f95f409a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f4164220 0x7f95f40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4073650 con 0x7f95f40d1bb0 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f95f406fed0 con 0x7f95f406cc40 2024-05-14T11:19:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95fb400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f95f4164630 con 0x7f95f4164220 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95f3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95f3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39854/0 (socket says 172.21.15.116:39854) 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95f3e00640 1 -- 172.21.15.116:0/1190425128 learned_addr learned my addr 172.21.15.116:0/1190425128 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95f9400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f4164220 0x7f95f40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.255+0000 7f95f8a00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f406cc40 0x7f95f409a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 -- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f4164220 msgr2=0x7f95f40af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f4164220 0x7f95f40af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 -- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f406cc40 msgr2=0x7f95f409a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f406cc40 0x7f95f409a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 -- 172.21.15.116:0/1190425128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f4153d70 con 0x7f95f40d1bb0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f3e00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f95e000d800 tx=0x7f95e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e000b6c0 con 0x7f95f40d1bb0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e0019070 con 0x7f95f40d1bb0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f4154050 con 0x7f95f40d1bb0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e000baa0 con 0x7f95f40d1bb0 2024-05-14T11:19:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f4154610 con 0x7f95f40d1bb0 2024-05-14T11:19:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f406a9c0 con 0x7f95f40d1bb0 2024-05-14T11:19:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95e000bc40 con 0x7f95f40d1bb0 2024-05-14T11:19:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 0x7f95d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.259+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f95e008f7b0 con 0x7f95f40d1bb0 2024-05-14T11:19:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.263+0000 7f95f8a00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 0x7f95d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.263+0000 7f95f8a00640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 0x7f95d4071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af000 tx=0x7f95e40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.267+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f95e005be20 con 0x7f95f40d1bb0 2024-05-14T11:19:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.419+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95f406ecf0 con 0x7f95d406f400 2024-05-14T11:19:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95f0c00640 1 -- 172.21.15.116:0/1190425128 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239899 (secure 0 0 0) 0x7f95f406ecf0 con 0x7f95d406f400 2024-05-14T11:19:01.431 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:01.432 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 msgr2=0x7f95d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 0x7f95d4071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af000 tx=0x7f95e40a7000 comp rx=0 tx=0).stop 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 msgr2=0x7f95f41530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f95e000d800 tx=0x7f95e000dcc0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 shutdown_connections 2024-05-14T11:19:01.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f95d406f400 0x7f95d4071910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95f4164220 0x7f95f40af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f40d1bb0 0x7f95f41530d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 --2- 172.21.15.116:0/1190425128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95f406cc40 0x7f95f409a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 >> 172.21.15.116:0/1190425128 conn(0x7f95f40b4620 msgr2=0x7f95f415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.427+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 shutdown_connections 2024-05-14T11:19:01.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.431+0000 7f95fb400640 1 -- 172.21.15.116:0/1190425128 wait complete. 2024-05-14T11:19:01.452 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-05-14T11:19:00.399636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235164,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739556,"statfs":{"total":773094113280,"available":772853305344,"internally_reserved":0,"allocated":6631424,"data_stored":3586155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498636"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903331+0000","last_change":"2024-05-14T11:18:54.903331+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903331+0000","last_undegraded":"2024-05-14T11:18:54.903331+0000","last_fullsized":"2024-05-14T11:18:54.903331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911599+0000","last_change":"2024-05-14T11:18:54.911599+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911599+0000","last_undegraded":"2024-05-14T11:18:54.911599+0000","last_fullsized":"2024-05-14T11:18:54.911599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913847+0000","last_change":"2024-05-14T11:18:54.913847+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913847+0000","last_undegraded":"2024-05-14T11:18:54.913847+0000","last_fullsized":"2024-05-14T11:18:54.913847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911506+0000","last_change":"2024-05-14T11:18:54.911506+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911506+0000","last_undegraded":"2024-05-14T11:18:54.911506+0000","last_fullsized":"2024-05-14T11:18:54.911506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908463+0000","last_change":"2024-05-14T11:18:54.908463+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908463+0000","last_undegraded":"2024-05-14T11:18:54.908463+0000","last_fullsized":"2024-05-14T11:18:54.908463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909689+0000","last_change":"2024-05-14T11:18:54.909689+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909689+0000","last_undegraded":"2024-05-14T11:18:54.909689+0000","last_fullsized":"2024-05-14T11:18:54.909689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909509+0000","last_change":"2024-05-14T11:18:54.909509+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909509+0000","last_undegraded":"2024-05-14T11:18:54.909509+0000","last_fullsized":"2024-05-14T11:18:54.909509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.904529+0000","last_change":"2024-05-14T11:18:54.904529+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.904529+0000","last_undegraded":"2024-05-14T11:18:54.904529+0000","last_fullsized":"2024-05-14T11:18:54.904529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906459+0000","last_change":"2024-05-14T11:18:54.906459+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906459+0000","last_undegraded":"2024-05-14T11:18:54.906459+0000","last_fullsized":"2024-05-14T11:18:54.906459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.907121+0000","last_change":"2024-05-14T11:18:54.907121+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.907121+0000","last_undegraded":"2024-05-14T11:18:54.907121+0000","last_fullsized":"2024-05-14T11:18:54.907121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903310+0000","last_change":"2024-05-14T11:18:54.903310+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903310+0000","last_undegraded":"2024-05-14T11:18:54.903310+0000","last_fullsized":"2024-05-14T11:18:54.903310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906957+0000","last_change":"2024-05-14T11:18:54.906957+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906957+0000","last_undegraded":"2024-05-14T11:18:54.906957+0000","last_fullsized":"2024-05-14T11:18:54.906957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916572+0000","last_change":"2024-05-14T11:18:54.916572+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916572+0000","last_undegraded":"2024-05-14T11:18:54.916572+0000","last_fullsized":"2024-05-14T11:18:54.916572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906397+0000","last_change":"2024-05-14T11:18:54.906397+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906397+0000","last_undegraded":"2024-05-14T11:18:54.906397+0000","last_fullsized":"2024-05-14T11:18:54.906397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916591+0000","last_change":"2024-05-14T11:18:54.916591+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916591+0000","last_undegraded":"2024-05-14T11:18:54.916591+0000","last_fullsized":"2024-05-14T11:18:54.916591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906439+0000","last_change":"2024-05-14T11:18:54.906439+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906439+0000","last_undegraded":"2024-05-14T11:18:54.906439+0000","last_fullsized":"2024-05-14T11:18:54.906439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906403+0000","last_change":"2024-05-14T11:18:54.906403+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906403+0000","last_undegraded":"2024-05-14T11:18:54.906403+0000","last_fullsized":"2024-05-14T11:18:54.906403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908536+0000","last_change":"2024-05-14T11:18:54.908536+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908536+0000","last_undegraded":"2024-05-14T11:18:54.908536+0000","last_fullsized":"2024-05-14T11:18:54.908536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911519+0000","last_change":"2024-05-14T11:18:54.911519+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911519+0000","last_undegraded":"2024-05-14T11:18:54.911519+0000","last_fullsized":"2024-05-14T11:18:54.911519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911393+0000","last_change":"2024-05-14T11:18:54.911393+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911393+0000","last_undegraded":"2024-05-14T11:18:54.911393+0000","last_fullsized":"2024-05-14T11:18:54.911393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916032+0000","last_change":"2024-05-14T11:18:54.916032+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916032+0000","last_undegraded":"2024-05-14T11:18:54.916032+0000","last_fullsized":"2024-05-14T11:18:54.916032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913339+0000","last_change":"2024-05-14T11:18:54.913339+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913339+0000","last_undegraded":"2024-05-14T11:18:54.913339+0000","last_fullsized":"2024-05-14T11:18:54.913339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902685+0000","last_change":"2024-05-14T11:18:54.902685+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902685+0000","last_undegraded":"2024-05-14T11:18:54.902685+0000","last_fullsized":"2024-05-14T11:18:54.902685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906488+0000","last_change":"2024-05-14T11:18:54.906488+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906488+0000","last_undegraded":"2024-05-14T11:18:54.906488+0000","last_fullsized":"2024-05-14T11:18:54.906488+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908508+0000","last_change":"2024-05-14T11:18:54.908508+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908508+0000","last_undegraded":"2024-05-14T11:18:54.908508+0000","last_fullsized":"2024-05-14T11:18:54.908508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908520+0000","last_change":"2024-05-14T11:18:54.908520+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908520+0000","last_undegraded":"2024-05-14T11:18:54.908520+0000","last_fullsized":"2024-05-14T11:18:54.908520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902676+0000","last_change":"2024-05-14T11:18:54.902676+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902676+0000","last_undegraded":"2024-05-14T11:18:54.902676+0000","last_fullsized":"2024-05-14T11:18:54.902676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908510+0000","last_change":"2024-05-14T11:18:54.908510+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908510+0000","last_undegraded":"2024-05-14T11:18:54.908510+0000","last_fullsized":"2024-05-14T11:18:54.908510+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":176,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-14T11:18:54.907696+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:18:54.907696+0000","last_peered":"2024-05-14T11:18:54.907696+0000","last_clean":"2024-05-14T11:18:54.907696+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:18:54.907696+0000","last_undegraded":"2024-05-14T11:18:54.907696+0000","last_fullsized":"2024-05-14T11:18:54.907696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":894926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":888717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:19:01.456 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:19:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.639+0000 7fe9eba00640 1 Processor -- start 2024-05-14T11:19:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.639+0000 7fe9eba00640 1 -- start start 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9eba00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe9e4156fa0 0x7fe9e406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9eba00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9e40af380 0x7fe9e40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9eba00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e4160510 con 0x7fe9e40d1670 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9eba00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe9e41573b0 con 0x7fe9e40af380 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9eba00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe9e4161120 con 0x7fe9e4156fa0 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9e40af380 0x7fe9e40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9e40af380 0x7fe9e40af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39866/0 (socket says 172.21.15.116:39866) 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 -- 172.21.15.116:0/54776047 learned_addr learned my addr 172.21.15.116:0/54776047 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9000640 1 --2- 172.21.15.116:0/54776047 >> v2:172.21.15.149:3300/0 conn(0x7fe9e4156fa0 0x7fe9e406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 -- 172.21.15.116:0/54776047 >> v2:172.21.15.149:3300/0 conn(0x7fe9e4156fa0 msgr2=0x7fe9e406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 --2- 172.21.15.116:0/54776047 >> v2:172.21.15.149:3300/0 conn(0x7fe9e4156fa0 0x7fe9e406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 -- 172.21.15.116:0/54776047 >> v1:172.21.15.149:6789/0 conn(0x7fe9e40d1670 legacy=0x7fe9e40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.643+0000 7fe9e9a00640 1 -- 172.21.15.116:0/54776047 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e41607f0 con 0x7fe9e40af380 2024-05-14T11:19:01.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9e9a00640 1 --2- 172.21.15.116:0/54776047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40af380 0x7fe9e40af790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4071f20 tx=0x7fe9dc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f00343ab41110 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9e3e00640 1 -- 172.21.15.116:0/54776047 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0a5880 con 0x7fe9e40af380 2024-05-14T11:19:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9e3e00640 1 -- 172.21.15.116:0/54776047 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9dc0a5a20 con 0x7fe9e40af380 2024-05-14T11:19:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9e3e00640 1 -- 172.21.15.116:0/54776047 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0a5df0 con 0x7fe9e40af380 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 -- 172.21.15.116:0/54776047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40af380 msgr2=0x7fe9e40af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 --2- 172.21.15.116:0/54776047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40af380 0x7fe9e40af790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4071f20 tx=0x7fe9dc01c9c0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 -- 172.21.15.116:0/54776047 shutdown_connections 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 --2- 172.21.15.116:0/54776047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40af380 0x7fe9e40af790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 --2- 172.21.15.116:0/54776047 >> v2:172.21.15.149:3300/0 conn(0x7fe9e4156fa0 0x7fe9e406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 -- 172.21.15.116:0/54776047 >> 172.21.15.116:0/54776047 conn(0x7fe9e40b4620 msgr2=0x7fe9e40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 -- 172.21.15.116:0/54776047 shutdown_connections 2024-05-14T11:19:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 -- 172.21.15.116:0/54776047 wait complete. 2024-05-14T11:19:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.647+0000 7fe9eba00640 1 Processor -- start 2024-05-14T11:19:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- start start 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 0x7fe9e4148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9e4156fa0 0x7fe9e416a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e40b0660 con 0x7fe9e40d1670 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e40b04e0 con 0x7fe9e4156fa0 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e4167800 con 0x7fe9e40af380 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e9000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 0x7fe9e4148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39882/0 (socket says 172.21.15.116:39882) 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 -- 172.21.15.116:0/3954714370 learned_addr learned my addr 172.21.15.116:0/3954714370 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e9a00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9e4156fa0 0x7fe9e416a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 -- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 msgr2=0x7fe9e4148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 0x7fe9e4148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 -- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9e4156fa0 msgr2=0x7fe9e416a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9e4156fa0 0x7fe9e416a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 -- 172.21.15.116:0/3954714370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e40b1430 con 0x7fe9e40d1670 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e9000640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 0x7fe9e4148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9dbe00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c3f0 tx=0x7fe9cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc017070 con 0x7fe9e40d1670 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc0052d0 con 0x7fe9e40d1670 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e4152490 con 0x7fe9e40d1670 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc003820 con 0x7fe9e40d1670 2024-05-14T11:19:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e4152a10 con 0x7fe9e40d1670 2024-05-14T11:19:01.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.651+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e406a9c0 con 0x7fe9e40d1670 2024-05-14T11:19:01.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.655+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9cc01a070 con 0x7fe9e40d1670 2024-05-14T11:19:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.655+0000 7fe9e1600640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 0x7fe9c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.655+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fe9cc08e730 con 0x7fe9e40d1670 2024-05-14T11:19:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.655+0000 7fe9e9000640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 0x7fe9c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:01.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.655+0000 7fe9e9000640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 0x7fe9c0071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40b0d60 tx=0x7fe9d4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:01.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.659+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe9e4152490 con 0x7fe9e40d1670 2024-05-14T11:19:01.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.815+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9e40b4e80 con 0x7fe9c006f430 2024-05-14T11:19:01.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.819+0000 7fe9e1600640 1 -- 172.21.15.116:0/3954714370 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239899 (secure 0 0 0) 0x7fe9e40b4e80 con 0x7fe9c006f430 2024-05-14T11:19:01.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:01.827 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:19:01.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 msgr2=0x7fe9c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 0x7fe9c0071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40b0d60 tx=0x7fe9d4009190 comp rx=0 tx=0).stop 2024-05-14T11:19:01.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 msgr2=0x7fe9e4169d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:01.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c3f0 tx=0x7fe9cc00c8b0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 shutdown_connections 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9c006f430 0x7fe9c0071940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9e4156fa0 0x7fe9e416a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9e40d1670 0x7fe9e4169d70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 --2- 172.21.15.116:0/3954714370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9e40af380 0x7fe9e4148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 >> 172.21.15.116:0/3954714370 conn(0x7fe9e40b4620 msgr2=0x7fe9e40b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 shutdown_connections 2024-05-14T11:19:01.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:01.823+0000 7fe9eba00640 1 -- 172.21.15.116:0/3954714370 wait complete. 2024-05-14T11:19:01.847 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-05-14T11:19:00.399636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235164,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739556,"statfs":{"total":773094113280,"available":772853305344,"internally_reserved":0,"allocated":6631424,"data_stored":3586155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.498636"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903331+0000","last_change":"2024-05-14T11:18:54.903331+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903331+0000","last_undegraded":"2024-05-14T11:18:54.903331+0000","last_fullsized":"2024-05-14T11:18:54.903331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911599+0000","last_change":"2024-05-14T11:18:54.911599+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911599+0000","last_undegraded":"2024-05-14T11:18:54.911599+0000","last_fullsized":"2024-05-14T11:18:54.911599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913847+0000","last_change":"2024-05-14T11:18:54.913847+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913847+0000","last_undegraded":"2024-05-14T11:18:54.913847+0000","last_fullsized":"2024-05-14T11:18:54.913847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911506+0000","last_change":"2024-05-14T11:18:54.911506+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911506+0000","last_undegraded":"2024-05-14T11:18:54.911506+0000","last_fullsized":"2024-05-14T11:18:54.911506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908463+0000","last_change":"2024-05-14T11:18:54.908463+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908463+0000","last_undegraded":"2024-05-14T11:18:54.908463+0000","last_fullsized":"2024-05-14T11:18:54.908463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909689+0000","last_change":"2024-05-14T11:18:54.909689+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909689+0000","last_undegraded":"2024-05-14T11:18:54.909689+0000","last_fullsized":"2024-05-14T11:18:54.909689+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.909509+0000","last_change":"2024-05-14T11:18:54.909509+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.909509+0000","last_undegraded":"2024-05-14T11:18:54.909509+0000","last_fullsized":"2024-05-14T11:18:54.909509+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.904529+0000","last_change":"2024-05-14T11:18:54.904529+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.904529+0000","last_undegraded":"2024-05-14T11:18:54.904529+0000","last_fullsized":"2024-05-14T11:18:54.904529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906459+0000","last_change":"2024-05-14T11:18:54.906459+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906459+0000","last_undegraded":"2024-05-14T11:18:54.906459+0000","last_fullsized":"2024-05-14T11:18:54.906459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.907121+0000","last_change":"2024-05-14T11:18:54.907121+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.907121+0000","last_undegraded":"2024-05-14T11:18:54.907121+0000","last_fullsized":"2024-05-14T11:18:54.907121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903310+0000","last_change":"2024-05-14T11:18:54.903310+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903310+0000","last_undegraded":"2024-05-14T11:18:54.903310+0000","last_fullsized":"2024-05-14T11:18:54.903310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906957+0000","last_change":"2024-05-14T11:18:54.906957+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906957+0000","last_undegraded":"2024-05-14T11:18:54.906957+0000","last_fullsized":"2024-05-14T11:18:54.906957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916572+0000","last_change":"2024-05-14T11:18:54.916572+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916572+0000","last_undegraded":"2024-05-14T11:18:54.916572+0000","last_fullsized":"2024-05-14T11:18:54.916572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906397+0000","last_change":"2024-05-14T11:18:54.906397+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906397+0000","last_undegraded":"2024-05-14T11:18:54.906397+0000","last_fullsized":"2024-05-14T11:18:54.906397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.903741+0000","last_change":"2024-05-14T11:18:54.903741+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.903741+0000","last_undegraded":"2024-05-14T11:18:54.903741+0000","last_fullsized":"2024-05-14T11:18:54.903741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916591+0000","last_change":"2024-05-14T11:18:54.916591+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916591+0000","last_undegraded":"2024-05-14T11:18:54.916591+0000","last_fullsized":"2024-05-14T11:18:54.916591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906439+0000","last_change":"2024-05-14T11:18:54.906439+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906439+0000","last_undegraded":"2024-05-14T11:18:54.906439+0000","last_fullsized":"2024-05-14T11:18:54.906439+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906403+0000","last_change":"2024-05-14T11:18:54.906403+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906403+0000","last_undegraded":"2024-05-14T11:18:54.906403+0000","last_fullsized":"2024-05-14T11:18:54.906403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908536+0000","last_change":"2024-05-14T11:18:54.908536+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908536+0000","last_undegraded":"2024-05-14T11:18:54.908536+0000","last_fullsized":"2024-05-14T11:18:54.908536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911519+0000","last_change":"2024-05-14T11:18:54.911519+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911519+0000","last_undegraded":"2024-05-14T11:18:54.911519+0000","last_fullsized":"2024-05-14T11:18:54.911519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.911393+0000","last_change":"2024-05-14T11:18:54.911393+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.911393+0000","last_undegraded":"2024-05-14T11:18:54.911393+0000","last_fullsized":"2024-05-14T11:18:54.911393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916032+0000","last_change":"2024-05-14T11:18:54.916032+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916032+0000","last_undegraded":"2024-05-14T11:18:54.916032+0000","last_fullsized":"2024-05-14T11:18:54.916032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.916012+0000","last_change":"2024-05-14T11:18:54.916012+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.916012+0000","last_undegraded":"2024-05-14T11:18:54.916012+0000","last_fullsized":"2024-05-14T11:18:54.916012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.913339+0000","last_change":"2024-05-14T11:18:54.913339+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.913339+0000","last_undegraded":"2024-05-14T11:18:54.913339+0000","last_fullsized":"2024-05-14T11:18:54.913339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902685+0000","last_change":"2024-05-14T11:18:54.902685+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902685+0000","last_undegraded":"2024-05-14T11:18:54.902685+0000","last_fullsized":"2024-05-14T11:18:54.902685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.906488+0000","last_change":"2024-05-14T11:18:54.906488+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.906488+0000","last_undegraded":"2024-05-14T11:18:54.906488+0000","last_fullsized":"2024-05-14T11:18:54.906488+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908508+0000","last_change":"2024-05-14T11:18:54.908508+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908508+0000","last_undegraded":"2024-05-14T11:18:54.908508+0000","last_fullsized":"2024-05-14T11:18:54.908508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908520+0000","last_change":"2024-05-14T11:18:54.908520+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908520+0000","last_undegraded":"2024-05-14T11:18:54.908520+0000","last_fullsized":"2024-05-14T11:18:54.908520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.902676+0000","last_change":"2024-05-14T11:18:54.902676+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.902676+0000","last_undegraded":"2024-05-14T11:18:54.902676+0000","last_fullsized":"2024-05-14T11:18:54.902676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-14T11:18:55.898180+0000","last_change":"2024-05-14T11:18:55.898180+0000","last_active":"2024-05-14T11:18:55.898180+0000","last_peered":"2024-05-14T11:18:55.898180+0000","last_clean":"2024-05-14T11:18:55.898180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:55.898180+0000","last_undegraded":"2024-05-14T11:18:55.898180+0000","last_fullsized":"2024-05-14T11:18:55.898180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-05-14T11:18:54.908510+0000","last_change":"2024-05-14T11:18:54.908510+0000","last_active":"2024-05-14T11:18:54.890228+0000","last_peered":"2024-05-14T11:18:54.890228+0000","last_clean":"2024-05-14T11:18:54.890228+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:18:54.908510+0000","last_undegraded":"2024-05-14T11:18:54.908510+0000","last_fullsized":"2024-05-14T11:18:54.908510+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":176,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-14T11:18:54.907696+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:18:54.907696+0000","last_peered":"2024-05-14T11:18:54.907696+0000","last_clean":"2024-05-14T11:18:54.907696+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:18:54.907696+0000","last_undegraded":"2024-05-14T11:18:54.907696+0000","last_fullsized":"2024-05-14T11:18:54.907696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":894926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":304558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":298349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":888717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3f is not active+clean 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3d is not active+clean 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3c is not active+clean 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3b is not active+clean 2024-05-14T11:19:01.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:PG 10.39 is not active+clean 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:PG 10.38 is not active+clean 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:PG 10.37 is not active+clean 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-05-14T11:19:01.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.35 is not active+clean 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.34 is not active+clean 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.33 is not active+clean 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.32 is not active+clean 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.853 INFO:tasks.ceph.ceph_manager.ceph:PG 10.30 is not active+clean 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2f is not active+clean 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2e is not active+clean 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2d is not active+clean 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2c is not active+clean 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.854 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2b is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:PG 10.29 is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:PG 10.28 is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:PG 10.27 is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:PG 10.26 is not active+clean 2024-05-14T11:19:01.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.25 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.24 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.23 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.22 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.21 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.20 is not active+clean 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.856 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1c is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.903331+0000', 'last_change': '2024-05-14T11:18:54.903331+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.903331+0000', 'last_undegraded': '2024-05-14T11:18:54.903331+0000', 'last_fullsized': '2024-05-14T11:18:54.903331+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.911599+0000', 'last_change': '2024-05-14T11:18:54.911599+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.911599+0000', 'last_undegraded': '2024-05-14T11:18:54.911599+0000', 'last_fullsized': '2024-05-14T11:18:54.911599+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-05-14T11:19:01.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.913847+0000', 'last_change': '2024-05-14T11:18:54.913847+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.913847+0000', 'last_undegraded': '2024-05-14T11:18:54.913847+0000', 'last_fullsized': '2024-05-14T11:18:54.913847+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1f is not active+clean 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.911506+0000', 'last_change': '2024-05-14T11:18:54.911506+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.911506+0000', 'last_undegraded': '2024-05-14T11:18:54.911506+0000', 'last_fullsized': '2024-05-14T11:18:54.911506+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 10.18 is not active+clean 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.908463+0000', 'last_change': '2024-05-14T11:18:54.908463+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.908463+0000', 'last_undegraded': '2024-05-14T11:18:54.908463+0000', 'last_fullsized': '2024-05-14T11:18:54.908463+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.909689+0000', 'last_change': '2024-05-14T11:18:54.909689+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.909689+0000', 'last_undegraded': '2024-05-14T11:18:54.909689+0000', 'last_fullsized': '2024-05-14T11:18:54.909689+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.909509+0000', 'last_change': '2024-05-14T11:18:54.909509+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.909509+0000', 'last_undegraded': '2024-05-14T11:18:54.909509+0000', 'last_fullsized': '2024-05-14T11:18:54.909509+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.904529+0000', 'last_change': '2024-05-14T11:18:54.904529+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.904529+0000', 'last_undegraded': '2024-05-14T11:18:54.904529+0000', 'last_fullsized': '2024-05-14T11:18:54.904529+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-05-14T11:19:01.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906459+0000', 'last_change': '2024-05-14T11:18:54.906459+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906459+0000', 'last_undegraded': '2024-05-14T11:18:54.906459+0000', 'last_fullsized': '2024-05-14T11:18:54.906459+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.903741+0000', 'last_change': '2024-05-14T11:18:54.903741+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.903741+0000', 'last_undegraded': '2024-05-14T11:18:54.903741+0000', 'last_fullsized': '2024-05-14T11:18:54.903741+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 10.16 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.907121+0000', 'last_change': '2024-05-14T11:18:54.907121+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.907121+0000', 'last_undegraded': '2024-05-14T11:18:54.907121+0000', 'last_fullsized': '2024-05-14T11:18:54.907121+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-05-14T11:19:01.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.903310+0000', 'last_change': '2024-05-14T11:18:54.903310+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.903310+0000', 'last_undegraded': '2024-05-14T11:18:54.903310+0000', 'last_fullsized': '2024-05-14T11:18:54.903310+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906957+0000', 'last_change': '2024-05-14T11:18:54.906957+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906957+0000', 'last_undegraded': '2024-05-14T11:18:54.906957+0000', 'last_fullsized': '2024-05-14T11:18:54.906957+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.916572+0000', 'last_change': '2024-05-14T11:18:54.916572+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.916572+0000', 'last_undegraded': '2024-05-14T11:18:54.916572+0000', 'last_fullsized': '2024-05-14T11:18:54.916572+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-05-14T11:19:01.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906397+0000', 'last_change': '2024-05-14T11:18:54.906397+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906397+0000', 'last_undegraded': '2024-05-14T11:18:54.906397+0000', 'last_fullsized': '2024-05-14T11:18:54.906397+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.903741+0000', 'last_change': '2024-05-14T11:18:54.903741+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.903741+0000', 'last_undegraded': '2024-05-14T11:18:54.903741+0000', 'last_fullsized': '2024-05-14T11:18:54.903741+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 10.c is not active+clean 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.916591+0000', 'last_change': '2024-05-14T11:18:54.916591+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.916591+0000', 'last_undegraded': '2024-05-14T11:18:54.916591+0000', 'last_fullsized': '2024-05-14T11:18:54.916591+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 10.d is not active+clean 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.862 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906439+0000', 'last_change': '2024-05-14T11:18:54.906439+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906439+0000', 'last_undegraded': '2024-05-14T11:18:54.906439+0000', 'last_fullsized': '2024-05-14T11:18:54.906439+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906403+0000', 'last_change': '2024-05-14T11:18:54.906403+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906403+0000', 'last_undegraded': '2024-05-14T11:18:54.906403+0000', 'last_fullsized': '2024-05-14T11:18:54.906403+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 10.f is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.908536+0000', 'last_change': '2024-05-14T11:18:54.908536+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.908536+0000', 'last_undegraded': '2024-05-14T11:18:54.908536+0000', 'last_fullsized': '2024-05-14T11:18:54.908536+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-05-14T11:19:01.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.911519+0000', 'last_change': '2024-05-14T11:18:54.911519+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.911519+0000', 'last_undegraded': '2024-05-14T11:18:54.911519+0000', 'last_fullsized': '2024-05-14T11:18:54.911519+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.916012+0000', 'last_change': '2024-05-14T11:18:54.916012+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.916012+0000', 'last_undegraded': '2024-05-14T11:18:54.916012+0000', 'last_fullsized': '2024-05-14T11:18:54.916012+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 10.a is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.911393+0000', 'last_change': '2024-05-14T11:18:54.911393+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.911393+0000', 'last_undegraded': '2024-05-14T11:18:54.911393+0000', 'last_fullsized': '2024-05-14T11:18:54.911393+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-05-14T11:19:01.864 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.916032+0000', 'last_change': '2024-05-14T11:18:54.916032+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.916032+0000', 'last_undegraded': '2024-05-14T11:18:54.916032+0000', 'last_fullsized': '2024-05-14T11:18:54.916032+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.916012+0000', 'last_change': '2024-05-14T11:18:54.916012+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.916012+0000', 'last_undegraded': '2024-05-14T11:18:54.916012+0000', 'last_fullsized': '2024-05-14T11:18:54.916012+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.913339+0000', 'last_change': '2024-05-14T11:18:54.913339+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.913339+0000', 'last_undegraded': '2024-05-14T11:18:54.913339+0000', 'last_fullsized': '2024-05-14T11:18:54.913339+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 10.7 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-05-14T11:19:01.865 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.902685+0000', 'last_change': '2024-05-14T11:18:54.902685+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.902685+0000', 'last_undegraded': '2024-05-14T11:18:54.902685+0000', 'last_fullsized': '2024-05-14T11:18:54.902685+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 10.0 is not active+clean 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.906488+0000', 'last_change': '2024-05-14T11:18:54.906488+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.906488+0000', 'last_undegraded': '2024-05-14T11:18:54.906488+0000', 'last_fullsized': '2024-05-14T11:18:54.906488+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.908508+0000', 'last_change': '2024-05-14T11:18:54.908508+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.908508+0000', 'last_undegraded': '2024-05-14T11:18:54.908508+0000', 'last_fullsized': '2024-05-14T11:18:54.908508+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.866 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.908520+0000', 'last_change': '2024-05-14T11:18:54.908520+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.908520+0000', 'last_undegraded': '2024-05-14T11:18:54.908520+0000', 'last_fullsized': '2024-05-14T11:18:54.908520+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.902676+0000', 'last_change': '2024-05-14T11:18:54.902676+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.902676+0000', 'last_undegraded': '2024-05-14T11:18:54.902676+0000', 'last_fullsized': '2024-05-14T11:18:54.902676+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-14T11:18:55.898180+0000', 'last_change': '2024-05-14T11:18:55.898180+0000', 'last_active': '2024-05-14T11:18:55.898180+0000', 'last_peered': '2024-05-14T11:18:55.898180+0000', 'last_clean': '2024-05-14T11:18:55.898180+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:55.898180+0000', 'last_undegraded': '2024-05-14T11:18:55.898180+0000', 'last_fullsized': '2024-05-14T11:18:55.898180+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:55.898180+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-05-14T11:19:01.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 57, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:18:54.908510+0000', 'last_change': '2024-05-14T11:18:54.908510+0000', 'last_active': '2024-05-14T11:18:54.890228+0000', 'last_peered': '2024-05-14T11:18:54.890228+0000', 'last_clean': '2024-05-14T11:18:54.890228+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:18:54.908510+0000', 'last_undegraded': '2024-05-14T11:18:54.908510+0000', 'last_fullsized': '2024-05-14T11:18:54.908510+0000', 'mapping_epoch': 57, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 57, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'last_clean_scrub_stamp': '2024-05-14T11:18:54.890228+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-14T11:19:01.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-14T11:19:02.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 Processor -- start 2024-05-14T11:19:02.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 -- start start 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f232415fe00 0x7f23240d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23240cfeb0 con 0x7f23240b70a0 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23240cf110 con 0x7f23240b9160 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.035+0000 7f232b400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f23240b6d80 con 0x7f232415fe00 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2329400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f232415fe00 0x7f23240d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2328a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f23240b70a0 0x7f23240b8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58998/0 (socket says 172.21.15.116:58998) 2024-05-14T11:19:02.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2328a00640 1 -- 172.21.15.116:0/1946088939 learned_addr learned my addr 172.21.15.116:0/1946088939 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:02.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2329400640 1 -- 172.21.15.116:0/1946088939 >> v1:172.21.15.149:6789/0 conn(0x7f23240b9160 legacy=0x7f23240d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2329400640 1 -- 172.21.15.116:0/1946088939 >> v1:172.21.15.116:6789/0 conn(0x7f23240b70a0 legacy=0x7f23240b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2329400640 1 -- 172.21.15.116:0/1946088939 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23240cf3d0 con 0x7f232415fe00 2024-05-14T11:19:02.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2323400640 1 -- 172.21.15.116:0/1946088939 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993141716 0 0) 0x7f23240cfeb0 con 0x7f23240b70a0 2024-05-14T11:19:02.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2329400640 1 --2- 172.21.15.116:0/1946088939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f23240d1a20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f231c009d90 tx=0x7f231c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb748d0353dd6194 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2323400640 1 -- 172.21.15.116:0/1946088939 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0b2070 con 0x7f232415fe00 2024-05-14T11:19:02.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2323400640 1 -- 172.21.15.116:0/1946088939 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c0a5830 con 0x7f232415fe00 2024-05-14T11:19:02.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.039+0000 7f2323400640 1 -- 172.21.15.116:0/1946088939 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0a5c00 con 0x7f232415fe00 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- 172.21.15.116:0/1946088939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 msgr2=0x7f23240d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 --2- 172.21.15.116:0/1946088939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f23240d1a20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f231c009d90 tx=0x7f231c01c970 comp rx=0 tx=0).stop 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- 172.21.15.116:0/1946088939 shutdown_connections 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 --2- 172.21.15.116:0/1946088939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f23240d1a20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- 172.21.15.116:0/1946088939 >> 172.21.15.116:0/1946088939 conn(0x7f23240d3740 msgr2=0x7f23240d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- 172.21.15.116:0/1946088939 shutdown_connections 2024-05-14T11:19:02.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- 172.21.15.116:0/1946088939 wait complete. 2024-05-14T11:19:02.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 Processor -- start 2024-05-14T11:19:02.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.043+0000 7f232b400640 1 -- start start 2024-05-14T11:19:02.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 0x7f23240c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23240b9160 0x7f232414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23240b6b40 con 0x7f232415fe00 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23240b69c0 con 0x7f23240b9160 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f232416e640 con 0x7f23240b70a0 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39886/0 (socket says 172.21.15.116:39886) 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 -- 172.21.15.116:0/2710284825 learned_addr learned my addr 172.21.15.116:0/2710284825 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2323e00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23240b9160 0x7f232414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2328a00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 0x7f23240c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 -- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 msgr2=0x7f23240c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 0x7f23240c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 -- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23240b9160 msgr2=0x7f232414e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23240b9160 0x7f232414e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 -- 172.21.15.116:0/2710284825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232414ed80 con 0x7f232415fe00 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2328a00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 0x7f23240c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:02.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2329400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f231c0b0040 tx=0x7f231c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0b2050 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c0be070 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0adc00 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2324155570 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2324155b30 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.047+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232406a9c0 con 0x7f232415fe00 2024-05-14T11:19:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.051+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f231c0c3460 con 0x7f232415fe00 2024-05-14T11:19:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.051+0000 7f2320c00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 0x7f2300071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.051+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f231c12a820 con 0x7f232415fe00 2024-05-14T11:19:02.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.051+0000 7f2328a00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 0x7f2300071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.051+0000 7f2328a00640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 0x7f2300071910 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f230c000f70 tx=0x7f230c007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.055+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f231c016560 con 0x7f232415fe00 2024-05-14T11:19:02.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.267+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2324075f50 con 0x7f232415fe00 2024-05-14T11:19:02.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.267+0000 7f2320c00640 1 -- 172.21.15.116:0/2710284825 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1657 (secure 0 0 0) 0x7f231c0f7570 con 0x7f232415fe00 2024-05-14T11:19:02.271 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:02.271 INFO:teuthology.orchestra.run.smithi116.stdout:{"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1715685359,"num_in_osds":8,"osd_in_since":1715685350,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":240807936,"bytes_avail":772853305344,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":34,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5721}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-14T11:17:40.380187+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 msgr2=0x7f2300071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 0x7f2300071910 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f230c000f70 tx=0x7f230c007dd0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 msgr2=0x7f232414e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f231c0b0040 tx=0x7f231c019710 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 shutdown_connections 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f230006f400 0x7f2300071910 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23240b9160 0x7f232414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f232415fe00 0x7f232414e030 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 --2- 172.21.15.116:0/2710284825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23240b70a0 0x7f23240c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 >> 172.21.15.116:0/2710284825 conn(0x7f23240d3740 msgr2=0x7f23240cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 shutdown_connections 2024-05-14T11:19:02.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.271+0000 7f232b400640 1 -- 172.21.15.116:0/2710284825 wait complete. 2024-05-14T11:19:02.293 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-14T11:19:02.294 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:19:02.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.435+0000 7f33c5c00640 1 Processor -- start 2024-05-14T11:19:02.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 -- start start 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 0x7f33c00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f33c00d1670 0x7f33c00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c0161600 0x7f33c0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00ae100 con 0x7f33c00d1670 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00ae870 con 0x7f33c015db30 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33c5c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33c006ff20 con 0x7f33c0161600 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f33c00d1670 0x7f33c00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f33c00d1670 0x7f33c00d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39890/0 (socket says 172.21.15.116:39890) 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 -- 172.21.15.116:0/894547439 learned_addr learned my addr 172.21.15.116:0/894547439 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33bf400640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c0161600 0x7f33c0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33bea00640 1 --2- 172.21.15.116:0/894547439 >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 0x7f33c00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 -- 172.21.15.116:0/894547439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c0161600 msgr2=0x7f33c0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c0161600 0x7f33c0161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 -- 172.21.15.116:0/894547439 >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 msgr2=0x7f33c00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:02.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 --2- 172.21.15.116:0/894547439 >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 0x7f33c00aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33b6000640 1 -- 172.21.15.116:0/894547439 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c00d1fc0 con 0x7f33c00d1670 2024-05-14T11:19:02.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.439+0000 7f33bea00640 1 --2- 172.21.15.116:0/894547439 >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 0x7f33c00aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:02.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33b6000640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c00d1a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f33c0071f20 tx=0x7f33a801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2e26e3ef3faf7a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33be000640 1 -- 172.21.15.116:0/894547439 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b0070 con 0x7f33c00d1670 2024-05-14T11:19:02.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33be000640 1 -- 172.21.15.116:0/894547439 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a80ab600 con 0x7f33c00d1670 2024-05-14T11:19:02.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33be000640 1 -- 172.21.15.116:0/894547439 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80ab9d0 con 0x7f33c00d1670 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 -- 172.21.15.116:0/894547439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 msgr2=0x7f33c00d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c00d1a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f33c0071f20 tx=0x7f33a801c490 comp rx=0 tx=0).stop 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 -- 172.21.15.116:0/894547439 shutdown_connections 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c00d1a80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 --2- 172.21.15.116:0/894547439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c0161600 0x7f33c0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 --2- 172.21.15.116:0/894547439 >> v2:172.21.15.149:3300/0 conn(0x7f33c015db30 0x7f33c00aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 -- 172.21.15.116:0/894547439 >> 172.21.15.116:0/894547439 conn(0x7f33c00b4620 msgr2=0x7f33c00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 -- 172.21.15.116:0/894547439 shutdown_connections 2024-05-14T11:19:02.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.443+0000 7f33c5c00640 1 -- 172.21.15.116:0/894547439 wait complete. 2024-05-14T11:19:02.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 Processor -- start 2024-05-14T11:19:02.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- start start 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c015db30 0x7f33c0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 0x7f33c014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0162360 con 0x7f33c00d1670 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00aefe0 con 0x7f33c0161600 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33c015df40 con 0x7f33c015db30 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39904/0 (socket says 172.21.15.116:39904) 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 -- 172.21.15.116:0/89051099 learned_addr learned my addr 172.21.15.116:0/89051099 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33b6000640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c015db30 0x7f33c0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bf400640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 0x7f33c014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 -- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c015db30 msgr2=0x7f33c0074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c015db30 0x7f33c0074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 -- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 msgr2=0x7f33c014cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 0x7f33c014cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 -- 172.21.15.116:0/89051099 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c01677f0 con 0x7f33c00d1670 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bf400640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 0x7f33c014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33bea00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000b6c0 con 0x7f33c00d1670 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b0019070 con 0x7f33c00d1670 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c0167ad0 con 0x7f33c00d1670 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000baa0 con 0x7f33c00d1670 2024-05-14T11:19:02.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.447+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c00b3d00 con 0x7f33c00d1670 2024-05-14T11:19:02.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c006a9c0 con 0x7f33c00d1670 2024-05-14T11:19:02.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f33b000bc40 con 0x7f33c00d1670 2024-05-14T11:19:02.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33b7400640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 0x7f339c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:02.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f33b008f870 con 0x7f33c00d1670 2024-05-14T11:19:02.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33b6000640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 0x7f339c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:02.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.451+0000 7f33b6000640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 0x7f339c071910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f33c00d2370 tx=0x7f33a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:02.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.459+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f33b005c5c0 con 0x7f33c00d1670 2024-05-14T11:19:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.611+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33c01f9740 con 0x7f339c06f400 2024-05-14T11:19:02.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.619+0000 7f33b7400640 1 -- 172.21.15.116:0/89051099 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7f33c01f9740 con 0x7f339c06f400 2024-05-14T11:19:02.624 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:02.626 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 msgr2=0x7f339c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 0x7f339c071910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f33c00d2370 tx=0x7f33a8008040 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 msgr2=0x7f33c0148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 shutdown_connections 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f339c06f400 0x7f339c071910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33c0161600 0x7f33c014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33c015db30 0x7f33c0074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 --2- 172.21.15.116:0/89051099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c00d1670 0x7f33c0148c30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:02.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 >> 172.21.15.116:0/89051099 conn(0x7f33c00b4620 msgr2=0x7f33c00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:02.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 shutdown_connections 2024-05-14T11:19:02.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:02.623+0000 7f33c5c00640 1 -- 172.21.15.116:0/89051099 wait complete. 2024-05-14T11:19:02.646 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-05-14T11:19:02.399997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243880,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730840,"statfs":{"total":773094113280,"available":772844380160,"internally_reserved":0,"allocated":7053312,"data_stored":3786164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.490469"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193193+0000","last_change":"2024-05-14T11:18:56.951042+0000","last_active":"2024-05-14T11:18:58.193193+0000","last_peered":"2024-05-14T11:18:58.193193+0000","last_clean":"2024-05-14T11:18:58.193193+0000","last_became_active":"2024-05-14T11:18:56.943065+0000","last_became_peered":"2024-05-14T11:18:56.943065+0000","last_unstale":"2024-05-14T11:18:58.193193+0000","last_undegraded":"2024-05-14T11:18:58.193193+0000","last_fullsized":"2024-05-14T11:18:58.193193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:23.983706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193279+0000","last_change":"2024-05-14T11:18:56.945310+0000","last_active":"2024-05-14T11:18:58.193279+0000","last_peered":"2024-05-14T11:18:58.193279+0000","last_clean":"2024-05-14T11:18:58.193279+0000","last_became_active":"2024-05-14T11:18:56.944979+0000","last_became_peered":"2024-05-14T11:18:56.944979+0000","last_unstale":"2024-05-14T11:18:58.193279+0000","last_undegraded":"2024-05-14T11:18:58.193279+0000","last_fullsized":"2024-05-14T11:18:58.193279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:55.438844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193556+0000","last_change":"2024-05-14T11:18:56.943118+0000","last_active":"2024-05-14T11:18:58.193556+0000","last_peered":"2024-05-14T11:18:58.193556+0000","last_clean":"2024-05-14T11:18:58.193556+0000","last_became_active":"2024-05-14T11:18:56.942810+0000","last_became_peered":"2024-05-14T11:18:56.942810+0000","last_unstale":"2024-05-14T11:18:58.193556+0000","last_undegraded":"2024-05-14T11:18:58.193556+0000","last_fullsized":"2024-05-14T11:18:58.193556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:41:29.460299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931409+0000","last_change":"2024-05-14T11:18:56.921038+0000","last_active":"2024-05-14T11:18:57.931409+0000","last_peered":"2024-05-14T11:18:57.931409+0000","last_clean":"2024-05-14T11:18:57.931409+0000","last_became_active":"2024-05-14T11:18:56.920553+0000","last_became_peered":"2024-05-14T11:18:56.920553+0000","last_unstale":"2024-05-14T11:18:57.931409+0000","last_undegraded":"2024-05-14T11:18:57.931409+0000","last_fullsized":"2024-05-14T11:18:57.931409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:28.737096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932170+0000","last_change":"2024-05-14T11:18:56.922653+0000","last_active":"2024-05-14T11:18:57.932170+0000","last_peered":"2024-05-14T11:18:57.932170+0000","last_clean":"2024-05-14T11:18:57.932170+0000","last_became_active":"2024-05-14T11:18:56.922374+0000","last_became_peered":"2024-05-14T11:18:56.922374+0000","last_unstale":"2024-05-14T11:18:57.932170+0000","last_undegraded":"2024-05-14T11:18:57.932170+0000","last_fullsized":"2024-05-14T11:18:57.932170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:01:50.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192754+0000","last_change":"2024-05-14T11:18:56.940719+0000","last_active":"2024-05-14T11:18:58.192754+0000","last_peered":"2024-05-14T11:18:58.192754+0000","last_clean":"2024-05-14T11:18:58.192754+0000","last_became_active":"2024-05-14T11:18:56.940415+0000","last_became_peered":"2024-05-14T11:18:56.940415+0000","last_unstale":"2024-05-14T11:18:58.192754+0000","last_undegraded":"2024-05-14T11:18:58.192754+0000","last_fullsized":"2024-05-14T11:18:58.192754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:39.321114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937202+0000","last_change":"2024-05-14T11:18:56.936013+0000","last_active":"2024-05-14T11:18:57.937202+0000","last_peered":"2024-05-14T11:18:57.937202+0000","last_clean":"2024-05-14T11:18:57.937202+0000","last_became_active":"2024-05-14T11:18:56.935678+0000","last_became_peered":"2024-05-14T11:18:56.935678+0000","last_unstale":"2024-05-14T11:18:57.937202+0000","last_undegraded":"2024-05-14T11:18:57.937202+0000","last_fullsized":"2024-05-14T11:18:57.937202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:49.249969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934733+0000","last_change":"2024-05-14T11:18:56.945673+0000","last_active":"2024-05-14T11:18:57.934733+0000","last_peered":"2024-05-14T11:18:57.934733+0000","last_clean":"2024-05-14T11:18:57.934733+0000","last_became_active":"2024-05-14T11:18:56.945394+0000","last_became_peered":"2024-05-14T11:18:56.945394+0000","last_unstale":"2024-05-14T11:18:57.934733+0000","last_undegraded":"2024-05-14T11:18:57.934733+0000","last_fullsized":"2024-05-14T11:18:57.934733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:52.599195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193824+0000","last_change":"2024-05-14T11:18:56.936632+0000","last_active":"2024-05-14T11:18:58.193824+0000","last_peered":"2024-05-14T11:18:58.193824+0000","last_clean":"2024-05-14T11:18:58.193824+0000","last_became_active":"2024-05-14T11:18:56.936305+0000","last_became_peered":"2024-05-14T11:18:56.936305+0000","last_unstale":"2024-05-14T11:18:58.193824+0000","last_undegraded":"2024-05-14T11:18:58.193824+0000","last_fullsized":"2024-05-14T11:18:58.193824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193647+0000","last_change":"2024-05-14T11:18:56.952775+0000","last_active":"2024-05-14T11:18:58.193647+0000","last_peered":"2024-05-14T11:18:58.193647+0000","last_clean":"2024-05-14T11:18:58.193647+0000","last_became_active":"2024-05-14T11:18:56.952461+0000","last_became_peered":"2024-05-14T11:18:56.952461+0000","last_unstale":"2024-05-14T11:18:58.193647+0000","last_undegraded":"2024-05-14T11:18:58.193647+0000","last_fullsized":"2024-05-14T11:18:58.193647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:43.057516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932011+0000","last_change":"2024-05-14T11:18:56.939518+0000","last_active":"2024-05-14T11:18:57.932011+0000","last_peered":"2024-05-14T11:18:57.932011+0000","last_clean":"2024-05-14T11:18:57.932011+0000","last_became_active":"2024-05-14T11:18:56.938957+0000","last_became_peered":"2024-05-14T11:18:56.938957+0000","last_unstale":"2024-05-14T11:18:57.932011+0000","last_undegraded":"2024-05-14T11:18:57.932011+0000","last_fullsized":"2024-05-14T11:18:57.932011+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:49.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935039+0000","last_change":"2024-05-14T11:18:56.941581+0000","last_active":"2024-05-14T11:18:57.935039+0000","last_peered":"2024-05-14T11:18:57.935039+0000","last_clean":"2024-05-14T11:18:57.935039+0000","last_became_active":"2024-05-14T11:18:56.941256+0000","last_became_peered":"2024-05-14T11:18:56.941256+0000","last_unstale":"2024-05-14T11:18:57.935039+0000","last_undegraded":"2024-05-14T11:18:57.935039+0000","last_fullsized":"2024-05-14T11:18:57.935039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:43:49.923077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934763+0000","last_change":"2024-05-14T11:18:56.937159+0000","last_active":"2024-05-14T11:18:57.934763+0000","last_peered":"2024-05-14T11:18:57.934763+0000","last_clean":"2024-05-14T11:18:57.934763+0000","last_became_active":"2024-05-14T11:18:56.936771+0000","last_became_peered":"2024-05-14T11:18:56.936771+0000","last_unstale":"2024-05-14T11:18:57.934763+0000","last_undegraded":"2024-05-14T11:18:57.934763+0000","last_fullsized":"2024-05-14T11:18:57.934763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:56.621403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193465+0000","last_change":"2024-05-14T11:18:56.945522+0000","last_active":"2024-05-14T11:18:58.193465+0000","last_peered":"2024-05-14T11:18:58.193465+0000","last_clean":"2024-05-14T11:18:58.193465+0000","last_became_active":"2024-05-14T11:18:56.943591+0000","last_became_peered":"2024-05-14T11:18:56.943591+0000","last_unstale":"2024-05-14T11:18:58.193465+0000","last_undegraded":"2024-05-14T11:18:58.193465+0000","last_fullsized":"2024-05-14T11:18:58.193465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:03.355624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193471+0000","last_change":"2024-05-14T11:18:56.945063+0000","last_active":"2024-05-14T11:18:58.193471+0000","last_peered":"2024-05-14T11:18:58.193471+0000","last_clean":"2024-05-14T11:18:58.193471+0000","last_became_active":"2024-05-14T11:18:56.944326+0000","last_became_peered":"2024-05-14T11:18:56.944326+0000","last_unstale":"2024-05-14T11:18:58.193471+0000","last_undegraded":"2024-05-14T11:18:58.193471+0000","last_fullsized":"2024-05-14T11:18:58.193471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:43:49.923077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937049+0000","last_change":"2024-05-14T11:18:56.939395+0000","last_active":"2024-05-14T11:18:57.937049+0000","last_peered":"2024-05-14T11:18:57.937049+0000","last_clean":"2024-05-14T11:18:57.937049+0000","last_became_active":"2024-05-14T11:18:56.939037+0000","last_became_peered":"2024-05-14T11:18:56.939037+0000","last_unstale":"2024-05-14T11:18:57.937049+0000","last_undegraded":"2024-05-14T11:18:57.937049+0000","last_fullsized":"2024-05-14T11:18:57.937049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:16:40.998989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193245+0000","last_change":"2024-05-14T11:18:56.942671+0000","last_active":"2024-05-14T11:18:58.193245+0000","last_peered":"2024-05-14T11:18:58.193245+0000","last_clean":"2024-05-14T11:18:58.193245+0000","last_became_active":"2024-05-14T11:18:56.934867+0000","last_became_peered":"2024-05-14T11:18:56.934867+0000","last_unstale":"2024-05-14T11:18:58.193245+0000","last_undegraded":"2024-05-14T11:18:58.193245+0000","last_fullsized":"2024-05-14T11:18:58.193245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:16.117129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931934+0000","last_change":"2024-05-14T11:18:56.927997+0000","last_active":"2024-05-14T11:18:57.931934+0000","last_peered":"2024-05-14T11:18:57.931934+0000","last_clean":"2024-05-14T11:18:57.931934+0000","last_became_active":"2024-05-14T11:18:56.927541+0000","last_became_peered":"2024-05-14T11:18:56.927541+0000","last_unstale":"2024-05-14T11:18:57.931934+0000","last_undegraded":"2024-05-14T11:18:57.931934+0000","last_fullsized":"2024-05-14T11:18:57.931934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:59.274859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931855+0000","last_change":"2024-05-14T11:18:56.929995+0000","last_active":"2024-05-14T11:18:57.931855+0000","last_peered":"2024-05-14T11:18:57.931855+0000","last_clean":"2024-05-14T11:18:57.931855+0000","last_became_active":"2024-05-14T11:18:56.929715+0000","last_became_peered":"2024-05-14T11:18:56.929715+0000","last_unstale":"2024-05-14T11:18:57.931855+0000","last_undegraded":"2024-05-14T11:18:57.931855+0000","last_fullsized":"2024-05-14T11:18:57.931855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:49.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932152+0000","last_change":"2024-05-14T11:18:56.928939+0000","last_active":"2024-05-14T11:18:57.932152+0000","last_peered":"2024-05-14T11:18:57.932152+0000","last_clean":"2024-05-14T11:18:57.932152+0000","last_became_active":"2024-05-14T11:18:56.928643+0000","last_became_peered":"2024-05-14T11:18:56.928643+0000","last_unstale":"2024-05-14T11:18:57.932152+0000","last_undegraded":"2024-05-14T11:18:57.932152+0000","last_fullsized":"2024-05-14T11:18:57.932152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:52.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934915+0000","last_change":"2024-05-14T11:18:56.947223+0000","last_active":"2024-05-14T11:18:57.934915+0000","last_peered":"2024-05-14T11:18:57.934915+0000","last_clean":"2024-05-14T11:18:57.934915+0000","last_became_active":"2024-05-14T11:18:56.946886+0000","last_became_peered":"2024-05-14T11:18:56.946886+0000","last_unstale":"2024-05-14T11:18:57.934915+0000","last_undegraded":"2024-05-14T11:18:57.934915+0000","last_fullsized":"2024-05-14T11:18:57.934915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:03.355624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931859+0000","last_change":"2024-05-14T11:18:56.941547+0000","last_active":"2024-05-14T11:18:57.931859+0000","last_peered":"2024-05-14T11:18:57.931859+0000","last_clean":"2024-05-14T11:18:57.931859+0000","last_became_active":"2024-05-14T11:18:56.941165+0000","last_became_peered":"2024-05-14T11:18:56.941165+0000","last_unstale":"2024-05-14T11:18:57.931859+0000","last_undegraded":"2024-05-14T11:18:57.931859+0000","last_fullsized":"2024-05-14T11:18:57.931859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:38.804025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931653+0000","last_change":"2024-05-14T11:18:56.947149+0000","last_active":"2024-05-14T11:18:57.931653+0000","last_peered":"2024-05-14T11:18:57.931653+0000","last_clean":"2024-05-14T11:18:57.931653+0000","last_became_active":"2024-05-14T11:18:56.946809+0000","last_became_peered":"2024-05-14T11:18:56.946809+0000","last_unstale":"2024-05-14T11:18:57.931653+0000","last_undegraded":"2024-05-14T11:18:57.931653+0000","last_fullsized":"2024-05-14T11:18:57.931653+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193237+0000","last_change":"2024-05-14T11:18:56.942473+0000","last_active":"2024-05-14T11:18:58.193237+0000","last_peered":"2024-05-14T11:18:58.193237+0000","last_clean":"2024-05-14T11:18:58.193237+0000","last_became_active":"2024-05-14T11:18:56.942165+0000","last_became_peered":"2024-05-14T11:18:56.942165+0000","last_unstale":"2024-05-14T11:18:58.193237+0000","last_undegraded":"2024-05-14T11:18:58.193237+0000","last_fullsized":"2024-05-14T11:18:58.193237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:18.885017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935137+0000","last_change":"2024-05-14T11:18:56.949711+0000","last_active":"2024-05-14T11:18:57.935137+0000","last_peered":"2024-05-14T11:18:57.935137+0000","last_clean":"2024-05-14T11:18:57.935137+0000","last_became_active":"2024-05-14T11:18:56.949436+0000","last_became_peered":"2024-05-14T11:18:56.949436+0000","last_unstale":"2024-05-14T11:18:57.935137+0000","last_undegraded":"2024-05-14T11:18:57.935137+0000","last_fullsized":"2024-05-14T11:18:57.935137+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:35:06.732283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935404+0000","last_change":"2024-05-14T11:18:56.949946+0000","last_active":"2024-05-14T11:18:57.935404+0000","last_peered":"2024-05-14T11:18:57.935404+0000","last_clean":"2024-05-14T11:18:57.935404+0000","last_became_active":"2024-05-14T11:18:56.948967+0000","last_became_peered":"2024-05-14T11:18:56.948967+0000","last_unstale":"2024-05-14T11:18:57.935404+0000","last_undegraded":"2024-05-14T11:18:57.935404+0000","last_fullsized":"2024-05-14T11:18:57.935404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:43.057516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932211+0000","last_change":"2024-05-14T11:18:56.926910+0000","last_active":"2024-05-14T11:18:57.932211+0000","last_peered":"2024-05-14T11:18:57.932211+0000","last_clean":"2024-05-14T11:18:57.932211+0000","last_became_active":"2024-05-14T11:18:56.926203+0000","last_became_peered":"2024-05-14T11:18:56.926203+0000","last_unstale":"2024-05-14T11:18:57.932211+0000","last_undegraded":"2024-05-14T11:18:57.932211+0000","last_fullsized":"2024-05-14T11:18:57.932211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:00:58.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192624+0000","last_change":"2024-05-14T11:18:56.935910+0000","last_active":"2024-05-14T11:18:58.192624+0000","last_peered":"2024-05-14T11:18:58.192624+0000","last_clean":"2024-05-14T11:18:58.192624+0000","last_became_active":"2024-05-14T11:18:56.935468+0000","last_became_peered":"2024-05-14T11:18:56.935468+0000","last_unstale":"2024-05-14T11:18:58.192624+0000","last_undegraded":"2024-05-14T11:18:58.192624+0000","last_fullsized":"2024-05-14T11:18:58.192624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:02.408142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193295+0000","last_change":"2024-05-14T11:18:56.936736+0000","last_active":"2024-05-14T11:18:58.193295+0000","last_peered":"2024-05-14T11:18:58.193295+0000","last_clean":"2024-05-14T11:18:58.193295+0000","last_became_active":"2024-05-14T11:18:56.936263+0000","last_became_peered":"2024-05-14T11:18:56.936263+0000","last_unstale":"2024-05-14T11:18:58.193295+0000","last_undegraded":"2024-05-14T11:18:58.193295+0000","last_fullsized":"2024-05-14T11:18:58.193295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:56.621403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934758+0000","last_change":"2024-05-14T11:18:56.942327+0000","last_active":"2024-05-14T11:18:57.934758+0000","last_peered":"2024-05-14T11:18:57.934758+0000","last_clean":"2024-05-14T11:18:57.934758+0000","last_became_active":"2024-05-14T11:18:56.941975+0000","last_became_peered":"2024-05-14T11:18:56.941975+0000","last_unstale":"2024-05-14T11:18:57.934758+0000","last_undegraded":"2024-05-14T11:18:57.934758+0000","last_fullsized":"2024-05-14T11:18:57.934758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:53.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930982+0000","last_change":"2024-05-14T11:18:56.923728+0000","last_active":"2024-05-14T11:18:57.930982+0000","last_peered":"2024-05-14T11:18:57.930982+0000","last_clean":"2024-05-14T11:18:57.930982+0000","last_became_active":"2024-05-14T11:18:56.923386+0000","last_became_peered":"2024-05-14T11:18:56.923386+0000","last_unstale":"2024-05-14T11:18:57.930982+0000","last_undegraded":"2024-05-14T11:18:57.930982+0000","last_fullsized":"2024-05-14T11:18:57.930982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:26.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192275+0000","last_change":"2024-05-14T11:18:56.930344+0000","last_active":"2024-05-14T11:18:58.192275+0000","last_peered":"2024-05-14T11:18:58.192275+0000","last_clean":"2024-05-14T11:18:58.192275+0000","last_became_active":"2024-05-14T11:18:56.930031+0000","last_became_peered":"2024-05-14T11:18:56.930031+0000","last_unstale":"2024-05-14T11:18:58.192275+0000","last_undegraded":"2024-05-14T11:18:58.192275+0000","last_fullsized":"2024-05-14T11:18:58.192275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931860+0000","last_change":"2024-05-14T11:18:56.937328+0000","last_active":"2024-05-14T11:18:57.931860+0000","last_peered":"2024-05-14T11:18:57.931860+0000","last_clean":"2024-05-14T11:18:57.931860+0000","last_became_active":"2024-05-14T11:18:56.937014+0000","last_became_peered":"2024-05-14T11:18:56.937014+0000","last_unstale":"2024-05-14T11:18:57.931860+0000","last_undegraded":"2024-05-14T11:18:57.931860+0000","last_fullsized":"2024-05-14T11:18:57.931860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:11.225500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193259+0000","last_change":"2024-05-14T11:18:55.929650+0000","last_active":"2024-05-14T11:18:58.193259+0000","last_peered":"2024-05-14T11:18:58.193259+0000","last_clean":"2024-05-14T11:18:58.193259+0000","last_became_active":"2024-05-14T11:18:55.929313+0000","last_became_peered":"2024-05-14T11:18:55.929313+0000","last_unstale":"2024-05-14T11:18:58.193259+0000","last_undegraded":"2024-05-14T11:18:58.193259+0000","last_fullsized":"2024-05-14T11:18:58.193259+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:50.636758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932432+0000","last_change":"2024-05-14T11:18:56.920196+0000","last_active":"2024-05-14T11:18:57.932432+0000","last_peered":"2024-05-14T11:18:57.932432+0000","last_clean":"2024-05-14T11:18:57.932432+0000","last_became_active":"2024-05-14T11:18:56.919798+0000","last_became_peered":"2024-05-14T11:18:56.919798+0000","last_unstale":"2024-05-14T11:18:57.932432+0000","last_undegraded":"2024-05-14T11:18:57.932432+0000","last_fullsized":"2024-05-14T11:18:57.932432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:01:50.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193449+0000","last_change":"2024-05-14T11:18:55.925360+0000","last_active":"2024-05-14T11:18:58.193449+0000","last_peered":"2024-05-14T11:18:58.193449+0000","last_clean":"2024-05-14T11:18:58.193449+0000","last_became_active":"2024-05-14T11:18:55.925002+0000","last_became_peered":"2024-05-14T11:18:55.925002+0000","last_unstale":"2024-05-14T11:18:58.193449+0000","last_undegraded":"2024-05-14T11:18:58.193449+0000","last_fullsized":"2024-05-14T11:18:58.193449+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:55.613451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193414+0000","last_change":"2024-05-14T11:18:56.930939+0000","last_active":"2024-05-14T11:18:58.193414+0000","last_peered":"2024-05-14T11:18:58.193414+0000","last_clean":"2024-05-14T11:18:58.193414+0000","last_became_active":"2024-05-14T11:18:56.929789+0000","last_became_peered":"2024-05-14T11:18:56.929789+0000","last_unstale":"2024-05-14T11:18:58.193414+0000","last_undegraded":"2024-05-14T11:18:58.193414+0000","last_fullsized":"2024-05-14T11:18:58.193414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.313987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193297+0000","last_change":"2024-05-14T11:18:55.929021+0000","last_active":"2024-05-14T11:18:58.193297+0000","last_peered":"2024-05-14T11:18:58.193297+0000","last_clean":"2024-05-14T11:18:58.193297+0000","last_became_active":"2024-05-14T11:18:55.928464+0000","last_became_peered":"2024-05-14T11:18:55.928464+0000","last_unstale":"2024-05-14T11:18:58.193297+0000","last_undegraded":"2024-05-14T11:18:58.193297+0000","last_fullsized":"2024-05-14T11:18:58.193297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:02.347672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193640+0000","last_change":"2024-05-14T11:18:56.952449+0000","last_active":"2024-05-14T11:18:58.193640+0000","last_peered":"2024-05-14T11:18:58.193640+0000","last_clean":"2024-05-14T11:18:58.193640+0000","last_became_active":"2024-05-14T11:18:56.952193+0000","last_became_peered":"2024-05-14T11:18:56.952193+0000","last_unstale":"2024-05-14T11:18:58.193640+0000","last_undegraded":"2024-05-14T11:18:58.193640+0000","last_fullsized":"2024-05-14T11:18:58.193640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:35:06.732283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193096+0000","last_change":"2024-05-14T11:18:55.925238+0000","last_active":"2024-05-14T11:18:58.193096+0000","last_peered":"2024-05-14T11:18:58.193096+0000","last_clean":"2024-05-14T11:18:58.193096+0000","last_became_active":"2024-05-14T11:18:55.924925+0000","last_became_peered":"2024-05-14T11:18:55.924925+0000","last_unstale":"2024-05-14T11:18:58.193096+0000","last_undegraded":"2024-05-14T11:18:58.193096+0000","last_fullsized":"2024-05-14T11:18:58.193096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:17.877065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937290+0000","last_change":"2024-05-14T11:18:56.932288+0000","last_active":"2024-05-14T11:18:57.937290+0000","last_peered":"2024-05-14T11:18:57.937290+0000","last_clean":"2024-05-14T11:18:57.937290+0000","last_became_active":"2024-05-14T11:18:56.931965+0000","last_became_peered":"2024-05-14T11:18:56.931965+0000","last_unstale":"2024-05-14T11:18:57.937290+0000","last_undegraded":"2024-05-14T11:18:57.937290+0000","last_fullsized":"2024-05-14T11:18:57.937290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:38.386584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932001+0000","last_change":"2024-05-14T11:18:55.926997+0000","last_active":"2024-05-14T11:18:57.932001+0000","last_peered":"2024-05-14T11:18:57.932001+0000","last_clean":"2024-05-14T11:18:57.932001+0000","last_became_active":"2024-05-14T11:18:55.926691+0000","last_became_peered":"2024-05-14T11:18:55.926691+0000","last_unstale":"2024-05-14T11:18:57.932001+0000","last_undegraded":"2024-05-14T11:18:57.932001+0000","last_fullsized":"2024-05-14T11:18:57.932001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:31.939814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192562+0000","last_change":"2024-05-14T11:18:56.933970+0000","last_active":"2024-05-14T11:18:58.192562+0000","last_peered":"2024-05-14T11:18:58.192562+0000","last_clean":"2024-05-14T11:18:58.192562+0000","last_became_active":"2024-05-14T11:18:56.932978+0000","last_became_peered":"2024-05-14T11:18:56.932978+0000","last_unstale":"2024-05-14T11:18:58.192562+0000","last_undegraded":"2024-05-14T11:18:58.192562+0000","last_fullsized":"2024-05-14T11:18:58.192562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:38.874693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931812+0000","last_change":"2024-05-14T11:18:55.922225+0000","last_active":"2024-05-14T11:18:57.931812+0000","last_peered":"2024-05-14T11:18:57.931812+0000","last_clean":"2024-05-14T11:18:57.931812+0000","last_became_active":"2024-05-14T11:18:55.921862+0000","last_became_peered":"2024-05-14T11:18:55.921862+0000","last_unstale":"2024-05-14T11:18:57.931812+0000","last_undegraded":"2024-05-14T11:18:57.931812+0000","last_fullsized":"2024-05-14T11:18:57.931812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:52.322782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931413+0000","last_change":"2024-05-14T11:18:56.924715+0000","last_active":"2024-05-14T11:18:57.931413+0000","last_peered":"2024-05-14T11:18:57.931413+0000","last_clean":"2024-05-14T11:18:57.931413+0000","last_became_active":"2024-05-14T11:18:56.924390+0000","last_became_peered":"2024-05-14T11:18:56.924390+0000","last_unstale":"2024-05-14T11:18:57.931413+0000","last_undegraded":"2024-05-14T11:18:57.931413+0000","last_fullsized":"2024-05-14T11:18:57.931413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:26.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931648+0000","last_change":"2024-05-14T11:18:55.926111+0000","last_active":"2024-05-14T11:18:57.931648+0000","last_peered":"2024-05-14T11:18:57.931648+0000","last_clean":"2024-05-14T11:18:57.931648+0000","last_became_active":"2024-05-14T11:18:55.925459+0000","last_became_peered":"2024-05-14T11:18:55.925459+0000","last_unstale":"2024-05-14T11:18:57.931648+0000","last_undegraded":"2024-05-14T11:18:57.931648+0000","last_fullsized":"2024-05-14T11:18:57.931648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:17.877065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932087+0000","last_change":"2024-05-14T11:18:56.941960+0000","last_active":"2024-05-14T11:18:57.932087+0000","last_peered":"2024-05-14T11:18:57.932087+0000","last_clean":"2024-05-14T11:18:57.932087+0000","last_became_active":"2024-05-14T11:18:56.937300+0000","last_became_peered":"2024-05-14T11:18:56.937300+0000","last_unstale":"2024-05-14T11:18:57.932087+0000","last_undegraded":"2024-05-14T11:18:57.932087+0000","last_fullsized":"2024-05-14T11:18:57.932087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:39.321114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931181+0000","last_change":"2024-05-14T11:18:55.922438+0000","last_active":"2024-05-14T11:18:57.931181+0000","last_peered":"2024-05-14T11:18:57.931181+0000","last_clean":"2024-05-14T11:18:57.931181+0000","last_became_active":"2024-05-14T11:18:55.921484+0000","last_became_peered":"2024-05-14T11:18:55.921484+0000","last_unstale":"2024-05-14T11:18:57.931181+0000","last_undegraded":"2024-05-14T11:18:57.931181+0000","last_fullsized":"2024-05-14T11:18:57.931181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:00.972284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193018+0000","last_change":"2024-05-14T11:18:56.930745+0000","last_active":"2024-05-14T11:18:58.193018+0000","last_peered":"2024-05-14T11:18:58.193018+0000","last_clean":"2024-05-14T11:18:58.193018+0000","last_became_active":"2024-05-14T11:18:56.930418+0000","last_became_peered":"2024-05-14T11:18:56.930418+0000","last_unstale":"2024-05-14T11:18:58.193018+0000","last_undegraded":"2024-05-14T11:18:58.193018+0000","last_fullsized":"2024-05-14T11:18:58.193018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:47.462747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193722+0000","last_change":"2024-05-14T11:18:55.938638+0000","last_active":"2024-05-14T11:18:58.193722+0000","last_peered":"2024-05-14T11:18:58.193722+0000","last_clean":"2024-05-14T11:18:58.193722+0000","last_became_active":"2024-05-14T11:18:55.937136+0000","last_became_peered":"2024-05-14T11:18:55.937136+0000","last_unstale":"2024-05-14T11:18:58.193722+0000","last_undegraded":"2024-05-14T11:18:58.193722+0000","last_fullsized":"2024-05-14T11:18:58.193722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:32:01.643061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932185+0000","last_change":"2024-05-14T11:18:56.941473+0000","last_active":"2024-05-14T11:18:57.932185+0000","last_peered":"2024-05-14T11:18:57.932185+0000","last_clean":"2024-05-14T11:18:57.932185+0000","last_became_active":"2024-05-14T11:18:56.941113+0000","last_became_peered":"2024-05-14T11:18:56.941113+0000","last_unstale":"2024-05-14T11:18:57.932185+0000","last_undegraded":"2024-05-14T11:18:57.932185+0000","last_fullsized":"2024-05-14T11:18:57.932185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.313987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932224+0000","last_change":"2024-05-14T11:18:55.940055+0000","last_active":"2024-05-14T11:18:57.932224+0000","last_peered":"2024-05-14T11:18:57.932224+0000","last_clean":"2024-05-14T11:18:57.932224+0000","last_became_active":"2024-05-14T11:18:55.939780+0000","last_became_peered":"2024-05-14T11:18:55.939780+0000","last_unstale":"2024-05-14T11:18:57.932224+0000","last_undegraded":"2024-05-14T11:18:57.932224+0000","last_fullsized":"2024-05-14T11:18:57.932224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:40:03.934809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932435+0000","last_change":"2024-05-14T11:18:56.939367+0000","last_active":"2024-05-14T11:18:57.932435+0000","last_peered":"2024-05-14T11:18:57.932435+0000","last_clean":"2024-05-14T11:18:57.932435+0000","last_became_active":"2024-05-14T11:18:56.938571+0000","last_became_peered":"2024-05-14T11:18:56.938571+0000","last_unstale":"2024-05-14T11:18:57.932435+0000","last_undegraded":"2024-05-14T11:18:57.932435+0000","last_fullsized":"2024-05-14T11:18:57.932435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:02.408142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193103+0000","last_change":"2024-05-14T11:18:55.937856+0000","last_active":"2024-05-14T11:18:58.193103+0000","last_peered":"2024-05-14T11:18:58.193103+0000","last_clean":"2024-05-14T11:18:58.193103+0000","last_became_active":"2024-05-14T11:18:55.937505+0000","last_became_peered":"2024-05-14T11:18:55.937505+0000","last_unstale":"2024-05-14T11:18:58.193103+0000","last_undegraded":"2024-05-14T11:18:58.193103+0000","last_fullsized":"2024-05-14T11:18:58.193103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:15.109177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932310+0000","last_change":"2024-05-14T11:18:56.943119+0000","last_active":"2024-05-14T11:18:57.932310+0000","last_peered":"2024-05-14T11:18:57.932310+0000","last_clean":"2024-05-14T11:18:57.932310+0000","last_became_active":"2024-05-14T11:18:56.942594+0000","last_became_peered":"2024-05-14T11:18:56.942594+0000","last_unstale":"2024-05-14T11:18:57.932310+0000","last_undegraded":"2024-05-14T11:18:57.932310+0000","last_fullsized":"2024-05-14T11:18:57.932310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:51.644710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192683+0000","last_change":"2024-05-14T11:18:55.933957+0000","last_active":"2024-05-14T11:18:58.192683+0000","last_peered":"2024-05-14T11:18:58.192683+0000","last_clean":"2024-05-14T11:18:58.192683+0000","last_became_active":"2024-05-14T11:18:55.933591+0000","last_became_peered":"2024-05-14T11:18:55.933591+0000","last_unstale":"2024-05-14T11:18:58.192683+0000","last_undegraded":"2024-05-14T11:18:58.192683+0000","last_fullsized":"2024-05-14T11:18:58.192683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:48.024667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930660+0000","last_change":"2024-05-14T11:18:56.931845+0000","last_active":"2024-05-14T11:18:57.930660+0000","last_peered":"2024-05-14T11:18:57.930660+0000","last_clean":"2024-05-14T11:18:57.930660+0000","last_became_active":"2024-05-14T11:18:56.931388+0000","last_became_peered":"2024-05-14T11:18:56.931388+0000","last_unstale":"2024-05-14T11:18:57.930660+0000","last_undegraded":"2024-05-14T11:18:57.930660+0000","last_fullsized":"2024-05-14T11:18:57.930660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:47:59.920388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193495+0000","last_change":"2024-05-14T11:18:55.922634+0000","last_active":"2024-05-14T11:18:58.193495+0000","last_peered":"2024-05-14T11:18:58.193495+0000","last_clean":"2024-05-14T11:18:58.193495+0000","last_became_active":"2024-05-14T11:18:55.921988+0000","last_became_peered":"2024-05-14T11:18:55.921988+0000","last_unstale":"2024-05-14T11:18:58.193495+0000","last_undegraded":"2024-05-14T11:18:58.193495+0000","last_fullsized":"2024-05-14T11:18:58.193495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:59.724825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934746+0000","last_change":"2024-05-14T11:18:56.939457+0000","last_active":"2024-05-14T11:18:57.934746+0000","last_peered":"2024-05-14T11:18:57.934746+0000","last_clean":"2024-05-14T11:18:57.934746+0000","last_became_active":"2024-05-14T11:18:56.939082+0000","last_became_peered":"2024-05-14T11:18:56.939082+0000","last_unstale":"2024-05-14T11:18:57.934746+0000","last_undegraded":"2024-05-14T11:18:57.934746+0000","last_fullsized":"2024-05-14T11:18:57.934746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:32.947765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934687+0000","last_change":"2024-05-14T11:18:55.938688+0000","last_active":"2024-05-14T11:18:57.934687+0000","last_peered":"2024-05-14T11:18:57.934687+0000","last_clean":"2024-05-14T11:18:57.934687+0000","last_became_active":"2024-05-14T11:18:55.938287+0000","last_became_peered":"2024-05-14T11:18:55.938287+0000","last_unstale":"2024-05-14T11:18:57.934687+0000","last_undegraded":"2024-05-14T11:18:57.934687+0000","last_fullsized":"2024-05-14T11:18:57.934687+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:37.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931624+0000","last_change":"2024-05-14T11:18:56.929290+0000","last_active":"2024-05-14T11:18:57.931624+0000","last_peered":"2024-05-14T11:18:57.931624+0000","last_clean":"2024-05-14T11:18:57.931624+0000","last_became_active":"2024-05-14T11:18:56.928967+0000","last_became_peered":"2024-05-14T11:18:56.928967+0000","last_unstale":"2024-05-14T11:18:57.931624+0000","last_undegraded":"2024-05-14T11:18:57.931624+0000","last_fullsized":"2024-05-14T11:18:57.931624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:28.737096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193305+0000","last_change":"2024-05-14T11:18:55.924561+0000","last_active":"2024-05-14T11:18:58.193305+0000","last_peered":"2024-05-14T11:18:58.193305+0000","last_clean":"2024-05-14T11:18:58.193305+0000","last_became_active":"2024-05-14T11:18:55.924148+0000","last_became_peered":"2024-05-14T11:18:55.924148+0000","last_unstale":"2024-05-14T11:18:58.193305+0000","last_undegraded":"2024-05-14T11:18:58.193305+0000","last_fullsized":"2024-05-14T11:18:58.193305+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:22.975754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192708+0000","last_change":"2024-05-14T11:18:56.936334+0000","last_active":"2024-05-14T11:18:58.192708+0000","last_peered":"2024-05-14T11:18:58.192708+0000","last_clean":"2024-05-14T11:18:58.192708+0000","last_became_active":"2024-05-14T11:18:56.934877+0000","last_became_peered":"2024-05-14T11:18:56.934877+0000","last_unstale":"2024-05-14T11:18:58.192708+0000","last_undegraded":"2024-05-14T11:18:58.192708+0000","last_fullsized":"2024-05-14T11:18:58.192708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:03.082658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930745+0000","last_change":"2024-05-14T11:18:55.934695+0000","last_active":"2024-05-14T11:18:57.930745+0000","last_peered":"2024-05-14T11:18:57.930745+0000","last_clean":"2024-05-14T11:18:57.930745+0000","last_became_active":"2024-05-14T11:18:55.934312+0000","last_became_peered":"2024-05-14T11:18:55.934312+0000","last_unstale":"2024-05-14T11:18:57.930745+0000","last_undegraded":"2024-05-14T11:18:57.930745+0000","last_fullsized":"2024-05-14T11:18:57.930745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:00.972284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193751+0000","last_change":"2024-05-14T11:18:56.932287+0000","last_active":"2024-05-14T11:18:58.193751+0000","last_peered":"2024-05-14T11:18:58.193751+0000","last_clean":"2024-05-14T11:18:58.193751+0000","last_became_active":"2024-05-14T11:18:56.931936+0000","last_became_peered":"2024-05-14T11:18:56.931936+0000","last_unstale":"2024-05-14T11:18:58.193751+0000","last_undegraded":"2024-05-14T11:18:58.193751+0000","last_fullsized":"2024-05-14T11:18:58.193751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:44.787619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935129+0000","last_change":"2024-05-14T11:18:55.938172+0000","last_active":"2024-05-14T11:18:57.935129+0000","last_peered":"2024-05-14T11:18:57.935129+0000","last_clean":"2024-05-14T11:18:57.935129+0000","last_became_active":"2024-05-14T11:18:55.937823+0000","last_became_peered":"2024-05-14T11:18:55.937823+0000","last_unstale":"2024-05-14T11:18:57.935129+0000","last_undegraded":"2024-05-14T11:18:57.935129+0000","last_fullsized":"2024-05-14T11:18:57.935129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:02.074706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931903+0000","last_change":"2024-05-14T11:18:56.921592+0000","last_active":"2024-05-14T11:18:57.931903+0000","last_peered":"2024-05-14T11:18:57.931903+0000","last_clean":"2024-05-14T11:18:57.931903+0000","last_became_active":"2024-05-14T11:18:56.920154+0000","last_became_peered":"2024-05-14T11:18:56.920154+0000","last_unstale":"2024-05-14T11:18:57.931903+0000","last_undegraded":"2024-05-14T11:18:57.931903+0000","last_fullsized":"2024-05-14T11:18:57.931903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:00:58.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193384+0000","last_change":"2024-05-14T11:18:55.936674+0000","last_active":"2024-05-14T11:18:58.193384+0000","last_peered":"2024-05-14T11:18:58.193384+0000","last_clean":"2024-05-14T11:18:58.193384+0000","last_became_active":"2024-05-14T11:18:55.936304+0000","last_became_peered":"2024-05-14T11:18:55.936304+0000","last_unstale":"2024-05-14T11:18:58.193384+0000","last_undegraded":"2024-05-14T11:18:58.193384+0000","last_fullsized":"2024-05-14T11:18:58.193384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:21.191213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932117+0000","last_change":"2024-05-14T11:18:56.921612+0000","last_active":"2024-05-14T11:18:57.932117+0000","last_peered":"2024-05-14T11:18:57.932117+0000","last_clean":"2024-05-14T11:18:57.932117+0000","last_became_active":"2024-05-14T11:18:56.921353+0000","last_became_peered":"2024-05-14T11:18:56.921353+0000","last_unstale":"2024-05-14T11:18:57.932117+0000","last_undegraded":"2024-05-14T11:18:57.932117+0000","last_fullsized":"2024-05-14T11:18:57.932117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:11.225500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193575+0000","last_change":"2024-05-14T11:18:55.937880+0000","last_active":"2024-05-14T11:18:58.193575+0000","last_peered":"2024-05-14T11:18:58.193575+0000","last_clean":"2024-05-14T11:18:58.193575+0000","last_became_active":"2024-05-14T11:18:55.937571+0000","last_became_peered":"2024-05-14T11:18:55.937571+0000","last_unstale":"2024-05-14T11:18:58.193575+0000","last_undegraded":"2024-05-14T11:18:58.193575+0000","last_fullsized":"2024-05-14T11:18:58.193575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:54.430892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932403+0000","last_change":"2024-05-14T11:18:56.932003+0000","last_active":"2024-05-14T11:18:57.932403+0000","last_peered":"2024-05-14T11:18:57.932403+0000","last_clean":"2024-05-14T11:18:57.932403+0000","last_became_active":"2024-05-14T11:18:56.931692+0000","last_became_peered":"2024-05-14T11:18:56.931692+0000","last_unstale":"2024-05-14T11:18:57.932403+0000","last_undegraded":"2024-05-14T11:18:57.932403+0000","last_fullsized":"2024-05-14T11:18:57.932403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:38.874693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932025+0000","last_change":"2024-05-14T11:18:55.935420+0000","last_active":"2024-05-14T11:18:57.932025+0000","last_peered":"2024-05-14T11:18:57.932025+0000","last_clean":"2024-05-14T11:18:57.932025+0000","last_became_active":"2024-05-14T11:18:55.935035+0000","last_became_peered":"2024-05-14T11:18:55.935035+0000","last_unstale":"2024-05-14T11:18:57.932025+0000","last_undegraded":"2024-05-14T11:18:57.932025+0000","last_fullsized":"2024-05-14T11:18:57.932025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:51.591243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192432+0000","last_change":"2024-05-14T11:18:56.930237+0000","last_active":"2024-05-14T11:18:58.192432+0000","last_peered":"2024-05-14T11:18:58.192432+0000","last_clean":"2024-05-14T11:18:58.192432+0000","last_became_active":"2024-05-14T11:18:56.929955+0000","last_became_peered":"2024-05-14T11:18:56.929955+0000","last_unstale":"2024-05-14T11:18:58.192432+0000","last_undegraded":"2024-05-14T11:18:58.192432+0000","last_fullsized":"2024-05-14T11:18:58.192432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:38.804025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192877+0000","last_change":"2024-05-14T11:18:55.928853+0000","last_active":"2024-05-14T11:18:58.192877+0000","last_peered":"2024-05-14T11:18:58.192877+0000","last_clean":"2024-05-14T11:18:58.192877+0000","last_became_active":"2024-05-14T11:18:55.928563+0000","last_became_peered":"2024-05-14T11:18:55.928563+0000","last_unstale":"2024-05-14T11:18:58.192877+0000","last_undegraded":"2024-05-14T11:18:58.192877+0000","last_fullsized":"2024-05-14T11:18:58.192877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:51.591243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934773+0000","last_change":"2024-05-14T11:18:56.944065+0000","last_active":"2024-05-14T11:18:57.934773+0000","last_peered":"2024-05-14T11:18:57.934773+0000","last_clean":"2024-05-14T11:18:57.934773+0000","last_became_active":"2024-05-14T11:18:56.943727+0000","last_became_peered":"2024-05-14T11:18:56.943727+0000","last_unstale":"2024-05-14T11:18:57.934773+0000","last_undegraded":"2024-05-14T11:18:57.934773+0000","last_fullsized":"2024-05-14T11:18:57.934773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:18.885017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934814+0000","last_change":"2024-05-14T11:18:55.938239+0000","last_active":"2024-05-14T11:18:57.934814+0000","last_peered":"2024-05-14T11:18:57.934814+0000","last_clean":"2024-05-14T11:18:57.934814+0000","last_became_active":"2024-05-14T11:18:55.937953+0000","last_became_peered":"2024-05-14T11:18:55.937953+0000","last_unstale":"2024-05-14T11:18:57.934814+0000","last_undegraded":"2024-05-14T11:18:57.934814+0000","last_fullsized":"2024-05-14T11:18:57.934814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:59.724825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931622+0000","last_change":"2024-05-14T11:18:56.921431+0000","last_active":"2024-05-14T11:18:57.931622+0000","last_peered":"2024-05-14T11:18:57.931622+0000","last_clean":"2024-05-14T11:18:57.931622+0000","last_became_active":"2024-05-14T11:18:56.920835+0000","last_became_peered":"2024-05-14T11:18:56.920835+0000","last_unstale":"2024-05-14T11:18:57.931622+0000","last_undegraded":"2024-05-14T11:18:57.931622+0000","last_fullsized":"2024-05-14T11:18:57.931622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:59.274859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192266+0000","last_change":"2024-05-14T11:18:55.924405+0000","last_active":"2024-05-14T11:18:58.192266+0000","last_peered":"2024-05-14T11:18:58.192266+0000","last_clean":"2024-05-14T11:18:58.192266+0000","last_became_active":"2024-05-14T11:18:55.924050+0000","last_became_peered":"2024-05-14T11:18:55.924050+0000","last_unstale":"2024-05-14T11:18:58.192266+0000","last_undegraded":"2024-05-14T11:18:58.192266+0000","last_fullsized":"2024-05-14T11:18:58.192266+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:31.939814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193150+0000","last_change":"2024-05-14T11:18:56.920715+0000","last_active":"2024-05-14T11:18:58.193150+0000","last_peered":"2024-05-14T11:18:58.193150+0000","last_clean":"2024-05-14T11:18:58.193150+0000","last_became_active":"2024-05-14T11:18:56.919534+0000","last_became_peered":"2024-05-14T11:18:56.919534+0000","last_unstale":"2024-05-14T11:18:58.193150+0000","last_undegraded":"2024-05-14T11:18:58.193150+0000","last_fullsized":"2024-05-14T11:18:58.193150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:52.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935209+0000","last_change":"2024-05-14T11:18:55.931519+0000","last_active":"2024-05-14T11:18:57.935209+0000","last_peered":"2024-05-14T11:18:57.935209+0000","last_clean":"2024-05-14T11:18:57.935209+0000","last_became_active":"2024-05-14T11:18:55.931047+0000","last_became_peered":"2024-05-14T11:18:55.931047+0000","last_unstale":"2024-05-14T11:18:57.935209+0000","last_undegraded":"2024-05-14T11:18:57.935209+0000","last_fullsized":"2024-05-14T11:18:57.935209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:01.400191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937236+0000","last_change":"2024-05-14T11:18:56.932499+0000","last_active":"2024-05-14T11:18:57.937236+0000","last_peered":"2024-05-14T11:18:57.937236+0000","last_clean":"2024-05-14T11:18:57.937236+0000","last_became_active":"2024-05-14T11:18:56.932184+0000","last_became_peered":"2024-05-14T11:18:56.932184+0000","last_unstale":"2024-05-14T11:18:57.937236+0000","last_undegraded":"2024-05-14T11:18:57.937236+0000","last_fullsized":"2024-05-14T11:18:57.937236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:17.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935232+0000","last_change":"2024-05-14T11:18:55.931003+0000","last_active":"2024-05-14T11:18:57.935232+0000","last_peered":"2024-05-14T11:18:57.935232+0000","last_clean":"2024-05-14T11:18:57.935232+0000","last_became_active":"2024-05-14T11:18:55.930685+0000","last_became_peered":"2024-05-14T11:18:55.930685+0000","last_unstale":"2024-05-14T11:18:57.935232+0000","last_undegraded":"2024-05-14T11:18:57.935232+0000","last_fullsized":"2024-05-14T11:18:57.935232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:46.454795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935086+0000","last_change":"2024-05-14T11:18:56.940401+0000","last_active":"2024-05-14T11:18:57.935086+0000","last_peered":"2024-05-14T11:18:57.935086+0000","last_clean":"2024-05-14T11:18:57.935086+0000","last_became_active":"2024-05-14T11:18:56.938765+0000","last_became_peered":"2024-05-14T11:18:56.938765+0000","last_unstale":"2024-05-14T11:18:57.935086+0000","last_undegraded":"2024-05-14T11:18:57.935086+0000","last_fullsized":"2024-05-14T11:18:57.935086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:41:29.460299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193416+0000","last_change":"2024-05-14T11:18:55.932175+0000","last_active":"2024-05-14T11:18:58.193416+0000","last_peered":"2024-05-14T11:18:58.193416+0000","last_clean":"2024-05-14T11:18:58.193416+0000","last_became_active":"2024-05-14T11:18:55.931873+0000","last_became_peered":"2024-05-14T11:18:55.931873+0000","last_unstale":"2024-05-14T11:18:58.193416+0000","last_undegraded":"2024-05-14T11:18:58.193416+0000","last_fullsized":"2024-05-14T11:18:58.193416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:52.322782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932548+0000","last_change":"2024-05-14T11:18:56.932030+0000","last_active":"2024-05-14T11:18:57.932548+0000","last_peered":"2024-05-14T11:18:57.932548+0000","last_clean":"2024-05-14T11:18:57.932548+0000","last_became_active":"2024-05-14T11:18:56.931709+0000","last_became_peered":"2024-05-14T11:18:56.931709+0000","last_unstale":"2024-05-14T11:18:57.932548+0000","last_undegraded":"2024-05-14T11:18:57.932548+0000","last_fullsized":"2024-05-14T11:18:57.932548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:03.082658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192796+0000","last_change":"2024-05-14T11:18:55.933868+0000","last_active":"2024-05-14T11:18:58.192796+0000","last_peered":"2024-05-14T11:18:58.192796+0000","last_clean":"2024-05-14T11:18:58.192796+0000","last_became_active":"2024-05-14T11:18:55.933556+0000","last_became_peered":"2024-05-14T11:18:55.933556+0000","last_unstale":"2024-05-14T11:18:58.192796+0000","last_undegraded":"2024-05-14T11:18:58.192796+0000","last_fullsized":"2024-05-14T11:18:58.192796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:10.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930847+0000","last_change":"2024-05-14T11:18:56.937480+0000","last_active":"2024-05-14T11:18:57.930847+0000","last_peered":"2024-05-14T11:18:57.930847+0000","last_clean":"2024-05-14T11:18:57.930847+0000","last_became_active":"2024-05-14T11:18:56.937141+0000","last_became_peered":"2024-05-14T11:18:56.937141+0000","last_unstale":"2024-05-14T11:18:57.930847+0000","last_undegraded":"2024-05-14T11:18:57.930847+0000","last_fullsized":"2024-05-14T11:18:57.930847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:05.923070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193714+0000","last_change":"2024-05-14T11:18:55.926593+0000","last_active":"2024-05-14T11:18:58.193714+0000","last_peered":"2024-05-14T11:18:58.193714+0000","last_clean":"2024-05-14T11:18:58.193714+0000","last_became_active":"2024-05-14T11:18:55.926285+0000","last_became_peered":"2024-05-14T11:18:55.926285+0000","last_unstale":"2024-05-14T11:18:58.193714+0000","last_undegraded":"2024-05-14T11:18:58.193714+0000","last_fullsized":"2024-05-14T11:18:58.193714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:46.454795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930899+0000","last_change":"2024-05-14T11:18:56.935536+0000","last_active":"2024-05-14T11:18:57.930899+0000","last_peered":"2024-05-14T11:18:57.930899+0000","last_clean":"2024-05-14T11:18:57.930899+0000","last_became_active":"2024-05-14T11:18:56.935218+0000","last_became_peered":"2024-05-14T11:18:56.935218+0000","last_unstale":"2024-05-14T11:18:57.930899+0000","last_undegraded":"2024-05-14T11:18:57.930899+0000","last_fullsized":"2024-05-14T11:18:57.930899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:05.923070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931917+0000","last_change":"2024-05-14T11:18:55.937650+0000","last_active":"2024-05-14T11:18:57.931917+0000","last_peered":"2024-05-14T11:18:57.931917+0000","last_clean":"2024-05-14T11:18:57.931917+0000","last_became_active":"2024-05-14T11:18:55.937251+0000","last_became_peered":"2024-05-14T11:18:55.937251+0000","last_unstale":"2024-05-14T11:18:57.931917+0000","last_undegraded":"2024-05-14T11:18:57.931917+0000","last_fullsized":"2024-05-14T11:18:57.931917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:55.613451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192867+0000","last_change":"2024-05-14T11:18:56.933546+0000","last_active":"2024-05-14T11:18:58.192867+0000","last_peered":"2024-05-14T11:18:58.192867+0000","last_clean":"2024-05-14T11:18:58.192867+0000","last_became_active":"2024-05-14T11:18:56.933204+0000","last_became_peered":"2024-05-14T11:18:56.933204+0000","last_unstale":"2024-05-14T11:18:58.192867+0000","last_undegraded":"2024-05-14T11:18:58.192867+0000","last_fullsized":"2024-05-14T11:18:58.192867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:42:00.732777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931714+0000","last_change":"2024-05-14T11:18:55.925879+0000","last_active":"2024-05-14T11:18:57.931714+0000","last_peered":"2024-05-14T11:18:57.931714+0000","last_clean":"2024-05-14T11:18:57.931714+0000","last_became_active":"2024-05-14T11:18:55.925042+0000","last_became_peered":"2024-05-14T11:18:55.925042+0000","last_unstale":"2024-05-14T11:18:57.931714+0000","last_undegraded":"2024-05-14T11:18:57.931714+0000","last_fullsized":"2024-05-14T11:18:57.931714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:02.347672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937123+0000","last_change":"2024-05-14T11:18:56.938790+0000","last_active":"2024-05-14T11:18:57.937123+0000","last_peered":"2024-05-14T11:18:57.937123+0000","last_clean":"2024-05-14T11:18:57.937123+0000","last_became_active":"2024-05-14T11:18:56.938311+0000","last_became_peered":"2024-05-14T11:18:56.938311+0000","last_unstale":"2024-05-14T11:18:57.937123+0000","last_undegraded":"2024-05-14T11:18:57.937123+0000","last_fullsized":"2024-05-14T11:18:57.937123+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:24.714908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192273+0000","last_change":"2024-05-14T11:18:55.928062+0000","last_active":"2024-05-14T11:18:58.192273+0000","last_peered":"2024-05-14T11:18:58.192273+0000","last_clean":"2024-05-14T11:18:58.192273+0000","last_became_active":"2024-05-14T11:18:55.927658+0000","last_became_peered":"2024-05-14T11:18:55.927658+0000","last_unstale":"2024-05-14T11:18:58.192273+0000","last_undegraded":"2024-05-14T11:18:58.192273+0000","last_fullsized":"2024-05-14T11:18:58.192273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.827500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932117+0000","last_change":"2024-05-14T11:18:56.936635+0000","last_active":"2024-05-14T11:18:57.932117+0000","last_peered":"2024-05-14T11:18:57.932117+0000","last_clean":"2024-05-14T11:18:57.932117+0000","last_became_active":"2024-05-14T11:18:56.936292+0000","last_became_peered":"2024-05-14T11:18:56.936292+0000","last_unstale":"2024-05-14T11:18:57.932117+0000","last_undegraded":"2024-05-14T11:18:57.932117+0000","last_fullsized":"2024-05-14T11:18:57.932117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:51.644710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931499+0000","last_change":"2024-05-14T11:18:55.923385+0000","last_active":"2024-05-14T11:18:57.931499+0000","last_peered":"2024-05-14T11:18:57.931499+0000","last_clean":"2024-05-14T11:18:57.931499+0000","last_became_active":"2024-05-14T11:18:55.923063+0000","last_became_peered":"2024-05-14T11:18:55.923063+0000","last_unstale":"2024-05-14T11:18:57.931499+0000","last_undegraded":"2024-05-14T11:18:57.931499+0000","last_fullsized":"2024-05-14T11:18:57.931499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:30:01.324648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192937+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:18:58.192937+0000","last_peered":"2024-05-14T11:18:58.192937+0000","last_clean":"2024-05-14T11:18:58.192937+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:18:58.192937+0000","last_undegraded":"2024-05-14T11:18:58.192937+0000","last_fullsized":"2024-05-14T11:18:58.192937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":916220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":326688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":325076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":325076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":327452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":915444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:19:05.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:19:05.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 Processor -- start 2024-05-14T11:19:05.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 -- start start 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7a8c0af380 0x7f7a8c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c0d1fc0 con 0x7f7a8c156fa0 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8c0706f0 con 0x7f7a8c0af380 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a92e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c1573b0 con 0x7f7a8c0d1670 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a90e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7a8c0af380 0x7f7a8c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a90e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7a8c0af380 0x7f7a8c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39906/0 (socket says 172.21.15.116:39906) 2024-05-14T11:19:05.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a8be00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7a8c156fa0 0x7f7a8c06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59002/0 (socket says 172.21.15.116:59002) 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.807+0000 7f7a90e00640 1 -- 172.21.15.116:0/3345381814 learned_addr learned my addr 172.21.15.116:0/3345381814 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a8b400640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a90e00640 1 -- 172.21.15.116:0/3345381814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 msgr2=0x7f7a8c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a90e00640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a90e00640 1 -- 172.21.15.116:0/3345381814 >> v1:172.21.15.116:6789/0 conn(0x7f7a8c156fa0 legacy=0x7f7a8c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a90e00640 1 -- 172.21.15.116:0/3345381814 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c160510 con 0x7f7a8c0af380 2024-05-14T11:19:05.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a8aa00640 1 -- 172.21.15.116:0/3345381814 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325319481 0 0) 0x7f7a7c0014a0 con 0x7f7a8c156fa0 2024-05-14T11:19:05.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a90e00640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c0af380 0x7f7a8c0af790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009870 tx=0x7f7a8401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad24cb9ee0d7f7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:05.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a8aa00640 1 -- 172.21.15.116:0/3345381814 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840ab650 con 0x7f7a8c0af380 2024-05-14T11:19:05.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a8aa00640 1 -- 172.21.15.116:0/3345381814 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a840b0070 con 0x7f7a8c0af380 2024-05-14T11:19:05.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.811+0000 7f7a8aa00640 1 -- 172.21.15.116:0/3345381814 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840aba10 con 0x7f7a8c0af380 2024-05-14T11:19:05.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- 172.21.15.116:0/3345381814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c0af380 msgr2=0x7f7a8c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c0af380 0x7f7a8c0af790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009870 tx=0x7f7a8401c560 comp rx=0 tx=0).stop 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- 172.21.15.116:0/3345381814 shutdown_connections 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c0af380 0x7f7a8c0af790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 --2- 172.21.15.116:0/3345381814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- 172.21.15.116:0/3345381814 >> 172.21.15.116:0/3345381814 conn(0x7f7a8c0b4620 msgr2=0x7f7a8c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- 172.21.15.116:0/3345381814 shutdown_connections 2024-05-14T11:19:05.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- 172.21.15.116:0/3345381814 wait complete. 2024-05-14T11:19:05.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 Processor -- start 2024-05-14T11:19:05.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.815+0000 7f7a92e00640 1 -- start start 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a8c0af380 0x7f7a8c1f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b0660 con 0x7f7a8c156fa0 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0b04e0 con 0x7f7a8c0af380 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c169810 con 0x7f7a8c0d1670 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39910/0 (socket says 172.21.15.116:39910) 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 -- 172.21.15.116:0/2359644056 learned_addr learned my addr 172.21.15.116:0/2359644056 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a8b400640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a8be00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a8c0af380 0x7f7a8c1f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 -- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 msgr2=0x7f7a8c14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 -- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a8c0af380 msgr2=0x7f7a8c1f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a8c0af380 0x7f7a8c1f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 -- 172.21.15.116:0/2359644056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1fa200 con 0x7f7a8c156fa0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a90e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a8401a190 tx=0x7f7a8401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840be5f0 con 0x7f7a8c156fa0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a840be790 con 0x7f7a8c156fa0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840beb80 con 0x7f7a8c156fa0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c0cdd50 con 0x7f7a8c156fa0 2024-05-14T11:19:05.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c09ef90 con 0x7f7a8c156fa0 2024-05-14T11:19:05.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c06a9c0 con 0x7f7a8c156fa0 2024-05-14T11:19:05.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.819+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a840b43f0 con 0x7f7a8c156fa0 2024-05-14T11:19:05.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.823+0000 7f7a83e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 0x7f7a6c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:05.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.823+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f7a84129f50 con 0x7f7a8c156fa0 2024-05-14T11:19:05.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.823+0000 7f7a8be00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 0x7f7a6c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:05.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.823+0000 7f7a8be00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 0x7f7a6c071940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0012f0 tx=0x7f7a7c005ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:05.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.827+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7a8c1fa200 con 0x7f7a8c156fa0 2024-05-14T11:19:05.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.979+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a8c0b2f30 con 0x7f7a6c06f430 2024-05-14T11:19:05.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.987+0000 7f7a83e00640 1 -- 172.21.15.116:0/2359644056 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7f7a8c0b2f30 con 0x7f7a6c06f430 2024-05-14T11:19:05.993 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:05.995 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 msgr2=0x7f7a6c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 0x7f7a6c071940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0012f0 tx=0x7f7a7c005ec0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 msgr2=0x7f7a8c15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a8401a190 tx=0x7f7a8401ada0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 shutdown_connections 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7a6c06f430 0x7f7a6c071940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c156fa0 0x7f7a8c15b950 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7a8c0d1670 0x7f7a8c14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 --2- 172.21.15.116:0/2359644056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a8c0af380 0x7f7a8c1f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:05.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 >> 172.21.15.116:0/2359644056 conn(0x7f7a8c0b4620 msgr2=0x7f7a8c15e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:05.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 shutdown_connections 2024-05-14T11:19:05.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:05.991+0000 7f7a92e00640 1 -- 172.21.15.116:0/2359644056 wait complete. 2024-05-14T11:19:06.013 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-05-14T11:19:04.400569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243880,"kb_used_data":6888,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730840,"statfs":{"total":773094113280,"available":772844380160,"internally_reserved":0,"allocated":7053312,"data_stored":3786164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002107"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193193+0000","last_change":"2024-05-14T11:18:56.951042+0000","last_active":"2024-05-14T11:18:58.193193+0000","last_peered":"2024-05-14T11:18:58.193193+0000","last_clean":"2024-05-14T11:18:58.193193+0000","last_became_active":"2024-05-14T11:18:56.943065+0000","last_became_peered":"2024-05-14T11:18:56.943065+0000","last_unstale":"2024-05-14T11:18:58.193193+0000","last_undegraded":"2024-05-14T11:18:58.193193+0000","last_fullsized":"2024-05-14T11:18:58.193193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:23.983706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193279+0000","last_change":"2024-05-14T11:18:56.945310+0000","last_active":"2024-05-14T11:18:58.193279+0000","last_peered":"2024-05-14T11:18:58.193279+0000","last_clean":"2024-05-14T11:18:58.193279+0000","last_became_active":"2024-05-14T11:18:56.944979+0000","last_became_peered":"2024-05-14T11:18:56.944979+0000","last_unstale":"2024-05-14T11:18:58.193279+0000","last_undegraded":"2024-05-14T11:18:58.193279+0000","last_fullsized":"2024-05-14T11:18:58.193279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:55.438844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193556+0000","last_change":"2024-05-14T11:18:56.943118+0000","last_active":"2024-05-14T11:18:58.193556+0000","last_peered":"2024-05-14T11:18:58.193556+0000","last_clean":"2024-05-14T11:18:58.193556+0000","last_became_active":"2024-05-14T11:18:56.942810+0000","last_became_peered":"2024-05-14T11:18:56.942810+0000","last_unstale":"2024-05-14T11:18:58.193556+0000","last_undegraded":"2024-05-14T11:18:58.193556+0000","last_fullsized":"2024-05-14T11:18:58.193556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:41:29.460299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931409+0000","last_change":"2024-05-14T11:18:56.921038+0000","last_active":"2024-05-14T11:18:57.931409+0000","last_peered":"2024-05-14T11:18:57.931409+0000","last_clean":"2024-05-14T11:18:57.931409+0000","last_became_active":"2024-05-14T11:18:56.920553+0000","last_became_peered":"2024-05-14T11:18:56.920553+0000","last_unstale":"2024-05-14T11:18:57.931409+0000","last_undegraded":"2024-05-14T11:18:57.931409+0000","last_fullsized":"2024-05-14T11:18:57.931409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:28.737096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932170+0000","last_change":"2024-05-14T11:18:56.922653+0000","last_active":"2024-05-14T11:18:57.932170+0000","last_peered":"2024-05-14T11:18:57.932170+0000","last_clean":"2024-05-14T11:18:57.932170+0000","last_became_active":"2024-05-14T11:18:56.922374+0000","last_became_peered":"2024-05-14T11:18:56.922374+0000","last_unstale":"2024-05-14T11:18:57.932170+0000","last_undegraded":"2024-05-14T11:18:57.932170+0000","last_fullsized":"2024-05-14T11:18:57.932170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:01:50.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192754+0000","last_change":"2024-05-14T11:18:56.940719+0000","last_active":"2024-05-14T11:18:58.192754+0000","last_peered":"2024-05-14T11:18:58.192754+0000","last_clean":"2024-05-14T11:18:58.192754+0000","last_became_active":"2024-05-14T11:18:56.940415+0000","last_became_peered":"2024-05-14T11:18:56.940415+0000","last_unstale":"2024-05-14T11:18:58.192754+0000","last_undegraded":"2024-05-14T11:18:58.192754+0000","last_fullsized":"2024-05-14T11:18:58.192754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:39.321114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937202+0000","last_change":"2024-05-14T11:18:56.936013+0000","last_active":"2024-05-14T11:18:57.937202+0000","last_peered":"2024-05-14T11:18:57.937202+0000","last_clean":"2024-05-14T11:18:57.937202+0000","last_became_active":"2024-05-14T11:18:56.935678+0000","last_became_peered":"2024-05-14T11:18:56.935678+0000","last_unstale":"2024-05-14T11:18:57.937202+0000","last_undegraded":"2024-05-14T11:18:57.937202+0000","last_fullsized":"2024-05-14T11:18:57.937202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:49.249969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934733+0000","last_change":"2024-05-14T11:18:56.945673+0000","last_active":"2024-05-14T11:18:57.934733+0000","last_peered":"2024-05-14T11:18:57.934733+0000","last_clean":"2024-05-14T11:18:57.934733+0000","last_became_active":"2024-05-14T11:18:56.945394+0000","last_became_peered":"2024-05-14T11:18:56.945394+0000","last_unstale":"2024-05-14T11:18:57.934733+0000","last_undegraded":"2024-05-14T11:18:57.934733+0000","last_fullsized":"2024-05-14T11:18:57.934733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:52.599195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193824+0000","last_change":"2024-05-14T11:18:56.936632+0000","last_active":"2024-05-14T11:18:58.193824+0000","last_peered":"2024-05-14T11:18:58.193824+0000","last_clean":"2024-05-14T11:18:58.193824+0000","last_became_active":"2024-05-14T11:18:56.936305+0000","last_became_peered":"2024-05-14T11:18:56.936305+0000","last_unstale":"2024-05-14T11:18:58.193824+0000","last_undegraded":"2024-05-14T11:18:58.193824+0000","last_fullsized":"2024-05-14T11:18:58.193824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193647+0000","last_change":"2024-05-14T11:18:56.952775+0000","last_active":"2024-05-14T11:18:58.193647+0000","last_peered":"2024-05-14T11:18:58.193647+0000","last_clean":"2024-05-14T11:18:58.193647+0000","last_became_active":"2024-05-14T11:18:56.952461+0000","last_became_peered":"2024-05-14T11:18:56.952461+0000","last_unstale":"2024-05-14T11:18:58.193647+0000","last_undegraded":"2024-05-14T11:18:58.193647+0000","last_fullsized":"2024-05-14T11:18:58.193647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:43.057516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932011+0000","last_change":"2024-05-14T11:18:56.939518+0000","last_active":"2024-05-14T11:18:57.932011+0000","last_peered":"2024-05-14T11:18:57.932011+0000","last_clean":"2024-05-14T11:18:57.932011+0000","last_became_active":"2024-05-14T11:18:56.938957+0000","last_became_peered":"2024-05-14T11:18:56.938957+0000","last_unstale":"2024-05-14T11:18:57.932011+0000","last_undegraded":"2024-05-14T11:18:57.932011+0000","last_fullsized":"2024-05-14T11:18:57.932011+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:49.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935039+0000","last_change":"2024-05-14T11:18:56.941581+0000","last_active":"2024-05-14T11:18:57.935039+0000","last_peered":"2024-05-14T11:18:57.935039+0000","last_clean":"2024-05-14T11:18:57.935039+0000","last_became_active":"2024-05-14T11:18:56.941256+0000","last_became_peered":"2024-05-14T11:18:56.941256+0000","last_unstale":"2024-05-14T11:18:57.935039+0000","last_undegraded":"2024-05-14T11:18:57.935039+0000","last_fullsized":"2024-05-14T11:18:57.935039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:43:49.923077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934763+0000","last_change":"2024-05-14T11:18:56.937159+0000","last_active":"2024-05-14T11:18:57.934763+0000","last_peered":"2024-05-14T11:18:57.934763+0000","last_clean":"2024-05-14T11:18:57.934763+0000","last_became_active":"2024-05-14T11:18:56.936771+0000","last_became_peered":"2024-05-14T11:18:56.936771+0000","last_unstale":"2024-05-14T11:18:57.934763+0000","last_undegraded":"2024-05-14T11:18:57.934763+0000","last_fullsized":"2024-05-14T11:18:57.934763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:56.621403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193465+0000","last_change":"2024-05-14T11:18:56.945522+0000","last_active":"2024-05-14T11:18:58.193465+0000","last_peered":"2024-05-14T11:18:58.193465+0000","last_clean":"2024-05-14T11:18:58.193465+0000","last_became_active":"2024-05-14T11:18:56.943591+0000","last_became_peered":"2024-05-14T11:18:56.943591+0000","last_unstale":"2024-05-14T11:18:58.193465+0000","last_undegraded":"2024-05-14T11:18:58.193465+0000","last_fullsized":"2024-05-14T11:18:58.193465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:03.355624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193471+0000","last_change":"2024-05-14T11:18:56.945063+0000","last_active":"2024-05-14T11:18:58.193471+0000","last_peered":"2024-05-14T11:18:58.193471+0000","last_clean":"2024-05-14T11:18:58.193471+0000","last_became_active":"2024-05-14T11:18:56.944326+0000","last_became_peered":"2024-05-14T11:18:56.944326+0000","last_unstale":"2024-05-14T11:18:58.193471+0000","last_undegraded":"2024-05-14T11:18:58.193471+0000","last_fullsized":"2024-05-14T11:18:58.193471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:43:49.923077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937049+0000","last_change":"2024-05-14T11:18:56.939395+0000","last_active":"2024-05-14T11:18:57.937049+0000","last_peered":"2024-05-14T11:18:57.937049+0000","last_clean":"2024-05-14T11:18:57.937049+0000","last_became_active":"2024-05-14T11:18:56.939037+0000","last_became_peered":"2024-05-14T11:18:56.939037+0000","last_unstale":"2024-05-14T11:18:57.937049+0000","last_undegraded":"2024-05-14T11:18:57.937049+0000","last_fullsized":"2024-05-14T11:18:57.937049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:16:40.998989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193245+0000","last_change":"2024-05-14T11:18:56.942671+0000","last_active":"2024-05-14T11:18:58.193245+0000","last_peered":"2024-05-14T11:18:58.193245+0000","last_clean":"2024-05-14T11:18:58.193245+0000","last_became_active":"2024-05-14T11:18:56.934867+0000","last_became_peered":"2024-05-14T11:18:56.934867+0000","last_unstale":"2024-05-14T11:18:58.193245+0000","last_undegraded":"2024-05-14T11:18:58.193245+0000","last_fullsized":"2024-05-14T11:18:58.193245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:16.117129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931934+0000","last_change":"2024-05-14T11:18:56.927997+0000","last_active":"2024-05-14T11:18:57.931934+0000","last_peered":"2024-05-14T11:18:57.931934+0000","last_clean":"2024-05-14T11:18:57.931934+0000","last_became_active":"2024-05-14T11:18:56.927541+0000","last_became_peered":"2024-05-14T11:18:56.927541+0000","last_unstale":"2024-05-14T11:18:57.931934+0000","last_undegraded":"2024-05-14T11:18:57.931934+0000","last_fullsized":"2024-05-14T11:18:57.931934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:59.274859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931855+0000","last_change":"2024-05-14T11:18:56.929995+0000","last_active":"2024-05-14T11:18:57.931855+0000","last_peered":"2024-05-14T11:18:57.931855+0000","last_clean":"2024-05-14T11:18:57.931855+0000","last_became_active":"2024-05-14T11:18:56.929715+0000","last_became_peered":"2024-05-14T11:18:56.929715+0000","last_unstale":"2024-05-14T11:18:57.931855+0000","last_undegraded":"2024-05-14T11:18:57.931855+0000","last_fullsized":"2024-05-14T11:18:57.931855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:49.032619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932152+0000","last_change":"2024-05-14T11:18:56.928939+0000","last_active":"2024-05-14T11:18:57.932152+0000","last_peered":"2024-05-14T11:18:57.932152+0000","last_clean":"2024-05-14T11:18:57.932152+0000","last_became_active":"2024-05-14T11:18:56.928643+0000","last_became_peered":"2024-05-14T11:18:56.928643+0000","last_unstale":"2024-05-14T11:18:57.932152+0000","last_undegraded":"2024-05-14T11:18:57.932152+0000","last_fullsized":"2024-05-14T11:18:57.932152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:52.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934915+0000","last_change":"2024-05-14T11:18:56.947223+0000","last_active":"2024-05-14T11:18:57.934915+0000","last_peered":"2024-05-14T11:18:57.934915+0000","last_clean":"2024-05-14T11:18:57.934915+0000","last_became_active":"2024-05-14T11:18:56.946886+0000","last_became_peered":"2024-05-14T11:18:56.946886+0000","last_unstale":"2024-05-14T11:18:57.934915+0000","last_undegraded":"2024-05-14T11:18:57.934915+0000","last_fullsized":"2024-05-14T11:18:57.934915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:03.355624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931859+0000","last_change":"2024-05-14T11:18:56.941547+0000","last_active":"2024-05-14T11:18:57.931859+0000","last_peered":"2024-05-14T11:18:57.931859+0000","last_clean":"2024-05-14T11:18:57.931859+0000","last_became_active":"2024-05-14T11:18:56.941165+0000","last_became_peered":"2024-05-14T11:18:56.941165+0000","last_unstale":"2024-05-14T11:18:57.931859+0000","last_undegraded":"2024-05-14T11:18:57.931859+0000","last_fullsized":"2024-05-14T11:18:57.931859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:38.804025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931653+0000","last_change":"2024-05-14T11:18:56.947149+0000","last_active":"2024-05-14T11:18:57.931653+0000","last_peered":"2024-05-14T11:18:57.931653+0000","last_clean":"2024-05-14T11:18:57.931653+0000","last_became_active":"2024-05-14T11:18:56.946809+0000","last_became_peered":"2024-05-14T11:18:56.946809+0000","last_unstale":"2024-05-14T11:18:57.931653+0000","last_undegraded":"2024-05-14T11:18:57.931653+0000","last_fullsized":"2024-05-14T11:18:57.931653+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193237+0000","last_change":"2024-05-14T11:18:56.942473+0000","last_active":"2024-05-14T11:18:58.193237+0000","last_peered":"2024-05-14T11:18:58.193237+0000","last_clean":"2024-05-14T11:18:58.193237+0000","last_became_active":"2024-05-14T11:18:56.942165+0000","last_became_peered":"2024-05-14T11:18:56.942165+0000","last_unstale":"2024-05-14T11:18:58.193237+0000","last_undegraded":"2024-05-14T11:18:58.193237+0000","last_fullsized":"2024-05-14T11:18:58.193237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:18.885017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935137+0000","last_change":"2024-05-14T11:18:56.949711+0000","last_active":"2024-05-14T11:18:57.935137+0000","last_peered":"2024-05-14T11:18:57.935137+0000","last_clean":"2024-05-14T11:18:57.935137+0000","last_became_active":"2024-05-14T11:18:56.949436+0000","last_became_peered":"2024-05-14T11:18:56.949436+0000","last_unstale":"2024-05-14T11:18:57.935137+0000","last_undegraded":"2024-05-14T11:18:57.935137+0000","last_fullsized":"2024-05-14T11:18:57.935137+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:35:06.732283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935404+0000","last_change":"2024-05-14T11:18:56.949946+0000","last_active":"2024-05-14T11:18:57.935404+0000","last_peered":"2024-05-14T11:18:57.935404+0000","last_clean":"2024-05-14T11:18:57.935404+0000","last_became_active":"2024-05-14T11:18:56.948967+0000","last_became_peered":"2024-05-14T11:18:56.948967+0000","last_unstale":"2024-05-14T11:18:57.935404+0000","last_undegraded":"2024-05-14T11:18:57.935404+0000","last_fullsized":"2024-05-14T11:18:57.935404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:43.057516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932211+0000","last_change":"2024-05-14T11:18:56.926910+0000","last_active":"2024-05-14T11:18:57.932211+0000","last_peered":"2024-05-14T11:18:57.932211+0000","last_clean":"2024-05-14T11:18:57.932211+0000","last_became_active":"2024-05-14T11:18:56.926203+0000","last_became_peered":"2024-05-14T11:18:56.926203+0000","last_unstale":"2024-05-14T11:18:57.932211+0000","last_undegraded":"2024-05-14T11:18:57.932211+0000","last_fullsized":"2024-05-14T11:18:57.932211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:00:58.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192624+0000","last_change":"2024-05-14T11:18:56.935910+0000","last_active":"2024-05-14T11:18:58.192624+0000","last_peered":"2024-05-14T11:18:58.192624+0000","last_clean":"2024-05-14T11:18:58.192624+0000","last_became_active":"2024-05-14T11:18:56.935468+0000","last_became_peered":"2024-05-14T11:18:56.935468+0000","last_unstale":"2024-05-14T11:18:58.192624+0000","last_undegraded":"2024-05-14T11:18:58.192624+0000","last_fullsized":"2024-05-14T11:18:58.192624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:02.408142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193295+0000","last_change":"2024-05-14T11:18:56.936736+0000","last_active":"2024-05-14T11:18:58.193295+0000","last_peered":"2024-05-14T11:18:58.193295+0000","last_clean":"2024-05-14T11:18:58.193295+0000","last_became_active":"2024-05-14T11:18:56.936263+0000","last_became_peered":"2024-05-14T11:18:56.936263+0000","last_unstale":"2024-05-14T11:18:58.193295+0000","last_undegraded":"2024-05-14T11:18:58.193295+0000","last_fullsized":"2024-05-14T11:18:58.193295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:56.621403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934758+0000","last_change":"2024-05-14T11:18:56.942327+0000","last_active":"2024-05-14T11:18:57.934758+0000","last_peered":"2024-05-14T11:18:57.934758+0000","last_clean":"2024-05-14T11:18:57.934758+0000","last_became_active":"2024-05-14T11:18:56.941975+0000","last_became_peered":"2024-05-14T11:18:56.941975+0000","last_unstale":"2024-05-14T11:18:57.934758+0000","last_undegraded":"2024-05-14T11:18:57.934758+0000","last_fullsized":"2024-05-14T11:18:57.934758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:53.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930982+0000","last_change":"2024-05-14T11:18:56.923728+0000","last_active":"2024-05-14T11:18:57.930982+0000","last_peered":"2024-05-14T11:18:57.930982+0000","last_clean":"2024-05-14T11:18:57.930982+0000","last_became_active":"2024-05-14T11:18:56.923386+0000","last_became_peered":"2024-05-14T11:18:56.923386+0000","last_unstale":"2024-05-14T11:18:57.930982+0000","last_undegraded":"2024-05-14T11:18:57.930982+0000","last_fullsized":"2024-05-14T11:18:57.930982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:26.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192275+0000","last_change":"2024-05-14T11:18:56.930344+0000","last_active":"2024-05-14T11:18:58.192275+0000","last_peered":"2024-05-14T11:18:58.192275+0000","last_clean":"2024-05-14T11:18:58.192275+0000","last_became_active":"2024-05-14T11:18:56.930031+0000","last_became_peered":"2024-05-14T11:18:56.930031+0000","last_unstale":"2024-05-14T11:18:58.192275+0000","last_undegraded":"2024-05-14T11:18:58.192275+0000","last_fullsized":"2024-05-14T11:18:58.192275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:17:09.525199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931860+0000","last_change":"2024-05-14T11:18:56.937328+0000","last_active":"2024-05-14T11:18:57.931860+0000","last_peered":"2024-05-14T11:18:57.931860+0000","last_clean":"2024-05-14T11:18:57.931860+0000","last_became_active":"2024-05-14T11:18:56.937014+0000","last_became_peered":"2024-05-14T11:18:56.937014+0000","last_unstale":"2024-05-14T11:18:57.931860+0000","last_undegraded":"2024-05-14T11:18:57.931860+0000","last_fullsized":"2024-05-14T11:18:57.931860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:11.225500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193259+0000","last_change":"2024-05-14T11:18:55.929650+0000","last_active":"2024-05-14T11:18:58.193259+0000","last_peered":"2024-05-14T11:18:58.193259+0000","last_clean":"2024-05-14T11:18:58.193259+0000","last_became_active":"2024-05-14T11:18:55.929313+0000","last_became_peered":"2024-05-14T11:18:55.929313+0000","last_unstale":"2024-05-14T11:18:58.193259+0000","last_undegraded":"2024-05-14T11:18:58.193259+0000","last_fullsized":"2024-05-14T11:18:58.193259+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:50.636758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932432+0000","last_change":"2024-05-14T11:18:56.920196+0000","last_active":"2024-05-14T11:18:57.932432+0000","last_peered":"2024-05-14T11:18:57.932432+0000","last_clean":"2024-05-14T11:18:57.932432+0000","last_became_active":"2024-05-14T11:18:56.919798+0000","last_became_peered":"2024-05-14T11:18:56.919798+0000","last_unstale":"2024-05-14T11:18:57.932432+0000","last_undegraded":"2024-05-14T11:18:57.932432+0000","last_fullsized":"2024-05-14T11:18:57.932432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:01:50.431676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193449+0000","last_change":"2024-05-14T11:18:55.925360+0000","last_active":"2024-05-14T11:18:58.193449+0000","last_peered":"2024-05-14T11:18:58.193449+0000","last_clean":"2024-05-14T11:18:58.193449+0000","last_became_active":"2024-05-14T11:18:55.925002+0000","last_became_peered":"2024-05-14T11:18:55.925002+0000","last_unstale":"2024-05-14T11:18:58.193449+0000","last_undegraded":"2024-05-14T11:18:58.193449+0000","last_fullsized":"2024-05-14T11:18:58.193449+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:55.613451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193414+0000","last_change":"2024-05-14T11:18:56.930939+0000","last_active":"2024-05-14T11:18:58.193414+0000","last_peered":"2024-05-14T11:18:58.193414+0000","last_clean":"2024-05-14T11:18:58.193414+0000","last_became_active":"2024-05-14T11:18:56.929789+0000","last_became_peered":"2024-05-14T11:18:56.929789+0000","last_unstale":"2024-05-14T11:18:58.193414+0000","last_undegraded":"2024-05-14T11:18:58.193414+0000","last_fullsized":"2024-05-14T11:18:58.193414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.313987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193297+0000","last_change":"2024-05-14T11:18:55.929021+0000","last_active":"2024-05-14T11:18:58.193297+0000","last_peered":"2024-05-14T11:18:58.193297+0000","last_clean":"2024-05-14T11:18:58.193297+0000","last_became_active":"2024-05-14T11:18:55.928464+0000","last_became_peered":"2024-05-14T11:18:55.928464+0000","last_unstale":"2024-05-14T11:18:58.193297+0000","last_undegraded":"2024-05-14T11:18:58.193297+0000","last_fullsized":"2024-05-14T11:18:58.193297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:02.347672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193640+0000","last_change":"2024-05-14T11:18:56.952449+0000","last_active":"2024-05-14T11:18:58.193640+0000","last_peered":"2024-05-14T11:18:58.193640+0000","last_clean":"2024-05-14T11:18:58.193640+0000","last_became_active":"2024-05-14T11:18:56.952193+0000","last_became_peered":"2024-05-14T11:18:56.952193+0000","last_unstale":"2024-05-14T11:18:58.193640+0000","last_undegraded":"2024-05-14T11:18:58.193640+0000","last_fullsized":"2024-05-14T11:18:58.193640+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:35:06.732283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193096+0000","last_change":"2024-05-14T11:18:55.925238+0000","last_active":"2024-05-14T11:18:58.193096+0000","last_peered":"2024-05-14T11:18:58.193096+0000","last_clean":"2024-05-14T11:18:58.193096+0000","last_became_active":"2024-05-14T11:18:55.924925+0000","last_became_peered":"2024-05-14T11:18:55.924925+0000","last_unstale":"2024-05-14T11:18:58.193096+0000","last_undegraded":"2024-05-14T11:18:58.193096+0000","last_fullsized":"2024-05-14T11:18:58.193096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:17.877065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937290+0000","last_change":"2024-05-14T11:18:56.932288+0000","last_active":"2024-05-14T11:18:57.937290+0000","last_peered":"2024-05-14T11:18:57.937290+0000","last_clean":"2024-05-14T11:18:57.937290+0000","last_became_active":"2024-05-14T11:18:56.931965+0000","last_became_peered":"2024-05-14T11:18:56.931965+0000","last_unstale":"2024-05-14T11:18:57.937290+0000","last_undegraded":"2024-05-14T11:18:57.937290+0000","last_fullsized":"2024-05-14T11:18:57.937290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:07:38.386584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932001+0000","last_change":"2024-05-14T11:18:55.926997+0000","last_active":"2024-05-14T11:18:57.932001+0000","last_peered":"2024-05-14T11:18:57.932001+0000","last_clean":"2024-05-14T11:18:57.932001+0000","last_became_active":"2024-05-14T11:18:55.926691+0000","last_became_peered":"2024-05-14T11:18:55.926691+0000","last_unstale":"2024-05-14T11:18:57.932001+0000","last_undegraded":"2024-05-14T11:18:57.932001+0000","last_fullsized":"2024-05-14T11:18:57.932001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:31.939814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192562+0000","last_change":"2024-05-14T11:18:56.933970+0000","last_active":"2024-05-14T11:18:58.192562+0000","last_peered":"2024-05-14T11:18:58.192562+0000","last_clean":"2024-05-14T11:18:58.192562+0000","last_became_active":"2024-05-14T11:18:56.932978+0000","last_became_peered":"2024-05-14T11:18:56.932978+0000","last_unstale":"2024-05-14T11:18:58.192562+0000","last_undegraded":"2024-05-14T11:18:58.192562+0000","last_fullsized":"2024-05-14T11:18:58.192562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:38.874693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931812+0000","last_change":"2024-05-14T11:18:55.922225+0000","last_active":"2024-05-14T11:18:57.931812+0000","last_peered":"2024-05-14T11:18:57.931812+0000","last_clean":"2024-05-14T11:18:57.931812+0000","last_became_active":"2024-05-14T11:18:55.921862+0000","last_became_peered":"2024-05-14T11:18:55.921862+0000","last_unstale":"2024-05-14T11:18:57.931812+0000","last_undegraded":"2024-05-14T11:18:57.931812+0000","last_fullsized":"2024-05-14T11:18:57.931812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:52.322782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931413+0000","last_change":"2024-05-14T11:18:56.924715+0000","last_active":"2024-05-14T11:18:57.931413+0000","last_peered":"2024-05-14T11:18:57.931413+0000","last_clean":"2024-05-14T11:18:57.931413+0000","last_became_active":"2024-05-14T11:18:56.924390+0000","last_became_peered":"2024-05-14T11:18:56.924390+0000","last_unstale":"2024-05-14T11:18:57.931413+0000","last_undegraded":"2024-05-14T11:18:57.931413+0000","last_fullsized":"2024-05-14T11:18:57.931413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:26.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931648+0000","last_change":"2024-05-14T11:18:55.926111+0000","last_active":"2024-05-14T11:18:57.931648+0000","last_peered":"2024-05-14T11:18:57.931648+0000","last_clean":"2024-05-14T11:18:57.931648+0000","last_became_active":"2024-05-14T11:18:55.925459+0000","last_became_peered":"2024-05-14T11:18:55.925459+0000","last_unstale":"2024-05-14T11:18:57.931648+0000","last_undegraded":"2024-05-14T11:18:57.931648+0000","last_fullsized":"2024-05-14T11:18:57.931648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:17.877065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932087+0000","last_change":"2024-05-14T11:18:56.941960+0000","last_active":"2024-05-14T11:18:57.932087+0000","last_peered":"2024-05-14T11:18:57.932087+0000","last_clean":"2024-05-14T11:18:57.932087+0000","last_became_active":"2024-05-14T11:18:56.937300+0000","last_became_peered":"2024-05-14T11:18:56.937300+0000","last_unstale":"2024-05-14T11:18:57.932087+0000","last_undegraded":"2024-05-14T11:18:57.932087+0000","last_fullsized":"2024-05-14T11:18:57.932087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:44:39.321114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931181+0000","last_change":"2024-05-14T11:18:55.922438+0000","last_active":"2024-05-14T11:18:57.931181+0000","last_peered":"2024-05-14T11:18:57.931181+0000","last_clean":"2024-05-14T11:18:57.931181+0000","last_became_active":"2024-05-14T11:18:55.921484+0000","last_became_peered":"2024-05-14T11:18:55.921484+0000","last_unstale":"2024-05-14T11:18:57.931181+0000","last_undegraded":"2024-05-14T11:18:57.931181+0000","last_fullsized":"2024-05-14T11:18:57.931181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:00.972284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193018+0000","last_change":"2024-05-14T11:18:56.930745+0000","last_active":"2024-05-14T11:18:58.193018+0000","last_peered":"2024-05-14T11:18:58.193018+0000","last_clean":"2024-05-14T11:18:58.193018+0000","last_became_active":"2024-05-14T11:18:56.930418+0000","last_became_peered":"2024-05-14T11:18:56.930418+0000","last_unstale":"2024-05-14T11:18:58.193018+0000","last_undegraded":"2024-05-14T11:18:58.193018+0000","last_fullsized":"2024-05-14T11:18:58.193018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:47.462747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193722+0000","last_change":"2024-05-14T11:18:55.938638+0000","last_active":"2024-05-14T11:18:58.193722+0000","last_peered":"2024-05-14T11:18:58.193722+0000","last_clean":"2024-05-14T11:18:58.193722+0000","last_became_active":"2024-05-14T11:18:55.937136+0000","last_became_peered":"2024-05-14T11:18:55.937136+0000","last_unstale":"2024-05-14T11:18:58.193722+0000","last_undegraded":"2024-05-14T11:18:58.193722+0000","last_fullsized":"2024-05-14T11:18:58.193722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:32:01.643061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932185+0000","last_change":"2024-05-14T11:18:56.941473+0000","last_active":"2024-05-14T11:18:57.932185+0000","last_peered":"2024-05-14T11:18:57.932185+0000","last_clean":"2024-05-14T11:18:57.932185+0000","last_became_active":"2024-05-14T11:18:56.941113+0000","last_became_peered":"2024-05-14T11:18:56.941113+0000","last_unstale":"2024-05-14T11:18:57.932185+0000","last_undegraded":"2024-05-14T11:18:57.932185+0000","last_fullsized":"2024-05-14T11:18:57.932185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.313987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932224+0000","last_change":"2024-05-14T11:18:55.940055+0000","last_active":"2024-05-14T11:18:57.932224+0000","last_peered":"2024-05-14T11:18:57.932224+0000","last_clean":"2024-05-14T11:18:57.932224+0000","last_became_active":"2024-05-14T11:18:55.939780+0000","last_became_peered":"2024-05-14T11:18:55.939780+0000","last_unstale":"2024-05-14T11:18:57.932224+0000","last_undegraded":"2024-05-14T11:18:57.932224+0000","last_fullsized":"2024-05-14T11:18:57.932224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:40:03.934809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932435+0000","last_change":"2024-05-14T11:18:56.939367+0000","last_active":"2024-05-14T11:18:57.932435+0000","last_peered":"2024-05-14T11:18:57.932435+0000","last_clean":"2024-05-14T11:18:57.932435+0000","last_became_active":"2024-05-14T11:18:56.938571+0000","last_became_peered":"2024-05-14T11:18:56.938571+0000","last_unstale":"2024-05-14T11:18:57.932435+0000","last_undegraded":"2024-05-14T11:18:57.932435+0000","last_fullsized":"2024-05-14T11:18:57.932435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:02.408142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193103+0000","last_change":"2024-05-14T11:18:55.937856+0000","last_active":"2024-05-14T11:18:58.193103+0000","last_peered":"2024-05-14T11:18:58.193103+0000","last_clean":"2024-05-14T11:18:58.193103+0000","last_became_active":"2024-05-14T11:18:55.937505+0000","last_became_peered":"2024-05-14T11:18:55.937505+0000","last_unstale":"2024-05-14T11:18:58.193103+0000","last_undegraded":"2024-05-14T11:18:58.193103+0000","last_fullsized":"2024-05-14T11:18:58.193103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:54:15.109177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932310+0000","last_change":"2024-05-14T11:18:56.943119+0000","last_active":"2024-05-14T11:18:57.932310+0000","last_peered":"2024-05-14T11:18:57.932310+0000","last_clean":"2024-05-14T11:18:57.932310+0000","last_became_active":"2024-05-14T11:18:56.942594+0000","last_became_peered":"2024-05-14T11:18:56.942594+0000","last_unstale":"2024-05-14T11:18:57.932310+0000","last_undegraded":"2024-05-14T11:18:57.932310+0000","last_fullsized":"2024-05-14T11:18:57.932310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:51.644710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192683+0000","last_change":"2024-05-14T11:18:55.933957+0000","last_active":"2024-05-14T11:18:58.192683+0000","last_peered":"2024-05-14T11:18:58.192683+0000","last_clean":"2024-05-14T11:18:58.192683+0000","last_became_active":"2024-05-14T11:18:55.933591+0000","last_became_peered":"2024-05-14T11:18:55.933591+0000","last_unstale":"2024-05-14T11:18:58.192683+0000","last_undegraded":"2024-05-14T11:18:58.192683+0000","last_fullsized":"2024-05-14T11:18:58.192683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:48.024667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930660+0000","last_change":"2024-05-14T11:18:56.931845+0000","last_active":"2024-05-14T11:18:57.930660+0000","last_peered":"2024-05-14T11:18:57.930660+0000","last_clean":"2024-05-14T11:18:57.930660+0000","last_became_active":"2024-05-14T11:18:56.931388+0000","last_became_peered":"2024-05-14T11:18:56.931388+0000","last_unstale":"2024-05-14T11:18:57.930660+0000","last_undegraded":"2024-05-14T11:18:57.930660+0000","last_fullsized":"2024-05-14T11:18:57.930660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:47:59.920388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193495+0000","last_change":"2024-05-14T11:18:55.922634+0000","last_active":"2024-05-14T11:18:58.193495+0000","last_peered":"2024-05-14T11:18:58.193495+0000","last_clean":"2024-05-14T11:18:58.193495+0000","last_became_active":"2024-05-14T11:18:55.921988+0000","last_became_peered":"2024-05-14T11:18:55.921988+0000","last_unstale":"2024-05-14T11:18:58.193495+0000","last_undegraded":"2024-05-14T11:18:58.193495+0000","last_fullsized":"2024-05-14T11:18:58.193495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:59.724825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934746+0000","last_change":"2024-05-14T11:18:56.939457+0000","last_active":"2024-05-14T11:18:57.934746+0000","last_peered":"2024-05-14T11:18:57.934746+0000","last_clean":"2024-05-14T11:18:57.934746+0000","last_became_active":"2024-05-14T11:18:56.939082+0000","last_became_peered":"2024-05-14T11:18:56.939082+0000","last_unstale":"2024-05-14T11:18:57.934746+0000","last_undegraded":"2024-05-14T11:18:57.934746+0000","last_fullsized":"2024-05-14T11:18:57.934746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:32.947765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934687+0000","last_change":"2024-05-14T11:18:55.938688+0000","last_active":"2024-05-14T11:18:57.934687+0000","last_peered":"2024-05-14T11:18:57.934687+0000","last_clean":"2024-05-14T11:18:57.934687+0000","last_became_active":"2024-05-14T11:18:55.938287+0000","last_became_peered":"2024-05-14T11:18:55.938287+0000","last_unstale":"2024-05-14T11:18:57.934687+0000","last_undegraded":"2024-05-14T11:18:57.934687+0000","last_fullsized":"2024-05-14T11:18:57.934687+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:37.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931624+0000","last_change":"2024-05-14T11:18:56.929290+0000","last_active":"2024-05-14T11:18:57.931624+0000","last_peered":"2024-05-14T11:18:57.931624+0000","last_clean":"2024-05-14T11:18:57.931624+0000","last_became_active":"2024-05-14T11:18:56.928967+0000","last_became_peered":"2024-05-14T11:18:56.928967+0000","last_unstale":"2024-05-14T11:18:57.931624+0000","last_undegraded":"2024-05-14T11:18:57.931624+0000","last_fullsized":"2024-05-14T11:18:57.931624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:28.737096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193305+0000","last_change":"2024-05-14T11:18:55.924561+0000","last_active":"2024-05-14T11:18:58.193305+0000","last_peered":"2024-05-14T11:18:58.193305+0000","last_clean":"2024-05-14T11:18:58.193305+0000","last_became_active":"2024-05-14T11:18:55.924148+0000","last_became_peered":"2024-05-14T11:18:55.924148+0000","last_unstale":"2024-05-14T11:18:58.193305+0000","last_undegraded":"2024-05-14T11:18:58.193305+0000","last_fullsized":"2024-05-14T11:18:58.193305+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:22.975754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192708+0000","last_change":"2024-05-14T11:18:56.936334+0000","last_active":"2024-05-14T11:18:58.192708+0000","last_peered":"2024-05-14T11:18:58.192708+0000","last_clean":"2024-05-14T11:18:58.192708+0000","last_became_active":"2024-05-14T11:18:56.934877+0000","last_became_peered":"2024-05-14T11:18:56.934877+0000","last_unstale":"2024-05-14T11:18:58.192708+0000","last_undegraded":"2024-05-14T11:18:58.192708+0000","last_fullsized":"2024-05-14T11:18:58.192708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:03.082658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930745+0000","last_change":"2024-05-14T11:18:55.934695+0000","last_active":"2024-05-14T11:18:57.930745+0000","last_peered":"2024-05-14T11:18:57.930745+0000","last_clean":"2024-05-14T11:18:57.930745+0000","last_became_active":"2024-05-14T11:18:55.934312+0000","last_became_peered":"2024-05-14T11:18:55.934312+0000","last_unstale":"2024-05-14T11:18:57.930745+0000","last_undegraded":"2024-05-14T11:18:57.930745+0000","last_fullsized":"2024-05-14T11:18:57.930745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:05:00.972284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193751+0000","last_change":"2024-05-14T11:18:56.932287+0000","last_active":"2024-05-14T11:18:58.193751+0000","last_peered":"2024-05-14T11:18:58.193751+0000","last_clean":"2024-05-14T11:18:58.193751+0000","last_became_active":"2024-05-14T11:18:56.931936+0000","last_became_peered":"2024-05-14T11:18:56.931936+0000","last_unstale":"2024-05-14T11:18:58.193751+0000","last_undegraded":"2024-05-14T11:18:58.193751+0000","last_fullsized":"2024-05-14T11:18:58.193751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:01:44.787619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935129+0000","last_change":"2024-05-14T11:18:55.938172+0000","last_active":"2024-05-14T11:18:57.935129+0000","last_peered":"2024-05-14T11:18:57.935129+0000","last_clean":"2024-05-14T11:18:57.935129+0000","last_became_active":"2024-05-14T11:18:55.937823+0000","last_became_peered":"2024-05-14T11:18:55.937823+0000","last_unstale":"2024-05-14T11:18:57.935129+0000","last_undegraded":"2024-05-14T11:18:57.935129+0000","last_fullsized":"2024-05-14T11:18:57.935129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:02.074706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931903+0000","last_change":"2024-05-14T11:18:56.921592+0000","last_active":"2024-05-14T11:18:57.931903+0000","last_peered":"2024-05-14T11:18:57.931903+0000","last_clean":"2024-05-14T11:18:57.931903+0000","last_became_active":"2024-05-14T11:18:56.920154+0000","last_became_peered":"2024-05-14T11:18:56.920154+0000","last_unstale":"2024-05-14T11:18:57.931903+0000","last_undegraded":"2024-05-14T11:18:57.931903+0000","last_fullsized":"2024-05-14T11:18:57.931903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:00:58.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193384+0000","last_change":"2024-05-14T11:18:55.936674+0000","last_active":"2024-05-14T11:18:58.193384+0000","last_peered":"2024-05-14T11:18:58.193384+0000","last_clean":"2024-05-14T11:18:58.193384+0000","last_became_active":"2024-05-14T11:18:55.936304+0000","last_became_peered":"2024-05-14T11:18:55.936304+0000","last_unstale":"2024-05-14T11:18:58.193384+0000","last_undegraded":"2024-05-14T11:18:58.193384+0000","last_fullsized":"2024-05-14T11:18:58.193384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:21.191213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932117+0000","last_change":"2024-05-14T11:18:56.921612+0000","last_active":"2024-05-14T11:18:57.932117+0000","last_peered":"2024-05-14T11:18:57.932117+0000","last_clean":"2024-05-14T11:18:57.932117+0000","last_became_active":"2024-05-14T11:18:56.921353+0000","last_became_peered":"2024-05-14T11:18:56.921353+0000","last_unstale":"2024-05-14T11:18:57.932117+0000","last_undegraded":"2024-05-14T11:18:57.932117+0000","last_fullsized":"2024-05-14T11:18:57.932117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:11.225500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193575+0000","last_change":"2024-05-14T11:18:55.937880+0000","last_active":"2024-05-14T11:18:58.193575+0000","last_peered":"2024-05-14T11:18:58.193575+0000","last_clean":"2024-05-14T11:18:58.193575+0000","last_became_active":"2024-05-14T11:18:55.937571+0000","last_became_peered":"2024-05-14T11:18:55.937571+0000","last_unstale":"2024-05-14T11:18:58.193575+0000","last_undegraded":"2024-05-14T11:18:58.193575+0000","last_fullsized":"2024-05-14T11:18:58.193575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:54.430892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932403+0000","last_change":"2024-05-14T11:18:56.932003+0000","last_active":"2024-05-14T11:18:57.932403+0000","last_peered":"2024-05-14T11:18:57.932403+0000","last_clean":"2024-05-14T11:18:57.932403+0000","last_became_active":"2024-05-14T11:18:56.931692+0000","last_became_peered":"2024-05-14T11:18:56.931692+0000","last_unstale":"2024-05-14T11:18:57.932403+0000","last_undegraded":"2024-05-14T11:18:57.932403+0000","last_fullsized":"2024-05-14T11:18:57.932403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:38.874693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932025+0000","last_change":"2024-05-14T11:18:55.935420+0000","last_active":"2024-05-14T11:18:57.932025+0000","last_peered":"2024-05-14T11:18:57.932025+0000","last_clean":"2024-05-14T11:18:57.932025+0000","last_became_active":"2024-05-14T11:18:55.935035+0000","last_became_peered":"2024-05-14T11:18:55.935035+0000","last_unstale":"2024-05-14T11:18:57.932025+0000","last_undegraded":"2024-05-14T11:18:57.932025+0000","last_fullsized":"2024-05-14T11:18:57.932025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:51.591243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192432+0000","last_change":"2024-05-14T11:18:56.930237+0000","last_active":"2024-05-14T11:18:58.192432+0000","last_peered":"2024-05-14T11:18:58.192432+0000","last_clean":"2024-05-14T11:18:58.192432+0000","last_became_active":"2024-05-14T11:18:56.929955+0000","last_became_peered":"2024-05-14T11:18:56.929955+0000","last_unstale":"2024-05-14T11:18:58.192432+0000","last_undegraded":"2024-05-14T11:18:58.192432+0000","last_fullsized":"2024-05-14T11:18:58.192432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:38.804025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192877+0000","last_change":"2024-05-14T11:18:55.928853+0000","last_active":"2024-05-14T11:18:58.192877+0000","last_peered":"2024-05-14T11:18:58.192877+0000","last_clean":"2024-05-14T11:18:58.192877+0000","last_became_active":"2024-05-14T11:18:55.928563+0000","last_became_peered":"2024-05-14T11:18:55.928563+0000","last_unstale":"2024-05-14T11:18:58.192877+0000","last_undegraded":"2024-05-14T11:18:58.192877+0000","last_fullsized":"2024-05-14T11:18:58.192877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:40:51.591243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934773+0000","last_change":"2024-05-14T11:18:56.944065+0000","last_active":"2024-05-14T11:18:57.934773+0000","last_peered":"2024-05-14T11:18:57.934773+0000","last_clean":"2024-05-14T11:18:57.934773+0000","last_became_active":"2024-05-14T11:18:56.943727+0000","last_became_peered":"2024-05-14T11:18:56.943727+0000","last_unstale":"2024-05-14T11:18:57.934773+0000","last_undegraded":"2024-05-14T11:18:57.934773+0000","last_fullsized":"2024-05-14T11:18:57.934773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:35:18.885017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.934814+0000","last_change":"2024-05-14T11:18:55.938239+0000","last_active":"2024-05-14T11:18:57.934814+0000","last_peered":"2024-05-14T11:18:57.934814+0000","last_clean":"2024-05-14T11:18:57.934814+0000","last_became_active":"2024-05-14T11:18:55.937953+0000","last_became_peered":"2024-05-14T11:18:55.937953+0000","last_unstale":"2024-05-14T11:18:57.934814+0000","last_undegraded":"2024-05-14T11:18:57.934814+0000","last_fullsized":"2024-05-14T11:18:57.934814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:41:59.724825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931622+0000","last_change":"2024-05-14T11:18:56.921431+0000","last_active":"2024-05-14T11:18:57.931622+0000","last_peered":"2024-05-14T11:18:57.931622+0000","last_clean":"2024-05-14T11:18:57.931622+0000","last_became_active":"2024-05-14T11:18:56.920835+0000","last_became_peered":"2024-05-14T11:18:56.920835+0000","last_unstale":"2024-05-14T11:18:57.931622+0000","last_undegraded":"2024-05-14T11:18:57.931622+0000","last_fullsized":"2024-05-14T11:18:57.931622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:48:59.274859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192266+0000","last_change":"2024-05-14T11:18:55.924405+0000","last_active":"2024-05-14T11:18:58.192266+0000","last_peered":"2024-05-14T11:18:58.192266+0000","last_clean":"2024-05-14T11:18:58.192266+0000","last_became_active":"2024-05-14T11:18:55.924050+0000","last_became_peered":"2024-05-14T11:18:55.924050+0000","last_unstale":"2024-05-14T11:18:58.192266+0000","last_undegraded":"2024-05-14T11:18:58.192266+0000","last_fullsized":"2024-05-14T11:18:58.192266+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:31.939814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193150+0000","last_change":"2024-05-14T11:18:56.920715+0000","last_active":"2024-05-14T11:18:58.193150+0000","last_peered":"2024-05-14T11:18:58.193150+0000","last_clean":"2024-05-14T11:18:58.193150+0000","last_became_active":"2024-05-14T11:18:56.919534+0000","last_became_peered":"2024-05-14T11:18:56.919534+0000","last_unstale":"2024-05-14T11:18:58.193150+0000","last_undegraded":"2024-05-14T11:18:58.193150+0000","last_fullsized":"2024-05-14T11:18:58.193150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:37:52.501692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935209+0000","last_change":"2024-05-14T11:18:55.931519+0000","last_active":"2024-05-14T11:18:57.935209+0000","last_peered":"2024-05-14T11:18:57.935209+0000","last_clean":"2024-05-14T11:18:57.935209+0000","last_became_active":"2024-05-14T11:18:55.931047+0000","last_became_peered":"2024-05-14T11:18:55.931047+0000","last_unstale":"2024-05-14T11:18:57.935209+0000","last_undegraded":"2024-05-14T11:18:57.935209+0000","last_fullsized":"2024-05-14T11:18:57.935209+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:38:01.400191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937236+0000","last_change":"2024-05-14T11:18:56.932499+0000","last_active":"2024-05-14T11:18:57.937236+0000","last_peered":"2024-05-14T11:18:57.937236+0000","last_clean":"2024-05-14T11:18:57.937236+0000","last_became_active":"2024-05-14T11:18:56.932184+0000","last_became_peered":"2024-05-14T11:18:56.932184+0000","last_unstale":"2024-05-14T11:18:57.937236+0000","last_undegraded":"2024-05-14T11:18:57.937236+0000","last_fullsized":"2024-05-14T11:18:57.937236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:17.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935232+0000","last_change":"2024-05-14T11:18:55.931003+0000","last_active":"2024-05-14T11:18:57.935232+0000","last_peered":"2024-05-14T11:18:57.935232+0000","last_clean":"2024-05-14T11:18:57.935232+0000","last_became_active":"2024-05-14T11:18:55.930685+0000","last_became_peered":"2024-05-14T11:18:55.930685+0000","last_unstale":"2024-05-14T11:18:57.935232+0000","last_undegraded":"2024-05-14T11:18:57.935232+0000","last_fullsized":"2024-05-14T11:18:57.935232+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:46.454795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.935086+0000","last_change":"2024-05-14T11:18:56.940401+0000","last_active":"2024-05-14T11:18:57.935086+0000","last_peered":"2024-05-14T11:18:57.935086+0000","last_clean":"2024-05-14T11:18:57.935086+0000","last_became_active":"2024-05-14T11:18:56.938765+0000","last_became_peered":"2024-05-14T11:18:56.938765+0000","last_unstale":"2024-05-14T11:18:57.935086+0000","last_undegraded":"2024-05-14T11:18:57.935086+0000","last_fullsized":"2024-05-14T11:18:57.935086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:41:29.460299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193416+0000","last_change":"2024-05-14T11:18:55.932175+0000","last_active":"2024-05-14T11:18:58.193416+0000","last_peered":"2024-05-14T11:18:58.193416+0000","last_clean":"2024-05-14T11:18:58.193416+0000","last_became_active":"2024-05-14T11:18:55.931873+0000","last_became_peered":"2024-05-14T11:18:55.931873+0000","last_unstale":"2024-05-14T11:18:58.193416+0000","last_undegraded":"2024-05-14T11:18:58.193416+0000","last_fullsized":"2024-05-14T11:18:58.193416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:55:52.322782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932548+0000","last_change":"2024-05-14T11:18:56.932030+0000","last_active":"2024-05-14T11:18:57.932548+0000","last_peered":"2024-05-14T11:18:57.932548+0000","last_clean":"2024-05-14T11:18:57.932548+0000","last_became_active":"2024-05-14T11:18:56.931709+0000","last_became_peered":"2024-05-14T11:18:56.931709+0000","last_unstale":"2024-05-14T11:18:57.932548+0000","last_undegraded":"2024-05-14T11:18:57.932548+0000","last_fullsized":"2024-05-14T11:18:57.932548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:03.082658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192796+0000","last_change":"2024-05-14T11:18:55.933868+0000","last_active":"2024-05-14T11:18:58.192796+0000","last_peered":"2024-05-14T11:18:58.192796+0000","last_clean":"2024-05-14T11:18:58.192796+0000","last_became_active":"2024-05-14T11:18:55.933556+0000","last_became_peered":"2024-05-14T11:18:55.933556+0000","last_unstale":"2024-05-14T11:18:58.192796+0000","last_undegraded":"2024-05-14T11:18:58.192796+0000","last_fullsized":"2024-05-14T11:18:58.192796+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:10.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930847+0000","last_change":"2024-05-14T11:18:56.937480+0000","last_active":"2024-05-14T11:18:57.930847+0000","last_peered":"2024-05-14T11:18:57.930847+0000","last_clean":"2024-05-14T11:18:57.930847+0000","last_became_active":"2024-05-14T11:18:56.937141+0000","last_became_peered":"2024-05-14T11:18:56.937141+0000","last_unstale":"2024-05-14T11:18:57.930847+0000","last_undegraded":"2024-05-14T11:18:57.930847+0000","last_fullsized":"2024-05-14T11:18:57.930847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:05.923070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.193714+0000","last_change":"2024-05-14T11:18:55.926593+0000","last_active":"2024-05-14T11:18:58.193714+0000","last_peered":"2024-05-14T11:18:58.193714+0000","last_clean":"2024-05-14T11:18:58.193714+0000","last_became_active":"2024-05-14T11:18:55.926285+0000","last_became_peered":"2024-05-14T11:18:55.926285+0000","last_unstale":"2024-05-14T11:18:58.193714+0000","last_undegraded":"2024-05-14T11:18:58.193714+0000","last_fullsized":"2024-05-14T11:18:58.193714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:46.454795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.930899+0000","last_change":"2024-05-14T11:18:56.935536+0000","last_active":"2024-05-14T11:18:57.930899+0000","last_peered":"2024-05-14T11:18:57.930899+0000","last_clean":"2024-05-14T11:18:57.930899+0000","last_became_active":"2024-05-14T11:18:56.935218+0000","last_became_peered":"2024-05-14T11:18:56.935218+0000","last_unstale":"2024-05-14T11:18:57.930899+0000","last_undegraded":"2024-05-14T11:18:57.930899+0000","last_fullsized":"2024-05-14T11:18:57.930899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:05.923070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931917+0000","last_change":"2024-05-14T11:18:55.937650+0000","last_active":"2024-05-14T11:18:57.931917+0000","last_peered":"2024-05-14T11:18:57.931917+0000","last_clean":"2024-05-14T11:18:57.931917+0000","last_became_active":"2024-05-14T11:18:55.937251+0000","last_became_peered":"2024-05-14T11:18:55.937251+0000","last_unstale":"2024-05-14T11:18:57.931917+0000","last_undegraded":"2024-05-14T11:18:57.931917+0000","last_fullsized":"2024-05-14T11:18:57.931917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:55.613451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192867+0000","last_change":"2024-05-14T11:18:56.933546+0000","last_active":"2024-05-14T11:18:58.192867+0000","last_peered":"2024-05-14T11:18:58.192867+0000","last_clean":"2024-05-14T11:18:58.192867+0000","last_became_active":"2024-05-14T11:18:56.933204+0000","last_became_peered":"2024-05-14T11:18:56.933204+0000","last_unstale":"2024-05-14T11:18:58.192867+0000","last_undegraded":"2024-05-14T11:18:58.192867+0000","last_fullsized":"2024-05-14T11:18:58.192867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:42:00.732777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931714+0000","last_change":"2024-05-14T11:18:55.925879+0000","last_active":"2024-05-14T11:18:57.931714+0000","last_peered":"2024-05-14T11:18:57.931714+0000","last_clean":"2024-05-14T11:18:57.931714+0000","last_became_active":"2024-05-14T11:18:55.925042+0000","last_became_peered":"2024-05-14T11:18:55.925042+0000","last_unstale":"2024-05-14T11:18:57.931714+0000","last_undegraded":"2024-05-14T11:18:57.931714+0000","last_fullsized":"2024-05-14T11:18:57.931714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:02.347672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.937123+0000","last_change":"2024-05-14T11:18:56.938790+0000","last_active":"2024-05-14T11:18:57.937123+0000","last_peered":"2024-05-14T11:18:57.937123+0000","last_clean":"2024-05-14T11:18:57.937123+0000","last_became_active":"2024-05-14T11:18:56.938311+0000","last_became_peered":"2024-05-14T11:18:56.938311+0000","last_unstale":"2024-05-14T11:18:57.937123+0000","last_undegraded":"2024-05-14T11:18:57.937123+0000","last_fullsized":"2024-05-14T11:18:57.937123+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:24.714908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192273+0000","last_change":"2024-05-14T11:18:55.928062+0000","last_active":"2024-05-14T11:18:58.192273+0000","last_peered":"2024-05-14T11:18:58.192273+0000","last_clean":"2024-05-14T11:18:58.192273+0000","last_became_active":"2024-05-14T11:18:55.927658+0000","last_became_peered":"2024-05-14T11:18:55.927658+0000","last_unstale":"2024-05-14T11:18:58.192273+0000","last_undegraded":"2024-05-14T11:18:58.192273+0000","last_fullsized":"2024-05-14T11:18:58.192273+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.827500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.932117+0000","last_change":"2024-05-14T11:18:56.936635+0000","last_active":"2024-05-14T11:18:57.932117+0000","last_peered":"2024-05-14T11:18:57.932117+0000","last_clean":"2024-05-14T11:18:57.932117+0000","last_became_active":"2024-05-14T11:18:56.936292+0000","last_became_peered":"2024-05-14T11:18:56.936292+0000","last_unstale":"2024-05-14T11:18:57.932117+0000","last_undegraded":"2024-05-14T11:18:57.932117+0000","last_fullsized":"2024-05-14T11:18:57.932117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:55.898180+0000","last_clean_scrub_stamp":"2024-05-14T11:18:55.898180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:51.644710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:57.931499+0000","last_change":"2024-05-14T11:18:55.923385+0000","last_active":"2024-05-14T11:18:57.931499+0000","last_peered":"2024-05-14T11:18:57.931499+0000","last_clean":"2024-05-14T11:18:57.931499+0000","last_became_active":"2024-05-14T11:18:55.923063+0000","last_became_peered":"2024-05-14T11:18:55.923063+0000","last_unstale":"2024-05-14T11:18:57.931499+0000","last_undegraded":"2024-05-14T11:18:57.931499+0000","last_fullsized":"2024-05-14T11:18:57.931499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:18:54.890228+0000","last_clean_scrub_stamp":"2024-05-14T11:18:54.890228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:30:01.324648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-14T11:18:58.192937+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:18:58.192937+0000","last_peered":"2024-05-14T11:18:58.192937+0000","last_clean":"2024-05-14T11:18:58.192937+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:18:58.192937+0000","last_undegraded":"2024-05-14T11:18:58.192937+0000","last_fullsized":"2024-05-14T11:18:58.192937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":916220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":729088,"data_stored":325098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":326688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":325076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":325076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":327452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":915444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:19:06.017 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:19:06.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:19:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.203+0000 7f2850800640 1 Processor -- start 2024-05-14T11:19:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 -- start start 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f284806f450 0x7f284806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f28480d1730 0x7f28480d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284806f860 con 0x7f28480cce60 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f284806db80 con 0x7f284806f450 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f2850800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f28480d2080 con 0x7f28480d1730 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f284806f450 0x7f284806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284e800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f28480cce60 0x7f28480cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59006/0 (socket says 172.21.15.116:59006) 2024-05-14T11:19:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284e800640 1 -- 172.21.15.116:0/689691497 learned_addr learned my addr 172.21.15.116:0/689691497 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284d400640 1 --2- 172.21.15.116:0/689691497 >> v2:172.21.15.149:3300/0 conn(0x7f28480d1730 0x7f28480d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284de00640 1 -- 172.21.15.116:0/689691497 >> v2:172.21.15.149:3300/0 conn(0x7f28480d1730 msgr2=0x7f28480d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284de00640 1 --2- 172.21.15.116:0/689691497 >> v2:172.21.15.149:3300/0 conn(0x7f28480d1730 0x7f28480d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284de00640 1 -- 172.21.15.116:0/689691497 >> v1:172.21.15.116:6789/0 conn(0x7f28480cce60 legacy=0x7f28480cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284de00640 1 -- 172.21.15.116:0/689691497 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284806fae0 con 0x7f284806f450 2024-05-14T11:19:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.207+0000 7f284ca00640 1 -- 172.21.15.116:0/689691497 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926707562 0 0) 0x7f28440014a0 con 0x7f28480cce60 2024-05-14T11:19:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f284de00640 1 --2- 172.21.15.116:0/689691497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284806f450 0x7f284806d640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f28480d2430 tx=0x7f283c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d6c4f0ad01a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f284ca00640 1 -- 172.21.15.116:0/689691497 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b0070 con 0x7f284806f450 2024-05-14T11:19:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f284ca00640 1 -- 172.21.15.116:0/689691497 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c0ab600 con 0x7f284806f450 2024-05-14T11:19:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f284ca00640 1 -- 172.21.15.116:0/689691497 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480ce190 con 0x7f284806f450 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 -- 172.21.15.116:0/689691497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284806f450 msgr2=0x7f284806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 --2- 172.21.15.116:0/689691497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284806f450 0x7f284806d640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f28480d2430 tx=0x7f283c01c480 comp rx=0 tx=0).stop 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 -- 172.21.15.116:0/689691497 shutdown_connections 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 --2- 172.21.15.116:0/689691497 >> v2:172.21.15.149:3300/0 conn(0x7f28480d1730 0x7f28480d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 --2- 172.21.15.116:0/689691497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284806f450 0x7f284806d640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 -- 172.21.15.116:0/689691497 >> 172.21.15.116:0/689691497 conn(0x7f28480b4720 msgr2=0x7f28480b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 -- 172.21.15.116:0/689691497 shutdown_connections 2024-05-14T11:19:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.211+0000 7f2850800640 1 -- 172.21.15.116:0/689691497 wait complete. 2024-05-14T11:19:06.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 Processor -- start 2024-05-14T11:19:06.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- start start 2024-05-14T11:19:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f284806f450 0x7f2848155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f28480cce60 0x7f2848165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f284816a610 con 0x7f28480d1730 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2848070660 con 0x7f284806f450 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f284816a790 con 0x7f28480cce60 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39922/0 (socket says 172.21.15.116:39922) 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 -- 172.21.15.116:0/536455531 learned_addr learned my addr 172.21.15.116:0/536455531 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284de00640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f284806f450 0x7f2848155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284d400640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f28480cce60 0x7f2848165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 -- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f28480cce60 msgr2=0x7f2848165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f28480cce60 0x7f2848165d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 -- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f284806f450 msgr2=0x7f2848155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f284806f450 0x7f2848155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 -- 172.21.15.116:0/536455531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2848160200 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f284e800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f284400c6b0 tx=0x7f284400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844018070 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2844004db0 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400a880 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2848167f30 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.215+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2848157e20 con 0x7f28480d1730 2024-05-14T11:19:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284806a9c0 con 0x7f28480d1730 2024-05-14T11:19:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f284400aa20 con 0x7f28480d1730 2024-05-14T11:19:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f2842000640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 0x7f282c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f284408e880 con 0x7f28480d1730 2024-05-14T11:19:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f284de00640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 0x7f282c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.219+0000 7f284de00640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 0x7f282c071910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ad000 tx=0x7f283c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.223+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f284806f860 con 0x7f28480d1730 2024-05-14T11:19:06.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.439+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2848069de0 con 0x7f28480d1730 2024-05-14T11:19:06.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.439+0000 7f2842000640 1 -- 172.21.15.116:0/536455531 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f284405af90 con 0x7f28480d1730 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:19:06.444 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 msgr2=0x7f282c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 0x7f282c071910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ad000 tx=0x7f283c0a6000 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 msgr2=0x7f28481662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f284400c6b0 tx=0x7f284400cb70 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 shutdown_connections 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f282c06f400 0x7f282c071910 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28480d1730 0x7f28481662d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f28480cce60 0x7f2848165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 --2- 172.21.15.116:0/536455531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f284806f450 0x7f2848155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 >> 172.21.15.116:0/536455531 conn(0x7f28480b4720 msgr2=0x7f28480b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:06.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 shutdown_connections 2024-05-14T11:19:06.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.443+0000 7f2850800640 1 -- 172.21.15.116:0/536455531 wait complete. 2024-05-14T11:19:06.467 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:06.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 Processor -- start 2024-05-14T11:19:06.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 -- start start 2024-05-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 0x7f2c4006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c40160510 con 0x7f2c400af380 2024-05-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c401607f0 con 0x7f2c400d1670 2024-05-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c48200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c401573b0 con 0x7f2c40156fa0 2024-05-14T11:19:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c46200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2c400af380 0x7f2c400af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59018/0 (socket says 172.21.15.116:59018) 2024-05-14T11:19:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c46200640 1 -- 172.21.15.116:0/2194662495 learned_addr learned my addr 172.21.15.116:0/2194662495 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c45800640 1 --2- 172.21.15.116:0/2194662495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 0x7f2c4006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.611+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988472326 0 0) 0x7f2c40160510 con 0x7f2c400af380 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c28003530 con 0x7f2c400af380 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433785354 0 0) 0x7f2c401607f0 con 0x7f2c400d1670 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c40160510 con 0x7f2c400d1670 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c45800640 1 -- 172.21.15.116:0/2194662495 >> v1:172.21.15.149:6789/0 conn(0x7f2c400d1670 legacy=0x7f2c400d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c45800640 1 -- 172.21.15.116:0/2194662495 >> v1:172.21.15.116:6789/0 conn(0x7f2c400af380 legacy=0x7f2c400af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c45800640 1 -- 172.21.15.116:0/2194662495 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c40160ad0 con 0x7f2c40156fa0 2024-05-14T11:19:06.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c45800640 1 --2- 172.21.15.116:0/2194662495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 0x7f2c4006ffe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c340097f0 tx=0x7f2c3401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89a2a1b40200f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340ab5f0 con 0x7f2c40156fa0 2024-05-14T11:19:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c340b0070 con 0x7f2c40156fa0 2024-05-14T11:19:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.615+0000 7f2c3fe00640 1 -- 172.21.15.116:0/2194662495 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340ab9b0 con 0x7f2c40156fa0 2024-05-14T11:19:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- 172.21.15.116:0/2194662495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 msgr2=0x7f2c4006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 --2- 172.21.15.116:0/2194662495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 0x7f2c4006ffe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c340097f0 tx=0x7f2c3401c4e0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- 172.21.15.116:0/2194662495 shutdown_connections 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 --2- 172.21.15.116:0/2194662495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c40156fa0 0x7f2c4006ffe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- 172.21.15.116:0/2194662495 >> 172.21.15.116:0/2194662495 conn(0x7f2c400b4620 msgr2=0x7f2c400b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- 172.21.15.116:0/2194662495 shutdown_connections 2024-05-14T11:19:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- 172.21.15.116:0/2194662495 wait complete. 2024-05-14T11:19:06.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 Processor -- start 2024-05-14T11:19:06.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.619+0000 7f2c48200640 1 -- start start 2024-05-14T11:19:06.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c400af380 0x7f2c40073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 0x7f2c40157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4016abb0 con 0x7f2c400d1670 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4016aa30 con 0x7f2c40156fa0 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4016ad30 con 0x7f2c400af380 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39930/0 (socket says 172.21.15.116:39930) 2024-05-14T11:19:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 -- 172.21.15.116:0/2957116202 learned_addr learned my addr 172.21.15.116:0/2957116202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 -- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c400af380 msgr2=0x7f2c40073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c400af380 0x7f2c40073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 -- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 msgr2=0x7f2c40157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c46200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 0x7f2c40157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 0x7f2c40157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 -- 172.21.15.116:0/2957116202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c40150dc0 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c46200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 0x7f2c40157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c44e00640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c3f0 tx=0x7f2c2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c017070 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0052d0 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c003840 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c40072ad0 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c40072d70 con 0x7f2c400d1670 2024-05-14T11:19:06.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4006a9c0 con 0x7f2c400d1670 2024-05-14T11:19:06.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.623+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2c2c01a070 con 0x7f2c400d1670 2024-05-14T11:19:06.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.627+0000 7f2c3d600640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 0x7f2c20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:06.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.627+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f2c2c08e6d0 con 0x7f2c400d1670 2024-05-14T11:19:06.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.627+0000 7f2c45800640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 0x7f2c20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:06.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.627+0000 7f2c45800640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 0x7f2c20071940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c340af000 tx=0x7f2c34004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:06.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.631+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2c2c005480 con 0x7f2c400d1670 2024-05-14T11:19:06.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c4015a570 con 0x7f2c400d1670 2024-05-14T11:19:06.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c3d600640 1 -- 172.21.15.116:0/2957116202 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5191 (secure 0 0 0) 0x7f2c2c05b4c0 con 0x7f2c400d1670 2024-05-14T11:19:06.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:06.826 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:18:59.326933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:06.826 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 34 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 msgr2=0x7f2c20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 0x7f2c20071940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c340af000 tx=0x7f2c34004740 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 msgr2=0x7f2c40150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00c3f0 tx=0x7f2c2c00c8b0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 shutdown_connections 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c2006f430 0x7f2c20071940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c40156fa0 0x7f2c40157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c400d1670 0x7f2c40150880 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 --2- 172.21.15.116:0/2957116202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c400af380 0x7f2c40073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 >> 172.21.15.116:0/2957116202 conn(0x7f2c400b4620 msgr2=0x7f2c4015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 shutdown_connections 2024-05-14T11:19:06.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:06.823+0000 7f2c48200640 1 -- 172.21.15.116:0/2957116202 wait complete. 2024-05-14T11:19:07.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:18:56.305831+0000', 'modified': '2024-05-14T11:18:59.326933+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5721': {'gid': 5721, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/1576287145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 1576287145}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 1576287145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5721, 'qdb_cluster': [5721]} 2024-05-14T11:19:07.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:19:07.250 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-14T11:19:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.395+0000 7f3875c00640 1 Processor -- start 2024-05-14T11:19:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.395+0000 7f3875c00640 1 -- start start 2024-05-14T11:19:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3875c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f387006f730 0x7f387006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3875c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f3870072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3875c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3870070080 con 0x7f38700b0770 2024-05-14T11:19:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3875c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f387006db20 con 0x7f387006f730 2024-05-14T11:19:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3875c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f387006cf10 con 0x7f3870072a30 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f387006f730 0x7f387006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f387006f730 0x7f387006fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39944/0 (socket says 172.21.15.116:39944) 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 -- 172.21.15.116:0/3058937350 learned_addr learned my addr 172.21.15.116:0/3058937350 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f386f400640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f3870072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 -- 172.21.15.116:0/3058937350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 msgr2=0x7f3870072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f3870072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 -- 172.21.15.116:0/3058937350 >> v1:172.21.15.149:6789/0 conn(0x7f38700b0770 legacy=0x7f387006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.399+0000 7f3866000640 1 -- 172.21.15.116:0/3058937350 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387006d1f0 con 0x7f387006f730 2024-05-14T11:19:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3866000640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387006fb40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f387006dca0 tx=0x7f385801eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4224b5c209e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f386e000640 1 -- 172.21.15.116:0/3058937350 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580b4070 con 0x7f387006f730 2024-05-14T11:19:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f386e000640 1 -- 172.21.15.116:0/3058937350 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38580a7b40 con 0x7f387006f730 2024-05-14T11:19:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f386e000640 1 -- 172.21.15.116:0/3058937350 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38580af440 con 0x7f387006f730 2024-05-14T11:19:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 -- 172.21.15.116:0/3058937350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 msgr2=0x7f387006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387006fb40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f387006dca0 tx=0x7f385801eaf0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 -- 172.21.15.116:0/3058937350 shutdown_connections 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f3870072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 --2- 172.21.15.116:0/3058937350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387006fb40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 -- 172.21.15.116:0/3058937350 >> 172.21.15.116:0/3058937350 conn(0x7f38700b4620 msgr2=0x7f38700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 -- 172.21.15.116:0/3058937350 shutdown_connections 2024-05-14T11:19:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.403+0000 7f3875c00640 1 -- 172.21.15.116:0/3058937350 wait complete. 2024-05-14T11:19:07.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 Processor -- start 2024-05-14T11:19:07.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- start start 2024-05-14T11:19:07.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f38701592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 0x7f3870159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3870073500 con 0x7f387006f730 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3870073380 con 0x7f38700b0770 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3870073680 con 0x7f3870072a30 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39950/0 (socket says 172.21.15.116:39950) 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 -- 172.21.15.116:0/3408252604 learned_addr learned my addr 172.21.15.116:0/3408252604 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3866000640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f38701592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386f400640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 0x7f3870159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 -- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 msgr2=0x7f38701592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f38701592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 -- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 msgr2=0x7f3870159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 0x7f3870159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 -- 172.21.15.116:0/3408252604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38700cee10 con 0x7f387006f730 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386f400640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 0x7f3870159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f386ea00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f386000c3c0 tx=0x7f386000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3867400640 1 -- 172.21.15.116:0/3408252604 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860017070 con 0x7f387006f730 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3867400640 1 -- 172.21.15.116:0/3408252604 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38600052d0 con 0x7f387006f730 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3867400640 1 -- 172.21.15.116:0/3408252604 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860003840 con 0x7f387006f730 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38700cf0f0 con 0x7f387006f730 2024-05-14T11:19:07.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.407+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38700cf6b0 con 0x7f387006f730 2024-05-14T11:19:07.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.411+0000 7f3867400640 1 -- 172.21.15.116:0/3408252604 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f386001a070 con 0x7f387006f730 2024-05-14T11:19:07.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.411+0000 7f3867400640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 0x7f384c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.411+0000 7f3867400640 1 -- 172.21.15.116:0/3408252604 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f386008e730 con 0x7f387006f730 2024-05-14T11:19:07.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.411+0000 7f3866000640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 0x7f384c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.411+0000 7f387d1c3000 1 Processor -- start 2024-05-14T11:19:07.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- start start 2024-05-14T11:19:07.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f3866000640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 0x7f384c071910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38580b1000 tx=0x7f38580a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dab1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36dab1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d36d94e620 con 0x55d36d5777c0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d36d869740 con 0x55d36d95ff80 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d36d8b7090 con 0x55d36d9496b0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39966/0 (socket says 172.21.15.116:39966) 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 -- 172.21.15.116:0/1255327871 learned_addr learned my addr 172.21.15.116:0/1255327871 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386f400640 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36dab1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f3866000640 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dab1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 -- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 msgr2=0x55d36dab1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36dab1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 -- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 msgr2=0x55d36dab1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dab1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 -- 172.21.15.116:0/1255327871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36da4b0a0 con 0x55d36d5777c0 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f386ea00640 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3860057390 tx=0x7f3860001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fc37c895afc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f385f400640 1 -- 172.21.15.116:0/1255327871 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860063b50 con 0x55d36d5777c0 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f385f400640 1 -- 172.21.15.116:0/1255327871 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f385f400640 1 -- 172.21.15.116:0/1255327871 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860056710 con 0x55d36d5777c0 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- 172.21.15.116:0/1255327871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 msgr2=0x55d36dab1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3860057390 tx=0x7f3860001090 comp rx=0 tx=0).stop 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- 172.21.15.116:0/1255327871 shutdown_connections 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dab1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36dab1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1255327871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36dab1320 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- 172.21.15.116:0/1255327871 >> 172.21.15.116:0/1255327871 conn(0x55d36da36340 msgr2=0x55d36da65d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- 172.21.15.116:0/1255327871 shutdown_connections 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.415+0000 7f387d1c3000 1 -- 172.21.15.116:0/1255327871 wait complete. 2024-05-14T11:19:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- start start 2024-05-14T11:19:07.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 Processor -- start 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d9496b0 0x55d36da6c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 0x55d36da96770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d36d8b3050 con 0x55d36d5777c0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d36d86fec0 con 0x55d36d9496b0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d36d856c90 con 0x55d36d95ff80 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 10 client.0 _ll_register_callbacks cb 0x55d36da52e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39972/0 (socket says 172.21.15.116:39972) 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 -- 172.21.15.116:0/1822967952 learned_addr learned my addr 172.21.15.116:0/1822967952 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3866000640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d9496b0 0x55d36da6c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386f400640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 0x55d36da96770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 -- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 msgr2=0x55d36da96770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 0x55d36da96770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 -- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d9496b0 msgr2=0x55d36da6c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d9496b0 0x55d36da6c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 -- 172.21.15.116:0/1822967952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36d9616f0 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386f400640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 0x55d36da96770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f386ea00640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3860000fc0 tx=0x7f3860000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49e8daab856ef6d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860085dd0 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f3860056710 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3860079050 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 10 client.5880 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55d36d95d6a0 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3860056910 con 0x55d36d5777c0 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 10 client.5880 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 10 client.5880 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.419+0000 7f387d1c3000 20 client.5880 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d55c7d0 0x55d36dac28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 4 client.5880 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d36d84fbe0 con 0x55d36d55c7d0 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3866000640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d55c7d0 0x55d36dac28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3853e00640 10 client.5880 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3866000640 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d55c7d0 0x55d36dac28e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e93a91ac661d02 server_cookie=35863bb2a95a534c in_seq=0 out_seq=0 2024-05-14T11:19:07.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3853e00640 1 -- 172.21.15.116:0/1822967952 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d36d84fbe0 con 0x55d36d55c7d0 2024-05-14T11:19:07.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3853e00640 10 client.5880 handle_command_reply: tid=0 2024-05-14T11:19:07.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 client.5880 shutdown 2024-05-14T11:19:07.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 msgr2=0x55d36da6c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3860000fc0 tx=0x7f3860000bd0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 shutdown_connections 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d9496b0 0x55d36da6c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d55c7d0 0x55d36dac28e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d95ff80 0x55d36da96770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 --2- 172.21.15.116:0/1822967952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da6c3e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f3853e00640 0 client.5880 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 >> 172.21.15.116:0/1822967952 conn(0x55d36da36340 msgr2=0x55d36da66490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 shutdown_connections 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 1 -- 172.21.15.116:0/1822967952 wait complete. 2024-05-14T11:19:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.423+0000 7f387d1c3000 20 client.5880 trim_cache size 0 max 16384 2024-05-14T11:19:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 Processor -- start 2024-05-14T11:19:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 -- start start 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36da96370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36da662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3870204ba0 con 0x55d36d5777c0 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d36d85a7d0 con 0x55d36d95ff80 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d36d8b3050 con 0x55d36d9496b0 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39978/0 (socket says 172.21.15.116:39978) 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 -- 172.21.15.116:0/2456088564 learned_addr learned my addr 172.21.15.116:0/2456088564 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386f400640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36da662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f3866000640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36da96370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 -- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 msgr2=0x55d36da96370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36da96370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 -- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 msgr2=0x55d36da662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36da662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 -- 172.21.15.116:0/2456088564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36d890af0 con 0x55d36d5777c0 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386f400640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36da662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f3866000640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36da96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f386ea00640 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f386005ad30 tx=0x7f3860088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ef6154ce010608 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f385ea00640 1 -- 172.21.15.116:0/2456088564 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860085dd0 con 0x55d36d5777c0 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f385ea00640 1 -- 172.21.15.116:0/2456088564 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f385ea00640 1 -- 172.21.15.116:0/2456088564 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386007bd10 con 0x55d36d5777c0 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 -- 172.21.15.116:0/2456088564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 msgr2=0x55d36da95e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.443+0000 7f387d1c3000 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f386005ad30 tx=0x7f3860088040 comp rx=0 tx=0).stop 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- 172.21.15.116:0/2456088564 shutdown_connections 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36da662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36da96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- 172.21.15.116:0/2456088564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36da95e30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- 172.21.15.116:0/2456088564 >> 172.21.15.116:0/2456088564 conn(0x55d36da36340 msgr2=0x55d36da95720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- 172.21.15.116:0/2456088564 shutdown_connections 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- 172.21.15.116:0/2456088564 wait complete. 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- start start 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 Processor -- start 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36db240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dade7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d36d8b5f80 con 0x55d36d5777c0 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d36d8b4020 con 0x55d36d95ff80 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d36d8c4490 con 0x55d36d9496b0 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 10 client.0 _ll_register_callbacks cb 0x7f386005a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39990/0 (socket says 172.21.15.116:39990) 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 -- 172.21.15.116:0/873331466 learned_addr learned my addr 172.21.15.116:0/873331466 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386f400640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dade7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f3866000640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36db240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 -- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 msgr2=0x55d36db240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36db240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 -- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 msgr2=0x55d36dade7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dade7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 -- 172.21.15.116:0/873331466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36d88fe40 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f386ea00640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f386005ae70 tx=0x7f3860000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556084f911f5c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f3853e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860063b50 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f3860060230 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.447+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3860060450 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 10 client.5886 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55d36d9374e0 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386005a000 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f386007bb10 con 0x55d36d5777c0 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 10 client.5886 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 10 client.5886 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 20 client.5886 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d54a830 0x55d36dadf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 4 client.5886 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:19:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "status"}) -- 0x55d36d8c4ca0 con 0x55d36d54a830 2024-05-14T11:19:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3866000640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d54a830 0x55d36dadf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 10 client.5886 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3866000640 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d54a830 0x55d36dadf160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118ee833b421cd0c server_cookie=1ab73b7cfc688630 in_seq=0 out_seq=0 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 1 -- 172.21.15.116:0/873331466 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55d36d8c4ca0 con 0x55d36d54a830 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 10 client.5886 handle_command_reply: tid=0 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 client.5886 shutdown 2024-05-14T11:19:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 msgr2=0x55d36db23b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f386005ae70 tx=0x7f3860000ee0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 shutdown_connections 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d36d95ff80 0x55d36dade7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55d36d54a830 0x55d36dadf160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d36d9496b0 0x55d36db240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 --2- 172.21.15.116:0/873331466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d36d5777c0 0x55d36db23b70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3853e00640 0 client.5886 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 >> 172.21.15.116:0/873331466 conn(0x55d36da36340 msgr2=0x55d36dabdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 shutdown_connections 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 1 -- 172.21.15.116:0/873331466 wait complete. 2024-05-14T11:19:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f387d1c3000 20 client.5886 trim_cache size 0 max 16384 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5721, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 10.536701974, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 34, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 60, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 60, 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 15.386302342 2024-05-14T11:19:07.460 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 msgr2=0x7f384c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 0x7f384c071910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f38580b1000 tx=0x7f38580a9070 comp rx=0 tx=0).stop 2024-05-14T11:19:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 msgr2=0x7f387014d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f386000c3c0 tx=0x7f386000c880 comp rx=0 tx=0).stop 2024-05-14T11:19:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 shutdown_connections 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f384c06f400 0x7f384c071910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38700b0770 0x7f3870159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3870072a30 0x7f38701592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 --2- 172.21.15.116:0/3408252604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f387006f730 0x7f387014d730 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 >> 172.21.15.116:0/3408252604 conn(0x7f38700b4620 msgr2=0x7f387015bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 shutdown_connections 2024-05-14T11:19:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.451+0000 7f3875c00640 1 -- 172.21.15.116:0/3408252604 wait complete. 2024-05-14T11:19:07.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:19:07.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 Processor -- start 2024-05-14T11:19:07.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 -- start start 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f22700d10b0 0x7f22700cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2270161500 0x7f22700ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22700d1e40 con 0x7f22700b2b70 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f22700d2120 con 0x7f22700d10b0 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f2275200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f22700d2400 con 0x7f2270161500 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f22700d10b0 0x7f22700cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f22700b2b70 0x7f22700b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60618/0 (socket says 172.21.15.116:60618) 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226fe00640 1 -- 172.21.15.116:0/1170449985 learned_addr learned my addr 172.21.15.116:0/1170449985 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226ea00640 1 --2- 172.21.15.116:0/1170449985 >> v2:172.21.15.149:3300/0 conn(0x7f2270161500 0x7f22700ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226f400640 1 -- 172.21.15.116:0/1170449985 >> v2:172.21.15.149:3300/0 conn(0x7f2270161500 msgr2=0x7f22700ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:07.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226f400640 1 --2- 172.21.15.116:0/1170449985 >> v2:172.21.15.149:3300/0 conn(0x7f2270161500 0x7f22700ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226f400640 1 -- 172.21.15.116:0/1170449985 >> v1:172.21.15.116:6789/0 conn(0x7f22700b2b70 legacy=0x7f22700b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.619+0000 7f226f400640 1 -- 172.21.15.116:0/1170449985 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22700d26e0 con 0x7f22700d10b0 2024-05-14T11:19:07.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f226f400640 1 --2- 172.21.15.116:0/1170449985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f22700cdcd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f227006d880 tx=0x7f226001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757b03cce882c102 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f226e000640 1 -- 172.21.15.116:0/1170449985 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22600b2070 con 0x7f22700d10b0 2024-05-14T11:19:07.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f226e000640 1 -- 172.21.15.116:0/1170449985 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22600a9070 con 0x7f22700d10b0 2024-05-14T11:19:07.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f226e000640 1 -- 172.21.15.116:0/1170449985 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22600ad880 con 0x7f22700d10b0 2024-05-14T11:19:07.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 -- 172.21.15.116:0/1170449985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 msgr2=0x7f22700cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 --2- 172.21.15.116:0/1170449985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f22700cdcd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f227006d880 tx=0x7f226001c450 comp rx=0 tx=0).stop 2024-05-14T11:19:07.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 -- 172.21.15.116:0/1170449985 shutdown_connections 2024-05-14T11:19:07.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 --2- 172.21.15.116:0/1170449985 >> v2:172.21.15.149:3300/0 conn(0x7f2270161500 0x7f22700ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 --2- 172.21.15.116:0/1170449985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f22700cdcd0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 -- 172.21.15.116:0/1170449985 >> 172.21.15.116:0/1170449985 conn(0x7f22700b55c0 msgr2=0x7f22700d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 -- 172.21.15.116:0/1170449985 shutdown_connections 2024-05-14T11:19:07.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.623+0000 7f2275200640 1 -- 172.21.15.116:0/1170449985 wait complete. 2024-05-14T11:19:07.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 Processor -- start 2024-05-14T11:19:07.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 -- start start 2024-05-14T11:19:07.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 0x7f227014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2270161500 0x7f227014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f227016ae80 con 0x7f22700d10b0 2024-05-14T11:19:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22700d1bc0 con 0x7f22700b2b70 2024-05-14T11:19:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f2275200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f227016b000 con 0x7f2270161500 2024-05-14T11:19:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38800/0 (socket says 172.21.15.116:38800) 2024-05-14T11:19:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 -- 172.21.15.116:0/539169149 learned_addr learned my addr 172.21.15.116:0/539169149 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 -- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2270161500 msgr2=0x7f227014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2270161500 0x7f227014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226f400640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 0x7f227014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 -- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 msgr2=0x7f227014e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 0x7f227014e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226ea00640 1 -- 172.21.15.116:0/539169149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701506b0 con 0x7f22700d10b0 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.627+0000 7f226f400640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 0x7f227014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f226ea00640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f225c00d620 tx=0x7f225c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00b620 con 0x7f22700d10b0 2024-05-14T11:19:07.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c019070 con 0x7f22700d10b0 2024-05-14T11:19:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2270150990 con 0x7f22700d10b0 2024-05-14T11:19:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00b9e0 con 0x7f22700d10b0 2024-05-14T11:19:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2270150f50 con 0x7f22700d10b0 2024-05-14T11:19:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270150410 con 0x7f22700d10b0 2024-05-14T11:19:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f225c00bb80 con 0x7f22700d10b0 2024-05-14T11:19:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 0x7f2250071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f225c08f900 con 0x7f22700d10b0 2024-05-14T11:19:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.631+0000 7f226f400640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 0x7f2250071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.635+0000 7f226f400640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 0x7f2250071910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22600af000 tx=0x7f2260008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:07.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.639+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2270150410 con 0x7f22700d10b0 2024-05-14T11:19:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.827+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2270050880 con 0x7f22700d10b0 2024-05-14T11:19:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.827+0000 7f2267400640 1 -- 172.21.15.116:0/539169149 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f225c05c660 con 0x7f22700d10b0 2024-05-14T11:19:07.832 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 msgr2=0x7f2250071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 0x7f2250071910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22600af000 tx=0x7f2260008040 comp rx=0 tx=0).stop 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 msgr2=0x7f227006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f225c00d620 tx=0x7f225c00dae0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 shutdown_connections 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225006f400 0x7f2250071910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2270161500 0x7f227014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22700d10b0 0x7f227006eee0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 --2- 172.21.15.116:0/539169149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22700b2b70 0x7f227014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 >> 172.21.15.116:0/539169149 conn(0x7f22700b55c0 msgr2=0x7f22700b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 shutdown_connections 2024-05-14T11:19:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:07.831+0000 7f2275200640 1 -- 172.21.15.116:0/539169149 wait complete. 2024-05-14T11:19:07.853 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:19:07.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:19:07.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:19:07.853 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:19:07.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:07.854 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:19:07.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:19:07.854 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:19:07.854 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:19:07.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:07.854 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.863 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42255sec preferred_lft 42255sec 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.864 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:19:07.865 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:19:07.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:07.866 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:19:07.929 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:19:07.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:07.931 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:19:07.983 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:19:07.984 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:19:08.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:19:08.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:19:08.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.117 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:19:08.125 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:19:08.125 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:19:08.126 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:19:08.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.126 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:19:08.187 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:19:08.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:19:08.187 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:08.239 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:19:08.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:19:08.240 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:08.291 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:19:08.351 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:19:08.405 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:19:08.405 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:19:08.405 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:19:08.405 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:19:08.406 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:19:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:19:08.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.408 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:19:08.464 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:19:08.464 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:19:08.511 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:19:08.531 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:19:08.581 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:19:08.581 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:19:08.581 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:19:08.581 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:19:08.582 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:19:08.583 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:19:08.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.584 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:19:08.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:19:08.611+0000 7f72b834c580 -1 init, newargv = 0x7f72a001b560 newargc=15 2024-05-14T11:19:08.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[40086]: starting ceph client 2024-05-14T11:19:08.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[40086]: starting fuse 2024-05-14T11:19:08.624 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:19:08.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:19:08.625 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:19:08.626 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:19:08.747 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.40086.asok 2024-05-14T11:19:08.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:19:08.758 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40086.asok 2024-05-14T11:19:08.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.758 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40086.asok status 2024-05-14T11:19:08.936 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:08.936 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:19:08.936 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:19:08.936 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "40086", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5898, 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 5898 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:19:08.937 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2941563678 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2941563678 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.5898 192.168.0.1:0/2941563678", 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/2941563678", 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 34, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 60, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:19:08.938 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:19:08.939 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:08.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.952 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:19:08.952 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:08.960 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:19:08.961 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:08.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:08.961 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:09.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:09.027 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-14T11:19:09.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:09.087 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-14T11:19:09.139 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:19:09.139 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:19:09.139 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:19:09.139 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:19:09.140 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-14T11:19:09.140 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:19:09.243 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:19:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:19:09.252 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:09.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.447+0000 7f13e9400640 1 Processor -- start 2024-05-14T11:19:09.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.447+0000 7f13e9400640 1 -- start start 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13e4156fa0 0x7f13e406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 0x7f13e40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f13e41573b0 con 0x7f13e4156fa0 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f13e4161120 con 0x7f13e40d1670 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40d1fc0 con 0x7f13e406e9e0 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13e4156fa0 0x7f13e406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13e4156fa0 0x7f13e406ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38806/0 (socket says 172.21.15.116:38806) 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 -- 172.21.15.116:0/3886409834 learned_addr learned my addr 172.21.15.116:0/3886409834 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e1e00640 1 --2- 172.21.15.116:0/3886409834 >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 0x7f13e40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e3200640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 -- 172.21.15.116:0/3886409834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 msgr2=0x7f13e406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 -- 172.21.15.116:0/3886409834 >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 msgr2=0x7f13e40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 --2- 172.21.15.116:0/3886409834 >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 0x7f13e40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 -- 172.21.15.116:0/3886409834 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4160510 con 0x7f13e4156fa0 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e1e00640 1 --2- 172.21.15.116:0/3886409834 >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 0x7f13e40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:09.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e3200640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:09.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e2800640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e406ffe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13e4070c70 tx=0x7f13d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9132550f3a15a3d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e1400640 1 -- 172.21.15.116:0/3886409834 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40aa610 con 0x7f13e4156fa0 2024-05-14T11:19:09.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e1400640 1 -- 172.21.15.116:0/3886409834 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d40af070 con 0x7f13e4156fa0 2024-05-14T11:19:09.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e1400640 1 -- 172.21.15.116:0/3886409834 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40aa9d0 con 0x7f13e4156fa0 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- 172.21.15.116:0/3886409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 msgr2=0x7f13e406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e406ffe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13e4070c70 tx=0x7f13d401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- 172.21.15.116:0/3886409834 shutdown_connections 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- 172.21.15.116:0/3886409834 >> v2:172.21.15.149:3300/0 conn(0x7f13e40d1670 0x7f13e40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 --2- 172.21.15.116:0/3886409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e406ffe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- 172.21.15.116:0/3886409834 >> 172.21.15.116:0/3886409834 conn(0x7f13e40b4620 msgr2=0x7f13e40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- 172.21.15.116:0/3886409834 shutdown_connections 2024-05-14T11:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 -- 172.21.15.116:0/3886409834 wait complete. 2024-05-14T11:19:09.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.451+0000 7f13e9400640 1 Processor -- start 2024-05-14T11:19:09.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- start start 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13e40d1670 0x7f13e414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4070520 con 0x7f13e4156fa0 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13e406f740 con 0x7f13e40d1670 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40706a0 con 0x7f13e406e9e0 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38822/0 (socket says 172.21.15.116:38822) 2024-05-14T11:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 -- 172.21.15.116:0/3247593021 learned_addr learned my addr 172.21.15.116:0/3247593021 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e2800640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e1e00640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13e40d1670 0x7f13e414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 -- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 msgr2=0x7f13e4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 -- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13e40d1670 msgr2=0x7f13e414daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13e40d1670 0x7f13e414daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 -- 172.21.15.116:0/3247593021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e4073b90 con 0x7f13e4156fa0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e3200640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00cd50 tx=0x7f13dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00f680 con 0x7f13e4156fa0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13dc00f840 con 0x7f13e4156fa0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00fc30 con 0x7f13e4156fa0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e41547d0 con 0x7f13e4156fa0 2024-05-14T11:19:09.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e4154d90 con 0x7f13e4156fa0 2024-05-14T11:19:09.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.455+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e406a9c0 con 0x7f13e4156fa0 2024-05-14T11:19:09.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.459+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13dc0213f0 con 0x7f13e4156fa0 2024-05-14T11:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.459+0000 7f13daa00640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 0x7f13c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.459+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f13dc090110 con 0x7f13e4156fa0 2024-05-14T11:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.459+0000 7f13e2800640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 0x7f13c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.459+0000 7f13e2800640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 0x7f13c4071910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f13d40ae000 tx=0x7f13d40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.463+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f13dc014030 con 0x7f13e4156fa0 2024-05-14T11:19:09.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.651+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e414e030 con 0x7f13e4156fa0 2024-05-14T11:19:09.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13daa00640 1 -- 172.21.15.116:0/3247593021 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5191 (secure 0 0 0) 0x7f13dc05c810 con 0x7f13e4156fa0 2024-05-14T11:19:09.656 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:09.657 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:18:59.326933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:09.657 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 34 2024-05-14T11:19:09.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 msgr2=0x7f13c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 0x7f13c4071910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f13d40ae000 tx=0x7f13d40a7000 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 msgr2=0x7f13e41589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00cd50 tx=0x7f13dc00a650 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 shutdown_connections 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f13c406f400 0x7f13c4071910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13e40d1670 0x7f13e414daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13e4156fa0 0x7f13e41589f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 --2- 172.21.15.116:0/3247593021 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13e406e9e0 0x7f13e4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 >> 172.21.15.116:0/3247593021 conn(0x7f13e40b4620 msgr2=0x7f13e4150560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 shutdown_connections 2024-05-14T11:19:09.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.655+0000 7f13e9400640 1 -- 172.21.15.116:0/3247593021 wait complete. 2024-05-14T11:19:09.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-14T11:19:09.857 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:19:09.857 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:19:09.858 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-14T11:19:09.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.911+0000 7fd81b370080 1 Processor -- start 2024-05-14T11:19:09.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.911+0000 7fd81b370080 1 -- start start 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 0x562464c55a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71fa0 0x562464c55f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562464c58d00 con 0x562464cfd980 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562464bd0aa0 con 0x562464c71fa0 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562464c711d0 con 0x562464c71420 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38838/0 (socket says 172.21.15.116:38838) 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 -- 172.21.15.116:0/2956401326 learned_addr learned my addr 172.21.15.116:0/2956401326 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd813c00640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71fa0 0x562464c55f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd814600640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 0x562464c55a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 -- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 msgr2=0x562464c55a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 0x562464c55a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 -- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71fa0 msgr2=0x562464c55f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71fa0 0x562464c55f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 -- 172.21.15.116:0/2956401326 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562464c5a380 con 0x562464cfd980 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd814600640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 0x562464c55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd815000640 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c460 tx=0x7fd80c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480a6d7a290a63c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd812800640 1 -- 172.21.15.116:0/2956401326 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c010650 con 0x562464cfd980 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd812800640 1 -- 172.21.15.116:0/2956401326 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80c016070 con 0x562464cfd980 2024-05-14T11:19:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd812800640 1 -- 172.21.15.116:0/2956401326 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c010a50 con 0x562464cfd980 2024-05-14T11:19:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- 172.21.15.116:0/2956401326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 msgr2=0x562464ce1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c460 tx=0x7fd80c00c920 comp rx=0 tx=0).stop 2024-05-14T11:19:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- 172.21.15.116:0/2956401326 shutdown_connections 2024-05-14T11:19:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71fa0 0x562464c55f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464ce1a60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 --2- 172.21.15.116:0/2956401326 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71420 0x562464c55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- 172.21.15.116:0/2956401326 >> 172.21.15.116:0/2956401326 conn(0x562464c15e10 msgr2=0x562464c73ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- 172.21.15.116:0/2956401326 shutdown_connections 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- 172.21.15.116:0/2956401326 wait complete. 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 Processor -- start 2024-05-14T11:19:09.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.915+0000 7fd81b370080 1 -- start start 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71420 0x562464d96c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71fa0 0x562464c4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464d950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562464c5a900 con 0x562464cfd980 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562464c5a620 con 0x562464c71420 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562464cfe510 con 0x562464c71fa0 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464d950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464d950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38842/0 (socket says 172.21.15.116:38842) 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 -- 172.21.15.116:0/2439484810 learned_addr learned my addr 172.21.15.116:0/2439484810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd814600640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71420 0x562464d96c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd813c00640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71fa0 0x562464c4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 -- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71fa0 msgr2=0x562464c4c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562464c71fa0 0x562464c4c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 -- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71420 msgr2=0x562464d96c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562464c71420 0x562464d96c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 -- 172.21.15.116:0/2439484810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562464d97320 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd815000640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464d950b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd80c007010 tx=0x7fd80c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd811400640 1 -- 172.21.15.116:0/2439484810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c026680 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd811400640 1 -- 172.21.15.116:0/2439484810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80c026840 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd811400640 1 -- 172.21.15.116:0/2439484810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c026c30 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562464d975c0 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.919+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562464d97b00 con 0x562464cfd980 2024-05-14T11:19:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd811400640 1 -- 172.21.15.116:0/2439484810 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd80c011420 con 0x562464cfd980 2024-05-14T11:19:09.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd811400640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7f406dde0 0x7fd7f40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd811400640 1 -- 172.21.15.116:0/2439484810 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fd80c090410 con 0x562464cfd980 2024-05-14T11:19:09.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd814600640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7f406dde0 0x7fd7f40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd81b370080 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x562464d9e6d0 0x562464da0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:09.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60) -- 0x562464d98240 con 0x562464d9e6d0 2024-05-14T11:19:09.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd814600640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7f406dde0 0x7fd7f40702f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd80400f4f0 tx=0x7fd8040017f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd813c00640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x562464d9e6d0 0x562464da0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:09.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd813c00640 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x562464d9e6d0 0x562464da0af0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd813c00640 1 -- 172.21.15.116:0/2439484810 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fd8000b7040 con 0x562464d9e6d0 2024-05-14T11:19:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x562464d9e6d0 msgr2=0x562464da0af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.923+0000 7fd81b370080 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x562464d9e6d0 0x562464da0af0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7f406dde0 msgr2=0x7fd7f40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7f406dde0 0x7fd7f40702f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd80400f4f0 tx=0x7fd8040017f0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 msgr2=0x562464d950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 --2- 172.21.15.116:0/2439484810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562464cfd980 0x562464d950b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd80c007010 tx=0x7fd80c0033b0 comp rx=0 tx=0).stop 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd815000640 1 -- 172.21.15.116:0/2439484810 reap_dead start 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 shutdown_connections 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 >> 172.21.15.116:0/2439484810 conn(0x562464c15e10 msgr2=0x562464cf9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 shutdown_connections 2024-05-14T11:19:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:09.927+0000 7fd81b370080 1 -- 172.21.15.116:0/2439484810 wait complete. 2024-05-14T11:19:09.930 INFO:teuthology.orchestra.run.smithi116.stdout:Xfiledir_test 2024-05-14T11:19:09.933 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:09.933 DEBUG:teuthology.orchestra.run.smithi116:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: "ancestors": [ 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1099511627776, 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "file", 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 2 2024-05-14T11:19:09.983 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: "dirino": 1, 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "dir_test", 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 4 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 10, 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: "old_pools": [] 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:09.984 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:09.988 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-14T11:19:10.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 Processor -- start 2024-05-14T11:19:10.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 -- start start 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa177f1f0 0x557aa177dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa1780020 0x557aa1807e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557aa16ddaa0 con 0x557aa1767a20 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557aa16e1ae0 con 0x557aa1780020 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd399a94080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557aa177e570 con 0x557aa177f1f0 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38850/0 (socket says 172.21.15.116:38850) 2024-05-14T11:19:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 -- 172.21.15.116:0/3012999774 learned_addr learned my addr 172.21.15.116:0/3012999774 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392200640 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa177f1f0 0x557aa177dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd393600640 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa1780020 0x557aa1807e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 -- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa177f1f0 msgr2=0x557aa177dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa177f1f0 0x557aa177dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 -- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa1780020 msgr2=0x557aa1807e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa1780020 0x557aa1807e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 -- 172.21.15.116:0/3012999774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa1762f10 con 0x557aa1767a20 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.087+0000 7fd392c00640 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd384001ee0 tx=0x7fd38400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b47137b01ac958 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd390e00640 1 -- 172.21.15.116:0/3012999774 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384012020 con 0x557aa1767a20 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd390e00640 1 -- 172.21.15.116:0/3012999774 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38400d6b0 con 0x557aa1767a20 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd390e00640 1 -- 172.21.15.116:0/3012999774 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38400da80 con 0x557aa1767a20 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- 172.21.15.116:0/3012999774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 msgr2=0x557aa177d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd384001ee0 tx=0x7fd38400b810 comp rx=0 tx=0).stop 2024-05-14T11:19:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- 172.21.15.116:0/3012999774 shutdown_connections 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa1780020 0x557aa1807e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa177f1f0 0x557aa177dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- 172.21.15.116:0/3012999774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1767a20 0x557aa177d570 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- 172.21.15.116:0/3012999774 >> 172.21.15.116:0/3012999774 conn(0x557aa1722e10 msgr2=0x557aa180ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- 172.21.15.116:0/3012999774 shutdown_connections 2024-05-14T11:19:10.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- 172.21.15.116:0/3012999774 wait complete. 2024-05-14T11:19:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 Processor -- start 2024-05-14T11:19:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- start start 2024-05-14T11:19:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa1767a20 0x557aa180f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa177f1f0 0x557aa17f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 0x557aa177cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557aa17629f0 con 0x557aa1780020 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557aa17622b0 con 0x557aa177f1f0 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd399a94080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557aa17f58f0 con 0x557aa1767a20 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd393600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 0x557aa177cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd393600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 0x557aa177cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38856/0 (socket says 172.21.15.116:38856) 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd393600640 1 -- 172.21.15.116:0/1016190058 learned_addr learned my addr 172.21.15.116:0/1016190058 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd392c00640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa1767a20 0x557aa180f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.091+0000 7fd392200640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa177f1f0 0x557aa17f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 -- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa1767a20 msgr2=0x557aa180f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557aa1767a20 0x557aa180f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 -- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa177f1f0 msgr2=0x557aa17f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557aa177f1f0 0x557aa17f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 -- 172.21.15.116:0/1016190058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa1759120 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd393600640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 0x557aa177cac0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd38c004220 tx=0x7fd38c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd38b400640 1 -- 172.21.15.116:0/1016190058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c018070 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd38b400640 1 -- 172.21.15.116:0/1016190058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38c0030d0 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557aa17593c0 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd38b400640 1 -- 172.21.15.116:0/1016190058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0014e0 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa1764f90 con 0x557aa1780020 2024-05-14T11:19:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd38b400640 1 -- 172.21.15.116:0/1016190058 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd38c01b070 con 0x557aa1780020 2024-05-14T11:19:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.095+0000 7fd38b400640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd37406de10 0x7fd374070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd38b400640 1 -- 172.21.15.116:0/1016190058 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fd38c08d5f0 con 0x557aa1780020 2024-05-14T11:19:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd392c00640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd37406de10 0x7fd374070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x557aa1766440 0x557aa18b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) -- 0x557aa18b1270 con 0x557aa1766440 2024-05-14T11:19:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd392200640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x557aa1766440 0x557aa18b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd392c00640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd37406de10 0x7fd374070320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd384001ee0 tx=0x7fd384009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd392200640 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x557aa1766440 0x557aa18b0d30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd392200640 1 -- 172.21.15.116:0/1016190058 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v60'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd3800a7a80 con 0x557aa1766440 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x557aa1766440 msgr2=0x557aa18b0d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x557aa1766440 0x557aa18b0d30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd37406de10 msgr2=0x7fd374070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd37406de10 0x7fd374070320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd384001ee0 tx=0x7fd384009000 comp rx=0 tx=0).stop 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 msgr2=0x557aa177cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 --2- 172.21.15.116:0/1016190058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557aa1780020 0x557aa177cac0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd38c004220 tx=0x7fd38c00d5f0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd393600640 1 -- 172.21.15.116:0/1016190058 reap_dead start 2024-05-14T11:19:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 shutdown_connections 2024-05-14T11:19:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 >> 172.21.15.116:0/1016190058 conn(0x557aa1722e10 msgr2=0x557aa177d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 shutdown_connections 2024-05-14T11:19:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.099+0000 7fd399a94080 1 -- 172.21.15.116:0/1016190058 wait complete. 2024-05-14T11:19:10.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:10.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-14T11:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 Processor -- start 2024-05-14T11:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 -- start start 2024-05-14T11:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb17c15dfc0 0x7fb17c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0d20d0 0x7fb17c0d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 0x7fb17c0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb17c0b67c0 con 0x7fb17c15dfc0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb17c15ef90 con 0x7fb17c0b6c50 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb17c0d2a20 con 0x7fb17c0d20d0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb17c15dfc0 0x7fb17c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb17c15dfc0 0x7fb17c15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38872/0 (socket says 172.21.15.116:38872) 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 -- 172.21.15.116:0/370432321 learned_addr learned my addr 172.21.15.116:0/370432321 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17a800640 1 --2- 172.21.15.116:0/370432321 >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 0x7fb17c0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb171400640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0d20d0 0x7fb17c0d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 -- 172.21.15.116:0/370432321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0d20d0 msgr2=0x7fb17c0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0d20d0 0x7fb17c0d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 -- 172.21.15.116:0/370432321 >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 msgr2=0x7fb17c0b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 --2- 172.21.15.116:0/370432321 >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 0x7fb17c0b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb179e00640 1 -- 172.21.15.116:0/370432321 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c15e9d0 con 0x7fb17c15dfc0 2024-05-14T11:19:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.259+0000 7fb17a800640 1 --2- 172.21.15.116:0/370432321 >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 0x7fb17c0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb179e00640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c15e3d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074530 tx=0x7fb16c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1058360fe08264 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb179400640 1 -- 172.21.15.116:0/370432321 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0ab620 con 0x7fb17c15dfc0 2024-05-14T11:19:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb179400640 1 -- 172.21.15.116:0/370432321 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c0b0070 con 0x7fb17c15dfc0 2024-05-14T11:19:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb179400640 1 -- 172.21.15.116:0/370432321 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0ab9e0 con 0x7fb17c15dfc0 2024-05-14T11:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 -- 172.21.15.116:0/370432321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 msgr2=0x7fb17c15e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c15e3d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb17c074530 tx=0x7fb16c01c460 comp rx=0 tx=0).stop 2024-05-14T11:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 -- 172.21.15.116:0/370432321 shutdown_connections 2024-05-14T11:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 --2- 172.21.15.116:0/370432321 >> v2:172.21.15.149:3300/0 conn(0x7fb17c0b6c50 0x7fb17c0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0d20d0 0x7fb17c0d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 --2- 172.21.15.116:0/370432321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c15e3d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 -- 172.21.15.116:0/370432321 >> 172.21.15.116:0/370432321 conn(0x7fb17c0b4fe0 msgr2=0x7fb17c0b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 -- 172.21.15.116:0/370432321 shutdown_connections 2024-05-14T11:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.263+0000 7fb17b200640 1 -- 172.21.15.116:0/370432321 wait complete. 2024-05-14T11:19:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 Processor -- start 2024-05-14T11:19:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- start start 2024-05-14T11:19:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 0x7fb17c072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c0d20d0 0x7fb17c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c15ba60 con 0x7fb17c15dfc0 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c15b8e0 con 0x7fb17c0d20d0 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb17c0cf640 con 0x7fb17c0b6c50 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38884/0 (socket says 172.21.15.116:38884) 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 -- 172.21.15.116:0/1513129334 learned_addr learned my addr 172.21.15.116:0/1513129334 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb179e00640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 0x7fb17c072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb171400640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c0d20d0 0x7fb17c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 -- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 msgr2=0x7fb17c072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 0x7fb17c072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 -- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c0d20d0 msgr2=0x7fb17c155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c0d20d0 0x7fb17c155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 -- 172.21.15.116:0/1513129334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c1653b0 con 0x7fb17c15dfc0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb179e00640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 0x7fb17c072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17a800640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb174007e10 tx=0x7fb17400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb172a00640 1 -- 172.21.15.116:0/1513129334 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17401a070 con 0x7fb17c15dfc0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb172a00640 1 -- 172.21.15.116:0/1513129334 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1740039a0 con 0x7fb17c15dfc0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb172a00640 1 -- 172.21.15.116:0/1513129334 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174003d90 con 0x7fb17c15dfc0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c165650 con 0x7fb17c15dfc0 2024-05-14T11:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.267+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c06ed70 con 0x7fb17c15dfc0 2024-05-14T11:19:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.271+0000 7fb172a00640 1 -- 172.21.15.116:0/1513129334 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb17401d070 con 0x7fb17c15dfc0 2024-05-14T11:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 Processor -- start 2024-05-14T11:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb172a00640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 0x7fb158071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 -- start start 2024-05-14T11:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb172a00640 1 -- 172.21.15.116:0/1513129334 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb17408f650 con 0x7fb17c15dfc0 2024-05-14T11:19:10.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 0x7fb158071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 0x7fb158071940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0b4560 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d187a7a0 0x5645d19cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1452570 0x5645d19cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5645d1868a80 con 0x5645d1863b10 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5645d17d11c0 con 0x5645d187a7a0 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5645d1789d60 con 0x5645d1452570 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38886/0 (socket says 172.21.15.116:38886) 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 -- 172.21.15.116:0/221681054 learned_addr learned my addr 172.21.15.116:0/221681054 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb179e00640 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d187a7a0 0x5645d19cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb17a800640 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1452570 0x5645d19cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 -- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1452570 msgr2=0x5645d19cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1452570 0x5645d19cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 -- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d187a7a0 msgr2=0x5645d19cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d187a7a0 0x5645d19cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 -- 172.21.15.116:0/221681054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d178e8c0 con 0x5645d1863b10 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb171400640 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb16400f9f0 tx=0x7fb164012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3491ed82deb96a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb16a600640 1 -- 172.21.15.116:0/221681054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16401b050 con 0x5645d1863b10 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb16a600640 1 -- 172.21.15.116:0/221681054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb164025070 con 0x5645d1863b10 2024-05-14T11:19:10.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb16a600640 1 -- 172.21.15.116:0/221681054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164014de0 con 0x5645d1863b10 2024-05-14T11:19:10.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 -- 172.21.15.116:0/221681054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 msgr2=0x5645d19cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.275+0000 7fb1882ce000 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb16400f9f0 tx=0x7fb164012000 comp rx=0 tx=0).stop 2024-05-14T11:19:10.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- 172.21.15.116:0/221681054 shutdown_connections 2024-05-14T11:19:10.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1452570 0x5645d19cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1863b10 0x5645d19cc130 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- 172.21.15.116:0/221681054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d187a7a0 0x5645d19cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- 172.21.15.116:0/221681054 >> 172.21.15.116:0/221681054 conn(0x5645d1491750 msgr2=0x5645d1980140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- 172.21.15.116:0/221681054 shutdown_connections 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- 172.21.15.116:0/221681054 wait complete. 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- start start 2024-05-14T11:19:10.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 Processor -- start 2024-05-14T11:19:10.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5645d179bcb0 con 0x5645d187a7a0 2024-05-14T11:19:10.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5645d188c5a0 con 0x5645d1452570 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5645d178d140 con 0x5645d1863b10 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb1882ce000 10 client.0 _ll_register_callbacks cb 0x5645d196d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38902/0 (socket says 172.21.15.116:38902) 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 -- 172.21.15.116:0/3551358251 learned_addr learned my addr 172.21.15.116:0/3551358251 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb179e00640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb171400640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 -- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 msgr2=0x5645d19d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 -- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 msgr2=0x5645d19d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 -- 172.21.15.116:0/3551358251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d177b920 con 0x5645d187a7a0 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb179e00640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:10.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb17a800640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d9a0 tx=0x7fb17405b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805d4fa4dabc825 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.279+0000 7fb15ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17408b060 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb174086b90 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb164025070 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb174073070 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 10 client.5934 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5645d1789fb0 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb164025070 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fb17405af10 con 0x5645d187a7a0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 10 client.5934 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 10 client.5934 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 20 client.5934 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1476ac0 0x5645d198d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 4 client.5934 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d1769f90 con 0x5645d1476ac0 2024-05-14T11:19:10.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb179e00640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1476ac0 0x5645d198d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb179e00640 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1476ac0 0x5645d198d740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf9f3e0fe1055e server_cookie=13d7482366255ac4 in_seq=0 out_seq=0 2024-05-14T11:19:10.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 10 client.5934 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 1 -- 172.21.15.116:0/3551358251 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645d1769f90 con 0x5645d1476ac0 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 10 client.5934 handle_command_reply: tid=0 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 client.5934 shutdown 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 msgr2=0x5645d198c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d9a0 tx=0x7fb17405b730 comp rx=0 tx=0).stop 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 shutdown_connections 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1476ac0 0x5645d198d740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d198c9a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 --2- 172.21.15.116:0/3551358251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 >> 172.21.15.116:0/3551358251 conn(0x5645d1491750 msgr2=0x5645d1980140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb15ea00640 0 client.5934 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 shutdown_connections 2024-05-14T11:19:10.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 1 -- 172.21.15.116:0/3551358251 wait complete. 2024-05-14T11:19:10.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.283+0000 7fb1882ce000 20 client.5934 trim_cache size 0 max 16384 2024-05-14T11:19:10.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 Processor -- start 2024-05-14T11:19:10.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- start start 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c205240 con 0x5645d187a7a0 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1fd270 con 0x5645d1452570 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb17c204bf0 con 0x5645d1863b10 2024-05-14T11:19:10.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38904/0 (socket says 172.21.15.116:38904) 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 -- 172.21.15.116:0/306585194 learned_addr learned my addr 172.21.15.116:0/306585194 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb179e00640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb171400640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 -- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 msgr2=0x5645d19e2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19e2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 -- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 msgr2=0x5645d19e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 -- 172.21.15.116:0/306585194 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d198cd10 con 0x5645d187a7a0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb179e00640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb171400640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb17a800640 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb174007d80 tx=0x7fb174073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618c69440c75e5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb169c00640 1 -- 172.21.15.116:0/306585194 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17408b040 con 0x5645d187a7a0 2024-05-14T11:19:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb169c00640 1 -- 172.21.15.116:0/306585194 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174060060 con 0x5645d187a7a0 2024-05-14T11:19:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb169c00640 1 -- 172.21.15.116:0/306585194 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174064b10 con 0x5645d187a7a0 2024-05-14T11:19:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- 172.21.15.116:0/306585194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 msgr2=0x5645d19cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb174007d80 tx=0x7fb174073000 comp rx=0 tx=0).stop 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- 172.21.15.116:0/306585194 shutdown_connections 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d187a7a0 0x5645d19cfee0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d1863b10 0x5645d19e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 --2- 172.21.15.116:0/306585194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1452570 0x5645d19e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- 172.21.15.116:0/306585194 >> 172.21.15.116:0/306585194 conn(0x5645d1491750 msgr2=0x5645d1982960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- 172.21.15.116:0/306585194 shutdown_connections 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.303+0000 7fb1882ce000 1 -- 172.21.15.116:0/306585194 wait complete. 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- start start 2024-05-14T11:19:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 Processor -- start 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1863b10 0x5645d19fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d187a7a0 0x5645d19fb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5645d17d57f0 con 0x5645d1452570 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5645d17d04d0 con 0x5645d1863b10 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5645d17de5c0 con 0x5645d187a7a0 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 10 client.0 _ll_register_callbacks cb 0x7fb174007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38908/0 (socket says 172.21.15.116:38908) 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 -- 172.21.15.116:0/816285674 learned_addr learned my addr 172.21.15.116:0/816285674 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb171400640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1863b10 0x5645d19fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb17a800640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d187a7a0 0x5645d19fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 -- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d187a7a0 msgr2=0x5645d19fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d187a7a0 0x5645d19fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 -- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1863b10 msgr2=0x5645d19fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1863b10 0x5645d19fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 -- 172.21.15.116:0/816285674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d18520d0 con 0x5645d1452570 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb179e00640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0097c0 tx=0x7fb16c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd19273688726ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb15ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0c5060 con 0x5645d1452570 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb16c0aba90 con 0x5645d1452570 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174060060 con 0x5645d1452570 2024-05-14T11:19:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb16c0cb050 con 0x5645d1452570 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 10 client.5940 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5645d1828bc0 con 0x5645d1452570 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.307+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174060060 con 0x5645d1452570 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fb16c0ce020 con 0x5645d1452570 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 10 client.5940 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 10 client.5940 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 20 client.5940 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1464b20 0x5645d1a3e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 4 client.5940 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x5645d17dedd0 con 0x5645d1464b20 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb171400640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1464b20 0x5645d1a3e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb15ea00640 10 client.5940 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb171400640 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1464b20 0x5645d1a3e5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5be6a19997aab3 server_cookie=16ae45323bc93104 in_seq=0 out_seq=0 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb15ea00640 1 -- 172.21.15.116:0/816285674 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5645d17dedd0 con 0x5645d1464b20 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb15ea00640 10 client.5940 handle_command_reply: tid=0 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 client.5940 shutdown 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 msgr2=0x5645d1982130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0097c0 tx=0x7fb16c0046d0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 shutdown_connections 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645d1863b10 0x5645d19fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5645d1464b20 0x5645d1a3e5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645d187a7a0 0x5645d19fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 --2- 172.21.15.116:0/816285674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645d1452570 0x5645d1982130 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 >> 172.21.15.116:0/816285674 conn(0x5645d1491750 msgr2=0x5645d19e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb15ea00640 0 client.5940 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.317 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fe6e9200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5898=0-4194304@1} caps={5898=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5898 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562bde2fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 shutdown_connections 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 1 -- 172.21.15.116:0/816285674 wait complete. 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb1882ce000 20 client.5940 trim_cache size 0 max 16384 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "00bb14ac-91ef-4223-9fbd-2450fe8d0cb9", 2024-05-14T11:19:10.318 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:19:10.319 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:10.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 msgr2=0x7fb158071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 0x7fb158071940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0b4560 tx=0x7fb164008040 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 msgr2=0x7fb17c0715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb174007e10 tx=0x7fb17400d4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 shutdown_connections 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb15806f430 0x7fb158071940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c0d20d0 0x7fb17c155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb17c15dfc0 0x7fb17c0715d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 --2- 172.21.15.116:0/1513129334 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb17c0b6c50 0x7fb17c072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 >> 172.21.15.116:0/1513129334 conn(0x7fb17c0b4fe0 msgr2=0x7fb17c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 shutdown_connections 2024-05-14T11:19:10.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.311+0000 7fb17b200640 1 -- 172.21.15.116:0/1513129334 wait complete. 2024-05-14T11:19:10.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:10.334 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-14T11:19:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 Processor -- start 2024-05-14T11:19:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 -- start start 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 0x7f61e0167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e0160510 con 0x7f61e0156fa0 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e01607f0 con 0x7f61e00d1670 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e8000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f61e00706f0 con 0x7f61e00b0390 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e5600640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f61e0156fa0 0x7f61e006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60638/0 (socket says 172.21.15.116:60638) 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e5600640 1 -- 172.21.15.116:0/81851485 learned_addr learned my addr 172.21.15.116:0/81851485 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e6000640 1 --2- 172.21.15.116:0/81851485 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 0x7f61e0167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081097423 0 0) 0x7f61e0160510 con 0x7f61e0156fa0 2024-05-14T11:19:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c8003530 con 0x7f61e0156fa0 2024-05-14T11:19:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914211199 0 0) 0x7f61e01607f0 con 0x7f61e00d1670 2024-05-14T11:19:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0160510 con 0x7f61e00d1670 2024-05-14T11:19:10.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e6000640 1 -- 172.21.15.116:0/81851485 >> v1:172.21.15.149:6789/0 conn(0x7f61e00d1670 legacy=0x7f61e00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e6000640 1 -- 172.21.15.116:0/81851485 >> v1:172.21.15.116:6789/0 conn(0x7f61e0156fa0 legacy=0x7f61e006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.483+0000 7f61e6000640 1 -- 172.21.15.116:0/81851485 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0160ad0 con 0x7f61e00b0390 2024-05-14T11:19:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61e6000640 1 --2- 172.21.15.116:0/81851485 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 0x7f61e0167800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009dc0 tx=0x7f61d801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad26072d82ad1a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80b2070 con 0x7f61e00b0390 2024-05-14T11:19:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d80a5750 con 0x7f61e00b0390 2024-05-14T11:19:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61dfe00640 1 -- 172.21.15.116:0/81851485 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80a5b00 con 0x7f61e00b0390 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61e8000640 1 -- 172.21.15.116:0/81851485 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 msgr2=0x7f61e0167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61e8000640 1 --2- 172.21.15.116:0/81851485 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 0x7f61e0167800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009dc0 tx=0x7f61d801c990 comp rx=0 tx=0).stop 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.487+0000 7f61e8000640 1 -- 172.21.15.116:0/81851485 shutdown_connections 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 --2- 172.21.15.116:0/81851485 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e00b0390 0x7f61e0167800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- 172.21.15.116:0/81851485 >> 172.21.15.116:0/81851485 conn(0x7f61e00b4620 msgr2=0x7f61e00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- 172.21.15.116:0/81851485 shutdown_connections 2024-05-14T11:19:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- 172.21.15.116:0/81851485 wait complete. 2024-05-14T11:19:10.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 Processor -- start 2024-05-14T11:19:10.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- start start 2024-05-14T11:19:10.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 0x7f61e0150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e0156fa0 0x7f61e0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006d2a0 con 0x7f61e00b0390 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006d120 con 0x7f61e00d1670 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.491+0000 7f61e8000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f61e006d420 con 0x7f61e0156fa0 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38918/0 (socket says 172.21.15.116:38918) 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 -- 172.21.15.116:0/347752766 learned_addr learned my addr 172.21.15.116:0/347752766 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e6000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e0156fa0 0x7f61e0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e4c00640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 0x7f61e0150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 -- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e0156fa0 msgr2=0x7f61e0072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e0156fa0 0x7f61e0072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 -- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 msgr2=0x7f61e0150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 0x7f61e0150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 -- 172.21.15.116:0/347752766 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0150e90 con 0x7f61e00b0390 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e4c00640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 0x7f61e0150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e5600640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c420 tx=0x7f61d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61dd600640 1 -- 172.21.15.116:0/347752766 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61e00b0390 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61dd600640 1 -- 172.21.15.116:0/347752766 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4005cf0 con 0x7f61e00b0390 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e0072f70 con 0x7f61e00b0390 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61dd600640 1 -- 172.21.15.116:0/347752766 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40072d0 con 0x7f61e00b0390 2024-05-14T11:19:10.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e014fb90 con 0x7f61e00b0390 2024-05-14T11:19:10.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.495+0000 7f61dd600640 1 -- 172.21.15.116:0/347752766 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61d401a070 con 0x7f61e00b0390 2024-05-14T11:19:10.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61dd600640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 0x7f61c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61dd600640 1 -- 172.21.15.116:0/347752766 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61d408e790 con 0x7f61e00b0390 2024-05-14T11:19:10.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e4c00640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 0x7f61c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 Processor -- start 2024-05-14T11:19:10.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 -- start start 2024-05-14T11:19:10.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e4c00640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 0x7f61c0071940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61e00d0290 tx=0x7f61cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe13a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdbc03e0 0x5614fe13add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614fdf3f100 con 0x5614fdfd3120 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614fdef7ca0 con 0x5614fdbc03e0 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5614fdf3b0c0 con 0x5614fdfc0f00 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38932/0 (socket says 172.21.15.116:38932) 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 -- 172.21.15.116:0/3311949604 learned_addr learned my addr 172.21.15.116:0/3311949604 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e6000640 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdbc03e0 0x5614fe13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e4c00640 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe13a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 -- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 msgr2=0x5614fe13a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe13a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 -- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdbc03e0 msgr2=0x5614fe13add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdbc03e0 0x5614fe13add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 -- 172.21.15.116:0/3311949604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614fdefc800 con 0x5614fdfd3120 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.499+0000 7f61e5600640 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61d405ad90 tx=0x7f61d40574d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a932ac1a56b9c842 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61d1c00640 1 -- 172.21.15.116:0/3311949604 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4063bb0 con 0x5614fdfd3120 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61d1c00640 1 -- 172.21.15.116:0/3311949604 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4007470 con 0x5614fdfd3120 2024-05-14T11:19:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61d1c00640 1 -- 172.21.15.116:0/3311949604 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d405a060 con 0x5614fdfd3120 2024-05-14T11:19:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- 172.21.15.116:0/3311949604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 msgr2=0x5614fe13a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61d405ad90 tx=0x7f61d40574d0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- 172.21.15.116:0/3311949604 shutdown_connections 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdbc03e0 0x5614fe13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe13a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/3311949604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfd3120 0x5614fe13a350 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- 172.21.15.116:0/3311949604 >> 172.21.15.116:0/3311949604 conn(0x5614fdc00400 msgr2=0x5614fe11d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- 172.21.15.116:0/3311949604 shutdown_connections 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- 172.21.15.116:0/3311949604 wait complete. 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- start start 2024-05-14T11:19:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 Processor -- start 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe141310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe0f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614fe00a2a0 con 0x5614fdbc03e0 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614fdffaf40 con 0x5614fdfd3120 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.503+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5614fdefb080 con 0x5614fdfc0f00 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 10 client.0 _ll_register_callbacks cb 0x5614fe0dbc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38940/0 (socket says 172.21.15.116:38940) 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 -- 172.21.15.116:0/238302451 learned_addr learned my addr 172.21.15.116:0/238302451 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e4c00640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e6000640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe0f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 -- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 msgr2=0x5614fe141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 -- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 msgr2=0x5614fe0f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe0f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 -- 172.21.15.116:0/238302451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614fdee9860 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e5600640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f61d405ab00 tx=0x7f61d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe828a33dea89c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d408a070 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61d405a060 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4007470 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61d405a280 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 10 client.5952 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5614fdef7ef0 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d4007470 con 0x5614fdbc03e0 2024-05-14T11:19:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f61d4074070 con 0x5614fdbc03e0 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 10 client.5952 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 10 client.5952 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 20 client.5952 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbe4930 0x5614fe0f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 4 client.5952 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614fded7df0 con 0x5614fdbe4930 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e4c00640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbe4930 0x5614fe0f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61e4c00640 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbe4930 0x5614fe0f4d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb18e00ecfdc4509 server_cookie=b8cf2be36f5ae505 in_seq=0 out_seq=0 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 10 client.5952 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 1 -- 172.21.15.116:0/238302451 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614fded7df0 con 0x5614fdbe4930 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 10 client.5952 handle_command_reply: tid=0 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 client.5952 shutdown 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 msgr2=0x5614fe140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f61d405ab00 tx=0x7f61d400cf90 comp rx=0 tx=0).stop 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 shutdown_connections 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe0f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbe4930 0x5614fe0f4d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/238302451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe140dd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61c6000640 0 client.5952 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 >> 172.21.15.116:0/238302451 conn(0x5614fdc00400 msgr2=0x5614fe11d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 shutdown_connections 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 1 -- 172.21.15.116:0/238302451 wait complete. 2024-05-14T11:19:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.507+0000 7f61ef5f0000 20 client.5952 trim_cache size 0 max 16384 2024-05-14T11:19:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 Processor -- start 2024-05-14T11:19:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- start start 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0204fc0 con 0x5614fdbc03e0 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00ce630 con 0x5614fdfd3120 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f61e0152e90 con 0x5614fdfc0f00 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38946/0 (socket says 172.21.15.116:38946) 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 -- 172.21.15.116:0/1576444523 learned_addr learned my addr 172.21.15.116:0/1576444523 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e6000640 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 -- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 msgr2=0x5614fe150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 -- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 msgr2=0x5614fe1514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 -- 172.21.15.116:0/1576444523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614fdfc0340 con 0x5614fdbc03e0 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e6000640 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e5600640 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f61d405a4b0 tx=0x7f61d408b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094878aac4c5660 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61d1200640 1 -- 172.21.15.116:0/1576444523 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4056770 con 0x5614fdbc03e0 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61d1200640 1 -- 172.21.15.116:0/1576444523 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d405a060 con 0x5614fdbc03e0 2024-05-14T11:19:10.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61d1200640 1 -- 172.21.15.116:0/1576444523 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d407bb70 con 0x5614fdbc03e0 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- 172.21.15.116:0/1576444523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 msgr2=0x5614fe0f6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f61d405a4b0 tx=0x7f61d408b000 comp rx=0 tx=0).stop 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- 172.21.15.116:0/1576444523 shutdown_connections 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdfc0f00 0x5614fe150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/1576444523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdbc03e0 0x5614fe0f6370 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- 172.21.15.116:0/1576444523 >> 172.21.15.116:0/1576444523 conn(0x5614fdc00400 msgr2=0x5614fe0f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- 172.21.15.116:0/1576444523 shutdown_connections 2024-05-14T11:19:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- 172.21.15.116:0/1576444523 wait complete. 2024-05-14T11:19:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- start start 2024-05-14T11:19:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 Processor -- start 2024-05-14T11:19:10.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdbc03e0 0x5614fe143e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614fdf43730 con 0x5614fdfc0f00 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614fdf3e410 con 0x5614fdfd3120 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5614fdf4c500 con 0x5614fdbc03e0 2024-05-14T11:19:10.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61ef5f0000 10 client.0 _ll_register_callbacks cb 0x7f61d405d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.531+0000 7f61e4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38958/0 (socket says 172.21.15.116:38958) 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 -- 172.21.15.116:0/981817731 learned_addr learned my addr 172.21.15.116:0/981817731 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e5600640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdbc03e0 0x5614fe143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e6000640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 -- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdbc03e0 msgr2=0x5614fe143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdbc03e0 0x5614fe143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 -- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 msgr2=0x5614fe1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 -- 172.21.15.116:0/981817731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614fdedb8e0 con 0x5614fdfc0f00 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e4c00640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f61cc014720 tx=0x7f61cc000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425b917e89ed14bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:10.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc027360 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f61cc027500 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d405a060 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61cc037050 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 10 client.5958 fetch_fsmap learned FSMap version 34 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5614fdbf8220 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d405a060 con 0x5614fdfc0f00 2024-05-14T11:19:10.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f61cc03a020 con 0x5614fdfc0f00 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 10 client.5958 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 10 client.5958 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 20 client.5958 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbd2990 0x5614fe1b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 4 client.5958 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:19:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614fdf4cd10 con 0x5614fdbd2990 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e5600640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbd2990 0x5614fe1b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 10 client.5958 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e5600640 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbd2990 0x5614fe1b4d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e67dbe84fb2c89 server_cookie=d73cd3e3d806592d in_seq=0 out_seq=0 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 1 -- 172.21.15.116:0/981817731 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x5614fdf4cd10 con 0x5614fdbd2990 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 10 client.5958 handle_command_reply: tid=0 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 client.5958 shutdown 2024-05-14T11:19:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 msgr2=0x5614fe144370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f61cc014720 tx=0x7f61cc000ff0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 shutdown_connections 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614fdfd3120 0x5614fe1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x5614fdbd2990 0x5614fe1b4d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614fdfc0f00 0x5614fe144370 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 --2- 172.21.15.116:0/981817731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614fdbc03e0 0x5614fe143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61c6000640 0 client.5958 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 >> 172.21.15.116:0/981817731 conn(0x5614fdc00400 msgr2=0x5614fe0f2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 shutdown_connections 2024-05-14T11:19:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 1 -- 172.21.15.116:0/981817731 wait complete. 2024-05-14T11:19:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61ef5f0000 20 client.5958 trim_cache size 0 max 16384 2024-05-14T11:19:10.549 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:10.549 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:19:10.549 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:19:10.549 INFO:teuthology.orchestra.run.smithi116.stdout: "00bb14ac-91ef-4223-9fbd-2450fe8d0cb9": { 2024-05-14T11:19:10.549 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-05-14T11:19:10.550 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "00bb14ac-91ef-4223-9fbd-2450fe8d0cb9", 2024-05-14T11:19:10.550 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive" 2024-05-14T11:19:10.550 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:19:10.550 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:19:10.550 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:10.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 msgr2=0x7f61c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 0x7f61c0071940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61e00d0290 tx=0x7f61cc009040 comp rx=0 tx=0).stop 2024-05-14T11:19:10.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 msgr2=0x7f61e01422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:10.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c420 tx=0x7f61d400c8e0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 shutdown_connections 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61c006f430 0x7f61c0071940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61e00d1670 0x7f61e0150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61e0156fa0 0x7f61e0072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 --2- 172.21.15.116:0/347752766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61e00b0390 0x7f61e01422c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.535+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 >> 172.21.15.116:0/347752766 conn(0x7f61e00b4620 msgr2=0x7f61e006e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.539+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 shutdown_connections 2024-05-14T11:19:10.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:10.539+0000 7f61e8000640 1 -- 172.21.15.116:0/347752766 wait complete. 2024-05-14T11:19:10.559 INFO:tasks.cephfs.filesystem:scrub status for tag:00bb14ac-91ef-4223-9fbd-2450fe8d0cb9 - {'path': '/', 'tag': '00bb14ac-91ef-4223-9fbd-2450fe8d0cb9', 'options': 'recursive'} 2024-05-14T11:19:11.579 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:19:11.583 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:19:13.724 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:19:13.723+0000 7f7ed0800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T11:19:40.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:40.561 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-14T11:19:40.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 Processor -- start 2024-05-14T11:19:40.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 -- start start 2024-05-14T11:19:40.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f658c0b0770 0x7f658c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f658c06f730 0x7f658c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f658c070080 con 0x7f658c15fc00 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f658c06db20 con 0x7f658c0b0770 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.723+0000 7f6594400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f658c06cc40 con 0x7f658c06f730 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f658c0b0770 0x7f658c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6592400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f658c15fc00 0x7f658c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45578/0 (socket says 172.21.15.116:45578) 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6592400640 1 -- 172.21.15.116:0/986757145 learned_addr learned my addr 172.21.15.116:0/986757145 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591000640 1 --2- 172.21.15.116:0/986757145 >> v2:172.21.15.149:3300/0 conn(0x7f658c06f730 0x7f658c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 -- 172.21.15.116:0/986757145 >> v2:172.21.15.149:3300/0 conn(0x7f658c06f730 msgr2=0x7f658c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 --2- 172.21.15.116:0/986757145 >> v2:172.21.15.149:3300/0 conn(0x7f658c06f730 0x7f658c06fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 -- 172.21.15.116:0/986757145 >> v1:172.21.15.116:6789/0 conn(0x7f658c15fc00 legacy=0x7f658c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 -- 172.21.15.116:0/986757145 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c06cf10 con 0x7f658c0b0770 2024-05-14T11:19:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f658be00640 1 -- 172.21.15.116:0/986757145 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381567244 0 0) 0x7f65840014a0 con 0x7f658c15fc00 2024-05-14T11:19:40.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f6591a00640 1 --2- 172.21.15.116:0/986757145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c0b0770 0x7f658c06c700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f658c06f4e0 tx=0x7f658001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5561f95bd124eae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f658be00640 1 -- 172.21.15.116:0/986757145 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800b0070 con 0x7f658c0b0770 2024-05-14T11:19:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f658be00640 1 -- 172.21.15.116:0/986757145 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65800ab600 con 0x7f658c0b0770 2024-05-14T11:19:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.727+0000 7f658be00640 1 -- 172.21.15.116:0/986757145 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800ab9d0 con 0x7f658c0b0770 2024-05-14T11:19:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- 172.21.15.116:0/986757145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c0b0770 msgr2=0x7f658c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 --2- 172.21.15.116:0/986757145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c0b0770 0x7f658c06c700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f658c06f4e0 tx=0x7f658001c3c0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- 172.21.15.116:0/986757145 shutdown_connections 2024-05-14T11:19:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 --2- 172.21.15.116:0/986757145 >> v2:172.21.15.149:3300/0 conn(0x7f658c06f730 0x7f658c06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 --2- 172.21.15.116:0/986757145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c0b0770 0x7f658c06c700 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- 172.21.15.116:0/986757145 >> 172.21.15.116:0/986757145 conn(0x7f658c0b4620 msgr2=0x7f658c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- 172.21.15.116:0/986757145 shutdown_connections 2024-05-14T11:19:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- 172.21.15.116:0/986757145 wait complete. 2024-05-14T11:19:40.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 Processor -- start 2024-05-14T11:19:40.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.731+0000 7f6594400640 1 -- start start 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f658c0b0770 0x7f658c145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f658c15fc00 0x7f658c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f658c073a40 con 0x7f658c06f730 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f658c06dfb0 con 0x7f658c15fc00 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f658c166f90 con 0x7f658c0b0770 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59402/0 (socket says 172.21.15.116:59402) 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 -- 172.21.15.116:0/4169910447 learned_addr learned my addr 172.21.15.116:0/4169910447 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6592400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f658c15fc00 0x7f658c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591000640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f658c0b0770 0x7f658c145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 -- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f658c0b0770 msgr2=0x7f658c145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f658c0b0770 0x7f658c145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 -- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f658c15fc00 msgr2=0x7f658c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f658c15fc00 0x7f658c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 -- 172.21.15.116:0/4169910447 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c1457a0 con 0x7f658c06f730 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6591a00640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6580019ff0 tx=0x7f658001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6589600640 1 -- 172.21.15.116:0/4169910447 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800b0040 con 0x7f658c06f730 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6589600640 1 -- 172.21.15.116:0/4169910447 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580004030 con 0x7f658c06f730 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6589600640 1 -- 172.21.15.116:0/4169910447 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800043e0 con 0x7f658c06f730 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c15b5b0 con 0x7f658c06f730 2024-05-14T11:19:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.735+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c0cd930 con 0x7f658c06f730 2024-05-14T11:19:40.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6589600640 1 -- 172.21.15.116:0/4169910447 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f65800b3070 con 0x7f658c06f730 2024-05-14T11:19:40.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6589600640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 0x7f656c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6589600640 1 -- 172.21.15.116:0/4169910447 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f65801292f0 con 0x7f658c06f730 2024-05-14T11:19:40.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591000640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 0x7f656c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 Processor -- start 2024-05-14T11:19:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 -- start start 2024-05-14T11:19:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591000640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 0x7f656c071940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f658c0b2f00 tx=0x7f6578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55719500ef00 0x55719518ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 0x55719518d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557194f8d100 con 0x557195021120 2024-05-14T11:19:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557194f45ca0 con 0x557194c0e3e0 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557194f890c0 con 0x55719500ef00 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59404/0 (socket says 172.21.15.116:59404) 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 -- 172.21.15.116:0/1070745347 learned_addr learned my addr 172.21.15.116:0/1070745347 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591000640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55719500ef00 0x55719518ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6592400640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 0x55719518d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 -- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55719500ef00 msgr2=0x55719518ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55719500ef00 0x55719518ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 -- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 msgr2=0x55719518d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 0x55719518d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6591a00640 1 -- 172.21.15.116:0/1070745347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557194f4a800 con 0x557195021120 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.739+0000 7f6592400640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 0x55719518d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f6591a00640 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f65800ae040 tx=0x7f6580019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8863e80f0ca318c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f657e000640 1 -- 172.21.15.116:0/1070745347 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800c2040 con 0x557195021120 2024-05-14T11:19:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f657e000640 1 -- 172.21.15.116:0/1070745347 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65800f54e0 con 0x557195021120 2024-05-14T11:19:40.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f657e000640 1 -- 172.21.15.116:0/1070745347 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800fe510 con 0x557195021120 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- 172.21.15.116:0/1070745347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 msgr2=0x55719518c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f65800ae040 tx=0x7f6580019160 comp rx=0 tx=0).stop 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- 172.21.15.116:0/1070745347 shutdown_connections 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557194c0e3e0 0x55719518d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55719500ef00 0x55719518ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 --2- 172.21.15.116:0/1070745347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557195021120 0x55719518c7b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- 172.21.15.116:0/1070745347 >> 172.21.15.116:0/1070745347 conn(0x557194c4e400 msgr2=0x55719516f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- 172.21.15.116:0/1070745347 shutdown_connections 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- 172.21.15.116:0/1070745347 wait complete. 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 -- start start 2024-05-14T11:19:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.743+0000 7f659bb51000 1 Processor -- start 2024-05-14T11:19:40.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195193a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x55719514aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5571950582a0 con 0x55719500ef00 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557195048f40 con 0x557195021120 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557194f49080 con 0x557194c0e3e0 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 10 client.0 _ll_register_callbacks cb 0x557195129c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59420/0 (socket says 172.21.15.116:59420) 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 -- 172.21.15.116:0/3001230841 learned_addr learned my addr 172.21.15.116:0/3001230841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591a00640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195193a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6592400640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x55719514aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 -- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 msgr2=0x557195193a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195193a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 -- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 msgr2=0x55719514aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x55719514aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 -- 172.21.15.116:0/3001230841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557194f37860 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591000640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f65780144e0 tx=0x7f65780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58eebb5ddf440b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578025070 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65800f54e0 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f65780158e0 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6578030050 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 10 client.5973 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x557194f45ef0 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65800f54e0 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f6578033020 con 0x55719500ef00 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 10 client.5973 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 10 client.5973 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 20 client.5973 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c32930 0x55719514b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 4 client.5973 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557194f25df0 con 0x557194c32930 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591a00640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c32930 0x55719514b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6591a00640 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c32930 0x55719514b950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e44abec43e15de server_cookie=eea415bbf2914344 in_seq=0 out_seq=0 2024-05-14T11:19:40.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 10 client.5973 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 1 -- 172.21.15.116:0/3001230841 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557194f25df0 con 0x557194c32930 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 10 client.5973 handle_command_reply: tid=0 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 client.5973 shutdown 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 msgr2=0x557195193f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f65780144e0 tx=0x7f65780046d0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 shutdown_connections 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x55719514aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c32930 0x55719514b950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x557195193f40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 --2- 172.21.15.116:0/3001230841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195193a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f6572a00640 0 client.5973 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.747+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 >> 172.21.15.116:0/3001230841 conn(0x557194c4e400 msgr2=0x55719516e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.751+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 shutdown_connections 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.751+0000 7f659bb51000 1 -- 172.21.15.116:0/3001230841 wait complete. 2024-05-14T11:19:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.751+0000 7f659bb51000 20 client.5973 trim_cache size 0 max 16384 2024-05-14T11:19:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 Processor -- start 2024-05-14T11:19:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 -- start start 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195198e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x5571951a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f658c15ee40 con 0x55719500ef00 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f658c0cf0c0 con 0x557195021120 2024-05-14T11:19:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f658c0cf710 con 0x557194c0e3e0 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59436/0 (socket says 172.21.15.116:59436) 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 -- 172.21.15.116:0/3233660230 learned_addr learned my addr 172.21.15.116:0/3233660230 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591a00640 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6592400640 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x5571951a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 -- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 msgr2=0x557195198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 -- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 msgr2=0x5571951a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x5571951a30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.767+0000 7f6591000640 1 -- 172.21.15.116:0/3233660230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719500e340 con 0x55719500ef00 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f6591000640 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6578015fd0 tx=0x7f6578022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141ac6140319d40e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f657d600640 1 -- 172.21.15.116:0/3233660230 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578025070 con 0x55719500ef00 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f657d600640 1 -- 172.21.15.116:0/3233660230 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657801c040 con 0x55719500ef00 2024-05-14T11:19:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f657d600640 1 -- 172.21.15.116:0/3233660230 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657801b040 con 0x55719500ef00 2024-05-14T11:19:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- 172.21.15.116:0/3233660230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 msgr2=0x5571951a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6578015fd0 tx=0x7f6578022000 comp rx=0 tx=0).stop 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- 172.21.15.116:0/3233660230 shutdown_connections 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x5571951a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951a2b90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- 172.21.15.116:0/3233660230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x557195198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- 172.21.15.116:0/3233660230 >> 172.21.15.116:0/3233660230 conn(0x557194c4e400 msgr2=0x55719514a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- 172.21.15.116:0/3233660230 shutdown_connections 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- 172.21.15.116:0/3233660230 wait complete. 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- start start 2024-05-14T11:19:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 Processor -- start 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x5571951ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x557195196d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557194f91730 con 0x55719500ef00 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557194f8c410 con 0x557195021120 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.771+0000 7f659bb51000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557194f9a500 con 0x557194c0e3e0 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 10 client.0 _ll_register_callbacks cb 0x7f657800b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x5571951ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59446/0 (socket says 172.21.15.116:59446) 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 -- 172.21.15.116:0/3059698877 learned_addr learned my addr 172.21.15.116:0/3059698877 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6592400640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x557195196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 -- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 msgr2=0x5571951ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x5571951ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 -- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 msgr2=0x557195196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x557195196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 -- 172.21.15.116:0/3059698877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557194f298e0 con 0x55719500ef00 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591000640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x557194ff31f0 tx=0x7f6578016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c483b75eb4ebb6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578040020 con 0x55719500ef00 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f6578015d30 con 0x55719500ef00 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657801c040 con 0x55719500ef00 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f657804a050 con 0x55719500ef00 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 10 client.5979 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x557194c46220 con 0x55719500ef00 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657801c040 con 0x55719500ef00 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f657804d020 con 0x55719500ef00 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 10 client.5979 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 10 client.5979 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 20 client.5979 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c20990 0x5571951976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 4 client.5979 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557194f9ad10 con 0x557194c20990 2024-05-14T11:19:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591a00640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c20990 0x5571951976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 10 client.5979 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6591a00640 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c20990 0x5571951976d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163fd1ac96be39d6 server_cookie=1f853e01cb3f5490 in_seq=0 out_seq=0 2024-05-14T11:19:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 1 -- 172.21.15.116:0/3059698877 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557194f9ad10 con 0x557194c20990 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 10 client.5979 handle_command_reply: tid=0 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 client.5979 shutdown 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 msgr2=0x5571951ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x557194ff31f0 tx=0x7f6578016000 comp rx=0 tx=0).stop 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 shutdown_connections 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557195021120 0x557195196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x557194c20990 0x5571951976d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55719500ef00 0x5571951ffa90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 --2- 172.21.15.116:0/3059698877 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557194c0e3e0 0x5571951ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6572a00640 0 client.5979 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 >> 172.21.15.116:0/3059698877 conn(0x557194c4e400 msgr2=0x55719516ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 shutdown_connections 2024-05-14T11:19:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 1 -- 172.21.15.116:0/3059698877 wait complete. 2024-05-14T11:19:40.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f659bb51000 20 client.5979 trim_cache size 0 max 16384 2024-05-14T11:19:40.779 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:40.779 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:19:40.779 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:19:40.779 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 msgr2=0x7f656c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 0x7f656c071940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f658c0b2f00 tx=0x7f6578008040 comp rx=0 tx=0).stop 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 msgr2=0x7f658c155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6580019ff0 tx=0x7f658001ac00 comp rx=0 tx=0).stop 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 shutdown_connections 2024-05-14T11:19:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f656c06f430 0x7f656c071940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f658c15fc00 0x7f658c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f658c0b0770 0x7f658c145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 --2- 172.21.15.116:0/4169910447 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f658c06f730 0x7f658c155200 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.775+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 >> 172.21.15.116:0/4169910447 conn(0x7f658c0b4620 msgr2=0x7f658c168550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.779+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 shutdown_connections 2024-05-14T11:19:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.779+0000 7f6594400640 1 -- 172.21.15.116:0/4169910447 wait complete. 2024-05-14T11:19:40.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:19:40.796 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:40.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.943+0000 7feb1ca00640 1 Processor -- start 2024-05-14T11:19:40.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.943+0000 7feb1ca00640 1 -- start start 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb1ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 0x7feb1806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb1ca00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb1815ce20 con 0x7feb180cbaf0 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb1ca00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb1806f850 con 0x7feb180d15e0 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb1ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb1806fb30 con 0x7feb1806ead0 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb17200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feb180cbaf0 0x7feb180cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45590/0 (socket says 172.21.15.116:45590) 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb17200640 1 -- 172.21.15.116:0/1300072763 learned_addr learned my addr 172.21.15.116:0/1300072763 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb0de00640 1 --2- 172.21.15.116:0/1300072763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 0x7feb1806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965504050 0 0) 0x7feb1815ce20 con 0x7feb180cbaf0 2024-05-14T11:19:40.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feafc005740 con 0x7feb180cbaf0 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773817769 0 0) 0x7feb1806f850 con 0x7feb180d15e0 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb1815ce20 con 0x7feb180d15e0 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 210888766 0 0) 0x7feafc005740 con 0x7feb180cbaf0 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb0de00640 1 -- 172.21.15.116:0/1300072763 >> v1:172.21.15.149:6789/0 conn(0x7feb180d15e0 legacy=0x7feb180d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb0de00640 1 -- 172.21.15.116:0/1300072763 >> v1:172.21.15.116:6789/0 conn(0x7feb180cbaf0 legacy=0x7feb180cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.947+0000 7feb0de00640 1 -- 172.21.15.116:0/1300072763 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1806fe10 con 0x7feb1806ead0 2024-05-14T11:19:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb0de00640 1 --2- 172.21.15.116:0/1300072763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 0x7feb1806eee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb00009820 tx=0x7feb0001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f5a382b9aeb6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000b1070 con 0x7feb1806ead0 2024-05-14T11:19:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000ac440 con 0x7feb1806ead0 2024-05-14T11:19:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb15e00640 1 -- 172.21.15.116:0/1300072763 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10001aa0 con 0x7feb1806ead0 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 -- 172.21.15.116:0/1300072763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 msgr2=0x7feb1806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 --2- 172.21.15.116:0/1300072763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 0x7feb1806eee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb00009820 tx=0x7feb0001c350 comp rx=0 tx=0).stop 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 -- 172.21.15.116:0/1300072763 shutdown_connections 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 --2- 172.21.15.116:0/1300072763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb1806ead0 0x7feb1806eee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 -- 172.21.15.116:0/1300072763 >> 172.21.15.116:0/1300072763 conn(0x7feb180b4530 msgr2=0x7feb180b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 -- 172.21.15.116:0/1300072763 shutdown_connections 2024-05-14T11:19:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.951+0000 7feb1ca00640 1 -- 172.21.15.116:0/1300072763 wait complete. 2024-05-14T11:19:40.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 Processor -- start 2024-05-14T11:19:40.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- start start 2024-05-14T11:19:40.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb1806ead0 0x7feb180983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb180cbaf0 0x7feb1815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb180cdc60 con 0x7feb180d15e0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb1806f630 con 0x7feb1806ead0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb180cdde0 con 0x7feb180cbaf0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59454/0 (socket says 172.21.15.116:59454) 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 -- 172.21.15.116:0/2546003923 learned_addr learned my addr 172.21.15.116:0/2546003923 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb16800640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb1806ead0 0x7feb180983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb0de00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb180cbaf0 0x7feb1815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 -- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb180cbaf0 msgr2=0x7feb1815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb180cbaf0 0x7feb1815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 -- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb1806ead0 msgr2=0x7feb180983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:40.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb1806ead0 0x7feb180983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:40.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 -- 172.21.15.116:0/2546003923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb180b18f0 con 0x7feb180d15e0 2024-05-14T11:19:40.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb17200640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c7a0 tx=0x7feb1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10018070 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1000a5e0 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000a9d0 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1815e1d0 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.955+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb18099a80 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb18050880 con 0x7feb180d15e0 2024-05-14T11:19:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb1000ab70 con 0x7feb180d15e0 2024-05-14T11:19:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb0f400640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 0x7feaf4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7feb1008e970 con 0x7feb180d15e0 2024-05-14T11:19:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb16800640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 0x7feaf4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.959+0000 7feb16800640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 0x7feaf4071940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feb08007fd0 tx=0x7feb08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:40.967+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feb1005afe0 con 0x7feb180d15e0 2024-05-14T11:19:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.155+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb180732e0 con 0x7feb180d15e0 2024-05-14T11:19:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.155+0000 7feb0f400640 1 -- 172.21.15.116:0/2546003923 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5191 (secure 0 0 0) 0x7feb1005f2e0 con 0x7feb180d15e0 2024-05-14T11:19:41.156 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:41.156 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:12.961108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:41.156 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 35 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 msgr2=0x7feaf4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 0x7feaf4071940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feb08007fd0 tx=0x7feb08009040 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 msgr2=0x7feb1815ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c7a0 tx=0x7feb1000cc60 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 shutdown_connections 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feaf406f430 0x7feaf4071940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb180d15e0 0x7feb1815ef10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb180cbaf0 0x7feb1815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 --2- 172.21.15.116:0/2546003923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb1806ead0 0x7feb180983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 >> 172.21.15.116:0/2546003923 conn(0x7feb180b4530 msgr2=0x7feb180686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 shutdown_connections 2024-05-14T11:19:41.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.159+0000 7feb1ca00640 1 -- 172.21.15.116:0/2546003923 wait complete. 2024-05-14T11:19:41.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-14T11:19:41.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 Processor -- start 2024-05-14T11:19:41.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 -- start start 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f34b80d1670 0x7f34b80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f34b806c700 0x7f34b806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b806ff20 con 0x7f34b815db30 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f34b80ae100 con 0x7f34b806c700 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34be600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f34b80ae870 con 0x7f34b80d1670 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f34b806c700 0x7f34b806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f34b806c700 0x7f34b806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59460/0 (socket says 172.21.15.116:59460) 2024-05-14T11:19:41.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 -- 172.21.15.116:0/3207517829 learned_addr learned my addr 172.21.15.116:0/3207517829 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34aea00640 1 --2- 172.21.15.116:0/3207517829 >> v2:172.21.15.149:3300/0 conn(0x7f34b80d1670 0x7f34b80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 -- 172.21.15.116:0/3207517829 >> v2:172.21.15.149:3300/0 conn(0x7f34b80d1670 msgr2=0x7f34b80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3207517829 >> v2:172.21.15.149:3300/0 conn(0x7f34b80d1670 0x7f34b80d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 -- 172.21.15.116:0/3207517829 >> v1:172.21.15.149:6789/0 conn(0x7f34b815db30 legacy=0x7f34b80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.327+0000 7f34b7e00640 1 -- 172.21.15.116:0/3207517829 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b80d1fc0 con 0x7f34b806c700 2024-05-14T11:19:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3207517829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b806c700 0x7f34b806cb10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f34b80d2370 tx=0x7f34b001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa4dd3fc2e38cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34b6a00640 1 -- 172.21.15.116:0/3207517829 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00a5600 con 0x7f34b806c700 2024-05-14T11:19:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34b6a00640 1 -- 172.21.15.116:0/3207517829 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00a57a0 con 0x7f34b806c700 2024-05-14T11:19:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34b6a00640 1 -- 172.21.15.116:0/3207517829 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00a5b70 con 0x7f34b806c700 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 -- 172.21.15.116:0/3207517829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b806c700 msgr2=0x7f34b806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 --2- 172.21.15.116:0/3207517829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b806c700 0x7f34b806cb10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f34b80d2370 tx=0x7f34b001c890 comp rx=0 tx=0).stop 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 -- 172.21.15.116:0/3207517829 shutdown_connections 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 --2- 172.21.15.116:0/3207517829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b806c700 0x7f34b806cb10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 --2- 172.21.15.116:0/3207517829 >> v2:172.21.15.149:3300/0 conn(0x7f34b80d1670 0x7f34b80d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 -- 172.21.15.116:0/3207517829 >> 172.21.15.116:0/3207517829 conn(0x7f34b80b4620 msgr2=0x7f34b80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 -- 172.21.15.116:0/3207517829 shutdown_connections 2024-05-14T11:19:41.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.331+0000 7f34be600640 1 -- 172.21.15.116:0/3207517829 wait complete. 2024-05-14T11:19:41.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 Processor -- start 2024-05-14T11:19:41.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 -- start start 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b806c700 0x7f34b80741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80d1670 0x7f34b814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f34b816a770 con 0x7f34b815db30 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34b815df40 con 0x7f34b806c700 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34be600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f34b816a8f0 con 0x7f34b80d1670 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59462/0 (socket says 172.21.15.116:59462) 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34b7e00640 1 -- 172.21.15.116:0/154200648 learned_addr learned my addr 172.21.15.116:0/154200648 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34aea00640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80d1670 0x7f34b814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.335+0000 7f34b7400640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b806c700 0x7f34b80741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 -- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80d1670 msgr2=0x7f34b814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80d1670 0x7f34b814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 -- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b806c700 msgr2=0x7f34b80741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b806c700 0x7f34b80741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 -- 172.21.15.116:0/154200648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b814ebb0 con 0x7f34b815db30 2024-05-14T11:19:41.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34b7e00640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f34b001af40 tx=0x7f34b0019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34afe00640 1 -- 172.21.15.116:0/154200648 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00bc020 con 0x7f34b815db30 2024-05-14T11:19:41.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34afe00640 1 -- 172.21.15.116:0/154200648 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00b8670 con 0x7f34b815db30 2024-05-14T11:19:41.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b814ee90 con 0x7f34b815db30 2024-05-14T11:19:41.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34afe00640 1 -- 172.21.15.116:0/154200648 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0009510 con 0x7f34b815db30 2024-05-14T11:19:41.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b814feb0 con 0x7f34b815db30 2024-05-14T11:19:41.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34afe00640 1 -- 172.21.15.116:0/154200648 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34b00bf020 con 0x7f34b815db30 2024-05-14T11:19:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.339+0000 7f34afe00640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 0x7f3494071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34afe00640 1 -- 172.21.15.116:0/154200648 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f34b012a300 con 0x7f34b815db30 2024-05-14T11:19:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 0x7f3494071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 Processor -- start 2024-05-14T11:19:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 -- start start 2024-05-14T11:19:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 0x7f3494071940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098e0 tx=0x7f34a8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569965da1a0 con 0x556996671bd0 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556996592b80 con 0x5569966612d0 2024-05-14T11:19:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5569965d6160 con 0x55699625b580 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59478/0 (socket says 172.21.15.116:59478) 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 -- 172.21.15.116:0/571684483 learned_addr learned my addr 172.21.15.116:0/571684483 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34aea00640 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7e00640 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 -- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 msgr2=0x5569967d58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967d58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 -- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 msgr2=0x5569967d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967d53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 -- 172.21.15.116:0/571684483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569965e6630 con 0x556996671bd0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34b7400640 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f34a8006d10 tx=0x7f34a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceb8b21044bdfb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34a7e00640 1 -- 172.21.15.116:0/571684483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a801b040 con 0x556996671bd0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34a7e00640 1 -- 172.21.15.116:0/571684483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8015a40 con 0x556996671bd0 2024-05-14T11:19:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.343+0000 7f34a7e00640 1 -- 172.21.15.116:0/571684483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8015c10 con 0x556996671bd0 2024-05-14T11:19:41.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- 172.21.15.116:0/571684483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 msgr2=0x5569967d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f34a8006d10 tx=0x7f34a8011000 comp rx=0 tx=0).stop 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- 172.21.15.116:0/571684483 shutdown_connections 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967d53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967d58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/571684483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x5569967d4e70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- 172.21.15.116:0/571684483 >> 172.21.15.116:0/571684483 conn(0x55699629a760 msgr2=0x5569967b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- 172.21.15.116:0/571684483 shutdown_connections 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- 172.21.15.116:0/571684483 wait complete. 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- start start 2024-05-14T11:19:41.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 Processor -- start 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556996579ab0 con 0x556996671bd0 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55699658ef20 con 0x5569966612d0 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556996639da0 con 0x55699625b580 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34c5d5e000 10 client.0 _ll_register_callbacks cb 0x556996775f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59482/0 (socket says 172.21.15.116:59482) 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 -- 172.21.15.116:0/3465565614 learned_addr learned my addr 172.21.15.116:0/3465565614 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7400640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34aea00640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 -- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 msgr2=0x5569967dc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967dc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 -- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 msgr2=0x5569967dca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967dca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.347+0000 7f34b7e00640 1 -- 172.21.15.116:0/3465565614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556996598ce0 con 0x556996671bd0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f34b001cd70 tx=0x7f34b00ff060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad4f5097af370e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00f6970 con 0x556996671bd0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f34b00ff290 con 0x556996671bd0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a8015a40 con 0x556996671bd0 2024-05-14T11:19:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f34b00fac70 con 0x556996671bd0 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 10 client.5997 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556996584740 con 0x556996671bd0 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a8015a40 con 0x556996671bd0 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f34b00f5bd0 con 0x556996671bd0 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 10 client.5997 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 10 client.5997 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 20 client.5997 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699627fad0 0x55699678f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 4 client.5997 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556996572e60 con 0x55699627fad0 2024-05-14T11:19:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34b7400640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699627fad0 0x55699678f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34b7400640 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699627fad0 0x55699678f0b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194ab5820a7a592c server_cookie=9935e7aadf23e121 in_seq=0 out_seq=0 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 10 client.5997 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 1 -- 172.21.15.116:0/3465565614 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556996572e60 con 0x55699627fad0 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 10 client.5997 handle_command_reply: tid=0 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 client.5997 shutdown 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 msgr2=0x55699678e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f34b001cd70 tx=0x7f34b00ff060 comp rx=0 tx=0).stop 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 shutdown_connections 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569966612d0 0x5569967dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699627fad0 0x55699678f0b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x55699678e310 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3465565614 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55699625b580 0x5569967dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34a4c00640 0 client.5997 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 >> 172.21.15.116:0/3465565614 conn(0x55699629a760 msgr2=0x5569967b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 shutdown_connections 2024-05-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3465565614 wait complete. 2024-05-14T11:19:41.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.351+0000 7f34c5d5e000 20 client.5997 trim_cache size 0 max 16384 2024-05-14T11:19:41.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 Processor -- start 2024-05-14T11:19:41.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 -- start start 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 0x5569968437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996843d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f34b8205040 con 0x556996671bd0 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34b814e310 con 0x55699625b580 2024-05-14T11:19:41.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f34b814e960 con 0x5569966612d0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59484/0 (socket says 172.21.15.116:59484) 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 -- 172.21.15.116:0/2384391997 learned_addr learned my addr 172.21.15.116:0/2384391997 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34aea00640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996843d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7400640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 0x5569968437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 -- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 msgr2=0x556996843d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996843d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 -- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 msgr2=0x5569968437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 0x5569968437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 -- 172.21.15.116:0/2384391997 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556996592dd0 con 0x556996671bd0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7400640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 0x5569968437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34aea00640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996843d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34b7e00640 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f34b00f6050 tx=0x7f34b012a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02b1d521c7ac54f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34a7400640 1 -- 172.21.15.116:0/2384391997 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00f62c0 con 0x556996671bd0 2024-05-14T11:19:41.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34a7400640 1 -- 172.21.15.116:0/2384391997 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00ff4e0 con 0x556996671bd0 2024-05-14T11:19:41.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.367+0000 7f34a7400640 1 -- 172.21.15.116:0/2384391997 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00f6970 con 0x556996671bd0 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- 172.21.15.116:0/2384391997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 msgr2=0x556996844270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f34b00f6050 tx=0x7f34b012a050 comp rx=0 tx=0).stop 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- 172.21.15.116:0/2384391997 shutdown_connections 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556996671bd0 0x556996844270 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996843d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/2384391997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699625b580 0x5569968437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- 172.21.15.116:0/2384391997 >> 172.21.15.116:0/2384391997 conn(0x55699629a760 msgr2=0x5569967eb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- 172.21.15.116:0/2384391997 shutdown_connections 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- 172.21.15.116:0/2384391997 wait complete. 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- start start 2024-05-14T11:19:41.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 Processor -- start 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996847e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556996671bd0 0x556996802fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569965835f0 con 0x55699625b580 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569965d4e80 con 0x556996671bd0 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556996595f60 con 0x5569966612d0 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34c5d5e000 10 client.0 _ll_register_callbacks cb 0x7f34b00f2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59486/0 (socket says 172.21.15.116:59486) 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 -- 172.21.15.116:0/3481571483 learned_addr learned my addr 172.21.15.116:0/3481571483 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34aea00640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996847e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7e00640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556996671bd0 0x556996802fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 -- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 msgr2=0x556996847e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996847e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 -- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556996671bd0 msgr2=0x556996802fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556996671bd0 0x556996802fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.371+0000 7f34b7400640 1 -- 172.21.15.116:0/3481571483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699665ac40 con 0x55699625b580 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34b7400640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e5a0 tx=0x7f34a8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be69a18b1835e555 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8009610 con 0x55699625b580 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f34a8024040 con 0x55699625b580 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b00ff4e0 con 0x55699625b580 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f34a803e050 con 0x55699625b580 2024-05-14T11:19:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 10 client.6003 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5569962933c0 con 0x55699625b580 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00ff4e0 con 0x55699625b580 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f34a8041020 con 0x55699625b580 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 10 client.6003 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 10 client.6003 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 20 client.6003 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699626db30 0x5569968507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 4 client.6003 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5569965e7db0 con 0x55699626db30 2024-05-14T11:19:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34aea00640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699626db30 0x5569968507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34aea00640 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699626db30 0x5569968507c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a59915d73473a35 server_cookie=97dc689eaf420247 in_seq=0 out_seq=0 2024-05-14T11:19:41.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 10 client.6003 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 1 -- 172.21.15.116:0/3481571483 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x5569965e7db0 con 0x55699626db30 2024-05-14T11:19:41.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 10 client.6003 handle_command_reply: tid=0 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 client.6003 shutdown 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 msgr2=0x556996847950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e5a0 tx=0x7f34a8004700 comp rx=0 tx=0).stop 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 shutdown_connections 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556996671bd0 0x556996802fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55699626db30 0x5569968507c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569966612d0 0x556996847e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 --2- 172.21.15.116:0/3481571483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55699625b580 0x556996847950 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34a4c00640 0 client.6003 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 >> 172.21.15.116:0/3481571483 conn(0x55699629a760 msgr2=0x5569967e2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 shutdown_connections 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 1 -- 172.21.15.116:0/3481571483 wait complete. 2024-05-14T11:19:41.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34c5d5e000 20 client.6003 trim_cache size 0 max 16384 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "backtrace", 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 3749028578, 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir_test/file" 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:19:41.378 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 msgr2=0x7f3494071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 0x7f3494071940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34a80098e0 tx=0x7f34a8009060 comp rx=0 tx=0).stop 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 msgr2=0x7f34b809a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f34b001af40 tx=0x7f34b0019630 comp rx=0 tx=0).stop 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 shutdown_connections 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f349406f430 0x7f3494071940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34b815db30 0x7f34b809a840 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80d1670 0x7f34b814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 --2- 172.21.15.116:0/154200648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b806c700 0x7f34b80741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 >> 172.21.15.116:0/154200648 conn(0x7f34b80b4620 msgr2=0x7f34b8150220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 shutdown_connections 2024-05-14T11:19:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.375+0000 7f34be600640 1 -- 172.21.15.116:0/154200648 wait complete. 2024-05-14T11:19:41.394 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:19:41.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 Processor -- start 2024-05-14T11:19:41.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 -- start start 2024-05-14T11:19:41.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc2600d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2600d1f90 con 0x7fc2600b9020 2024-05-14T11:19:41.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2600d2270 con 0x7fc2600ce9d0 2024-05-14T11:19:41.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc265e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc2600d1d40 con 0x7fc2601621b0 2024-05-14T11:19:41.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc256000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc2600b9020 0x7fc2600ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45600/0 (socket says 172.21.15.116:45600) 2024-05-14T11:19:41.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc256000640 1 -- 172.21.15.116:0/3281915658 learned_addr learned my addr 172.21.15.116:0/3281915658 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25ea00640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc2600d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880598877 0 0) 0x7fc2600d1f90 con 0x7fc2600b9020 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc244003530 con 0x7fc2600b9020 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 364158636 0 0) 0x7fc244003530 con 0x7fc2600b9020 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 msgr2=0x7fc2600d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc2600d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 >> v1:172.21.15.149:6789/0 conn(0x7fc2600ce9d0 legacy=0x7fc2600b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2600d2550 con 0x7fc2600b9020 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fc248001700 con 0x7fc2600b9020 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 >> v1:172.21.15.116:6789/0 conn(0x7fc2600b9020 legacy=0x7fc2600ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc244003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc244004010 0x7fc244004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004d50 0x7fc244005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600d1d40 con 0x7fc244004010 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc24400bc10 con 0x7fc244004d50 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc24400be10 con 0x7fc2601621b0 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.543+0000 7fc256000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc244004010 0x7fc244004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc25ea00640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc244003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc25f400640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004d50 0x7fc244005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 -- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 msgr2=0x7fc244003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc2601621b0 0x7fc244003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 -- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004d50 msgr2=0x7fc244005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004d50 0x7fc244005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 -- 172.21.15.116:0/3281915658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc244003530 con 0x7fc244004010 2024-05-14T11:19:41.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc256000640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc244004010 0x7fc244004810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc2600743b0 tx=0x7fc24801f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd1708a01228bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248007d80 con 0x7fc244004010 2024-05-14T11:19:41.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc248005ae0 con 0x7fc244004010 2024-05-14T11:19:41.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.547+0000 7fc25e000640 1 -- 172.21.15.116:0/3281915658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248007070 con 0x7fc244004010 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- 172.21.15.116:0/3281915658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc244004010 msgr2=0x7fc244004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 --2- 172.21.15.116:0/3281915658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc244004010 0x7fc244004810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc2600743b0 tx=0x7fc24801f2d0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc25f400640 1 -- 172.21.15.116:0/3281915658 reap_dead start 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- 172.21.15.116:0/3281915658 shutdown_connections 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- 172.21.15.116:0/3281915658 >> 172.21.15.116:0/3281915658 conn(0x7fc2600d0770 msgr2=0x7fc2600d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- 172.21.15.116:0/3281915658 shutdown_connections 2024-05-14T11:19:41.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- 172.21.15.116:0/3281915658 wait complete. 2024-05-14T11:19:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 Processor -- start 2024-05-14T11:19:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.551+0000 7fc265e00640 1 -- start start 2024-05-14T11:19:41.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 0x7fc2601f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc26015e560 0x7fc2600ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600b95b0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600b9430 con 0x7fc244004010 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc2600d2d50 con 0x7fc26015e560 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59508/0 (socket says 172.21.15.116:59508) 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 -- 172.21.15.116:0/1724495747 learned_addr learned my addr 172.21.15.116:0/1724495747 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc256000640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc26015e560 0x7fc2600ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25ea00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 0x7fc2601f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 -- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc26015e560 msgr2=0x7fc2600ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc26015e560 0x7fc2600ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 -- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 msgr2=0x7fc2601f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 0x7fc2601f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 -- 172.21.15.116:0/1724495747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2600b75f0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25ea00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 0x7fc2601f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:41.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc25f400640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc258007fd0 tx=0x7fc25800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580052e0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc258018070 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2601f7dd0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258005480 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2601f8390 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26006a9c0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.555+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc258003840 con 0x7fc2600b6ca0 2024-05-14T11:19:41.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.559+0000 7fc257e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.559+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fc25808f0f0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.559+0000 7fc25ea00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.559+0000 7fc25ea00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 0x7fc23c071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b4b80 tx=0x7fc25000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.563+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc2600b75f0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2600b5460 con 0x7fc2600b6ca0 2024-05-14T11:19:41.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc257e00640 1 -- 172.21.15.116:0/1724495747 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fc25805bdb0 con 0x7fc2600b6ca0 2024-05-14T11:19:41.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:41.754 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 msgr2=0x7fc23c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 0x7fc23c071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b4b80 tx=0x7fc25000b040 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 msgr2=0x7fc2600b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc258007fd0 tx=0x7fc25800f9d0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 shutdown_connections 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2600b6ca0 0x7fc2600b70b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc26015e560 0x7fc2600ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 --2- 172.21.15.116:0/1724495747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc244004010 0x7fc2601f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 >> 172.21.15.116:0/1724495747 conn(0x7fc2600d0770 msgr2=0x7fc260161920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 shutdown_connections 2024-05-14T11:19:41.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.755+0000 7fc265e00640 1 -- 172.21.15.116:0/1724495747 wait complete. 2024-05-14T11:19:41.775 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-14T11:19:41.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:41.775 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-14T11:19:41.854 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:19:41.858 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:19:41.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.927+0000 7f9050a00640 1 Processor -- start 2024-05-14T11:19:41.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.927+0000 7f9050a00640 1 -- start start 2024-05-14T11:19:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9050a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 0x7f904c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9050a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c162530 con 0x7f904c06d710 2024-05-14T11:19:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9050a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c0d04c0 con 0x7f904c0cda30 2024-05-14T11:19:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9050a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0d0270 con 0x7f904c0cd620 2024-05-14T11:19:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f904a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f904c06d710 0x7f904c161e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45616/0 (socket says 172.21.15.116:45616) 2024-05-14T11:19:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f904a800640 1 -- 172.21.15.116:0/2370484866 learned_addr learned my addr 172.21.15.116:0/2370484866 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9049e00640 1 --2- 172.21.15.116:0/2370484866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 0x7f904c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034143353 0 0) 0x7f904c162530 con 0x7f904c06d710 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9030003530 con 0x7f904c06d710 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260171514 0 0) 0x7f904c0d04c0 con 0x7f904c0cda30 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f904c162530 con 0x7f904c0cda30 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9049e00640 1 -- 172.21.15.116:0/2370484866 >> v1:172.21.15.149:6789/0 conn(0x7f904c0cda30 legacy=0x7f904c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9049e00640 1 -- 172.21.15.116:0/2370484866 >> v1:172.21.15.116:6789/0 conn(0x7f904c06d710 legacy=0x7f904c161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.931+0000 7f9049e00640 1 -- 172.21.15.116:0/2370484866 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0d07a0 con 0x7f904c0cd620 2024-05-14T11:19:41.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9049e00640 1 --2- 172.21.15.116:0/2370484866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 0x7f904c06f6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097f0 tx=0x7f903c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aaf9ddb948b50f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0b0070 con 0x7f904c0cd620 2024-05-14T11:19:41.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c0ab600 con 0x7f904c0cd620 2024-05-14T11:19:41.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9048a00640 1 -- 172.21.15.116:0/2370484866 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0ab9b0 con 0x7f904c0cd620 2024-05-14T11:19:41.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 -- 172.21.15.116:0/2370484866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 msgr2=0x7f904c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 --2- 172.21.15.116:0/2370484866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 0x7f904c06f6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097f0 tx=0x7f903c01c4e0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 -- 172.21.15.116:0/2370484866 shutdown_connections 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 --2- 172.21.15.116:0/2370484866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0cd620 0x7f904c06f6b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 -- 172.21.15.116:0/2370484866 >> 172.21.15.116:0/2370484866 conn(0x7f904c0cde80 msgr2=0x7f904c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 -- 172.21.15.116:0/2370484866 shutdown_connections 2024-05-14T11:19:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 -- 172.21.15.116:0/2370484866 wait complete. 2024-05-14T11:19:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.935+0000 7f9050a00640 1 Processor -- start 2024-05-14T11:19:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- start start 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f904c0cd620 0x7f904c0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0b5950 0x7f904c0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f904c070000 con 0x7f904c06d710 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0b0c30 con 0x7f904c0cd620 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f904c16c8f0 con 0x7f904c0b5950 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59524/0 (socket says 172.21.15.116:59524) 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 -- 172.21.15.116:0/1129563550 learned_addr learned my addr 172.21.15.116:0/1129563550 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049400640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f904c0cd620 0x7f904c0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f904a800640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0b5950 0x7f904c0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 -- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0b5950 msgr2=0x7f904c0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0b5950 0x7f904c0aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 -- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f904c0cd620 msgr2=0x7f904c0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f904c0cd620 0x7f904c0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 -- 172.21.15.116:0/1129563550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c1fcfd0 con 0x7f904c06d710 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9049e00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f903c0ae040 tx=0x7f903c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9042000640 1 -- 172.21.15.116:0/1129563550 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0b0040 con 0x7f904c06d710 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9042000640 1 -- 172.21.15.116:0/1129563550 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c0b9070 con 0x7f904c06d710 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c15ef30 con 0x7f904c06d710 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9042000640 1 -- 172.21.15.116:0/1129563550 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0ab9b0 con 0x7f904c06d710 2024-05-14T11:19:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.939+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c15f4f0 con 0x7f904c06d710 2024-05-14T11:19:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f9042000640 1 -- 172.21.15.116:0/1129563550 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f903c0abb50 con 0x7f904c06d710 2024-05-14T11:19:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f9042000640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 0x7f902c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f9042000640 1 -- 172.21.15.116:0/1129563550 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f903c128cf0 con 0x7f904c06d710 2024-05-14T11:19:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f9049400640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 0x7f902c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f90580cf000 1 Processor -- start 2024-05-14T11:19:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f90580cf000 1 -- start start 2024-05-14T11:19:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.943+0000 7f9049400640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 0x7f902c071940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f904c15d7b0 tx=0x7f9038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c14056280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555c13e5bf10 con 0x555c13edcc70 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555c13e14df0 con 0x555c13edde70 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555c13e57ed0 con 0x555c13ef0ae0 2024-05-14T11:19:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59538/0 (socket says 172.21.15.116:59538) 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 -- 172.21.15.116:0/944634834 learned_addr learned my addr 172.21.15.116:0/944634834 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9049e00640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c14056280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9049400640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 -- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 msgr2=0x555c14056280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c14056280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 -- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 msgr2=0x555c140567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 -- 172.21.15.116:0/944634834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c13ea19a0 con 0x555c13edcc70 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9049400640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f904a800640 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f90440021c0 tx=0x7f904400f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38e40819224a57c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9036600640 1 -- 172.21.15.116:0/944634834 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440070d0 con 0x555c13edcc70 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9036600640 1 -- 172.21.15.116:0/944634834 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044018070 con 0x555c13edcc70 2024-05-14T11:19:41.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f9036600640 1 -- 172.21.15.116:0/944634834 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044007470 con 0x555c13edcc70 2024-05-14T11:19:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- 172.21.15.116:0/944634834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 msgr2=0x555c14056d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f90440021c0 tx=0x7f904400f660 comp rx=0 tx=0).stop 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- 172.21.15.116:0/944634834 shutdown_connections 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c14056d00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 --2- 172.21.15.116:0/944634834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c14056280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- 172.21.15.116:0/944634834 >> 172.21.15.116:0/944634834 conn(0x555c13b1c070 msgr2=0x555c1400ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- 172.21.15.116:0/944634834 shutdown_connections 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- 172.21.15.116:0/944634834 wait complete. 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 -- start start 2024-05-14T11:19:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.947+0000 7f90580cf000 1 Processor -- start 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c1403a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c1403aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555c13e269e0 con 0x555c13edde70 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555c13dfbc20 con 0x555c13edcc70 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555c13e19260 con 0x555c13ef0ae0 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 10 client.0 _ll_register_callbacks cb 0x555c13df46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:41.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59554/0 (socket says 172.21.15.116:59554) 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 -- 172.21.15.116:0/1218889806 learned_addr learned my addr 172.21.15.116:0/1218889806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049e00640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c1403a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f904a800640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c1403aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 -- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 msgr2=0x555c1403aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c1403aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 -- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 msgr2=0x555c1403a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c1403a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 -- 172.21.15.116:0/1218889806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c13e163c0 con 0x555c13edde70 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049400640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9038014750 tx=0x7f9038000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5981c032c4dd0a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90380135c0 con 0x555c13edde70 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f9038013760 con 0x555c13edde70 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9044018070 con 0x555c13edde70 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f903802e050 con 0x555c13edde70 2024-05-14T11:19:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 10 client.6021 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x555c13e0e800 con 0x555c13edde70 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044018070 con 0x555c13edde70 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f9038031020 con 0x555c13edde70 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 10 client.6021 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 10 client.6021 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 20 client.6021 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13b01860 0x555c14067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 4 client.6021 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c13df4b60 con 0x555c13b01860 2024-05-14T11:19:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049e00640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13b01860 0x555c14067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f9049e00640 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13b01860 0x555c14067840 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41bb6e24698b8c6d server_cookie=bf0a9862a50d86c5 in_seq=0 out_seq=0 2024-05-14T11:19:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.951+0000 7f902aa00640 10 client.6021 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f902aa00640 1 -- 172.21.15.116:0/1218889806 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c13df4b60 con 0x555c13b01860 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f902aa00640 10 client.6021 handle_command_reply: tid=0 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 client.6021 shutdown 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 msgr2=0x555c1403a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9038014750 tx=0x7f9038000ff0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 shutdown_connections 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13b01860 0x555c14067840 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c1403aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c1403a9b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f902aa00640 0 client.6021 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 --2- 172.21.15.116:0/1218889806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c1403a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 >> 172.21.15.116:0/1218889806 conn(0x555c13b1c070 msgr2=0x555c14055a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 shutdown_connections 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 1 -- 172.21.15.116:0/1218889806 wait complete. 2024-05-14T11:19:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.955+0000 7f90580cf000 20 client.6021 trim_cache size 0 max 16384 2024-05-14T11:19:41.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.971+0000 7f90580cf000 1 Processor -- start 2024-05-14T11:19:41.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.971+0000 7f90580cf000 1 -- start start 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c140c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f904c205830 con 0x555c13edde70 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1fa9c0 con 0x555c13edcc70 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f904c1fb010 con 0x555c13ef0ae0 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59560/0 (socket says 172.21.15.116:59560) 2024-05-14T11:19:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 -- 172.21.15.116:0/2826207848 learned_addr learned my addr 172.21.15.116:0/2826207848 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f904a800640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049e00640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c140c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 -- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 msgr2=0x555c140c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 -- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 msgr2=0x555c140c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c140c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 -- 172.21.15.116:0/2826207848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c13e15040 con 0x555c13edde70 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f904a800640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049e00640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c140c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9049400640 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f903800fe50 tx=0x7f9038019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be27884d323334a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9035c00640 1 -- 172.21.15.116:0/2826207848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90380135c0 con 0x555c13edde70 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9035c00640 1 -- 172.21.15.116:0/2826207848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9038026070 con 0x555c13edde70 2024-05-14T11:19:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f9035c00640 1 -- 172.21.15.116:0/2826207848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038013960 con 0x555c13edde70 2024-05-14T11:19:41.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- 172.21.15.116:0/2826207848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 msgr2=0x555c140c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f903800fe50 tx=0x7f9038019000 comp rx=0 tx=0).stop 2024-05-14T11:19:41.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- 172.21.15.116:0/2826207848 shutdown_connections 2024-05-14T11:19:41.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edde70 0x555c140c5680 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 --2- 172.21.15.116:0/2826207848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edcc70 0x555c140c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- 172.21.15.116:0/2826207848 >> 172.21.15.116:0/2826207848 conn(0x555c13b1c070 msgr2=0x555c140627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- 172.21.15.116:0/2826207848 shutdown_connections 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- 172.21.15.116:0/2826207848 wait complete. 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 -- start start 2024-05-14T11:19:41.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.975+0000 7f90580cf000 1 Processor -- start 2024-05-14T11:19:41.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140ae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555c13e5b220 con 0x555c13edcc70 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555c13ddf8c0 con 0x555c13edde70 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555c13e60540 con 0x555c13ef0ae0 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 10 client.0 _ll_register_callbacks cb 0x7f9038017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59572/0 (socket says 172.21.15.116:59572) 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 -- 172.21.15.116:0/3196500077 learned_addr learned my addr 172.21.15.116:0/3196500077 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f904a800640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049400640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 -- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 msgr2=0x555c140ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 -- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 msgr2=0x555c140ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 -- 172.21.15.116:0/3196500077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c13e27ca0 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049e00640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f903c0f25d0 tx=0x7f903c0f9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c182eee5701b942d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0fde80 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f903c0f45c0 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9038026070 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f903c0abe40 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 10 client.6027 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x555c13b15150 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9038026070 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f903c0b3070 con 0x555c13edcc70 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 10 client.6027 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 10 client.6027 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 20 client.6027 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13aef8c0 0x555c140aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:41.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 4 client.6027 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-14T11:19:41.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x555c13e69b20 con 0x555c13aef8c0 2024-05-14T11:19:41.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049400640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13aef8c0 0x555c140aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f9049400640 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13aef8c0 0x555c140aea90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16d3302aaadc968 server_cookie=f05b78b5c939cf4b in_seq=0 out_seq=0 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 10 client.6027 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 1 -- 172.21.15.116:0/3196500077 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c13e69b20 con 0x555c13aef8c0 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f902aa00640 10 client.6027 handle_command_reply: tid=0 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.979+0000 7f90580cf000 1 client.6027 shutdown 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 msgr2=0x555c1400de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f903c0f25d0 tx=0x7f903c0f9040 comp rx=0 tx=0).stop 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 shutdown_connections 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c13edde70 0x555c140ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x555c13aef8c0 0x555c140aea90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555c13ef0ae0 0x555c140ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 --2- 172.21.15.116:0/3196500077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c13edcc70 0x555c1400de70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f902aa00640 0 client.6027 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 >> 172.21.15.116:0/3196500077 conn(0x555c13b1c070 msgr2=0x555c1400a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 shutdown_connections 2024-05-14T11:19:41.983 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:19:41.983+0000 7fe6e9200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-14T11:19:09.141597+0000 1=1+0) caps={5898=pAsLsXsFs/-@2},l=5898(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x562bde2fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 1 -- 172.21.15.116:0/3196500077 wait complete. 2024-05-14T11:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f90580cf000 20 client.6027 trim_cache size 0 max 16384 2024-05-14T11:19:41.984 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:41.984 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:19:41.984 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "240d0d44-cf48-4078-965f-72b434cc0a72", 2024-05-14T11:19:41.984 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:19:41.984 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 msgr2=0x7f902c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 0x7f902c071940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f904c15d7b0 tx=0x7f9038009040 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 msgr2=0x7f904c1fca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f903c0ae040 tx=0x7f903c019280 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 shutdown_connections 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f902c06f430 0x7f902c071940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f904c0cd620 0x7f904c0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f904c0b5950 0x7f904c0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 --2- 172.21.15.116:0/1129563550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904c06d710 0x7f904c1fca90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:41.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 >> 172.21.15.116:0/1129563550 conn(0x7f904c0cde80 msgr2=0x7f904c150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:41.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 shutdown_connections 2024-05-14T11:19:41.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:41.983+0000 7f9050a00640 1 -- 172.21.15.116:0/1129563550 wait complete. 2024-05-14T11:19:42.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:42.000 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-14T11:19:42.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.191+0000 7f8deb200640 1 Processor -- start 2024-05-14T11:19:42.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.191+0000 7f8deb200640 1 -- start start 2024-05-14T11:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8deb200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8dec15eb20 0x7f8dec157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8deb200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 0x7f8dec0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8deb200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dec15e080 con 0x7f8dec0d01b0 2024-05-14T11:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8deb200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f8dec15e690 con 0x7f8dec15eb20 2024-05-14T11:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8deb200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec0d0e30 con 0x7f8dec0d1bd0 2024-05-14T11:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8dec15eb20 0x7f8dec157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8dea800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 0x7f8dec0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8dec15eb20 0x7f8dec157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:37254/0 (socket says 172.21.15.116:37254) 2024-05-14T11:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de1400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f8dec0d01b0 0x7f8dec0d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:49364/0 (socket says 172.21.15.116:49364) 2024-05-14T11:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 -- 172.21.15.116:0/2707180573 learned_addr learned my addr 172.21.15.116:0/2707180573 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 -- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 msgr2=0x7f8dec0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 0x7f8dec0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 -- 172.21.15.116:0/2707180573 >> v1:172.21.15.149:6789/0 conn(0x7f8dec0d01b0 legacy=0x7f8dec0d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9e00640 1 -- 172.21.15.116:0/2707180573 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec0d0b50 con 0x7f8dec15eb20 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8dea800640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 0x7f8dec0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.195+0000 7f8de9400640 1 -- 172.21.15.116:0/2707180573 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664187550 0 0) 0x7f8dec15e080 con 0x7f8dec0d01b0 2024-05-14T11:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8de9e00640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec15eb20 0x7f8dec157ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009e50 tx=0x7f8ddc01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31f39b741925a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8de9400640 1 -- 172.21.15.116:0/2707180573 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0a57c0 con 0x7f8dec15eb20 2024-05-14T11:19:42.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8de9400640 1 -- 172.21.15.116:0/2707180573 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0a5960 con 0x7f8dec15eb20 2024-05-14T11:19:42.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8de9400640 1 -- 172.21.15.116:0/2707180573 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0a5d30 con 0x7f8dec15eb20 2024-05-14T11:19:42.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 -- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec15eb20 msgr2=0x7f8dec157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec15eb20 0x7f8dec157ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009e50 tx=0x7f8ddc01c8e0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 -- 172.21.15.116:0/2707180573 shutdown_connections 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec0d1bd0 0x7f8dec0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 --2- 172.21.15.116:0/2707180573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec15eb20 0x7f8dec157ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 -- 172.21.15.116:0/2707180573 >> 172.21.15.116:0/2707180573 conn(0x7f8dec0b4510 msgr2=0x7f8dec0b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 -- 172.21.15.116:0/2707180573 shutdown_connections 2024-05-14T11:19:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.199+0000 7f8deb200640 1 -- 172.21.15.116:0/2707180573 wait complete. 2024-05-14T11:19:42.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 Processor -- start 2024-05-14T11:19:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- start start 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec0d01b0 0x7f8dec0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec15eb20 0x7f8dec14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec161e60 con 0x7f8dec1600b0 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec161ce0 con 0x7f8dec0d01b0 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec161fe0 con 0x7f8dec15eb20 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59586/0 (socket says 172.21.15.116:59586) 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 -- 172.21.15.116:0/1107161495 learned_addr learned my addr 172.21.15.116:0/1107161495 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8de9e00640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec0d01b0 0x7f8dec0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8de1400640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec15eb20 0x7f8dec14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 -- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec15eb20 msgr2=0x7f8dec14a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec15eb20 0x7f8dec14a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 -- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec0d01b0 msgr2=0x7f8dec0742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec0d01b0 0x7f8dec0742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 -- 172.21.15.116:0/1107161495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec160a00 con 0x7f8dec1600b0 2024-05-14T11:19:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8dea800640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e960 tx=0x7f8de400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8de2a00640 1 -- 172.21.15.116:0/1107161495 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017070 con 0x7f8dec1600b0 2024-05-14T11:19:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8de2a00640 1 -- 172.21.15.116:0/1107161495 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de400ca00 con 0x7f8dec1600b0 2024-05-14T11:19:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec14aa00 con 0x7f8dec1600b0 2024-05-14T11:19:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8de2a00640 1 -- 172.21.15.116:0/1107161495 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de400ce10 con 0x7f8dec1600b0 2024-05-14T11:19:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.203+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec0ce040 con 0x7f8dec1600b0 2024-05-14T11:19:42.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8de2a00640 1 -- 172.21.15.116:0/1107161495 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8de4013470 con 0x7f8dec1600b0 2024-05-14T11:19:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8de2a00640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 0x7f8dc8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8de2a00640 1 -- 172.21.15.116:0/1107161495 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f8de408efe0 con 0x7f8dec1600b0 2024-05-14T11:19:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8de9e00640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 0x7f8dc8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8df809e000 1 Processor -- start 2024-05-14T11:19:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8df809e000 1 -- start start 2024-05-14T11:19:42.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.207+0000 7f8de9e00640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 0x7f8dc8071940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0ae000 tx=0x7f8ddc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558fefeaf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 0x558fefeafdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558fefcb4100 con 0x558fefd48120 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fefc6cca0 con 0x558fef9353e0 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558fefcb00c0 con 0x558fefd35f00 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59588/0 (socket says 172.21.15.116:59588) 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 -- 172.21.15.116:0/2092338856 learned_addr learned my addr 172.21.15.116:0/2092338856 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de1400640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558fefeaf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8dea800640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 0x558fefeafdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 -- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 msgr2=0x558fefeaf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558fefeaf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 -- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 msgr2=0x558fefeafdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 0x558fefeafdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 -- 172.21.15.116:0/2092338856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fefc71800 con 0x558fefd48120 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8dea800640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 0x558fefeafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8de9e00640 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc01d180 tx=0x7f8ddc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c31bc583a231a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8dda600640 1 -- 172.21.15.116:0/2092338856 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0b6420 con 0x558fefd48120 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8dda600640 1 -- 172.21.15.116:0/2092338856 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0b65c0 con 0x558fefd48120 2024-05-14T11:19:42.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8dda600640 1 -- 172.21.15.116:0/2092338856 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0b6970 con 0x558fefd48120 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- 172.21.15.116:0/2092338856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 msgr2=0x558fefeaf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc01d180 tx=0x7f8ddc004ac0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- 172.21.15.116:0/2092338856 shutdown_connections 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fef9353e0 0x558fefeafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558fefeaf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 --2- 172.21.15.116:0/2092338856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefeaf350 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- 172.21.15.116:0/2092338856 >> 172.21.15.116:0/2092338856 conn(0x558fef975400 msgr2=0x558fefe92310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- 172.21.15.116:0/2092338856 shutdown_connections 2024-05-14T11:19:42.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.211+0000 7f8df809e000 1 -- 172.21.15.116:0/2092338856 wait complete. 2024-05-14T11:19:42.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- start start 2024-05-14T11:19:42.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 Processor -- start 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558fefeb5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558fefeb6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558fefd7f2a0 con 0x558fefd48120 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fefd6ff40 con 0x558fefd35f00 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558fefc70080 con 0x558fef9353e0 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 10 client.0 _ll_register_callbacks cb 0x558fefe50c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59590/0 (socket says 172.21.15.116:59590) 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 -- 172.21.15.116:0/300282601 learned_addr learned my addr 172.21.15.116:0/300282601 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8de1400640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558fefeb6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8de9e00640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558fefeb5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 -- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 msgr2=0x558fefeb5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558fefeb5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 -- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 msgr2=0x558fefeb6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558fefeb6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 -- 172.21.15.116:0/300282601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fefc5e860 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8de9e00640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558fefeb5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8de1400640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558fefeb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dea800640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e930 tx=0x7f8de405acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf8ada2f6bf57b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4020030 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f8de405a9c0 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ddc0b65c0 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8de405b560 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 10 client.6036 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x558fefc6cef0 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0b65c0 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f8de4056ed0 con 0x558fefd48120 2024-05-14T11:19:42.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 10 client.6036 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 10 client.6036 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 20 client.6036 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef959930 0x558fefe69cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 4 client.6036 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fefc4cdf0 con 0x558fef959930 2024-05-14T11:19:42.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.215+0000 7f8de9e00640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef959930 0x558fefe69cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8dcea00640 10 client.6036 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8de9e00640 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef959930 0x558fefe69cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06d52c150f66721 server_cookie=64d52d663bdbbf9 in_seq=0 out_seq=0 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8dcea00640 1 -- 172.21.15.116:0/300282601 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558fefc4cdf0 con 0x558fef959930 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8dcea00640 10 client.6036 handle_command_reply: tid=0 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 client.6036 shutdown 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 msgr2=0x558fefe68fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e930 tx=0x7f8de405acd0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 shutdown_connections 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558fefeb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef959930 0x558fefe69cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558fefe68fb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 --2- 172.21.15.116:0/300282601 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558fefeb5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8dcea00640 0 client.6036 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 >> 172.21.15.116:0/300282601 conn(0x558fef975400 msgr2=0x558fefe92200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 shutdown_connections 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 1 -- 172.21.15.116:0/300282601 wait complete. 2024-05-14T11:19:42.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.219+0000 7f8df809e000 20 client.6036 trim_cache size 0 max 16384 2024-05-14T11:19:42.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.235+0000 7f8df809e000 1 Processor -- start 2024-05-14T11:19:42.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.235+0000 7f8df809e000 1 -- start start 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558feff1d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558feff1d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec205630 con 0x558fefd48120 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec0cc570 con 0x558fefd35f00 2024-05-14T11:19:42.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec204fe0 con 0x558fef9353e0 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59598/0 (socket says 172.21.15.116:59598) 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 -- 172.21.15.116:0/4079559869 learned_addr learned my addr 172.21.15.116:0/4079559869 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8de9e00640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558feff1d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8de1400640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558feff1d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 -- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 msgr2=0x558feff1d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558feff1d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 -- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 msgr2=0x558feff1d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558feff1d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 -- 172.21.15.116:0/4079559869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fefd35340 con 0x558fefd48120 2024-05-14T11:19:42.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8de1400640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558feff1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dea800640 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8de408f280 tx=0x7f8de405d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b97c0a5a189d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dd9c00640 1 -- 172.21.15.116:0/4079559869 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4020070 con 0x558fefd48120 2024-05-14T11:19:42.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dd9c00640 1 -- 172.21.15.116:0/4079559869 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de40643c0 con 0x558fefd48120 2024-05-14T11:19:42.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8dd9c00640 1 -- 172.21.15.116:0/4079559869 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de405a9c0 con 0x558fefd48120 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- 172.21.15.116:0/4079559869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 msgr2=0x558feff1dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8de408f280 tx=0x7f8de405d000 comp rx=0 tx=0).stop 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- 172.21.15.116:0/4079559869 shutdown_connections 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd35f00 0x558feff1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fefd48120 0x558feff1dd60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 --2- 172.21.15.116:0/4079559869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fef9353e0 0x558feff1d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- 172.21.15.116:0/4079559869 >> 172.21.15.116:0/4079559869 conn(0x558fef975400 msgr2=0x558fefe913a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- 172.21.15.116:0/4079559869 shutdown_connections 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.239+0000 7f8df809e000 1 -- 172.21.15.116:0/4079559869 wait complete. 2024-05-14T11:19:42.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- start start 2024-05-14T11:19:42.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 Processor -- start 2024-05-14T11:19:42.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558feff21a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd48120 0x558fefeb9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558fefcb8730 con 0x558fef9353e0 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fefcb3410 con 0x558fefd48120 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558fefcc1500 con 0x558fefd35f00 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 10 client.0 _ll_register_callbacks cb 0x7f8de4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59610/0 (socket says 172.21.15.116:59610) 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 -- 172.21.15.116:0/4118903709 learned_addr learned my addr 172.21.15.116:0/4118903709 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de1400640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558feff21a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dea800640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd48120 0x558fefeb9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 -- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 msgr2=0x558feff21a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558feff21a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 -- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd48120 msgr2=0x558fefeb9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd48120 0x558fefeb9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 -- 172.21.15.116:0/4118903709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fefc508e0 con 0x558fef9353e0 2024-05-14T11:19:42.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8de9e00640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc004b00 tx=0x7f8ddc01b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b0a2e2086c1699 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:42.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0d3070 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de40643c0 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f8ddc0b65a0 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ddc0db050 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 10 client.6042 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x558fef96d220 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de40643c0 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f8ddc0de020 con 0x558fef9353e0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 10 client.6042 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 10 client.6042 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.243+0000 7f8df809e000 20 client.6042 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef947990 0x558fefeba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 4 client.6042 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fefcc1d10 con 0x558fef947990 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8de1400640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef947990 0x558fefeba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8de1400640 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef947990 0x558fefeba100 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79324002bf49ce6e server_cookie=b74bdefccf0cb557 in_seq=0 out_seq=0 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8dcea00640 10 client.6042 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8dcea00640 1 -- 172.21.15.116:0/4118903709 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558fefcc1d10 con 0x558fef947990 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8dcea00640 10 client.6042 handle_command_reply: tid=0 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 client.6042 shutdown 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 msgr2=0x558feff21500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc004b00 tx=0x7f8ddc01b120 comp rx=0 tx=0).stop 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 shutdown_connections 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fefd48120 0x558fefeb9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x558fef947990 0x558fefeba100 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558fefd35f00 0x558feff21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 --2- 172.21.15.116:0/4118903709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558fef9353e0 0x558feff21500 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8dcea00640 0 client.6042 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 >> 172.21.15.116:0/4118903709 conn(0x558fef975400 msgr2=0x558fefe91100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 shutdown_connections 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 1 -- 172.21.15.116:0/4118903709 wait complete. 2024-05-14T11:19:42.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8df809e000 20 client.6042 trim_cache size 0 max 16384 2024-05-14T11:19:42.248 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:19:42.248 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:19:42.248 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:19:42.248 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 msgr2=0x7f8dc8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 0x7f8dc8071940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0ae000 tx=0x7f8ddc0a7070 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 msgr2=0x7f8dec1604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e960 tx=0x7f8de400ee20 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 shutdown_connections 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8dc806f430 0x7f8dc8071940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dec1600b0 0x7f8dec1604c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8dec15eb20 0x7f8dec14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 --2- 172.21.15.116:0/1107161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8dec0d01b0 0x7f8dec0742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 >> 172.21.15.116:0/1107161495 conn(0x7f8dec0b4510 msgr2=0x7f8dec154390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 shutdown_connections 2024-05-14T11:19:42.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.247+0000 7f8deb200640 1 -- 172.21.15.116:0/1107161495 wait complete. 2024-05-14T11:19:42.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:19:42.265 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.411+0000 7f02ffa00640 1 Processor -- start 2024-05-14T11:19:42.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.411+0000 7f02ffa00640 1 -- start start 2024-05-14T11:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02ffa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f02f80b7720 0x7f02f806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02ffa00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f80cda30 con 0x7f02f80cc530 2024-05-14T11:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02ffa00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f80cdd10 con 0x7f02f80b6930 2024-05-14T11:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02ffa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f02f80cdff0 con 0x7f02f80b7720 2024-05-14T11:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fd000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f02f80cc530 0x7f02f80b7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45624/0 (socket says 172.21.15.116:45624) 2024-05-14T11:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fd000640 1 -- 172.21.15.116:0/4006882941 learned_addr learned my addr 172.21.15.116:0/4006882941 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440273027 0 0) 0x7f02f80cda30 con 0x7f02f80cc530 2024-05-14T11:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02efe00640 1 --2- 172.21.15.116:0/4006882941 >> v2:172.21.15.149:3300/0 conn(0x7f02f80b7720 0x7f02f806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02dc003530 con 0x7f02f80cc530 2024-05-14T11:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3189877646 0 0) 0x7f02dc003530 con 0x7f02f80cc530 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 >> v2:172.21.15.149:3300/0 conn(0x7f02f80b7720 msgr2=0x7f02f806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 --2- 172.21.15.116:0/4006882941 >> v2:172.21.15.149:3300/0 conn(0x7f02f80b7720 0x7f02f806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 >> v1:172.21.15.149:6789/0 conn(0x7f02f80b6930 legacy=0x7f02f806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f80ce2d0 con 0x7f02f80cc530 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f02e8001700 con 0x7f02f80cc530 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 >> v1:172.21.15.116:6789/0 conn(0x7f02f80cc530 legacy=0x7f02f80b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02dc003a60 0x7f02dc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02dc004810 0x7f02dc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02f80cdff0 con 0x7f02dc0053b0 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc00c340 con 0x7f02dc004810 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f02dc00c540 con 0x7f02dc003a60 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fd000640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02dc003a60 0x7f02dc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02efe00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02dc004810 0x7f02dc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 -- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02dc003a60 msgr2=0x7f02dc003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02dc003a60 0x7f02dc003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 -- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02dc004810 msgr2=0x7f02dc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02dc004810 0x7f02dc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.415+0000 7f02fda00640 1 -- 172.21.15.116:0/4006882941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc003530 con 0x7f02dc0053b0 2024-05-14T11:19:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02fda00640 1 -- 172.21.15.116:0/4006882941 reap_dead start 2024-05-14T11:19:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02fda00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02dc00ba60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02f80cede0 tx=0x7f02f001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31698f21ccc27bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0001ae0 con 0x7f02dc0053b0 2024-05-14T11:19:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f0003980 con 0x7f02dc0053b0 2024-05-14T11:19:42.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02f7e00640 1 -- 172.21.15.116:0/4006882941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0003d70 con 0x7f02dc0053b0 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 -- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 msgr2=0x7f02dc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02dc00ba60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02f80cede0 tx=0x7f02f001c750 comp rx=0 tx=0).stop 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 -- 172.21.15.116:0/4006882941 shutdown_connections 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 --2- 172.21.15.116:0/4006882941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02dc00ba60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 -- 172.21.15.116:0/4006882941 >> 172.21.15.116:0/4006882941 conn(0x7f02f8071270 msgr2=0x7f02f8071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 -- 172.21.15.116:0/4006882941 shutdown_connections 2024-05-14T11:19:42.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.419+0000 7f02ffa00640 1 -- 172.21.15.116:0/4006882941 wait complete. 2024-05-14T11:19:42.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 Processor -- start 2024-05-14T11:19:42.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 -- start start 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02f8149360 0x7f02f80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80cbc20 0x7f02f80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02f80b73a0 con 0x7f02dc0053b0 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02f80b7220 con 0x7f02f80cbc20 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02ffa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f02f80b7520 con 0x7f02f8149360 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59626/0 (socket says 172.21.15.116:59626) 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 -- 172.21.15.116:0/1261098476 learned_addr learned my addr 172.21.15.116:0/1261098476 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fda00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80cbc20 0x7f02f80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02efe00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02f8149360 0x7f02f80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 -- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02f8149360 msgr2=0x7f02f80cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02f8149360 0x7f02f80cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 -- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80cbc20 msgr2=0x7f02f80d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80cbc20 0x7f02f80d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 -- 172.21.15.116:0/1261098476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f80cc030 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.423+0000 7f02fd000640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02e8007370 tx=0x7f02e8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e801a070 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e80016b0 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e800fbe0 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f814cc30 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f80b37f0 con 0x7f02dc0053b0 2024-05-14T11:19:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f806a9c0 con 0x7f02dc0053b0 2024-05-14T11:19:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f02e801d360 con 0x7f02dc0053b0 2024-05-14T11:19:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 0x7f02d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f02e808ed80 con 0x7f02dc0053b0 2024-05-14T11:19:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.427+0000 7f02efe00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 0x7f02d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.431+0000 7f02efe00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 0x7f02d4071910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f02f8152e70 tx=0x7f02e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.435+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f02f80cc030 con 0x7f02dc0053b0 2024-05-14T11:19:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.623+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02f8153180 con 0x7f02dc0053b0 2024-05-14T11:19:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.623+0000 7f02f5600640 1 -- 172.21.15.116:0/1261098476 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5191 (secure 0 0 0) 0x7f02e805bad0 con 0x7f02dc0053b0 2024-05-14T11:19:42.623 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:42.623 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:12.961108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 35 2024-05-14T11:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 msgr2=0x7f02d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 0x7f02d4071910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f02f8152e70 tx=0x7f02e0008040 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 msgr2=0x7f02f8149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02e8007370 tx=0x7f02e8007830 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 shutdown_connections 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02d406f400 0x7f02d4071910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80cbc20 0x7f02f80d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02f8149360 0x7f02f80cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 --2- 172.21.15.116:0/1261098476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02dc0053b0 0x7f02f8149db0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 >> 172.21.15.116:0/1261098476 conn(0x7f02f8071270 msgr2=0x7f02f81536a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 shutdown_connections 2024-05-14T11:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.627+0000 7f02ffa00640 1 -- 172.21.15.116:0/1261098476 wait complete. 2024-05-14T11:19:42.645 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-14T11:19:42.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 Processor -- start 2024-05-14T11:19:42.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 -- start start 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 0x7f5ce806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce8160510 con 0x7f5ce80d1670 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce81607f0 con 0x7f5ce80af380 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ceea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce81573b0 con 0x7f5ce8156fa0 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5ce80d1670 0x7f5ce80d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45634/0 (socket says 172.21.15.116:45634) 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7400640 1 -- 172.21.15.116:0/1451603653 learned_addr learned my addr 172.21.15.116:0/1451603653 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:42.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/1451603653 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 0x7f5ce806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822788649 0 0) 0x7f5ce8160510 con 0x7f5ce80d1670 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ccc003530 con 0x7f5ce80d1670 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645466751 0 0) 0x7f5ce81607f0 con 0x7f5ce80af380 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ce8160510 con 0x7f5ce80af380 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1451603653 >> v1:172.21.15.149:6789/0 conn(0x7f5ce80af380 legacy=0x7f5ce80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1451603653 >> v1:172.21.15.116:6789/0 conn(0x7f5ce80d1670 legacy=0x7f5ce80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.795+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1451603653 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce8160ad0 con 0x7f5ce8156fa0 2024-05-14T11:19:42.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/1451603653 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 0x7f5ce806ffe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80097f0 tx=0x7f5cd801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531d18c3ea16d082 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80ab5f0 con 0x7f5ce8156fa0 2024-05-14T11:19:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd80b0070 con 0x7f5ce8156fa0 2024-05-14T11:19:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ce6a00640 1 -- 172.21.15.116:0/1451603653 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80ab9b0 con 0x7f5ce8156fa0 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ceea00640 1 -- 172.21.15.116:0/1451603653 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 msgr2=0x7f5ce806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ceea00640 1 --2- 172.21.15.116:0/1451603653 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 0x7f5ce806ffe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80097f0 tx=0x7f5cd801c4e0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ceea00640 1 -- 172.21.15.116:0/1451603653 shutdown_connections 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ceea00640 1 --2- 172.21.15.116:0/1451603653 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce8156fa0 0x7f5ce806ffe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.799+0000 7f5ceea00640 1 -- 172.21.15.116:0/1451603653 >> 172.21.15.116:0/1451603653 conn(0x7f5ce80b4620 msgr2=0x7f5ce80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- 172.21.15.116:0/1451603653 shutdown_connections 2024-05-14T11:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- 172.21.15.116:0/1451603653 wait complete. 2024-05-14T11:19:42.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 Processor -- start 2024-05-14T11:19:42.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- start start 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 0x7f5ce8150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ce8156fa0 0x7f5ce8138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce806c880 con 0x7f5ce80d1670 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce806c700 con 0x7f5ce8156fa0 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.803+0000 7f5ceea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce806ca00 con 0x7f5ce80af380 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59630/0 (socket says 172.21.15.116:59630) 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 -- 172.21.15.116:0/314661341 learned_addr learned my addr 172.21.15.116:0/314661341 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ceca00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ce8156fa0 0x7f5ce8138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 0x7f5ce8150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 -- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 msgr2=0x7f5ce8150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 0x7f5ce8150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 -- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ce8156fa0 msgr2=0x7f5ce8138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ce8156fa0 0x7f5ce8138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 -- 172.21.15.116:0/314661341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce80cf020 con 0x7f5ce80d1670 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 0x7f5ce8150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ce7400640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c460 tx=0x7f5cd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5cdfe00640 1 -- 172.21.15.116:0/314661341 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4017070 con 0x7f5ce80d1670 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5cdfe00640 1 -- 172.21.15.116:0/314661341 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd4005cf0 con 0x7f5ce80d1670 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce80cf300 con 0x7f5ce80d1670 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5cdfe00640 1 -- 172.21.15.116:0/314661341 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40072f0 con 0x7f5ce80d1670 2024-05-14T11:19:42.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce814e6e0 con 0x7f5ce80d1670 2024-05-14T11:19:42.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.807+0000 7f5cdfe00640 1 -- 172.21.15.116:0/314661341 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5cd401a070 con 0x7f5ce80d1670 2024-05-14T11:19:42.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cdfe00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 0x7f5cc8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cdfe00640 1 -- 172.21.15.116:0/314661341 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5cd408e910 con 0x7f5ce80d1670 2024-05-14T11:19:42.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 0x7f5cc8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 Processor -- start 2024-05-14T11:19:42.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 -- start start 2024-05-14T11:19:42.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 0x7f5cc8071910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80af000 tx=0x7f5cd80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f5d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f5d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55611f3de1a0 con 0x55611f4652d0 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55611f396b80 con 0x55611f05f580 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55611f3da160 con 0x55611f475bd0 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59640/0 (socket says 172.21.15.116:59640) 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 -- 172.21.15.116:0/885191747 learned_addr learned my addr 172.21.15.116:0/885191747 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f5d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ceca00640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f5d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 -- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 msgr2=0x55611f5d8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f5d8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 -- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 msgr2=0x55611f5d98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f5d98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 -- 172.21.15.116:0/885191747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611f3ea630 con 0x55611f4652d0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f5d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ceca00640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f5d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5ce7400640 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd405a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce2124a68b6be7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.811+0000 7f5cd3e00640 1 -- 172.21.15.116:0/885191747 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd405ae50 con 0x55611f4652d0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cd3e00640 1 -- 172.21.15.116:0/885191747 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd405d070 con 0x55611f4652d0 2024-05-14T11:19:42.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cd3e00640 1 -- 172.21.15.116:0/885191747 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4007690 con 0x55611f4652d0 2024-05-14T11:19:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- 172.21.15.116:0/885191747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 msgr2=0x55611f5d93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd405a680 comp rx=0 tx=0).stop 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- 172.21.15.116:0/885191747 shutdown_connections 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f5d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f5d93b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- 172.21.15.116:0/885191747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f5d8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- 172.21.15.116:0/885191747 >> 172.21.15.116:0/885191747 conn(0x55611f09e760 msgr2=0x55611f5bbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- 172.21.15.116:0/885191747 shutdown_connections 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- 172.21.15.116:0/885191747 wait complete. 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- start start 2024-05-14T11:19:42.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 Processor -- start 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f05f580 0x55611f594a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f475bd0 0x55611f58bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55611f37dab0 con 0x55611f4652d0 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55611f392f20 con 0x55611f475bd0 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55611f43dda0 con 0x55611f05f580 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5cf6026000 10 client.0 _ll_register_callbacks cb 0x55611f579f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59654/0 (socket says 172.21.15.116:59654) 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.815+0000 7f5ce7400640 1 -- 172.21.15.116:0/3939208071 learned_addr learned my addr 172.21.15.116:0/3939208071 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f05f580 0x55611f594a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ceca00640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f475bd0 0x55611f58bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 -- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f05f580 msgr2=0x55611f594a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f05f580 0x55611f594a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 -- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f475bd0 msgr2=0x55611f58bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f475bd0 0x55611f58bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 -- 172.21.15.116:0/3939208071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611f39cce0 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400cfd0 tx=0x7f5cd4007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092cf21ca715a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd405ae50 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd405d070 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5cd405a0b0 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5cd405a2b0 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 10 client.6060 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55611f388740 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd405d070 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f5cd40567c0 con 0x55611f4652d0 2024-05-14T11:19:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 10 client.6060 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 10 client.6060 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 20 client.6060 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f083ad0 0x55611f58c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 4 client.6060 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611f376e60 con 0x55611f083ad0 2024-05-14T11:19:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f083ad0 0x55611f58c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 10 client.6060 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f083ad0 0x55611f58c930 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e247cf9da8035645 server_cookie=6d5ba114ffec077d in_seq=0 out_seq=0 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3939208071 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55611f376e60 con 0x55611f083ad0 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 10 client.6060 handle_command_reply: tid=0 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 client.6060 shutdown 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 msgr2=0x55611f58b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400cfd0 tx=0x7f5cd4007040 comp rx=0 tx=0).stop 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 shutdown_connections 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f475bd0 0x55611f58bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f083ad0 0x55611f58c930 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f58b650 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3939208071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f05f580 0x55611f594a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cd0c00640 0 client.6060 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 >> 172.21.15.116:0/3939208071 conn(0x55611f09e760 msgr2=0x55611f5ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 shutdown_connections 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 1 -- 172.21.15.116:0/3939208071 wait complete. 2024-05-14T11:19:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.819+0000 7f5cf6026000 20 client.6060 trim_cache size 0 max 16384 2024-05-14T11:19:42.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 Processor -- start 2024-05-14T11:19:42.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 -- start start 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 0x55611f64bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f64c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce80ce9e0 con 0x55611f05f580 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce80cdd40 con 0x55611f4652d0 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce80ce390 con 0x55611f475bd0 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59664/0 (socket says 172.21.15.116:59664) 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1270333581 learned_addr learned my addr 172.21.15.116:0/1270333581 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7400640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 0x55611f64bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ceca00640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f64c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 msgr2=0x55611f64c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f64c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 msgr2=0x55611f64bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 0x55611f64bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 -- 172.21.15.116:0/1270333581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611f396dd0 con 0x55611f05f580 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7400640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 0x55611f64bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.835+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80190f0 tx=0x7f5cd80a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1248d8e5100c359a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cd3400640 1 -- 172.21.15.116:0/1270333581 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80bb070 con 0x55611f05f580 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cd3400640 1 -- 172.21.15.116:0/1270333581 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd80b0040 con 0x55611f05f580 2024-05-14T11:19:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cd3400640 1 -- 172.21.15.116:0/1270333581 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80bfad0 con 0x55611f05f580 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- 172.21.15.116:0/1270333581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 msgr2=0x55611f64b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80190f0 tx=0x7f5cd80a9000 comp rx=0 tx=0).stop 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- 172.21.15.116:0/1270333581 shutdown_connections 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f4652d0 0x55611f64bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f64c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- 172.21.15.116:0/1270333581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f05f580 0x55611f64b610 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- 172.21.15.116:0/1270333581 >> 172.21.15.116:0/1270333581 conn(0x55611f09e760 msgr2=0x55611f5f2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- 172.21.15.116:0/1270333581 shutdown_connections 2024-05-14T11:19:42.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- 172.21.15.116:0/1270333581 wait complete. 2024-05-14T11:19:42.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- start start 2024-05-14T11:19:42.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 Processor -- start 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f64f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f60a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55611f3875f0 con 0x55611f4652d0 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55611f3d8e80 con 0x55611f05f580 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55611f399f60 con 0x55611f475bd0 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.839+0000 7f5cf6026000 10 client.0 _ll_register_callbacks cb 0x7f5cd405a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f60a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59670/0 (socket says 172.21.15.116:59670) 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 -- 172.21.15.116:0/3220923484 learned_addr learned my addr 172.21.15.116:0/3220923484 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f64f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 -- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 msgr2=0x55611f60a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f60a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 -- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 msgr2=0x55611f64f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f64f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 -- 172.21.15.116:0/3220923484 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611f45ec40 con 0x55611f4652d0 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7400640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40580c0 tx=0x7f5cd4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1d9b89732a24f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40574a0 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f5cd4059b20 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd80b0040 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5cd405ae50 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 10 client.6066 fetch_fsmap learned FSMap version 35 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55611f0973c0 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd80b0040 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f5cd4059d20 con 0x55611f4652d0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 10 client.6066 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 10 client.6066 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 20 client.6066 populate_metadata read hostname 'smithi116' 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f071b30 0x55611f60ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 4 client.6066 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 --> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55611f3ebdb0 con 0x55611f071b30 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f071b30 0x55611f60ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:42.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ce7e00640 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f071b30 0x55611f60ab50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fecc64d440d683 server_cookie=8d557c6619608400 in_seq=0 out_seq=0 2024-05-14T11:19:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 10 client.6066 ms_handle_connect on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 1 -- 172.21.15.116:0/3220923484 <== mds.0 v2:172.21.15.116:6834/1576287145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55611f3ebdb0 con 0x55611f071b30 2024-05-14T11:19:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 10 client.6066 handle_command_reply: tid=0 2024-05-14T11:19:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 client.6066 shutdown 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 msgr2=0x55611f64fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40580c0 tx=0x7f5cd4009040 comp rx=0 tx=0).stop 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 shutdown_connections 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:6834/1576287145,v1:172.21.15.116:6836/1576287145] conn(0x55611f071b30 0x55611f60ab50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55611f475bd0 0x55611f60a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55611f4652d0 0x55611f64fcb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 --2- 172.21.15.116:0/3220923484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55611f05f580 0x55611f64f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cd0c00640 0 client.6066 ms_handle_reset on v2:172.21.15.116:6834/1576287145 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 >> 172.21.15.116:0/3220923484 conn(0x55611f09e760 msgr2=0x55611f5f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 shutdown_connections 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 1 -- 172.21.15.116:0/3220923484 wait complete. 2024-05-14T11:19:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5cf6026000 20 client.6066 trim_cache size 0 max 16384 2024-05-14T11:19:42.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:42.846 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-05-14T11:19:42.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 msgr2=0x7f5cc8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 0x7f5cc8071910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80af000 tx=0x7f5cd80a7000 comp rx=0 tx=0).stop 2024-05-14T11:19:42.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 msgr2=0x7f5ce809a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c460 tx=0x7f5cd400c920 comp rx=0 tx=0).stop 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 shutdown_connections 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5cc806f400 0x7f5cc8071910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ce8156fa0 0x7f5ce8138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce80d1670 0x7f5ce809a8b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.843+0000 7f5ceea00640 1 --2- 172.21.15.116:0/314661341 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5ce80af380 0x7f5ce8150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.847+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 >> 172.21.15.116:0/314661341 conn(0x7f5ce80b4620 msgr2=0x7f5ce814d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.847+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 shutdown_connections 2024-05-14T11:19:42.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:42.847+0000 7f5ceea00640 1 -- 172.21.15.116:0/314661341 wait complete. 2024-05-14T11:19:42.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:19:43.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.007+0000 7f35fbc00640 1 Processor -- start 2024-05-14T11:19:43.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 -- start start 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 0x7f35f40714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35f4153c90 0x7f35f4071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f40ccaa0 con 0x7f35f40d1810 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f35f40b1f00 con 0x7f35f4153c90 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35fbc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f35f415e070 con 0x7f35f4072470 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35f4153c90 0x7f35f4071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35f4153c90 0x7f35f4071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59680/0 (socket says 172.21.15.116:59680) 2024-05-14T11:19:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 -- 172.21.15.116:0/956506138 learned_addr learned my addr 172.21.15.116:0/956506138 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f9200640 1 --2- 172.21.15.116:0/956506138 >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 0x7f35f40714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 -- 172.21.15.116:0/956506138 >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 msgr2=0x7f35f40714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 --2- 172.21.15.116:0/956506138 >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 0x7f35f40714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 -- 172.21.15.116:0/956506138 >> v1:172.21.15.149:6789/0 conn(0x7f35f40d1810 legacy=0x7f35f40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f3e00640 1 -- 172.21.15.116:0/956506138 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f40d2330 con 0x7f35f4153c90 2024-05-14T11:19:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.011+0000 7f35f9200640 1 --2- 172.21.15.116:0/956506138 >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 0x7f35f40714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35f3e00640 1 --2- 172.21.15.116:0/956506138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f4071a00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f35f4072400 tx=0x7f35e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1377ea71fdddad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35f3400640 1 -- 172.21.15.116:0/956506138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00b0070 con 0x7f35f4153c90 2024-05-14T11:19:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35f3400640 1 -- 172.21.15.116:0/956506138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e00ab600 con 0x7f35f4153c90 2024-05-14T11:19:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35f3400640 1 -- 172.21.15.116:0/956506138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00ab9d0 con 0x7f35f4153c90 2024-05-14T11:19:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 -- 172.21.15.116:0/956506138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 msgr2=0x7f35f4071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 --2- 172.21.15.116:0/956506138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f4071a00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f35f4072400 tx=0x7f35e001c520 comp rx=0 tx=0).stop 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 -- 172.21.15.116:0/956506138 shutdown_connections 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 --2- 172.21.15.116:0/956506138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f4071a00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 --2- 172.21.15.116:0/956506138 >> v2:172.21.15.149:3300/0 conn(0x7f35f4072470 0x7f35f40714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 -- 172.21.15.116:0/956506138 >> 172.21.15.116:0/956506138 conn(0x7f35f4073ca0 msgr2=0x7f35f40740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 -- 172.21.15.116:0/956506138 shutdown_connections 2024-05-14T11:19:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.015+0000 7f35fbc00640 1 -- 172.21.15.116:0/956506138 wait complete. 2024-05-14T11:19:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 Processor -- start 2024-05-14T11:19:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- start start 2024-05-14T11:19:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35f4072470 0x7f35f41490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35f40d1810 0x7f35f40c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f35f40c8ec0 con 0x7f35f4153c90 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35f415db50 con 0x7f35f4072470 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f35f40c9040 con 0x7f35f40d1810 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59694/0 (socket says 172.21.15.116:59694) 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 -- 172.21.15.116:0/2209682719 learned_addr learned my addr 172.21.15.116:0/2209682719 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9200640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35f4072470 0x7f35f41490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f3e00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35f40d1810 0x7f35f40c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 -- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35f40d1810 msgr2=0x7f35f40c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35f40d1810 0x7f35f40c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 -- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35f4072470 msgr2=0x7f35f41490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35f4072470 0x7f35f41490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 -- 172.21.15.116:0/2209682719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f4151e50 con 0x7f35f4153c90 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f9c00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ba10 tx=0x7f35ec00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec019070 con 0x7f35f4153c90 2024-05-14T11:19:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec00cba0 con 0x7f35f4153c90 2024-05-14T11:19:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec014440 con 0x7f35f4153c90 2024-05-14T11:19:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f40ca050 con 0x7f35f4153c90 2024-05-14T11:19:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.019+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f4149600 con 0x7f35f4153c90 2024-05-14T11:19:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f406a9c0 con 0x7f35f4153c90 2024-05-14T11:19:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35ec01c070 con 0x7f35f4153c90 2024-05-14T11:19:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35f0c00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 0x7f35d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f35ec08f190 con 0x7f35f4153c90 2024-05-14T11:19:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35f9200640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 0x7f35d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.023+0000 7f35f9200640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 0x7f35d4071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006fd0 tx=0x7f35e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.027+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f35ec05b800 con 0x7f35f4153c90 2024-05-14T11:19:43.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f35f406fa50 con 0x7f35f4153c90 2024-05-14T11:19:43.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35f0c00640 1 -- 172.21.15.116:0/2209682719 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f35ec05fb00 con 0x7f35f4153c90 2024-05-14T11:19:43.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:43.221 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 msgr2=0x7f35d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 0x7f35d4071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006fd0 tx=0x7f35e8008040 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 msgr2=0x7f35f40c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ba10 tx=0x7f35ec00bed0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 shutdown_connections 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35d406f400 0x7f35d4071910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35f4153c90 0x7f35f40c9930 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35f40d1810 0x7f35f40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 --2- 172.21.15.116:0/2209682719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35f4072470 0x7f35f41490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:43.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 >> 172.21.15.116:0/2209682719 conn(0x7f35f4073ca0 msgr2=0x7f35f406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:43.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 shutdown_connections 2024-05-14T11:19:43.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:43.223+0000 7f35fbc00640 1 -- 172.21.15.116:0/2209682719 wait complete. 2024-05-14T11:19:43.242 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-05-14T11:19:48.243 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:19:48.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 Processor -- start 2024-05-14T11:19:48.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 -- start start 2024-05-14T11:19:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 0x7f76e80aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 0x7f76e80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e806ff20 con 0x7f76e8161600 2024-05-14T11:19:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f76e806d710 con 0x7f76e80d1670 2024-05-14T11:19:48.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76efa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f76e80ae100 con 0x7f76e815db30 2024-05-14T11:19:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76eda00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f76e8161600 0x7f76e8161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53582/0 (socket says 172.21.15.116:53582) 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76eda00640 1 -- 172.21.15.116:0/1885110688 learned_addr learned my addr 172.21.15.116:0/1885110688 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76e7e00640 1 --2- 172.21.15.116:0/1885110688 >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 0x7f76e80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76ed000640 1 --2- 172.21.15.116:0/1885110688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 0x7f76e80aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355618826 0 0) 0x7f76e806ff20 con 0x7f76e8161600 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d0003530 con 0x7f76e8161600 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76ed000640 1 -- 172.21.15.116:0/1885110688 >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 msgr2=0x7f76e80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76ed000640 1 --2- 172.21.15.116:0/1885110688 >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 0x7f76e80d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76ed000640 1 -- 172.21.15.116:0/1885110688 >> v1:172.21.15.116:6789/0 conn(0x7f76e8161600 legacy=0x7f76e8161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4187313610 0 0) 0x7f76d0003530 con 0x7f76e8161600 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76ed000640 1 -- 172.21.15.116:0/1885110688 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80d1fc0 con 0x7f76e815db30 2024-05-14T11:19:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.403+0000 7f76e7e00640 1 --2- 172.21.15.116:0/1885110688 >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 0x7f76e80d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76ed000640 1 --2- 172.21.15.116:0/1885110688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 0x7f76e80aeaa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb8277ba97153f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0b0070 con 0x7f76e815db30 2024-05-14T11:19:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc0ab400 con 0x7f76e815db30 2024-05-14T11:19:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76e7400640 1 -- 172.21.15.116:0/1885110688 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0001a80 con 0x7f76e815db30 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 -- 172.21.15.116:0/1885110688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 msgr2=0x7f76e80aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 --2- 172.21.15.116:0/1885110688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 0x7f76e80aeaa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0097c0 tx=0x7f76dc01c330 comp rx=0 tx=0).stop 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 -- 172.21.15.116:0/1885110688 shutdown_connections 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 --2- 172.21.15.116:0/1885110688 >> v2:172.21.15.149:3300/0 conn(0x7f76e80d1670 0x7f76e80d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 --2- 172.21.15.116:0/1885110688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e815db30 0x7f76e80aeaa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 -- 172.21.15.116:0/1885110688 >> 172.21.15.116:0/1885110688 conn(0x7f76e80b4620 msgr2=0x7f76e80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 -- 172.21.15.116:0/1885110688 shutdown_connections 2024-05-14T11:19:48.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.407+0000 7f76efa00640 1 -- 172.21.15.116:0/1885110688 wait complete. 2024-05-14T11:19:48.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 Processor -- start 2024-05-14T11:19:48.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 -- start start 2024-05-14T11:19:48.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e80d1670 0x7f76e8147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 0x7f76e8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76e815df40 con 0x7f76e815db30 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80aefe0 con 0x7f76e80d1670 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76efa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f76e816af80 con 0x7f76e8161600 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59866/0 (socket says 172.21.15.116:59866) 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76e7e00640 1 -- 172.21.15.116:0/1004769414 learned_addr learned my addr 172.21.15.116:0/1004769414 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.411+0000 7f76ed000640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e80d1670 0x7f76e8147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76eda00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 0x7f76e8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 -- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 msgr2=0x7f76e8159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 0x7f76e8159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 -- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e80d1670 msgr2=0x7f76e8147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e80d1670 0x7f76e8147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 -- 172.21.15.116:0/1004769414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e815bfe0 con 0x7f76e815db30 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76eda00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 0x7f76e8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e7e00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f76d400e8e0 tx=0x7f76d400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4017070 con 0x7f76e815db30 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d400c8e0 con 0x7f76e815db30 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e815aa00 con 0x7f76e815db30 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d400ccd0 con 0x7f76e815db30 2024-05-14T11:19:48.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e815afc0 con 0x7f76e815db30 2024-05-14T11:19:48.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e806a9c0 con 0x7f76e815db30 2024-05-14T11:19:48.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.415+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76d401a070 con 0x7f76e815db30 2024-05-14T11:19:48.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.419+0000 7f76e4c00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 0x7f76c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:48.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.419+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f76d408f350 con 0x7f76e815db30 2024-05-14T11:19:48.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.419+0000 7f76ed000640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 0x7f76c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:48.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.419+0000 7f76ed000640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 0x7f76c8071940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ad000 tx=0x7f76dc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:48.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.423+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f76d405b940 con 0x7f76e815db30 2024-05-14T11:19:48.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.615+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f76e815a790 con 0x7f76e815db30 2024-05-14T11:19:48.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.615+0000 7f76e4c00640 1 -- 172.21.15.116:0/1004769414 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f76d405fc40 con 0x7f76e815db30 2024-05-14T11:19:48.615 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:48.615 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 msgr2=0x7f76c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 0x7f76c8071940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ad000 tx=0x7f76dc0a6000 comp rx=0 tx=0).stop 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 msgr2=0x7f76e815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f76d400e8e0 tx=0x7f76d400eda0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 shutdown_connections 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f76c806f430 0x7f76c8071940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76e8161600 0x7f76e8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76e815db30 0x7f76e815baa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 --2- 172.21.15.116:0/1004769414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e80d1670 0x7f76e8147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:48.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 >> 172.21.15.116:0/1004769414 conn(0x7f76e80b4620 msgr2=0x7f76e81675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:48.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 shutdown_connections 2024-05-14T11:19:48.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:48.619+0000 7f76efa00640 1 -- 172.21.15.116:0/1004769414 wait complete. 2024-05-14T11:19:48.636 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-14T11:19:48.636 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:19:48.636 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:19:48.754 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:19:48.800 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:19:48.800 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:19:48.801 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:19:48.801 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:48.809 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:19:48.809 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:48.861 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:19:49.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 Processor -- start 2024-05-14T11:19:49.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 -- start start 2024-05-14T11:19:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f52d4074b70 0x7f52d40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d415d340 con 0x7f52d40af830 2024-05-14T11:19:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f52d415d620 con 0x7f52d4074b70 2024-05-14T11:19:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52db400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52d415d900 con 0x7f52d40af420 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52cbe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f52d4074b70 0x7f52d40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52d8a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52d9400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f52d40af830 0x7f52d406edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54138/0 (socket says 172.21.15.116:54138) 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.055+0000 7f52d9400640 1 -- 172.21.15.116:0/2032790583 learned_addr learned my addr 172.21.15.116:0/2032790583 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d3e00640 1 -- 172.21.15.116:0/2032790583 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950225206 0 0) 0x7f52d415d340 con 0x7f52d40af830 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d3e00640 1 -- 172.21.15.116:0/2032790583 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b8003530 con 0x7f52d40af830 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52cbe00640 1 -- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 msgr2=0x7f52d4074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52cbe00640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4074630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52cbe00640 1 -- 172.21.15.116:0/2032790583 >> v1:172.21.15.149:6789/0 conn(0x7f52d40af830 legacy=0x7f52d406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52cbe00640 1 -- 172.21.15.116:0/2032790583 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d415dbe0 con 0x7f52d4074b70 2024-05-14T11:19:49.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d8a00640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:49.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52cbe00640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d40d1670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0097c0 tx=0x7f52bc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f77b416b9c2ea1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d3e00640 1 -- 172.21.15.116:0/2032790583 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0b1070 con 0x7f52d4074b70 2024-05-14T11:19:49.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d3e00640 1 -- 172.21.15.116:0/2032790583 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc0ac410 con 0x7f52d4074b70 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.059+0000 7f52d3e00640 1 -- 172.21.15.116:0/2032790583 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0ac7c0 con 0x7f52d4074b70 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 msgr2=0x7f52d40d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d40d1670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0097c0 tx=0x7f52bc01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- 172.21.15.116:0/2032790583 shutdown_connections 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d40d1670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 --2- 172.21.15.116:0/2032790583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- 172.21.15.116:0/2032790583 >> 172.21.15.116:0/2032790583 conn(0x7f52d40b4620 msgr2=0x7f52d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:49.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- 172.21.15.116:0/2032790583 shutdown_connections 2024-05-14T11:19:49.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- 172.21.15.116:0/2032790583 wait complete. 2024-05-14T11:19:49.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 Processor -- start 2024-05-14T11:19:49.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.063+0000 7f52db400640 1 -- start start 2024-05-14T11:19:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d409a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52d416a420 con 0x7f52d40af830 2024-05-14T11:19:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52d416a2a0 con 0x7f52d4074b70 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52d416a5a0 con 0x7f52d40af420 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59874/0 (socket says 172.21.15.116:59874) 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 -- 172.21.15.116:0/3960539511 learned_addr learned my addr 172.21.15.116:0/3960539511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d8a00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d409a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52cbe00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 -- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 msgr2=0x7f52d4150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 -- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 msgr2=0x7f52d409a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d409a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 -- 172.21.15.116:0/3960539511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d40ced00 con 0x7f52d40af830 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d8a00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d409a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52cbe00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d9400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00caa0 tx=0x7f52cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00aba0 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc00ad40 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc012600 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d40cefe0 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d40cf5a0 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d406a9c0 con 0x7f52d40af830 2024-05-14T11:19:49.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.067+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52cc0127a0 con 0x7f52d40af830 2024-05-14T11:19:49.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.071+0000 7f52d1600640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 0x7f52b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.071+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f52cc08fa70 con 0x7f52d40af830 2024-05-14T11:19:49.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.071+0000 7f52d8a00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 0x7f52b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.071+0000 7f52d8a00640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 0x7f52b0071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52d4070ac0 tx=0x7f52c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.075+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f52cc05c730 con 0x7f52d40af830 2024-05-14T11:19:49.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.243+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52d40b0590 con 0x7f52d40af830 2024-05-14T11:19:49.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.243+0000 7f52d1600640 1 -- 172.21.15.116:0/3960539511 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f52cc05c030 con 0x7f52d40af830 2024-05-14T11:19:49.242 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 msgr2=0x7f52b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 0x7f52b0071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52d4070ac0 tx=0x7f52c4008040 comp rx=0 tx=0).stop 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 msgr2=0x7f52d415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00caa0 tx=0x7f52cc00cf60 comp rx=0 tx=0).stop 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 shutdown_connections 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52b006f430 0x7f52b0071940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52d40af830 0x7f52d415b910 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52d40af420 0x7f52d4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 --2- 172.21.15.116:0/3960539511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52d4074b70 0x7f52d409a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 >> 172.21.15.116:0/3960539511 conn(0x7f52d40b4620 msgr2=0x7f52d415f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 shutdown_connections 2024-05-14T11:19:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.247+0000 7f52db400640 1 -- 172.21.15.116:0/3960539511 wait complete. 2024-05-14T11:19:49.264 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:19:49.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:49.264 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:19:49.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[40086]: fuse finished with error 0 and tester_r 0 2024-05-14T11:19:49.284 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:19:49.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:19:49.284 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:19:49.444 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:19:49.444 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:19:49.444 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:49.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:19:49.454 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:19:49.454 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:19:49.454 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:49.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:19:49.500 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:19:49.501 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:19:49.501 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:49.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:19:49.510 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:49.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 Processor -- start 2024-05-14T11:19:49.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 -- start start 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f30a00d2d50 0x7f30a00ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 0x7f30a00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a00cec10 con 0x7f30a01621b0 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f30a00d1d40 con 0x7f30a00d2d50 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a8800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30a00d3160 con 0x7f30a00ce800 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a5e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f30a01621b0 0x7f30a00b9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53584/0 (socket says 172.21.15.116:53584) 2024-05-14T11:19:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a5e00640 1 -- 172.21.15.116:0/1932254797 learned_addr learned my addr 172.21.15.116:0/1932254797 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a5400640 1 --2- 172.21.15.116:0/1932254797 >> v2:172.21.15.149:3300/0 conn(0x7f30a00d2d50 0x7f30a00ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a6800640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 0x7f30a00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a4a00640 1 -- 172.21.15.116:0/1932254797 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265138425 0 0) 0x7f3094001120 con 0x7f30a01621b0 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.691+0000 7f30a4a00640 1 -- 172.21.15.116:0/1932254797 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3088003530 con 0x7f30a01621b0 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a5400640 1 -- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 msgr2=0x7f30a00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:49.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a5400640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 0x7f30a00b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a5400640 1 -- 172.21.15.116:0/1932254797 >> v1:172.21.15.116:6789/0 conn(0x7f30a01621b0 legacy=0x7f30a00b9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a5400640 1 -- 172.21.15.116:0/1932254797 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a00d1f90 con 0x7f30a00d2d50 2024-05-14T11:19:49.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a6800640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 0x7f30a00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a5400640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00d2d50 0x7f30a00ce2c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f30900097c0 tx=0x7f309001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4b7b01abc74bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a4a00640 1 -- 172.21.15.116:0/1932254797 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30900b0070 con 0x7f30a00d2d50 2024-05-14T11:19:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a4a00640 1 -- 172.21.15.116:0/1932254797 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30900ab600 con 0x7f30a00d2d50 2024-05-14T11:19:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.695+0000 7f30a4a00640 1 -- 172.21.15.116:0/1932254797 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30900ab9d0 con 0x7f30a00d2d50 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00d2d50 msgr2=0x7f30a00ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00d2d50 0x7f30a00ce2c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f30900097c0 tx=0x7f309001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- 172.21.15.116:0/1932254797 shutdown_connections 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00ce800 0x7f30a00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 --2- 172.21.15.116:0/1932254797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00d2d50 0x7f30a00ce2c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- 172.21.15.116:0/1932254797 >> 172.21.15.116:0/1932254797 conn(0x7f30a00d0770 msgr2=0x7f30a00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:49.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- 172.21.15.116:0/1932254797 shutdown_connections 2024-05-14T11:19:49.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- 172.21.15.116:0/1932254797 wait complete. 2024-05-14T11:19:49.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 Processor -- start 2024-05-14T11:19:49.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 -- start start 2024-05-14T11:19:49.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00ce800 0x7f30a006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.699+0000 7f30a8800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00d2d50 0x7f30a014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f30a015e3f0 con 0x7f30a01621b0 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30a015e270 con 0x7f30a00ce800 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30a015e570 con 0x7f30a00d2d50 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59880/0 (socket says 172.21.15.116:59880) 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 -- 172.21.15.116:0/2492898383 learned_addr learned my addr 172.21.15.116:0/2492898383 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a5e00640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00ce800 0x7f30a006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a5400640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00d2d50 0x7f30a014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 -- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00d2d50 msgr2=0x7f30a014b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00d2d50 0x7f30a014b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 -- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00ce800 msgr2=0x7f30a006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00ce800 0x7f30a006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 -- 172.21.15.116:0/2492898383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a00b67a0 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a6800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c018030 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c00cd30 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c003bc0 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a0166cc0 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a00b6cf0 con 0x7f30a01621b0 2024-05-14T11:19:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a006f580 con 0x7f30a01621b0 2024-05-14T11:19:49.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.703+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f309c01b070 con 0x7f30a01621b0 2024-05-14T11:19:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.707+0000 7f309a000640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 0x7f3084071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.707+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f309c08fd40 con 0x7f30a01621b0 2024-05-14T11:19:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.707+0000 7f30a5e00640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 0x7f3084071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.707+0000 7f30a5e00640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 0x7f3084071910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3094007fd0 tx=0x7f3094009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:49.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.711+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f309c022070 con 0x7f30a01621b0 2024-05-14T11:19:49.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.895+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30a006a9c0 con 0x7f30a01621b0 2024-05-14T11:19:49.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f309a000640 1 -- 172.21.15.116:0/2492898383 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5192 (secure 0 0 0) 0x7f309c010040 con 0x7f30a01621b0 2024-05-14T11:19:49.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:49.897 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:44.961667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:49.897 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 36 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 msgr2=0x7f3084071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 0x7f3084071910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3094007fd0 tx=0x7f3094009040 comp rx=0 tx=0).stop 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 msgr2=0x7f30a0156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c490 comp rx=0 tx=0).stop 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 shutdown_connections 2024-05-14T11:19:49.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f308406f400 0x7f3084071910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30a01621b0 0x7f30a0156c90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30a00d2d50 0x7f30a014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 --2- 172.21.15.116:0/2492898383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30a00ce800 0x7f30a006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 >> 172.21.15.116:0/2492898383 conn(0x7f30a00d0770 msgr2=0x7f30a0160690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 shutdown_connections 2024-05-14T11:19:49.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:49.899+0000 7f30a8800640 1 -- 172.21.15.116:0/2492898383 wait complete. 2024-05-14T11:19:49.917 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.063+0000 7f69ef800640 1 Processor -- start 2024-05-14T11:19:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 -- start start 2024-05-14T11:19:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e80d1670 0x7f69e80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e80d1fc0 con 0x7f69e8156fa0 2024-05-14T11:19:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f69e80706f0 con 0x7f69e80d1670 2024-05-14T11:19:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ef800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69e81573b0 con 0x7f69e80af380 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e80d1670 0x7f69e80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ece00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f69e8156fa0 0x7f69e806ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53586/0 (socket says 172.21.15.116:53586) 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ece00640 1 -- 172.21.15.116:0/3244886181 learned_addr learned my addr 172.21.15.116:0/3244886181 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ed800640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7e00640 1 -- 172.21.15.116:0/3244886181 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 msgr2=0x7f69e80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7e00640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e80af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7e00640 1 -- 172.21.15.116:0/3244886181 >> v1:172.21.15.116:6789/0 conn(0x7f69e8156fa0 legacy=0x7f69e806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7e00640 1 -- 172.21.15.116:0/3244886181 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e8160510 con 0x7f69e80d1670 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69e7400640 1 -- 172.21.15.116:0/3244886181 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325634627 0 0) 0x7f69d0001120 con 0x7f69e8156fa0 2024-05-14T11:19:50.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.067+0000 7f69ed800640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:50.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69e7e00640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e80d1670 0x7f69e80d1a80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f69d80097c0 tx=0x7f69d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bf8a3317176b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69e7400640 1 -- 172.21.15.116:0/3244886181 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b2070 con 0x7f69e80d1670 2024-05-14T11:19:50.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69e7400640 1 -- 172.21.15.116:0/3244886181 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80ad640 con 0x7f69e80d1670 2024-05-14T11:19:50.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69e7400640 1 -- 172.21.15.116:0/3244886181 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ada10 con 0x7f69e80d1670 2024-05-14T11:19:50.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 -- 172.21.15.116:0/3244886181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e80d1670 msgr2=0x7f69e80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e80d1670 0x7f69e80d1a80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f69d80097c0 tx=0x7f69d801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 -- 172.21.15.116:0/3244886181 shutdown_connections 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 --2- 172.21.15.116:0/3244886181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e80d1670 0x7f69e80d1a80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 -- 172.21.15.116:0/3244886181 >> 172.21.15.116:0/3244886181 conn(0x7f69e80b4620 msgr2=0x7f69e80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 -- 172.21.15.116:0/3244886181 shutdown_connections 2024-05-14T11:19:50.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.071+0000 7f69ef800640 1 -- 172.21.15.116:0/3244886181 wait complete. 2024-05-14T11:19:50.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 Processor -- start 2024-05-14T11:19:50.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- start start 2024-05-14T11:19:50.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e80d1670 0x7f69e81510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80b0660 con 0x7f69e8156fa0 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80b04e0 con 0x7f69e80d1670 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69e8169810 con 0x7f69e80af380 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59904/0 (socket says 172.21.15.116:59904) 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 -- 172.21.15.116:0/3959999174 learned_addr learned my addr 172.21.15.116:0/3959999174 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ece00640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69e7e00640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e80d1670 0x7f69e81510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 -- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 msgr2=0x7f69e8074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e8074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 -- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e80d1670 msgr2=0x7f69e81510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e80d1670 0x7f69e81510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 -- 172.21.15.116:0/3959999174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e815c3f0 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ed800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007e70 tx=0x7f69e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e001a070 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e0003bc0 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000de20 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e8073e20 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.075+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e815bb50 con 0x7f69e8156fa0 2024-05-14T11:19:50.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.079+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e806a9c0 con 0x7f69e8156fa0 2024-05-14T11:19:50.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.079+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69e001d070 con 0x7f69e8156fa0 2024-05-14T11:19:50.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.079+0000 7f69e4c00640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 0x7f69c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.079+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f69e008f710 con 0x7f69e8156fa0 2024-05-14T11:19:50.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.079+0000 7f69ece00640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 0x7f69c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.083+0000 7f69ece00640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 0x7f69c4071910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69d0000f70 tx=0x7f69d0007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.083+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f69e0010070 con 0x7f69e8156fa0 2024-05-14T11:19:50.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.275+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69e8150ab0 con 0x7f69e8156fa0 2024-05-14T11:19:50.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.275+0000 7f69e4c00640 1 -- 172.21.15.116:0/3959999174 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5192 (secure 0 0 0) 0x7f69e005c3d0 con 0x7f69e8156fa0 2024-05-14T11:19:50.275 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:50.275 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:44.961667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:50.275 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 36 2024-05-14T11:19:50.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 msgr2=0x7f69c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 0x7f69c4071910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69d0000f70 tx=0x7f69d0007c10 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 msgr2=0x7f69e80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007e70 tx=0x7f69e000d4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 shutdown_connections 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69c406f400 0x7f69c4071910 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e80d1670 0x7f69e81510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e8156fa0 0x7f69e80731c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 --2- 172.21.15.116:0/3959999174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69e80af380 0x7f69e8074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 >> 172.21.15.116:0/3959999174 conn(0x7f69e80b4620 msgr2=0x7f69e80b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 shutdown_connections 2024-05-14T11:19:50.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.279+0000 7f69ef800640 1 -- 172.21.15.116:0/3959999174 wait complete. 2024-05-14T11:19:50.296 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:19:50.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.443+0000 7f0fc4200640 1 Processor -- start 2024-05-14T11:19:50.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.443+0000 7f0fc4200640 1 -- start start 2024-05-14T11:19:50.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc4200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc4200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 0x7f0fbc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc4200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc06ff20 con 0x7f0fbc161600 2024-05-14T11:19:50.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc4200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc06d710 con 0x7f0fbc15db30 2024-05-14T11:19:50.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc4200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0ae100 con 0x7f0fbc0d1670 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc0e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 0x7f0fbc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc2200640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f0fbc161600 0x7f0fbc161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54154/0 (socket says 172.21.15.116:54154) 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc2200640 1 -- 172.21.15.116:0/2312835010 learned_addr learned my addr 172.21.15.116:0/2312835010 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59050/0 (socket says 172.21.15.116:59050) 2024-05-14T11:19:50.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc0e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 0x7f0fbc0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:52244/0 (socket says 172.21.15.116:52244) 2024-05-14T11:19:50.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 -- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 msgr2=0x7f0fbc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:50.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 --2- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 0x7f0fbc0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 -- 172.21.15.116:0/2312835010 >> v1:172.21.15.149:6789/0 conn(0x7f0fbc161600 legacy=0x7f0fbc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fbbe00640 1 -- 172.21.15.116:0/2312835010 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353427049 0 0) 0x7f0fbc06ff20 con 0x7f0fbc161600 2024-05-14T11:19:50.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.447+0000 7f0fc1800640 1 -- 172.21.15.116:0/2312835010 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0d1fc0 con 0x7f0fbc15db30 2024-05-14T11:19:50.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc1800640 1 --2- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00097c0 tx=0x7f0fb001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968d25113ec71188 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fbbe00640 1 -- 172.21.15.116:0/2312835010 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00b0070 con 0x7f0fbc15db30 2024-05-14T11:19:50.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fbbe00640 1 -- 172.21.15.116:0/2312835010 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00ab600 con 0x7f0fbc15db30 2024-05-14T11:19:50.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fbbe00640 1 -- 172.21.15.116:0/2312835010 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00ab9d0 con 0x7f0fbc15db30 2024-05-14T11:19:50.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 -- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc15db30 msgr2=0x7f0fbc0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00097c0 tx=0x7f0fb001c410 comp rx=0 tx=0).stop 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 -- 172.21.15.116:0/2312835010 shutdown_connections 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc0d1670 0x7f0fbc0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2312835010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc15db30 0x7f0fbc0aeaa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 -- 172.21.15.116:0/2312835010 >> 172.21.15.116:0/2312835010 conn(0x7f0fbc0b4620 msgr2=0x7f0fbc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 -- 172.21.15.116:0/2312835010 shutdown_connections 2024-05-14T11:19:50.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.451+0000 7f0fc4200640 1 -- 172.21.15.116:0/2312835010 wait complete. 2024-05-14T11:19:50.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 Processor -- start 2024-05-14T11:19:50.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 -- start start 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc0d1670 0x7f0fbc160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc15db30 0x7f0fbc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15df40 con 0x7f0fbc161600 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0aefe0 con 0x7f0fbc0d1670 2024-05-14T11:19:50.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc16af80 con 0x7f0fbc15db30 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59914/0 (socket says 172.21.15.116:59914) 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 -- 172.21.15.116:0/2918322204 learned_addr learned my addr 172.21.15.116:0/2918322204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc0e00640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc15db30 0x7f0fbc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc1800640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc0d1670 0x7f0fbc160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 -- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc15db30 msgr2=0x7f0fbc158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc15db30 0x7f0fbc158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 -- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc0d1670 msgr2=0x7f0fbc160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc0d1670 0x7f0fbc160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 -- 172.21.15.116:0/2918322204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc1590c0 con 0x7f0fbc161600 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc2200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c680 tx=0x7f0fb400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4018070 con 0x7f0fbc161600 2024-05-14T11:19:50.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4004d70 con 0x7f0fbc161600 2024-05-14T11:19:50.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc15c860 con 0x7f0fbc161600 2024-05-14T11:19:50.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.455+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400a880 con 0x7f0fbc161600 2024-05-14T11:19:50.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc153b90 con 0x7f0fbc161600 2024-05-14T11:19:50.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc06a9c0 con 0x7f0fbc161600 2024-05-14T11:19:50.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0fb401b070 con 0x7f0fbc161600 2024-05-14T11:19:50.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fb9600640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 0x7f0f9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f0fb408e850 con 0x7f0fbc161600 2024-05-14T11:19:50.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.459+0000 7f0fc1800640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 0x7f0f9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.463+0000 7f0fc1800640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 0x7f0f9c071910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00ad000 tx=0x7f0fb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.463+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0fb405af50 con 0x7f0fbc161600 2024-05-14T11:19:50.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.623+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fbc15c5f0 con 0x7f0fbc161600 2024-05-14T11:19:50.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fb9600640 1 -- 172.21.15.116:0/2918322204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16932 (secure 0 0 0) 0x7f0fb405f250 con 0x7f0fbc161600 2024-05-14T11:19:50.625 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:50.625 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:57.918127+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:18:54.005073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-14T11:18:55.275835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 msgr2=0x7f0f9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 0x7f0f9c071910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00ad000 tx=0x7f0fb0008040 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 msgr2=0x7f0fbc1513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c680 tx=0x7f0fb400cb40 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 shutdown_connections 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f9c06f400 0x7f0f9c071910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fbc161600 0x7f0fbc1513d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0fbc15db30 0x7f0fbc158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 --2- 172.21.15.116:0/2918322204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc0d1670 0x7f0fbc160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 >> 172.21.15.116:0/2918322204 conn(0x7f0fbc0b4620 msgr2=0x7f0fbc0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:50.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 shutdown_connections 2024-05-14T11:19:50.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.627+0000 7f0fc4200640 1 -- 172.21.15.116:0/2918322204 wait complete. 2024-05-14T11:19:50.646 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:19:50.647 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:50.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 Processor -- start 2024-05-14T11:19:50.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 -- start start 2024-05-14T11:19:50.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f68d0164050 0x7f68d015fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d006eb90 con 0x7f68d006e780 2024-05-14T11:19:50.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d0164510 con 0x7f68d00cf900 2024-05-14T11:19:50.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d8200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f68d00cd1f0 con 0x7f68d0164050 2024-05-14T11:19:50.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d5800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f68d006e780 0x7f68d0163940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53594/0 (socket says 172.21.15.116:53594) 2024-05-14T11:19:50.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d5800640 1 -- 172.21.15.116:0/3489782920 learned_addr learned my addr 172.21.15.116:0/3489782920 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68d4e00640 1 --2- 172.21.15.116:0/3489782920 >> v2:172.21.15.149:3300/0 conn(0x7f68d0164050 0x7f68d015fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980222713 0 0) 0x7f68d006eb90 con 0x7f68d006e780 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b8003530 con 0x7f68d006e780 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3473721076 0 0) 0x7f68d0164510 con 0x7f68d00cf900 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68d006eb90 con 0x7f68d00cf900 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987738536 0 0) 0x7f68b8003530 con 0x7f68d006e780 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 >> v2:172.21.15.149:3300/0 conn(0x7f68d0164050 msgr2=0x7f68d015fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 --2- 172.21.15.116:0/3489782920 >> v2:172.21.15.149:3300/0 conn(0x7f68d0164050 0x7f68d015fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 >> v1:172.21.15.149:6789/0 conn(0x7f68d00cf900 legacy=0x7f68d00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d00cd4d0 con 0x7f68d006e780 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f68c4001880 con 0x7f68d006e780 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.795+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 >> v1:172.21.15.116:6789/0 conn(0x7f68d006e780 legacy=0x7f68d0163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8003c40 0x7f68b8004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 0x7f68b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68b8005330 0x7f68b800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68d00cd1f0 con 0x7f68b8004810 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68b800c2d0 con 0x7f68b8003c40 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f68b800c4d0 con 0x7f68b8005330 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 0x7f68b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d5800640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8003c40 0x7f68b8004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d6200640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68b8005330 0x7f68b800b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 -- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68b8005330 msgr2=0x7f68b800b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68b8005330 0x7f68b800b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 -- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8003c40 msgr2=0x7f68b8004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8003c40 0x7f68b8004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 -- 172.21.15.116:0/3489782920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b8003530 con 0x7f68b8004810 2024-05-14T11:19:50.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d6200640 1 -- 172.21.15.116:0/3489782920 reap_dead start 2024-05-14T11:19:50.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68d4e00640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 0x7f68b8004c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ab0 tx=0x7f68bc01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b82f5fa97ef8bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b2070 con 0x7f68b8004810 2024-05-14T11:19:50.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0a5630 con 0x7f68b8004810 2024-05-14T11:19:50.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.799+0000 7f68cfe00640 1 -- 172.21.15.116:0/3489782920 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0096b0 con 0x7f68b8004810 2024-05-14T11:19:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 msgr2=0x7f68b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 0x7f68b8004c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ab0 tx=0x7f68bc01ca40 comp rx=0 tx=0).stop 2024-05-14T11:19:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- 172.21.15.116:0/3489782920 shutdown_connections 2024-05-14T11:19:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 --2- 172.21.15.116:0/3489782920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b8004810 0x7f68b8004c20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- 172.21.15.116:0/3489782920 >> 172.21.15.116:0/3489782920 conn(0x7f68d00ce570 msgr2=0x7f68d00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- 172.21.15.116:0/3489782920 shutdown_connections 2024-05-14T11:19:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- 172.21.15.116:0/3489782920 wait complete. 2024-05-14T11:19:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 Processor -- start 2024-05-14T11:19:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.803+0000 7f68d8200640 1 -- start start 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8004810 0x7f68d01fade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 0x7f68d01fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68d00cc140 con 0x7f68d01f70a0 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68d00c5340 con 0x7f68b8004810 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f68d00cc2c0 con 0x7f68d01fa3c0 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59920/0 (socket says 172.21.15.116:59920) 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 -- 172.21.15.116:0/2002093314 learned_addr learned my addr 172.21.15.116:0/2002093314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d6200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 0x7f68d01fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d5800640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8004810 0x7f68d01fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 -- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 msgr2=0x7f68d01fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 0x7f68d01fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 -- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8004810 msgr2=0x7f68d01fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8004810 0x7f68d01fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 -- 172.21.15.116:0/2002093314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d01f9c20 con 0x7f68d01f70a0 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d6200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 0x7f68d01fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d4e00640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ab0 tx=0x7f68b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b2040 con 0x7f68d01f70a0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc00f000 con 0x7f68d01f70a0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d00b6fc0 con 0x7f68d01f70a0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0096b0 con 0x7f68d01f70a0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d00b7580 con 0x7f68d01f70a0 2024-05-14T11:19:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.807+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d00b6d30 con 0x7f68d01f70a0 2024-05-14T11:19:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.811+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68bc0c0030 con 0x7f68d01f70a0 2024-05-14T11:19:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.811+0000 7f68cd600640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 0x7f68b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.811+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f68bc128f40 con 0x7f68d01f70a0 2024-05-14T11:19:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.811+0000 7f68d5800640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 0x7f68b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.811+0000 7f68d5800640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 0x7f68b0071910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000f00 tx=0x7f68c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:50.815+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f68d01f9c20 con 0x7f68d01f70a0 2024-05-14T11:19:51.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68d006a9c0 con 0x7f68d01f70a0 2024-05-14T11:19:51.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68cd600640 1 -- 172.21.15.116:0/2002093314 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5192 (secure 0 0 0) 0x7f68bc016780 con 0x7f68d01f70a0 2024-05-14T11:19:51.005 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:51.005 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:44.961667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:51.006 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 36 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 msgr2=0x7f68b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 0x7f68b0071910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000f00 tx=0x7f68c4009040 comp rx=0 tx=0).stop 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 msgr2=0x7f68d00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009ab0 tx=0x7f68b8005840 comp rx=0 tx=0).stop 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 shutdown_connections 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f68b006f400 0x7f68b0071910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f68d01fa3c0 0x7f68d01fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68d01f70a0 0x7f68d00b4e70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 --2- 172.21.15.116:0/2002093314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68b8004810 0x7f68d01fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 >> 172.21.15.116:0/2002093314 conn(0x7f68d00ce570 msgr2=0x7f68d0070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 shutdown_connections 2024-05-14T11:19:51.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.007+0000 7f68d8200640 1 -- 172.21.15.116:0/2002093314 wait complete. 2024-05-14T11:19:51.026 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:51.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 Processor -- start 2024-05-14T11:19:51.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 -- start start 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f01181621b0 0x7f01180d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f01180ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01180cec10 con 0x7f01180ce800 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f01180d1d40 con 0x7f01181621b0 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01180d1f90 con 0x7f01180b9020 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f01180ce800 0x7f01180b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53600/0 (socket says 172.21.15.116:53600) 2024-05-14T11:19:51.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011de00640 1 -- 172.21.15.116:0/2120548748 learned_addr learned my addr 172.21.15.116:0/2120548748 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011ca00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f01180ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f011d400640 1 --2- 172.21.15.116:0/2120548748 >> v2:172.21.15.149:3300/0 conn(0x7f01181621b0 0x7f01180d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980244992 0 0) 0x7f01100014a0 con 0x7f01180ce800 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0100003530 con 0x7f01180ce800 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 437382219 0 0) 0x7f0100003530 con 0x7f01180ce800 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 msgr2=0x7f01180ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f01180ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 >> v2:172.21.15.149:3300/0 conn(0x7f01181621b0 msgr2=0x7f01180d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 --2- 172.21.15.116:0/2120548748 >> v2:172.21.15.149:3300/0 conn(0x7f01181621b0 0x7f01180d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01180d2270 con 0x7f01180ce800 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f0110001c50 con 0x7f01180ce800 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.175+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 >> v1:172.21.15.116:6789/0 conn(0x7f01180ce800 legacy=0x7f01180b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0100003a10 0x7f0100003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0100004810 0x7f0100004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f0100005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01180d1d40 con 0x7f0100003a10 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01180d1f90 con 0x7f0100004810 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f010000ba80 con 0x7f01180b9020 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0100003a10 0x7f0100003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011ca00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0100004810 0x7f0100004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011de00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f0100005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 -- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 msgr2=0x7f0100005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01180b9020 0x7f0100005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 -- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0100004810 msgr2=0x7f0100004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0100004810 0x7f0100004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 -- 172.21.15.116:0/2120548748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0100003530 con 0x7f0100003a10 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f011d400640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0100003a10 0x7f0100003e80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ab0 tx=0x7f010c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f32f5701854ec45 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0096b0 con 0x7f0100003a10 2024-05-14T11:19:51.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f010c0a6470 con 0x7f0100003a10 2024-05-14T11:19:51.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.179+0000 7f0117e00640 1 -- 172.21.15.116:0/2120548748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0a6820 con 0x7f0100003a10 2024-05-14T11:19:51.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- 172.21.15.116:0/2120548748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0100003a10 msgr2=0x7f0100003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 --2- 172.21.15.116:0/2120548748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0100003a10 0x7f0100003e80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ab0 tx=0x7f010c01ca90 comp rx=0 tx=0).stop 2024-05-14T11:19:51.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011de00640 1 -- 172.21.15.116:0/2120548748 reap_dead start 2024-05-14T11:19:51.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- 172.21.15.116:0/2120548748 shutdown_connections 2024-05-14T11:19:51.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- 172.21.15.116:0/2120548748 >> 172.21.15.116:0/2120548748 conn(0x7f01180d0770 msgr2=0x7f01180d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- 172.21.15.116:0/2120548748 shutdown_connections 2024-05-14T11:19:51.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- 172.21.15.116:0/2120548748 wait complete. 2024-05-14T11:19:51.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 Processor -- start 2024-05-14T11:19:51.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.183+0000 7f011fe00640 1 -- start start 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0100003a10 0x7f0118150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 0x7f011806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01180b3bd0 con 0x7f0118071560 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01180b3a50 con 0x7f01180b6700 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f011815ea80 con 0x7f0100003a10 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59932/0 (socket says 172.21.15.116:59932) 2024-05-14T11:19:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 -- 172.21.15.116:0/3589502787 learned_addr learned my addr 172.21.15.116:0/3589502787 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011ca00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 0x7f011806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011d400640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0100003a10 0x7f0118150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 -- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0100003a10 msgr2=0x7f0118150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0100003a10 0x7f0118150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 -- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 msgr2=0x7f011806eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 0x7f011806eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 -- 172.21.15.116:0/3589502787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0118071eb0 con 0x7f0118071560 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011ca00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 0x7f011806eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011de00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01180b4bb0 tx=0x7f011000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000eb90 con 0x7f0118071560 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0110019070 con 0x7f0118071560 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0118150780 con 0x7f0118071560 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110013420 con 0x7f0118071560 2024-05-14T11:19:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0118152450 con 0x7f0118071560 2024-05-14T11:19:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.187+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011806a9c0 con 0x7f0118071560 2024-05-14T11:19:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.191+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01100135c0 con 0x7f0118071560 2024-05-14T11:19:51.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.191+0000 7f0115600640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 0x7f00f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.191+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f011008f7c0 con 0x7f0118071560 2024-05-14T11:19:51.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.191+0000 7f011d400640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 0x7f00f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.191+0000 7f011d400640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 0x7f00f8071940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ab0 tx=0x7f010c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.195+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0118071eb0 con 0x7f0118071560 2024-05-14T11:19:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.383+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01180b1f50 con 0x7f0118071560 2024-05-14T11:19:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f0115600640 1 -- 172.21.15.116:0/3589502787 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5192 (secure 0 0 0) 0x7f01180cec10 con 0x7f0118071560 2024-05-14T11:19:51.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:51.385 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:18:56.305831+0000","modified":"2024-05-14T11:19:44.961667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.116:6836/1576287145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1576287145},{"type":"v1","addr":"172.21.15.116:6836","nonce":1576287145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":4}]} 2024-05-14T11:19:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 36 2024-05-14T11:19:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 msgr2=0x7f00f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 0x7f00f8071940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ab0 tx=0x7f010c0046d0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 msgr2=0x7f0118071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01180b4bb0 tx=0x7f011000e530 comp rx=0 tx=0).stop 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 shutdown_connections 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00f806f430 0x7f00f8071940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01180b6700 0x7f011806eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0118071560 0x7f0118071970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 --2- 172.21.15.116:0/3589502787 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0100003a10 0x7f0118150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 >> 172.21.15.116:0/3589502787 conn(0x7f01180d0770 msgr2=0x7f0118160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 shutdown_connections 2024-05-14T11:19:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.387+0000 7f011fe00640 1 -- 172.21.15.116:0/3589502787 wait complete. 2024-05-14T11:19:51.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:19:51.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 Processor -- start 2024-05-14T11:19:51.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 -- start start 2024-05-14T11:19:51.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 0x7fa28c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa28c0d1670 0x7fa28c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa28c0d1fc0 con 0x7fa28c06e9e0 2024-05-14T11:19:51.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa28c1573b0 con 0x7fa28c0d1670 2024-05-14T11:19:51.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa293200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa28c161120 con 0x7fa28c156fa0 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa28be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 0x7fa28c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa28c0d1670 0x7fa28c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa28c0d1670 0x7fa28c0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59128/0 (socket says 172.21.15.116:59128) 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 -- 172.21.15.116:0/2695598508 learned_addr learned my addr 172.21.15.116:0/2695598508 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa28be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 0x7fa28c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:52308/0 (socket says 172.21.15.116:52308) 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 -- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 msgr2=0x7fa28c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 --2- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 0x7fa28c06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 -- 172.21.15.116:0/2695598508 >> v1:172.21.15.149:6789/0 conn(0x7fa28c06e9e0 legacy=0x7fa28c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.555+0000 7fa283400640 1 -- 172.21.15.116:0/2695598508 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c160510 con 0x7fa28c0d1670 2024-05-14T11:19:51.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa283400640 1 --2- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c0d1670 0x7fa28c0d1a80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009870 tx=0x7fa27c01e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4cf941bbcce8ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa28b400640 1 -- 172.21.15.116:0/2695598508 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0aa610 con 0x7fa28c0d1670 2024-05-14T11:19:51.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa28b400640 1 -- 172.21.15.116:0/2695598508 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0aa7b0 con 0x7fa28c0d1670 2024-05-14T11:19:51.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa28b400640 1 -- 172.21.15.116:0/2695598508 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0aab80 con 0x7fa28c0d1670 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 -- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c0d1670 msgr2=0x7fa28c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 --2- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c0d1670 0x7fa28c0d1a80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009870 tx=0x7fa27c01e6e0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 -- 172.21.15.116:0/2695598508 shutdown_connections 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 --2- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c0d1670 0x7fa28c0d1a80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 --2- 172.21.15.116:0/2695598508 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c156fa0 0x7fa28c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 -- 172.21.15.116:0/2695598508 >> 172.21.15.116:0/2695598508 conn(0x7fa28c0b4620 msgr2=0x7fa28c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 -- 172.21.15.116:0/2695598508 shutdown_connections 2024-05-14T11:19:51.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.559+0000 7fa293200640 1 -- 172.21.15.116:0/2695598508 wait complete. 2024-05-14T11:19:51.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 Processor -- start 2024-05-14T11:19:51.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 -- start start 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c06e9e0 0x7fa28c0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c0d1670 0x7fa28c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c070520 con 0x7fa28c156fa0 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c0b07a0 con 0x7fa28c06e9e0 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa293200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa28c0706a0 con 0x7fa28c0d1670 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa291200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa291200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59944/0 (socket says 172.21.15.116:59944) 2024-05-14T11:19:51.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa291200640 1 -- 172.21.15.116:0/3890619798 learned_addr learned my addr 172.21.15.116:0/3890619798 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa283400640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c0d1670 0x7fa28c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.563+0000 7fa28be00640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c06e9e0 0x7fa28c0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 -- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c0d1670 msgr2=0x7fa28c1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c0d1670 0x7fa28c1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 -- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c06e9e0 msgr2=0x7fa28c0733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c06e9e0 0x7fa28c0733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 -- 172.21.15.116:0/3890619798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c074a80 con 0x7fa28c156fa0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa291200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa28400c730 tx=0x7fa28400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284003b80 con 0x7fa28c156fa0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa284003d20 con 0x7fa28c156fa0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28400aa70 con 0x7fa28c156fa0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c0744f0 con 0x7fa28c156fa0 2024-05-14T11:19:51.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c158900 con 0x7fa28c156fa0 2024-05-14T11:19:51.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c06a9c0 con 0x7fa28c156fa0 2024-05-14T11:19:51.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.567+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa28400ac10 con 0x7fa28c156fa0 2024-05-14T11:19:51.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.571+0000 7fa288c00640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 0x7fa268071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.571+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fa28408f180 con 0x7fa28c156fa0 2024-05-14T11:19:51.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.571+0000 7fa28be00640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 0x7fa268071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.571+0000 7fa28be00640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 0x7fa268071940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0b0d60 tx=0x7fa274009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.575+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa28c0d1fc0 con 0x7fa28c156fa0 2024-05-14T11:19:51.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.731+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa28c074780 con 0x7fa28c156fa0 2024-05-14T11:19:51.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.731+0000 7fa288c00640 1 -- 172.21.15.116:0/3890619798 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16932 (secure 0 0 0) 0x7fa28405b890 con 0x7fa28c156fa0 2024-05-14T11:19:51.730 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:51.731 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:18:57.918127+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:18:54.005073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-14T11:18:55.275835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 msgr2=0x7fa268071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 0x7fa268071940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0b0d60 tx=0x7fa274009190 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 msgr2=0x7fa28c152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa28400c730 tx=0x7fa28400cbf0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 shutdown_connections 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa26806f430 0x7fa268071940 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c156fa0 0x7fa28c152100 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa28c0d1670 0x7fa28c1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 --2- 172.21.15.116:0/3890619798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c06e9e0 0x7fa28c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 >> 172.21.15.116:0/3890619798 conn(0x7fa28c0b4620 msgr2=0x7fa28c0b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 shutdown_connections 2024-05-14T11:19:51.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.735+0000 7fa293200640 1 -- 172.21.15.116:0/3890619798 wait complete. 2024-05-14T11:19:51.752 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:19:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.899+0000 7f542a000640 1 Processor -- start 2024-05-14T11:19:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.899+0000 7f542a000640 1 -- start start 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f542a000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54240b0770 0x7f542406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f542a000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f542406f730 0x7f542406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f542a000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5424070080 con 0x7f542415fc00 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f542a000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f542406db20 con 0x7f54240b0770 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f542a000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f542406cc40 con 0x7f542406f730 2024-05-14T11:19:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54240b0770 0x7f542406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54240b0770 0x7f542406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59958/0 (socket says 172.21.15.116:59958) 2024-05-14T11:19:51.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 -- 172.21.15.116:0/1584535225 learned_addr learned my addr 172.21.15.116:0/1584535225 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:51.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f542415fc00 0x7f5424160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53614/0 (socket says 172.21.15.116:53614) 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f541aa00640 1 --2- 172.21.15.116:0/1584535225 >> v2:172.21.15.149:3300/0 conn(0x7f542406f730 0x7f542406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 -- 172.21.15.116:0/1584535225 >> v2:172.21.15.149:3300/0 conn(0x7f542406f730 msgr2=0x7f542406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 --2- 172.21.15.116:0/1584535225 >> v2:172.21.15.149:3300/0 conn(0x7f542406f730 0x7f542406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 -- 172.21.15.116:0/1584535225 >> v1:172.21.15.116:6789/0 conn(0x7f542415fc00 legacy=0x7f5424160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5423400640 1 -- 172.21.15.116:0/1584535225 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542406cf10 con 0x7f54240b0770 2024-05-14T11:19:51.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.903+0000 7f5422a00640 1 -- 172.21.15.116:0/1584535225 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784290105 0 0) 0x7f541c0014a0 con 0x7f542415fc00 2024-05-14T11:19:51.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f5423400640 1 --2- 172.21.15.116:0/1584535225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54240b0770 0x7f542406c700 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f542406f4e0 tx=0x7f541401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb8928bbe587312 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f5422a00640 1 -- 172.21.15.116:0/1584535225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140ab660 con 0x7f54240b0770 2024-05-14T11:19:51.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f5422a00640 1 -- 172.21.15.116:0/1584535225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54140b0070 con 0x7f54240b0770 2024-05-14T11:19:51.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f5422a00640 1 -- 172.21.15.116:0/1584535225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140aba20 con 0x7f54240b0770 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 -- 172.21.15.116:0/1584535225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54240b0770 msgr2=0x7f542406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 --2- 172.21.15.116:0/1584535225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54240b0770 0x7f542406c700 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f542406f4e0 tx=0x7f541401c460 comp rx=0 tx=0).stop 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 -- 172.21.15.116:0/1584535225 shutdown_connections 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 --2- 172.21.15.116:0/1584535225 >> v2:172.21.15.149:3300/0 conn(0x7f542406f730 0x7f542406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 --2- 172.21.15.116:0/1584535225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54240b0770 0x7f542406c700 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 -- 172.21.15.116:0/1584535225 >> 172.21.15.116:0/1584535225 conn(0x7f54240b4620 msgr2=0x7f54240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 -- 172.21.15.116:0/1584535225 shutdown_connections 2024-05-14T11:19:51.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.907+0000 7f542a000640 1 -- 172.21.15.116:0/1584535225 wait complete. 2024-05-14T11:19:51.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 Processor -- start 2024-05-14T11:19:51.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- start start 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54240b0770 0x7f5424154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f542415fc00 0x7f5424154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5424073a40 con 0x7f542406f730 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f542406dfb0 con 0x7f542415fc00 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f54240d1670 con 0x7f54240b0770 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59962/0 (socket says 172.21.15.116:59962) 2024-05-14T11:19:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 -- 172.21.15.116:0/3563644426 learned_addr learned my addr 172.21.15.116:0/3563644426 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423e00640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f542415fc00 0x7f5424154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f541aa00640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54240b0770 0x7f5424154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 -- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54240b0770 msgr2=0x7f5424154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54240b0770 0x7f5424154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 -- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f542415fc00 msgr2=0x7f5424154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f542415fc00 0x7f5424154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 -- 172.21.15.116:0/3563644426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5424166a40 con 0x7f542406f730 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f5423400640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f54140ad040 tx=0x7f541401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140ba070 con 0x7f542406f730 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54140abc20 con 0x7f542406f730 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542415f7c0 con 0x7f542406f730 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414004020 con 0x7f542406f730 2024-05-14T11:19:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.911+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5424165f60 con 0x7f542406f730 2024-05-14T11:19:51.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542406a9c0 con 0x7f542406f730 2024-05-14T11:19:51.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54140041c0 con 0x7f542406f730 2024-05-14T11:19:51.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f541be00640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 0x7f5400071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:51.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f54141296d0 con 0x7f542406f730 2024-05-14T11:19:51.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f541aa00640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 0x7f5400071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:51.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.915+0000 7f541aa00640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 0x7f5400071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f542415c2b0 tx=0x7f540c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:51.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:51.919+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5414016110 con 0x7f542406f730 2024-05-14T11:19:52.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.115+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f542406f510 con 0x7f542406f730 2024-05-14T11:19:52.113 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:19:52.115+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:19:52.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.115+0000 7f541be00640 1 -- 172.21.15.116:0/3563644426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f54140044b0 con 0x7f542406f730 2024-05-14T11:19:52.115 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:19:52.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 msgr2=0x7f5400071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 0x7f5400071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f542415c2b0 tx=0x7f540c008040 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 msgr2=0x7f5424165370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f54140ad040 tx=0x7f541401aca0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 shutdown_connections 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f540006f400 0x7f5400071910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f542415fc00 0x7f5424154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54240b0770 0x7f5424154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 --2- 172.21.15.116:0/3563644426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f542406f730 0x7f5424165370 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 >> 172.21.15.116:0/3563644426 conn(0x7f54240b4620 msgr2=0x7f5424073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 shutdown_connections 2024-05-14T11:19:52.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.119+0000 7f542a000640 1 -- 172.21.15.116:0/3563644426 wait complete. 2024-05-14T11:19:52.127 INFO:tasks.ceph.mds.a.smithi116.stderr: -5596> 2024-05-14T11:18:57.887+0000 7fe6ee200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:19:52.131 INFO:tasks.ceph.mds.a.smithi116.stderr: -3636> 2024-05-14T11:19:10.311+0000 7fe6e9200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5898=0-4194304@1} caps={5898=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5898 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562bde2fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:19:52.133 INFO:tasks.ceph.mds.a.smithi116.stderr: -1683> 2024-05-14T11:19:41.983+0000 7fe6e9200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-14T11:19:09.141597+0000 1=1+0) caps={5898=pAsLsXsFs/-@2},l=5898(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x562bde2fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:19:52.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:19:52.251 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:19:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 Processor -- start 2024-05-14T11:19:52.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 -- start start 2024-05-14T11:19:52.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9ea006f730 0x7f9ea006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:52.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea0070080 con 0x7f9ea0072a30 2024-05-14T11:19:52.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea006cf10 con 0x7f9ea00b0770 2024-05-14T11:19:52.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9ea5a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea006db20 con 0x7f9ea006f730 2024-05-14T11:19:52.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9e9f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9ea0072a30 0x7f9ea0072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53630/0 (socket says 172.21.15.116:53630) 2024-05-14T11:19:52.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.295+0000 7f9e9f400640 1 -- 172.21.15.116:0/1144128370 learned_addr learned my addr 172.21.15.116:0/1144128370 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9e000640 1 --2- 172.21.15.116:0/1144128370 >> v2:172.21.15.149:3300/0 conn(0x7f9ea006f730 0x7f9ea006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073152479 0 0) 0x7f9e980014a0 con 0x7f9ea0072a30 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e84003450 con 0x7f9ea0072a30 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038383521 0 0) 0x7f9ea006cf10 con 0x7f9ea00b0770 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e980014a0 con 0x7f9ea00b0770 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9e000640 1 -- 172.21.15.116:0/1144128370 >> v1:172.21.15.149:6789/0 conn(0x7f9ea00b0770 legacy=0x7f9ea006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128349598 0 0) 0x7f9e84003450 con 0x7f9ea0072a30 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9e000640 1 -- 172.21.15.116:0/1144128370 >> v1:172.21.15.116:6789/0 conn(0x7f9ea0072a30 legacy=0x7f9ea0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9e000640 1 -- 172.21.15.116:0/1144128370 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea006d1f0 con 0x7f9ea006f730 2024-05-14T11:19:52.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.299+0000 7f9e9e000640 1 --2- 172.21.15.116:0/1144128370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea006f730 0x7f9ea006fb40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0097c0 tx=0x7f9e8c01c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660e7e6eeae85f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:52.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0b0070 con 0x7f9ea006f730 2024-05-14T11:19:52.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e8c0ab400 con 0x7f9ea006f730 2024-05-14T11:19:52.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9e9d600640 1 -- 172.21.15.116:0/1144128370 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98001c70 con 0x7f9ea006f730 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- 172.21.15.116:0/1144128370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea006f730 msgr2=0x7f9ea006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/1144128370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea006f730 0x7f9ea006fb40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0097c0 tx=0x7f9e8c01c300 comp rx=0 tx=0).stop 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- 172.21.15.116:0/1144128370 shutdown_connections 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/1144128370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea006f730 0x7f9ea006fb40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- 172.21.15.116:0/1144128370 >> 172.21.15.116:0/1144128370 conn(0x7f9ea00b4620 msgr2=0x7f9ea00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- 172.21.15.116:0/1144128370 shutdown_connections 2024-05-14T11:19:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- 172.21.15.116:0/1144128370 wait complete. 2024-05-14T11:19:52.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 Processor -- start 2024-05-14T11:19:52.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.303+0000 7f9ea5a00640 1 -- start start 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea0072a30 0x7f9ea01f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 0x7f9ea015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0073ae0 con 0x7f9ea006f730 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0073960 con 0x7f9ea0072a30 2024-05-14T11:19:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea0169fb0 con 0x7f9ea00b0770 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59984/0 (socket says 172.21.15.116:59984) 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 -- 172.21.15.116:0/2437646370 learned_addr learned my addr 172.21.15.116:0/2437646370 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9f400640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 0x7f9ea015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9e000640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea0072a30 0x7f9ea01f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 -- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 msgr2=0x7f9ea015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 0x7f9ea015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 -- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea0072a30 msgr2=0x7f9ea01f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea0072a30 0x7f9ea01f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 -- 172.21.15.116:0/2437646370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea01f9b50 con 0x7f9ea006f730 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9f400640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 0x7f9ea015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e9ea00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c430 tx=0x7f9e9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90017070 con 0x7f9ea006f730 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900052f0 con 0x7f9ea006f730 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea01533f0 con 0x7f9ea006f730 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90003840 con 0x7f9ea006f730 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea01539b0 con 0x7f9ea006f730 2024-05-14T11:19:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.307+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea006a9c0 con 0x7f9ea006f730 2024-05-14T11:19:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.311+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e9001a070 con 0x7f9ea006f730 2024-05-14T11:19:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.311+0000 7f9e96a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 0x7f9e80071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.311+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6187+0+0 (secure 0 0 0) 0x7f9e9008e7c0 con 0x7f9ea006f730 2024-05-14T11:19:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.311+0000 7f9e9e000640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 0x7f9e80071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.311+0000 7f9e9e000640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 0x7f9e80071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0ad000 tx=0x7f9e8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.315+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9e9005ae80 con 0x7f9ea006f730 2024-05-14T11:19:52.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:52.511+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ea006f510 con 0x7f9ea006f730 2024-05-14T11:19:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.127+0000 7f9e96a00640 1 -- 172.21.15.116:0/2437646370 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f9e9005f180 con 0x7f9ea006f730 2024-05-14T11:19:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 msgr2=0x7f9e80071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 0x7f9e80071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0ad000 tx=0x7f9e8c008040 comp rx=0 tx=0).stop 2024-05-14T11:19:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 msgr2=0x7f9ea0152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c430 tx=0x7f9e9000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 shutdown_connections 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9e8006f430 0x7f9e80071940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ea0072a30 0x7f9ea01f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ea00b0770 0x7f9ea015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 --2- 172.21.15.116:0/2437646370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea006f730 0x7f9ea0152af0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 >> 172.21.15.116:0/2437646370 conn(0x7f9ea00b4620 msgr2=0x7f9ea00b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 shutdown_connections 2024-05-14T11:19:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.131+0000 7f9ea5a00640 1 -- 172.21.15.116:0/2437646370 wait complete. 2024-05-14T11:19:53.154 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 Processor -- start 2024-05-14T11:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 -- start start 2024-05-14T11:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22ec161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f22ec161c70 0x7f22ec0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec162080 con 0x7f22ec0cd990 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec0ce1a0 con 0x7f22ec161c70 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22f1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec15f4a0 con 0x7f22ec1583b0 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22ebe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f22ec0cd990 0x7f22ec0b7230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53632/0 (socket says 172.21.15.116:53632) 2024-05-14T11:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.335+0000 7f22ebe00640 1 -- 172.21.15.116:0/3751771650 learned_addr learned my addr 172.21.15.116:0/3751771650 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eaa00640 1 --2- 172.21.15.116:0/3751771650 >> v2:172.21.15.149:3300/0 conn(0x7f22ec161c70 0x7f22ec0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22ec161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283982035 0 0) 0x7f22e40014a0 con 0x7f22ec0cd990 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d0003530 con 0x7f22ec0cd990 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3040262432 0 0) 0x7f22d0003530 con 0x7f22ec0cd990 2024-05-14T11:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 msgr2=0x7f22ec161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22ec161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 >> v2:172.21.15.149:3300/0 conn(0x7f22ec161c70 msgr2=0x7f22ec0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 --2- 172.21.15.116:0/3751771650 >> v2:172.21.15.149:3300/0 conn(0x7f22ec161c70 0x7f22ec0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec15f780 con 0x7f22ec0cd990 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f22e4001c50 con 0x7f22ec0cd990 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 >> v1:172.21.15.116:6789/0 conn(0x7f22ec0cd990 legacy=0x7f22ec0b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22d0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0004d50 0x7f22d0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0ce1a0 con 0x7f22d0003a10 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec15f4a0 con 0x7f22d0004d50 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22d000ba80 con 0x7f22ec1583b0 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22d0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ebe00640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0004d50 0x7f22d0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eaa00640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 -- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 msgr2=0x7f22d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec1583b0 0x7f22d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 -- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0004d50 msgr2=0x7f22d0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0004d50 0x7f22d0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22eb400640 1 -- 172.21.15.116:0/3751771650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d0003530 con 0x7f22d0003a10 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.339+0000 7f22ebe00640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0004d50 0x7f22d0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22eb400640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22d0003e80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ab0 tx=0x7f22dc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb9129b620648bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0096b0 con 0x7f22d0003a10 2024-05-14T11:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc0a6450 con 0x7f22d0003a10 2024-05-14T11:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22ea000640 1 -- 172.21.15.116:0/3751771650 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a6800 con 0x7f22d0003a10 2024-05-14T11:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 -- 172.21.15.116:0/3751771650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 msgr2=0x7f22d0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 --2- 172.21.15.116:0/3751771650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22d0003e80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ab0 tx=0x7f22dc01ca70 comp rx=0 tx=0).stop 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22ebe00640 1 -- 172.21.15.116:0/3751771650 reap_dead start 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 -- 172.21.15.116:0/3751771650 shutdown_connections 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 -- 172.21.15.116:0/3751771650 >> 172.21.15.116:0/3751771650 conn(0x7f22ec0cee00 msgr2=0x7f22ec0cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 -- 172.21.15.116:0/3751771650 shutdown_connections 2024-05-14T11:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.343+0000 7f22f1000640 1 -- 172.21.15.116:0/3751771650 wait complete. 2024-05-14T11:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 Processor -- start 2024-05-14T11:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- start start 2024-05-14T11:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 0x7f22ec0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec156660 0x7f22ec156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec160910 con 0x7f22d0003a10 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec160790 con 0x7f22ec0cb390 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec160a90 con 0x7f22ec156660 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59996/0 (socket says 172.21.15.116:59996) 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 -- 172.21.15.116:0/2605196649 learned_addr learned my addr 172.21.15.116:0/2605196649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 -- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec156660 msgr2=0x7f22ec156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec156660 0x7f22ec156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 -- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 msgr2=0x7f22ec0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 0x7f22ec0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 0x7f22ec0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 -- 172.21.15.116:0/2605196649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1fb7a0 con 0x7f22d0003a10 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 0x7f22ec0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22eb400640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f22dc000c10 tx=0x7f22d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0be070 con 0x7f22d0003a10 2024-05-14T11:19:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc00f000 con 0x7f22d0003a10 2024-05-14T11:19:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec158b40 con 0x7f22d0003a10 2024-05-14T11:19:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a6e00 con 0x7f22d0003a10 2024-05-14T11:19:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.347+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec159100 con 0x7f22d0003a10 2024-05-14T11:19:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec050880 con 0x7f22d0003a10 2024-05-14T11:19:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22dc0c1070 con 0x7f22d0003a10 2024-05-14T11:19:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22e3400640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 0x7f22cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6332+0+0 (secure 0 0 0) 0x7f22dc128ff0 con 0x7f22d0003a10 2024-05-14T11:19:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 0x7f22cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.351+0000 7f22eaa00640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 0x7f22cc071940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22d80090e0 tx=0x7f22d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:53.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.355+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f22dc0ac020 con 0x7f22d0003a10 2024-05-14T11:19:53.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:53.515+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22ec0ce890 con 0x7f22d0003a10 2024-05-14T11:19:54.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.118+0000 7f22e3400640 1 -- 172.21.15.116:0/2605196649 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) ==== 181+0+0 (secure 0 0 0) 0x7f22dc016780 con 0x7f22d0003a10 2024-05-14T11:19:54.119 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:19:54.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 msgr2=0x7f22cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 0x7f22cc071940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22d80090e0 tx=0x7f22d8008040 comp rx=0 tx=0).stop 2024-05-14T11:19:54.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 msgr2=0x7f22ec1fb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f22dc000c10 tx=0x7f22d0005840 comp rx=0 tx=0).stop 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 shutdown_connections 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f22cc06f430 0x7f22cc071940 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22ec0cb390 0x7f22ec0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22ec156660 0x7f22ec156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 --2- 172.21.15.116:0/2605196649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22d0003a10 0x7f22ec1fb260 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 >> 172.21.15.116:0/2605196649 conn(0x7f22ec0cee00 msgr2=0x7f22ec15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 shutdown_connections 2024-05-14T11:19:54.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.122+0000 7f22f1000640 1 -- 172.21.15.116:0/2605196649 wait complete. 2024-05-14T11:19:54.147 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:19:54.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 Processor -- start 2024-05-14T11:19:54.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 -- start start 2024-05-14T11:19:54.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 0x7f173806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:54.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1738143010 con 0x7f17380d0290 2024-05-14T11:19:54.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173815d360 con 0x7f1738070a50 2024-05-14T11:19:54.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f1740800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f173815dc90 con 0x7f1738070640 2024-05-14T11:19:54.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173e800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f17380d0290 0x7f1738142900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53638/0 (socket says 172.21.15.116:53638) 2024-05-14T11:19:54.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173e800640 1 -- 172.21.15.116:0/4205739120 learned_addr learned my addr 172.21.15.116:0/4205739120 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173de00640 1 --2- 172.21.15.116:0/4205739120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 0x7f173806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1574834795 0 0) 0x7f1738143010 con 0x7f17380d0290 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1720003530 con 0x7f17380d0290 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449861084 0 0) 0x7f173815d360 con 0x7f1738070a50 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1738143010 con 0x7f1738070a50 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173de00640 1 -- 172.21.15.116:0/4205739120 >> v1:172.21.15.149:6789/0 conn(0x7f1738070a50 legacy=0x7f17381421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173de00640 1 -- 172.21.15.116:0/4205739120 >> v1:172.21.15.116:6789/0 conn(0x7f17380d0290 legacy=0x7f1738142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2395762158 0 0) 0x7f1720003530 con 0x7f17380d0290 2024-05-14T11:19:54.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.298+0000 7f173de00640 1 -- 172.21.15.116:0/4205739120 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173815d640 con 0x7f1738070640 2024-05-14T11:19:54.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.302+0000 7f173de00640 1 --2- 172.21.15.116:0/4205739120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 0x7f173806e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097f0 tx=0x7f172c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625b5e49afaa8ae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.302+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0b0070 con 0x7f1738070640 2024-05-14T11:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.302+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c0ab400 con 0x7f1738070640 2024-05-14T11:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.302+0000 7f173ca00640 1 -- 172.21.15.116:0/4205739120 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734001a80 con 0x7f1738070640 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- 172.21.15.116:0/4205739120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 msgr2=0x7f173806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 --2- 172.21.15.116:0/4205739120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 0x7f173806e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097f0 tx=0x7f172c01c400 comp rx=0 tx=0).stop 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- 172.21.15.116:0/4205739120 shutdown_connections 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 --2- 172.21.15.116:0/4205739120 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1738070640 0x7f173806e870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- 172.21.15.116:0/4205739120 >> 172.21.15.116:0/4205739120 conn(0x7f17380cde80 msgr2=0x7f17380ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- 172.21.15.116:0/4205739120 shutdown_connections 2024-05-14T11:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- 172.21.15.116:0/4205739120 wait complete. 2024-05-14T11:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 Processor -- start 2024-05-14T11:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 -- start start 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1738070a50 0x7f1738149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.306+0000 7f1740800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 0x7f173815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17380cd980 con 0x7f1738070640 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17380cd800 con 0x7f1738070a50 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f173816b5d0 con 0x7f17380d0290 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60000/0 (socket says 172.21.15.116:60000) 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 -- 172.21.15.116:0/3369486783 learned_addr learned my addr 172.21.15.116:0/3369486783 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173e800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 0x7f173815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173d400640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1738070a50 0x7f1738149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 -- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 msgr2=0x7f173815bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 0x7f173815bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 -- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1738070a50 msgr2=0x7f1738149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1738070a50 0x7f1738149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 -- 172.21.15.116:0/3369486783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17381fd280 con 0x7f1738070640 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173e800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 0x7f173815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f173de00640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ae040 tx=0x7f172c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0b0040 con 0x7f1738070640 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c003620 con 0x7f1738070640 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173813d640 con 0x7f1738070640 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0039d0 con 0x7f1738070640 2024-05-14T11:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17381fd540 con 0x7f1738070640 2024-05-14T11:19:54.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173806aa30 con 0x7f1738070640 2024-05-14T11:19:54.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.310+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f172c0bd020 con 0x7f1738070640 2024-05-14T11:19:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.314+0000 7f1732000640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 0x7f171c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.314+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5865+0+0 (secure 0 0 0) 0x7f172c128b50 con 0x7f1738070640 2024-05-14T11:19:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.314+0000 7f173d400640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 0x7f171c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.314+0000 7f173d400640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 0x7f171c071910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f173815c6d0 tx=0x7f1728009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.318+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f172c015ff0 con 0x7f1738070640 2024-05-14T11:19:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:54.478+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f173806cd30 con 0x7f1738070640 2024-05-14T11:19:55.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1732000640 1 -- 172.21.15.116:0/3369486783 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) ==== 169+0+0 (secure 0 0 0) 0x7f172c0f5990 con 0x7f1738070640 2024-05-14T11:19:55.125 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:19:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 msgr2=0x7f171c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 0x7f171c071910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f173815c6d0 tx=0x7f1728009040 comp rx=0 tx=0).stop 2024-05-14T11:19:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 msgr2=0x7f173813d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ae040 tx=0x7f172c0191a0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 shutdown_connections 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f171c06f400 0x7f171c071910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1738070a50 0x7f1738149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17380d0290 0x7f173815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 --2- 172.21.15.116:0/3369486783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1738070640 0x7f173813d100 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 >> 172.21.15.116:0/3369486783 conn(0x7f17380cde80 msgr2=0x7f173813adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 shutdown_connections 2024-05-14T11:19:55.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.126+0000 7f1740800640 1 -- 172.21.15.116:0/3369486783 wait complete. 2024-05-14T11:19:55.152 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:19:55.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 Processor -- start 2024-05-14T11:19:55.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 -- start start 2024-05-14T11:19:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff04c143480 0x7ff04c143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff04c162640 con 0x7ff04c06cf90 2024-05-14T11:19:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff04c1621b0 con 0x7ff04c143480 2024-05-14T11:19:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff050a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c15f0b0 con 0x7ff04c06ff90 2024-05-14T11:19:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff049e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff049400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff04c143480 0x7ff04c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff049e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:52392/0 (socket says 172.21.15.116:52392) 2024-05-14T11:19:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff049e00640 1 -- 172.21.15.116:0/791653245 learned_addr learned my addr 172.21.15.116:0/791653245 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.306+0000 7ff04a800640 1 --1- 172.21.15.116:0/791653245 >> v1:172.21.15.149:6789/0 conn(0x7ff04c06cf90 0x7ff04c06d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54206/0 (socket says 172.21.15.116:54206) 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049400640 1 -- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 msgr2=0x7ff04c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049400640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049400640 1 -- 172.21.15.116:0/791653245 >> v1:172.21.15.149:6789/0 conn(0x7ff04c06cf90 legacy=0x7ff04c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff048a00640 1 -- 172.21.15.116:0/791653245 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247970626 0 0) 0x7ff04c162640 con 0x7ff04c06cf90 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049400640 1 -- 172.21.15.116:0/791653245 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c0d1210 con 0x7ff04c143480 2024-05-14T11:19:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049e00640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff049400640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c143890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0380097c0 tx=0x7ff03801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7af4ed9ae08d090 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff048a00640 1 -- 172.21.15.116:0/791653245 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0070 con 0x7ff04c143480 2024-05-14T11:19:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff048a00640 1 -- 172.21.15.116:0/791653245 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0380ab600 con 0x7ff04c143480 2024-05-14T11:19:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.310+0000 7ff048a00640 1 -- 172.21.15.116:0/791653245 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380ab9d0 con 0x7ff04c143480 2024-05-14T11:19:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 msgr2=0x7ff04c143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c143890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0380097c0 tx=0x7ff03801c440 comp rx=0 tx=0).stop 2024-05-14T11:19:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- 172.21.15.116:0/791653245 shutdown_connections 2024-05-14T11:19:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c143890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 --2- 172.21.15.116:0/791653245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06ff90 0x7ff04c0703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- 172.21.15.116:0/791653245 >> 172.21.15.116:0/791653245 conn(0x7ff04c071370 msgr2=0x7ff04c071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- 172.21.15.116:0/791653245 shutdown_connections 2024-05-14T11:19:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- 172.21.15.116:0/791653245 wait complete. 2024-05-14T11:19:55.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 Processor -- start 2024-05-14T11:19:55.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.314+0000 7ff050a00640 1 -- start start 2024-05-14T11:19:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06cf90 0x7ff04c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c0708e0 con 0x7ff04c06ff90 2024-05-14T11:19:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c140e60 con 0x7ff04c143480 2024-05-14T11:19:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c13c6a0 con 0x7ff04c06cf90 2024-05-14T11:19:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60002/0 (socket says 172.21.15.116:60002) 2024-05-14T11:19:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 -- 172.21.15.116:0/1024225079 learned_addr learned my addr 172.21.15.116:0/1024225079 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:55.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff04a800640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049e00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06cf90 0x7ff04c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 -- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06cf90 msgr2=0x7ff04c13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06cf90 0x7ff04c13e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 -- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 msgr2=0x7ff04c142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 -- 172.21.15.116:0/1024225079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c13ebe0 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff049400640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff0380ae040 tx=0x7ff038019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0040 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff038004020 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038006e20 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c13eec0 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c13ab60 con 0x7ff04c06ff90 2024-05-14T11:19:55.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04c06a9c0 con 0x7ff04c06ff90 2024-05-14T11:19:55.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.318+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff0380c0060 con 0x7ff04c06ff90 2024-05-14T11:19:55.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.322+0000 7ff042000640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.322+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7ff0381288a0 con 0x7ff04c06ff90 2024-05-14T11:19:55.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.322+0000 7ff049e00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.322+0000 7ff049e00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 0x7ff02c071910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0b20f0 tx=0x7ff03c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.326+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff038016090 con 0x7ff04c06ff90 2024-05-14T11:19:55.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.490+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff04c13e970 con 0x7ff04c06ff90 2024-05-14T11:19:55.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.494+0000 7ff042000640 1 -- 172.21.15.116:0/1024225079 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7ff0380f5900 con 0x7ff04c06ff90 2024-05-14T11:19:55.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 msgr2=0x7ff02c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 0x7ff02c071910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0b20f0 tx=0x7ff03c008040 comp rx=0 tx=0).stop 2024-05-14T11:19:55.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 msgr2=0x7ff04c1563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff0380ae040 tx=0x7ff038019240 comp rx=0 tx=0).stop 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 shutdown_connections 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff04c143480 0x7ff04c142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff04c06ff90 0x7ff04c1563a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 --2- 172.21.15.116:0/1024225079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff04c06cf90 0x7ff04c13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 >> 172.21.15.116:0/1024225079 conn(0x7ff04c071370 msgr2=0x7ff04c15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 shutdown_connections 2024-05-14T11:19:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.498+0000 7ff050a00640 1 -- 172.21.15.116:0/1024225079 wait complete. 2024-05-14T11:19:55.516 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-14T11:19:55.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.662+0000 7f7260800640 1 Processor -- start 2024-05-14T11:19:55.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.662+0000 7f7260800640 1 -- start start 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f7260800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f725806f730 0x7f725806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f7260800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7258072a30 0x7f7258072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f7260800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7258070080 con 0x7f72580b0770 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f7260800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f725806db20 con 0x7f725806f730 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f7260800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f725806cf10 con 0x7f7258072a30 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f725806f730 0x7f725806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f725806f730 0x7f725806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60004/0 (socket says 172.21.15.116:60004) 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f72580b0770 0x7f725806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53654/0 (socket says 172.21.15.116:53654) 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 -- 172.21.15.116:0/3591104178 learned_addr learned my addr 172.21.15.116:0/3591104178 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725e800640 1 --2- 172.21.15.116:0/3591104178 >> v2:172.21.15.149:3300/0 conn(0x7f7258072a30 0x7f7258072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 -- 172.21.15.116:0/3591104178 >> v2:172.21.15.149:3300/0 conn(0x7f7258072a30 msgr2=0x7f7258072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:55.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 --2- 172.21.15.116:0/3591104178 >> v2:172.21.15.149:3300/0 conn(0x7f7258072a30 0x7f7258072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 -- 172.21.15.116:0/3591104178 >> v1:172.21.15.116:6789/0 conn(0x7f72580b0770 legacy=0x7f725806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725ca00640 1 -- 172.21.15.116:0/3591104178 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858299875 0 0) 0x7f724c001120 con 0x7f72580b0770 2024-05-14T11:19:55.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.666+0000 7f725d400640 1 -- 172.21.15.116:0/3591104178 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725806d1f0 con 0x7f725806f730 2024-05-14T11:19:55.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f725d400640 1 --2- 172.21.15.116:0/3591104178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f725806fb40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f72580729c0 tx=0x7f724801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690db51ce1a5e563 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f725ca00640 1 -- 172.21.15.116:0/3591104178 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72480b4070 con 0x7f725806f730 2024-05-14T11:19:55.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f725ca00640 1 -- 172.21.15.116:0/3591104178 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72480a7b80 con 0x7f725806f730 2024-05-14T11:19:55.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f725ca00640 1 -- 172.21.15.116:0/3591104178 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72480af440 con 0x7f725806f730 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 -- 172.21.15.116:0/3591104178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 msgr2=0x7f725806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 --2- 172.21.15.116:0/3591104178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f725806fb40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f72580729c0 tx=0x7f724801eb30 comp rx=0 tx=0).stop 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 -- 172.21.15.116:0/3591104178 shutdown_connections 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 --2- 172.21.15.116:0/3591104178 >> v2:172.21.15.149:3300/0 conn(0x7f7258072a30 0x7f7258072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 --2- 172.21.15.116:0/3591104178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f725806fb40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 -- 172.21.15.116:0/3591104178 >> 172.21.15.116:0/3591104178 conn(0x7f72580b4620 msgr2=0x7f72580b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 -- 172.21.15.116:0/3591104178 shutdown_connections 2024-05-14T11:19:55.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.670+0000 7f7260800640 1 -- 172.21.15.116:0/3591104178 wait complete. 2024-05-14T11:19:55.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 Processor -- start 2024-05-14T11:19:55.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- start start 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 0x7f725815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 0x7f725815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7258073500 con 0x7f725806f730 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7258073380 con 0x7f72580b0770 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7258073680 con 0x7f7258072a30 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60020/0 (socket says 172.21.15.116:60020) 2024-05-14T11:19:55.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 -- 172.21.15.116:0/2404194106 learned_addr learned my addr 172.21.15.116:0/2404194106 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725e800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 0x7f725815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725d400640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 0x7f725815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 -- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 msgr2=0x7f725815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 0x7f725815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 -- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 msgr2=0x7f725815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 0x7f725815ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 -- 172.21.15.116:0/2404194106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7258158a70 con 0x7f725806f730 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725e800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 0x7f725815ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725d400640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 0x7f725815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f725de00640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c3c0 tx=0x7f724c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0074e0 con 0x7f725806f730 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c007680 con 0x7f725806f730 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7258157ef0 con 0x7f725806f730 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c004a30 con 0x7f725806f730 2024-05-14T11:19:55.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.674+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72581584b0 con 0x7f725806f730 2024-05-14T11:19:55.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725806a9c0 con 0x7f725806f730 2024-05-14T11:19:55.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f724c01a020 con 0x7f725806f730 2024-05-14T11:19:55.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f7252000640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 0x7f723c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:55.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f724c08e9a0 con 0x7f725806f730 2024-05-14T11:19:55.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f725d400640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 0x7f723c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:55.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.678+0000 7f725d400640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 0x7f723c071910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72480b1000 tx=0x7f72480a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:55.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.682+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7258158a70 con 0x7f725806f730 2024-05-14T11:19:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:55.902+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f72580b2b90 con 0x7f725806f730 2024-05-14T11:19:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.126+0000 7f7252000640 1 -- 172.21.15.116:0/2404194106 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v207) ==== 169+0+0 (secure 0 0 0) 0x7f7258070080 con 0x7f725806f730 2024-05-14T11:19:56.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 msgr2=0x7f723c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 0x7f723c071910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72480b1000 tx=0x7f72480a9070 comp rx=0 tx=0).stop 2024-05-14T11:19:56.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 msgr2=0x7f72581495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c3c0 tx=0x7f724c00c880 comp rx=0 tx=0).stop 2024-05-14T11:19:56.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 shutdown_connections 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f723c06f400 0x7f723c071910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72580b0770 0x7f725815ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7258072a30 0x7f725815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 --2- 172.21.15.116:0/2404194106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f725806f730 0x7f72581495f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 >> 172.21.15.116:0/2404194106 conn(0x7f72580b4620 msgr2=0x7f72580d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 shutdown_connections 2024-05-14T11:19:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.130+0000 7f7260800640 1 -- 172.21.15.116:0/2404194106 wait complete. 2024-05-14T11:19:56.149 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-14T11:19:56.149 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-14T11:19:56.149 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:19:56.150 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:19:56.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.294+0000 7f466ea00640 1 Processor -- start 2024-05-14T11:19:56.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.294+0000 7f466ea00640 1 -- start start 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f466ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46680d1670 0x7f46680d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f466ea00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46680d1fc0 con 0x7f4668156fa0 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f466ea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4668160510 con 0x7f46680af380 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f466ea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f46680706f0 con 0x7f46680d1670 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46680d1670 0x7f46680d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4668156fa0 0x7f466806ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53662/0 (socket says 172.21.15.116:53662) 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46680d1670 0x7f46680d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60030/0 (socket says 172.21.15.116:60030) 2024-05-14T11:19:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667e00640 1 -- 172.21.15.116:0/2929649741 learned_addr learned my addr 172.21.15.116:0/2929649741 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667400640 1 -- 172.21.15.116:0/2929649741 >> v1:172.21.15.149:6789/0 conn(0x7f46680af380 legacy=0x7f46680af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667400640 1 -- 172.21.15.116:0/2929649741 >> v1:172.21.15.116:6789/0 conn(0x7f4668156fa0 legacy=0x7f466806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.298+0000 7f4667400640 1 -- 172.21.15.116:0/2929649741 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46681607f0 con 0x7f46680d1670 2024-05-14T11:19:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f4667400640 1 --2- 172.21.15.116:0/2929649741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680d1670 0x7f46680d1a80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f46580097c0 tx=0x7f465801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f0aa331cab51b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f4666a00640 1 -- 172.21.15.116:0/2929649741 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580b0070 con 0x7f46680d1670 2024-05-14T11:19:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f4666a00640 1 -- 172.21.15.116:0/2929649741 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46580ab600 con 0x7f46680d1670 2024-05-14T11:19:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f4666a00640 1 -- 172.21.15.116:0/2929649741 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580ab9d0 con 0x7f46680d1670 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 -- 172.21.15.116:0/2929649741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680d1670 msgr2=0x7f46680d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 --2- 172.21.15.116:0/2929649741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680d1670 0x7f46680d1a80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f46580097c0 tx=0x7f465801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 -- 172.21.15.116:0/2929649741 shutdown_connections 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 --2- 172.21.15.116:0/2929649741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680d1670 0x7f46680d1a80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 -- 172.21.15.116:0/2929649741 >> 172.21.15.116:0/2929649741 conn(0x7f46680b4620 msgr2=0x7f46680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 -- 172.21.15.116:0/2929649741 shutdown_connections 2024-05-14T11:19:56.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.302+0000 7f466ea00640 1 -- 172.21.15.116:0/2929649741 wait complete. 2024-05-14T11:19:56.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 Processor -- start 2024-05-14T11:19:56.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- start start 2024-05-14T11:19:56.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46680d1670 0x7f46681f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 0x7f46681501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f466816abb0 con 0x7f46680af380 2024-05-14T11:19:56.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f466816aa30 con 0x7f46680d1670 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f466816ad30 con 0x7f4668156fa0 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60040/0 (socket says 172.21.15.116:60040) 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 -- 172.21.15.116:0/810064110 learned_addr learned my addr 172.21.15.116:0/810064110 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667400640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46680d1670 0x7f46681f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ca00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 0x7f46681501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 -- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 msgr2=0x7f46681501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 0x7f46681501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 -- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46680d1670 msgr2=0x7f46681f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46680d1670 0x7f46681f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 -- 172.21.15.116:0/810064110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4668156070 con 0x7f46680af380 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ca00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 0x7f46681501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f4667e00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f465000c430 tx=0x7f465000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650017070 con 0x7f46680af380 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4650005d10 con 0x7f46680af380 2024-05-14T11:19:56.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4668156350 con 0x7f46680af380 2024-05-14T11:19:56.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650007310 con 0x7f46680af380 2024-05-14T11:19:56.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.306+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4668074680 con 0x7f46680af380 2024-05-14T11:19:56.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4668156870 con 0x7f46680af380 2024-05-14T11:19:56.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f465001a070 con 0x7f46680af380 2024-05-14T11:19:56.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f465fe00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 0x7f4644071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f465008e4e0 con 0x7f46680af380 2024-05-14T11:19:56.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f4667400640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 0x7f4644071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.310+0000 7f4667400640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 0x7f4644071940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f46580ad000 tx=0x7f46580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.314+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f46680d1fc0 con 0x7f46680af380 2024-05-14T11:19:56.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.474+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f466806a9c0 con 0x7f46680af380 2024-05-14T11:19:56.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.474+0000 7f465fe00640 1 -- 172.21.15.116:0/810064110 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+13344 (secure 0 0 0) 0x7f465005b540 con 0x7f46680af380 2024-05-14T11:19:56.476 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:56.476 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":64,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:19:55.122623+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6836/1576287145":"2024-05-15T11:19:52.116116+0000","172.21.15.116:6834/1576287145":"2024-05-15T11:19:52.116116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 msgr2=0x7f4644071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 0x7f4644071940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f46580ad000 tx=0x7f46580a6000 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 msgr2=0x7f4668166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f465000c430 tx=0x7f465000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 shutdown_connections 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f464406f430 0x7f4644071940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46680d1670 0x7f46681f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4668156fa0 0x7f46681501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 --2- 172.21.15.116:0/810064110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46680af380 0x7f4668166970 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 >> 172.21.15.116:0/810064110 conn(0x7f46680b4620 msgr2=0x7f46680b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:56.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 shutdown_connections 2024-05-14T11:19:56.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.478+0000 7f466ea00640 1 -- 172.21.15.116:0/810064110 wait complete. 2024-05-14T11:19:56.497 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:19:56.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:19:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.646+0000 7f240e000640 1 Processor -- start 2024-05-14T11:19:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.646+0000 7f240e000640 1 -- start start 2024-05-14T11:19:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f240e000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f240806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f240e000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24080d1730 0x7f24080d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f240e000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240806f860 con 0x7f24080cce60 2024-05-14T11:19:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f240e000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f240806db80 con 0x7f24080d1730 2024-05-14T11:19:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f240e000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f24080d2080 con 0x7f240806f450 2024-05-14T11:19:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24080d1730 0x7f24080d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f2407e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f24080cce60 0x7f24080cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53672/0 (socket says 172.21.15.116:53672) 2024-05-14T11:19:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f2407e00640 1 -- 172.21.15.116:0/3440683673 learned_addr learned my addr 172.21.15.116:0/3440683673 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24080d1730 0x7f24080d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60046/0 (socket says 172.21.15.116:60046) 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f2407400640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f240806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 -- 172.21.15.116:0/3440683673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 msgr2=0x7f240806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f240806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 -- 172.21.15.116:0/3440683673 >> v1:172.21.15.116:6789/0 conn(0x7f24080cce60 legacy=0x7f24080cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f23fea00640 1 -- 172.21.15.116:0/3440683673 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240806fae0 con 0x7f24080d1730 2024-05-14T11:19:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.650+0000 7f2406a00640 1 -- 172.21.15.116:0/3440683673 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664591710 0 0) 0x7f24000014a0 con 0x7f24080cce60 2024-05-14T11:19:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f23fea00640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f24080d1b40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23f00097c0 tx=0x7f23f001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d131dd3109377 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f2406a00640 1 -- 172.21.15.116:0/3440683673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00b2070 con 0x7f24080d1730 2024-05-14T11:19:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f2406a00640 1 -- 172.21.15.116:0/3440683673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f00ad640 con 0x7f24080d1730 2024-05-14T11:19:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f2406a00640 1 -- 172.21.15.116:0/3440683673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080ce190 con 0x7f24080d1730 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 -- 172.21.15.116:0/3440683673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 msgr2=0x7f24080d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f24080d1b40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23f00097c0 tx=0x7f23f001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 -- 172.21.15.116:0/3440683673 shutdown_connections 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f24080d1b40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 --2- 172.21.15.116:0/3440683673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f240806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 -- 172.21.15.116:0/3440683673 >> 172.21.15.116:0/3440683673 conn(0x7f24080b4720 msgr2=0x7f24080b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:56.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 -- 172.21.15.116:0/3440683673 shutdown_connections 2024-05-14T11:19:56.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.654+0000 7f240e000640 1 -- 172.21.15.116:0/3440683673 wait complete. 2024-05-14T11:19:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 Processor -- start 2024-05-14T11:19:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- start start 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f24081fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 0x7f24080c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f240816a610 con 0x7f24080d1730 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2408070660 con 0x7f24080cce60 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f240816a790 con 0x7f240806f450 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60054/0 (socket says 172.21.15.116:60054) 2024-05-14T11:19:56.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 -- 172.21.15.116:0/469590674 learned_addr learned my addr 172.21.15.116:0/469590674 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f23fea00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 0x7f24080c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407400640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f24081fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 -- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 msgr2=0x7f24081fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f24081fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 -- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 msgr2=0x7f24080c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 0x7f24080c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 -- 172.21.15.116:0/469590674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24080c7590 con 0x7f24080d1730 2024-05-14T11:19:56.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f23fea00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 0x7f24080c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407400640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f24081fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f2407e00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f240000c6b0 tx=0x7f240000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400018070 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2400004db0 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24081fc7a0 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240000a880 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.658+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24080c7870 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240806a9c0 con 0x7f24080d1730 2024-05-14T11:19:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f240001b070 con 0x7f24080d1730 2024-05-14T11:19:56.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f23ffe00640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 0x7f23e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f240008e870 con 0x7f24080d1730 2024-05-14T11:19:56.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f2407400640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 0x7f23e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.662+0000 7f2407400640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 0x7f23e4071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24080c83b0 tx=0x7f23f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.666+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f240806f860 con 0x7f24080d1730 2024-05-14T11:19:56.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2408069de0 con 0x7f24080d1730 2024-05-14T11:19:56.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f23ffe00640 1 -- 172.21.15.116:0/469590674 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7f240005b830 con 0x7f24080d1730 2024-05-14T11:19:56.825 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 msgr2=0x7f23e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 0x7f23e4071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24080c83b0 tx=0x7f23f8008040 comp rx=0 tx=0).stop 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 msgr2=0x7f2408166ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f240000c6b0 tx=0x7f240000cb70 comp rx=0 tx=0).stop 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 shutdown_connections 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f23e406f400 0x7f23e4071910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24080cce60 0x7f24080c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24080d1730 0x7f2408166ba0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 --2- 172.21.15.116:0/469590674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f240806f450 0x7f24081fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:56.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 >> 172.21.15.116:0/469590674 conn(0x7f24080b4720 msgr2=0x7f24080ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:56.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 shutdown_connections 2024-05-14T11:19:56.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.826+0000 7f240e000640 1 -- 172.21.15.116:0/469590674 wait complete. 2024-05-14T11:19:56.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-14T11:19:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 Processor -- start 2024-05-14T11:19:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 -- start start 2024-05-14T11:19:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b900ae9f0 0x7f6b900aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b901600a0 con 0x7f6b90164220 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9006f5d0 con 0x7f6b900d1d80 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b95200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9006f8b0 con 0x7f6b900ae9f0 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b8e800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b900ae9f0 0x7f6b900aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b8de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6b90164220 0x7f6b900d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53686/0 (socket says 172.21.15.116:53686) 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.994+0000 7f6b8de00640 1 -- 172.21.15.116:0/2501280187 learned_addr learned my addr 172.21.15.116:0/2501280187 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8e800640 1 -- 172.21.15.116:0/2501280187 >> v1:172.21.15.149:6789/0 conn(0x7f6b900d1d80 legacy=0x7f6b900ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8e800640 1 -- 172.21.15.116:0/2501280187 >> v1:172.21.15.116:6789/0 conn(0x7f6b90164220 legacy=0x7f6b900d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8e800640 1 -- 172.21.15.116:0/2501280187 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9006fb90 con 0x7f6b900ae9f0 2024-05-14T11:19:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8d400640 1 -- 172.21.15.116:0/2501280187 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8316622 0 0) 0x7f6b901600a0 con 0x7f6b90164220 2024-05-14T11:19:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8e800640 1 --2- 172.21.15.116:0/2501280187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b900aee00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009d90 tx=0x7f6b8801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d4db75b9d74ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8d400640 1 -- 172.21.15.116:0/2501280187 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880b2070 con 0x7f6b900ae9f0 2024-05-14T11:19:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8d400640 1 -- 172.21.15.116:0/2501280187 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b880a5970 con 0x7f6b900ae9f0 2024-05-14T11:19:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:56.998+0000 7f6b8d400640 1 -- 172.21.15.116:0/2501280187 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880a5d40 con 0x7f6b900ae9f0 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- 172.21.15.116:0/2501280187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 msgr2=0x7f6b900aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 --2- 172.21.15.116:0/2501280187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b900aee00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009d90 tx=0x7f6b8801c8b0 comp rx=0 tx=0).stop 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- 172.21.15.116:0/2501280187 shutdown_connections 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 --2- 172.21.15.116:0/2501280187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b900aee00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- 172.21.15.116:0/2501280187 >> 172.21.15.116:0/2501280187 conn(0x7f6b900b4620 msgr2=0x7f6b900b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- 172.21.15.116:0/2501280187 shutdown_connections 2024-05-14T11:19:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- 172.21.15.116:0/2501280187 wait complete. 2024-05-14T11:19:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 Processor -- start 2024-05-14T11:19:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.002+0000 7f6b95200640 1 -- start start 2024-05-14T11:19:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 0x7f6b900af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6b90162e00 0x7f6b900afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90164630 con 0x7f6b900ae9f0 2024-05-14T11:19:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90073650 con 0x7f6b90164220 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6b901647b0 con 0x7f6b90162e00 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60070/0 (socket says 172.21.15.116:60070) 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 -- 172.21.15.116:0/4133524444 learned_addr learned my addr 172.21.15.116:0/4133524444 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b85400640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 0x7f6b900af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8e800640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6b90162e00 0x7f6b900afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 -- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6b90162e00 msgr2=0x7f6b900afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6b90162e00 0x7f6b900afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 -- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 msgr2=0x7f6b900af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 0x7f6b900af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 -- 172.21.15.116:0/4133524444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b90161d60 con 0x7f6b900ae9f0 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b85400640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 0x7f6b900af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b8de00640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800c3c0 tx=0x7f6b7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b780074e0 con 0x7f6b900ae9f0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b780047f0 con 0x7f6b900ae9f0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b90162040 con 0x7f6b900ae9f0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78007680 con 0x7f6b900ae9f0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b90162600 con 0x7f6b900ae9f0 2024-05-14T11:19:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9006a9c0 con 0x7f6b900ae9f0 2024-05-14T11:19:57.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.006+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6b78011070 con 0x7f6b900ae9f0 2024-05-14T11:19:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.010+0000 7f6b86a00640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 0x7f6b6c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.010+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f6b7808ece0 con 0x7f6b900ae9f0 2024-05-14T11:19:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.010+0000 7f6b85400640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 0x7f6b6c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.010+0000 7f6b85400640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 0x7f6b6c071940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b90154140 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:57.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.014+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6b7805bca0 con 0x7f6b900ae9f0 2024-05-14T11:19:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:57.230+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f6b9006f310 con 0x7f6b900ae9f0 2024-05-14T11:19:58.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.134+0000 7f6b86a00640 1 -- 172.21.15.116:0/4133524444 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v209) ==== 169+0+0 (secure 0 0 0) 0x7f6b7805b5a0 con 0x7f6b900ae9f0 2024-05-14T11:19:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 msgr2=0x7f6b6c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 0x7f6b6c071940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b90154140 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2024-05-14T11:19:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 msgr2=0x7f6b90149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800c3c0 tx=0x7f6b7800c880 comp rx=0 tx=0).stop 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 shutdown_connections 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6b6c06f430 0x7f6b6c071940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b90164220 0x7f6b900af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6b90162e00 0x7f6b900afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 --2- 172.21.15.116:0/4133524444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b900ae9f0 0x7f6b90149bb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 >> 172.21.15.116:0/4133524444 conn(0x7f6b900b4620 msgr2=0x7f6b90153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 shutdown_connections 2024-05-14T11:19:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.138+0000 7f6b95200640 1 -- 172.21.15.116:0/4133524444 wait complete. 2024-05-14T11:19:58.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 Processor -- start 2024-05-14T11:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 -- start start 2024-05-14T11:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f56bc0cd1c0 0x7f56bc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56bc06f0a0 con 0x7f56bc0d1730 2024-05-14T11:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f56bc0cbb60 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.314+0000 7f56c1200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc06f420 con 0x7f56bc06c700 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56ba800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56ba800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:49158/0 (socket says 172.21.15.116:49158) 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56ba800640 1 -- 172.21.15.116:0/534528134 learned_addr learned my addr 172.21.15.116:0/534528134 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b9e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f56bc0cd1c0 0x7f56bc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b8a00640 1 -- 172.21.15.116:0/534528134 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253875430 0 0) 0x7f56bc06f0a0 con 0x7f56bc0d1730 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b8a00640 1 -- 172.21.15.116:0/534528134 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56a0003530 con 0x7f56bc0d1730 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b9e00640 1 -- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 msgr2=0x7f56bc06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b9e00640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b9e00640 1 -- 172.21.15.116:0/534528134 >> v1:172.21.15.149:6789/0 conn(0x7f56bc0d1730 legacy=0x7f56bc0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56b9e00640 1 -- 172.21.15.116:0/534528134 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc06f700 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.318+0000 7f56ba800640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56b9e00640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0cd5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009d90 tx=0x7f56ac01c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52bb94dee76880ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56b8a00640 1 -- 172.21.15.116:0/534528134 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b2070 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56b8a00640 1 -- 172.21.15.116:0/534528134 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac0a5850 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56b8a00640 1 -- 172.21.15.116:0/534528134 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0a5c20 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 msgr2=0x7f56bc0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0cd5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009d90 tx=0x7f56ac01c790 comp rx=0 tx=0).stop 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- 172.21.15.116:0/534528134 shutdown_connections 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc06c700 0x7f56bc06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 --2- 172.21.15.116:0/534528134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0cd5d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- 172.21.15.116:0/534528134 >> 172.21.15.116:0/534528134 conn(0x7f56bc0b4720 msgr2=0x7f56bc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- 172.21.15.116:0/534528134 shutdown_connections 2024-05-14T11:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- 172.21.15.116:0/534528134 wait complete. 2024-05-14T11:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 Processor -- start 2024-05-14T11:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.322+0000 7f56c1200640 1 -- start start 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc0d1730 0x7f56bc0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc0cddb0 con 0x7f56bc06c700 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc15cf10 con 0x7f56bc0cd1c0 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc0cdf30 con 0x7f56bc0d1730 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58314/0 (socket says 172.21.15.116:58314) 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 -- 172.21.15.116:0/863651898 learned_addr learned my addr 172.21.15.116:0/863651898 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9400640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56ba800640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc0d1730 0x7f56bc0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 -- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc0d1730 msgr2=0x7f56bc0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc0d1730 0x7f56bc0c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 -- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 msgr2=0x7f56bc0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 -- 172.21.15.116:0/863651898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc15a240 con 0x7f56bc06c700 2024-05-14T11:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b9e00640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f56ac01a3c0 tx=0x7f56ac019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b2040 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac0bf070 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b8690 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc15f1a0 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc15f760 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.326+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc15a410 con 0x7f56bc06c700 2024-05-14T11:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.330+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56ac0b8830 con 0x7f56bc06c700 2024-05-14T11:19:58.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.330+0000 7f56b2000640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 0x7f569c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.330+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f56ac129e40 con 0x7f56bc06c700 2024-05-14T11:19:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.330+0000 7f56b9400640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 0x7f569c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.330+0000 7f56b9400640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 0x7f569c071910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ca9f0 tx=0x7f56a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.334+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f56bc06d860 con 0x7f56bc06c700 2024-05-14T11:19:58.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.494+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f56bc06a9c0 con 0x7f56bc06c700 2024-05-14T11:19:58.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56b2000640 1 -- 172.21.15.116:0/863651898 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f56ac0f6840 con 0x7f56bc06c700 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 msgr2=0x7f569c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 0x7f569c071910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ca9f0 tx=0x7f56a8009040 comp rx=0 tx=0).stop 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 msgr2=0x7f56bc154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f56ac01a3c0 tx=0x7f56ac019530 comp rx=0 tx=0).stop 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 shutdown_connections 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f569c06f400 0x7f569c071910 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56bc0cd1c0 0x7f56bc0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56bc0d1730 0x7f56bc0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 --2- 172.21.15.116:0/863651898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56bc06c700 0x7f56bc154d10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 >> 172.21.15.116:0/863651898 conn(0x7f56bc0b4720 msgr2=0x7f56bc073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:58.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 shutdown_connections 2024-05-14T11:19:58.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.498+0000 7f56c1200640 1 -- 172.21.15.116:0/863651898 wait complete. 2024-05-14T11:19:58.519 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:19:58.519 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:19:58.606 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:19:58.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:19:58.657 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:19:58.657 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:58.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:19:58.666 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:19:58.666 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:58.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:19:58.676 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:19:58.676 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:58.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:19:58.719 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:19:58.719 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:19:58.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:19:58.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:19:58.728 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:19:58.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 Processor -- start 2024-05-14T11:19:58.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 -- start start 2024-05-14T11:19:58.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 0x7f01f4161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01f40cd400 0x7f01f40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f40c94a0 con 0x7f01f406dc50 2024-05-14T11:19:58.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f01f40ccc90 con 0x7f01f40cd400 2024-05-14T11:19:58.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01f4161d60 con 0x7f01f415dc10 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01ea000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01f40cd400 0x7f01f40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01ea000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01f40cd400 0x7f01f40cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58324/0 (socket says 172.21.15.116:58324) 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01ea000640 1 -- 172.21.15.116:0/1260697213 learned_addr learned my addr 172.21.15.116:0/1260697213 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.910+0000 7f01f2a00640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 0x7f01f4161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01ea000640 1 -- 172.21.15.116:0/1260697213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 msgr2=0x7f01f4161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01ea000640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 0x7f01f4161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01ea000640 1 -- 172.21.15.116:0/1260697213 >> v1:172.21.15.149:6789/0 conn(0x7f01f406dc50 legacy=0x7f01f406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01ea000640 1 -- 172.21.15.116:0/1260697213 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f4161fe0 con 0x7f01f40cd400 2024-05-14T11:19:58.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f2a00640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 0x7f01f4161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:58.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01ea000640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f40cd400 0x7f01f40cd810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009870 tx=0x7f01dc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d551d18e8821ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f2000640 1 -- 172.21.15.116:0/1260697213 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0ab5f0 con 0x7f01f40cd400 2024-05-14T11:19:58.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f2000640 1 -- 172.21.15.116:0/1260697213 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc0b0070 con 0x7f01f40cd400 2024-05-14T11:19:58.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f2000640 1 -- 172.21.15.116:0/1260697213 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0ab9b0 con 0x7f01f40cd400 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f9600640 1 -- 172.21.15.116:0/1260697213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f40cd400 msgr2=0x7f01f40cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.914+0000 7f01f9600640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f40cd400 0x7f01f40cd810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009870 tx=0x7f01dc01c500 comp rx=0 tx=0).stop 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- 172.21.15.116:0/1260697213 shutdown_connections 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f40cd400 0x7f01f40cd810 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 --2- 172.21.15.116:0/1260697213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f415dc10 0x7f01f4161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- 172.21.15.116:0/1260697213 >> 172.21.15.116:0/1260697213 conn(0x7f01f4073ca0 msgr2=0x7f01f40740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- 172.21.15.116:0/1260697213 shutdown_connections 2024-05-14T11:19:58.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- 172.21.15.116:0/1260697213 wait complete. 2024-05-14T11:19:58.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 Processor -- start 2024-05-14T11:19:58.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- start start 2024-05-14T11:19:58.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 0x7f01f4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f40cd400 0x7f01f41fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01f416abc0 con 0x7f01f415dc10 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01f415e020 con 0x7f01f406dc50 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.918+0000 7f01f9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01f416ad40 con 0x7f01f40cd400 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58328/0 (socket says 172.21.15.116:58328) 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 -- 172.21.15.116:0/400779819 learned_addr learned my addr 172.21.15.116:0/400779819 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01ea000640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f40cd400 0x7f01f41fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f2a00640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 0x7f01f4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 -- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f40cd400 msgr2=0x7f01f41fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f40cd400 0x7f01f41fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 -- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 msgr2=0x7f01f4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 0x7f01f4156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 -- 172.21.15.116:0/400779819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f4157320 con 0x7f01f415dc10 2024-05-14T11:19:58.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f2a00640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 0x7f01f4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:19:58.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f3400640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c740 tx=0x7f01ec00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003b80 con 0x7f01f415dc10 2024-05-14T11:19:58.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec00ce50 con 0x7f01f415dc10 2024-05-14T11:19:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00a850 con 0x7f01f415dc10 2024-05-14T11:19:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f415ac60 con 0x7f01f415dc10 2024-05-14T11:19:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f4160fb0 con 0x7f01f415dc10 2024-05-14T11:19:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f415a9d0 con 0x7f01f415dc10 2024-05-14T11:19:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.922+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01ec00a9f0 con 0x7f01f415dc10 2024-05-14T11:19:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.926+0000 7f01eb400640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 0x7f01d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.926+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f01ec08ee60 con 0x7f01f415dc10 2024-05-14T11:19:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.926+0000 7f01f2a00640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 0x7f01d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.926+0000 7f01f2a00640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 0x7f01d0071910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f01e400b990 tx=0x7f01e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:58.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:58.930+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f01ec05bfc0 con 0x7f01f415dc10 2024-05-14T11:19:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.118+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f406a9c0 con 0x7f01f415dc10 2024-05-14T11:19:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.118+0000 7f01eb400640 1 -- 172.21.15.116:0/400779819 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3878 (secure 0 0 0) 0x7f01ec05b8c0 con 0x7f01f415dc10 2024-05-14T11:19:59.120 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:19:59.120 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/2866351604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":2866351604},{"type":"v1","addr":"172.21.15.116:6837","nonce":2866351604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/552692741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":552692741},{"type":"v1","addr":"172.21.15.149:6834","nonce":552692741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1834446777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1834446777},{"type":"v1","addr":"172.21.15.149:6835","nonce":1834446777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/1991309500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1991309500},{"type":"v1","addr":"172.21.15.116:6836","nonce":1991309500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-05-14T11:19:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 38 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 msgr2=0x7f01d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 0x7f01d0071910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f01e400b990 tx=0x7f01e4008040 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 msgr2=0x7f01f41fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c740 tx=0x7f01ec00cc00 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 shutdown_connections 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01d006f400 0x7f01d0071910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f415dc10 0x7f01f41fcef0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01f40cd400 0x7f01f41fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 --2- 172.21.15.116:0/400779819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01f406dc50 0x7f01f4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 >> 172.21.15.116:0/400779819 conn(0x7f01f4073ca0 msgr2=0x7f01f40706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 shutdown_connections 2024-05-14T11:19:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.122+0000 7f01f9600640 1 -- 172.21.15.116:0/400779819 wait complete. 2024-05-14T11:19:59.142 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:19:59.142 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:19:59.142 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:19:59.142 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:19:59.142 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:19:59.142 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:19:59.142 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:19:59.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:19:59.143 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:19:59.143 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:19:59.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:19:59.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:19:59.143 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:19:59.143 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:19:59.143 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:19:59.144 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:19:59.157 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:19:59.157 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:19:59.160 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:19:59.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:19:59.161 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:19:59.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:19:59.162 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:19:59.163 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:19:59.208 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:19:59.208 DEBUG:teuthology.parallel:result is None 2024-05-14T11:19:59.209 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:19:59.209 DEBUG:teuthology.parallel:result is None 2024-05-14T11:19:59.210 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:19:59.210 DEBUG:teuthology.parallel:result is None 2024-05-14T11:19:59.212 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:19:59.213 DEBUG:teuthology.parallel:result is None 2024-05-14T11:19:59.213 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:19:59.312 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:19:59.312 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:19:59.313 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:19:59.315 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:19:59.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.370+0000 7f2f43200640 1 Processor -- start 2024-05-14T11:19:59.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.370+0000 7f2f43200640 1 -- start start 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f43200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2f3c0d1670 0x7f2f3c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f43200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 0x7f2f3c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f43200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3c0d1fc0 con 0x7f2f3c156fa0 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f43200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3c0706f0 con 0x7f2f3c0d1670 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f43200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3c1573b0 con 0x7f2f3c0af380 2024-05-14T11:19:59.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2f3c0d1670 0x7f2f3c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f3be00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2f3c156fa0 0x7f2f3c06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59160/0 (socket says 172.21.15.116:59160) 2024-05-14T11:19:59.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f3be00640 1 -- 172.21.15.116:0/1432918376 learned_addr learned my addr 172.21.15.116:0/1432918376 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:19:59.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f41200640 1 --2- 172.21.15.116:0/1432918376 >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 0x7f2f3c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 -- 172.21.15.116:0/1432918376 >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 msgr2=0x7f2f3c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:19:59.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 --2- 172.21.15.116:0/1432918376 >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 0x7f2f3c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 -- 172.21.15.116:0/1432918376 >> v1:172.21.15.116:6789/0 conn(0x7f2f3c156fa0 legacy=0x7f2f3c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 -- 172.21.15.116:0/1432918376 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c160510 con 0x7f2f3c0d1670 2024-05-14T11:19:59.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f41200640 1 --2- 172.21.15.116:0/1432918376 >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 0x7f2f3c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:19:59.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.374+0000 7f2f33e00640 1 --2- 172.21.15.116:0/1432918376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c0d1670 0x7f2f3c0d1a80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c074e70 tx=0x7f2f2401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c6c8526bf8dde9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:59.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f3b400640 1 -- 172.21.15.116:0/1432918376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f240b0070 con 0x7f2f3c0d1670 2024-05-14T11:19:59.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f3b400640 1 -- 172.21.15.116:0/1432918376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f240ab600 con 0x7f2f3c0d1670 2024-05-14T11:19:59.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f3b400640 1 -- 172.21.15.116:0/1432918376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f240ab9d0 con 0x7f2f3c0d1670 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- 172.21.15.116:0/1432918376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c0d1670 msgr2=0x7f2f3c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 --2- 172.21.15.116:0/1432918376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c0d1670 0x7f2f3c0d1a80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c074e70 tx=0x7f2f2401c480 comp rx=0 tx=0).stop 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- 172.21.15.116:0/1432918376 shutdown_connections 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 --2- 172.21.15.116:0/1432918376 >> v2:172.21.15.149:3300/0 conn(0x7f2f3c0af380 0x7f2f3c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 --2- 172.21.15.116:0/1432918376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c0d1670 0x7f2f3c0d1a80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- 172.21.15.116:0/1432918376 >> 172.21.15.116:0/1432918376 conn(0x7f2f3c0b4620 msgr2=0x7f2f3c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- 172.21.15.116:0/1432918376 shutdown_connections 2024-05-14T11:19:59.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- 172.21.15.116:0/1432918376 wait complete. 2024-05-14T11:19:59.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 Processor -- start 2024-05-14T11:19:59.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.378+0000 7f2f43200640 1 -- start start 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 0x7f2f3c074210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f3c156fa0 0x7f2f3c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c0b0660 con 0x7f2f3c14d620 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c0b04e0 con 0x7f2f3c156fa0 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c169810 con 0x7f2f3c0af380 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f41200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 0x7f2f3c074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f3c156fa0 0x7f2f3c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f41200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58388/0 (socket says 172.21.15.116:58388) 2024-05-14T11:19:59.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f41200640 1 -- 172.21.15.116:0/3480234648 learned_addr learned my addr 172.21.15.116:0/3480234648 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 -- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f3c156fa0 msgr2=0x7f2f3c09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f3c156fa0 0x7f2f3c09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 -- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 msgr2=0x7f2f3c0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 -- 172.21.15.116:0/3480234648 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c0ceac0 con 0x7f2f3c0af380 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f41200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f3be00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 0x7f2f3c074210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c3c0 tx=0x7f2f2c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c017070 con 0x7f2f3c0af380 2024-05-14T11:19:59.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c005d10 con 0x7f2f3c0af380 2024-05-14T11:19:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c074a40 con 0x7f2f3c0af380 2024-05-14T11:19:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c007310 con 0x7f2f3c0af380 2024-05-14T11:19:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c0cdd80 con 0x7f2f3c0af380 2024-05-14T11:19:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.382+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c06a9c0 con 0x7f2f3c0af380 2024-05-14T11:19:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.386+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f2c01a070 con 0x7f2f3c0af380 2024-05-14T11:19:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.386+0000 7f2f38c00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 0x7f2f18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:19:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.386+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 5335+0+0 (secure 0 0 0) 0x7f2f2c08e1e0 con 0x7f2f3c0af380 2024-05-14T11:19:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.386+0000 7f2f33e00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 0x7f2f18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:19:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.386+0000 7f2f33e00640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 0x7f2f18071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f240ad000 tx=0x7f2f24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:19:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.390+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2f3c0d1fc0 con 0x7f2f3c0af380 2024-05-14T11:19:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:19:59.558+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2f3c0d1050 con 0x7f2f3c0af380 2024-05-14T11:20:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.158+0000 7f2f38c00640 1 -- 172.21.15.116:0/3480234648 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) ==== 97+0+0 (secure 0 0 0) 0x7f2f2c05ad10 con 0x7f2f3c0af380 2024-05-14T11:20:00.159 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:20:00.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 msgr2=0x7f2f18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 0x7f2f18071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f240ad000 tx=0x7f2f24008040 comp rx=0 tx=0).stop 2024-05-14T11:20:00.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 msgr2=0x7f2f3c074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 0x7f2f3c074210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c3c0 tx=0x7f2f2c00c880 comp rx=0 tx=0).stop 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 shutdown_connections 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f1806f430 0x7f2f18071940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f3c156fa0 0x7f2f3c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f3c14d620 0x7f2f3c0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 --2- 172.21.15.116:0/3480234648 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2f3c0af380 0x7f2f3c074210 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 >> 172.21.15.116:0/3480234648 conn(0x7f2f3c0b4620 msgr2=0x7f2f3c1506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 shutdown_connections 2024-05-14T11:20:00.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.162+0000 7f2f43200640 1 -- 172.21.15.116:0/3480234648 wait complete. 2024-05-14T11:20:00.184 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:20:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.330+0000 7fe68e400640 1 Processor -- start 2024-05-14T11:20:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.330+0000 7fe68e400640 1 -- start start 2024-05-14T11:20:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68e400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880ccfa0 0x7fe6880ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68e400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe6880b8d80 0x7fe6880d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68e400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6880b6600 con 0x7fe6880cd490 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68e400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe6880b6b30 con 0x7fe6880b8d80 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68e400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe6880b6d80 con 0x7fe6880ccfa0 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe6880b8d80 0x7fe6880d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe6880b8d80 0x7fe6880d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58394/0 (socket says 172.21.15.116:58394) 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 -- 172.21.15.116:0/433441512 learned_addr learned my addr 172.21.15.116:0/433441512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe687e00640 1 --1- 172.21.15.116:0/433441512 >> v1:172.21.15.149:6789/0 conn(0x7fe6880cd490 0x7fe6880b8670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:44716/0 (socket says 172.21.15.116:44716) 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68d000640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880ccfa0 0x7fe6880ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 -- 172.21.15.116:0/433441512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880ccfa0 msgr2=0x7fe6880ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880ccfa0 0x7fe6880ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 -- 172.21.15.116:0/433441512 >> v1:172.21.15.149:6789/0 conn(0x7fe6880cd490 legacy=0x7fe6880b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe68da00640 1 -- 172.21.15.116:0/433441512 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6880b7060 con 0x7fe6880b8d80 2024-05-14T11:20:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.334+0000 7fe687400640 1 -- 172.21.15.116:0/433441512 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719226410 0 0) 0x7fe6880b6600 con 0x7fe6880cd490 2024-05-14T11:20:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68da00640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880b8d80 0x7fe6880d0fa0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe680009e50 tx=0x7fe68001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d60c7f4c45e9fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe687400640 1 -- 172.21.15.116:0/433441512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a5720 con 0x7fe6880b8d80 2024-05-14T11:20:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe687400640 1 -- 172.21.15.116:0/433441512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6800a58c0 con 0x7fe6880b8d80 2024-05-14T11:20:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe687400640 1 -- 172.21.15.116:0/433441512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a5c90 con 0x7fe6880b8d80 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 -- 172.21.15.116:0/433441512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880b8d80 msgr2=0x7fe6880d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880b8d80 0x7fe6880d0fa0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe680009e50 tx=0x7fe68001ca50 comp rx=0 tx=0).stop 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 -- 172.21.15.116:0/433441512 shutdown_connections 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880b8d80 0x7fe6880d0fa0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 --2- 172.21.15.116:0/433441512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880ccfa0 0x7fe6880ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 -- 172.21.15.116:0/433441512 >> 172.21.15.116:0/433441512 conn(0x7fe6880b56e0 msgr2=0x7fe6880d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 -- 172.21.15.116:0/433441512 shutdown_connections 2024-05-14T11:20:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.338+0000 7fe68e400640 1 -- 172.21.15.116:0/433441512 wait complete. 2024-05-14T11:20:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 Processor -- start 2024-05-14T11:20:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- start start 2024-05-14T11:20:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880b8d80 0x7fe688162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 0x7fe68815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe68816cca0 con 0x7fe6880cd490 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe68816cb20 con 0x7fe6880ccfa0 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe68816ce20 con 0x7fe6880b8d80 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe687e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 0x7fe68815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880b8d80 0x7fe688162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe687e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 0x7fe68815b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:50812/0 (socket says 172.21.15.116:50812) 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe687e00640 1 -- 172.21.15.116:0/1752887223 learned_addr learned my addr 172.21.15.116:0/1752887223 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58410/0 (socket says 172.21.15.116:58410) 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 -- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880b8d80 msgr2=0x7fe688162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880b8d80 0x7fe688162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 -- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 msgr2=0x7fe68815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 0x7fe68815b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 -- 172.21.15.116:0/1752887223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68815a760 con 0x7fe6880cd490 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68da00640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe6880d2f40 tx=0x7fe6800197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680003ba0 con 0x7fe6880cd490 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe680003d40 con 0x7fe6880cd490 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68815aa40 con 0x7fe6880cd490 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680004230 con 0x7fe6880cd490 2024-05-14T11:20:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.342+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68815c610 con 0x7fe6880cd490 2024-05-14T11:20:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe688050880 con 0x7fe6880cd490 2024-05-14T11:20:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6800c1060 con 0x7fe6880cd490 2024-05-14T11:20:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe684c00640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 0x7fe668071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5249+0+0 (secure 0 0 0) 0x7fe680129400 con 0x7fe6880cd490 2024-05-14T11:20:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe68d000640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 0x7fe668071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.346+0000 7fe68d000640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 0x7fe668071940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ac0 tx=0x7fe67c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.350+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe680016640 con 0x7fe6880cd490 2024-05-14T11:20:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe68815d9d0 con 0x7fe6880cd490 2024-05-14T11:20:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe684c00640 1 -- 172.21.15.116:0/1752887223 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fe6800f5dc0 con 0x7fe6880cd490 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.570 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:20:00.571 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.573 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.574 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.575 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:20:00.577 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:20:00.578 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.580 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:20:00.581 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 msgr2=0x7fe668071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 0x7fe668071940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ac0 tx=0x7fe67c009190 comp rx=0 tx=0).stop 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 msgr2=0x7fe68815bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe6880d2f40 tx=0x7fe6800197f0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 shutdown_connections 2024-05-14T11:20:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe66806f430 0x7fe668071940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6880ccfa0 0x7fe68815b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6880cd490 0x7fe68815bde0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 --2- 172.21.15.116:0/1752887223 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6880b8d80 0x7fe688162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 >> 172.21.15.116:0/1752887223 conn(0x7fe6880b56e0 msgr2=0x7fe6880d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.570+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 shutdown_connections 2024-05-14T11:20:00.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.574+0000 7fe68e400640 1 -- 172.21.15.116:0/1752887223 wait complete. 2024-05-14T11:20:00.592 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:20:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 Processor -- start 2024-05-14T11:20:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 -- start start 2024-05-14T11:20:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ffa4c0af420 0x7ffa4c074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa4c15d340 con 0x7ffa4c0d1d80 2024-05-14T11:20:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa4c15d620 con 0x7ffa4c074b70 2024-05-14T11:20:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa51800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ffa4c15d900 con 0x7ffa4c0af420 2024-05-14T11:20:00.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffa4c0d1d80 0x7ffa4c06edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59164/0 (socket says 172.21.15.116:59164) 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4b400640 1 -- 172.21.15.116:0/299722100 learned_addr learned my addr 172.21.15.116:0/299722100 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4aa00640 1 --2- 172.21.15.116:0/299722100 >> v2:172.21.15.149:3300/0 conn(0x7ffa4c0af420 0x7ffa4c074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738961515 0 0) 0x7ffa4c15d340 con 0x7ffa4c0d1d80 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa30003530 con 0x7ffa4c0d1d80 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4aa00640 1 -- 172.21.15.116:0/299722100 >> v1:172.21.15.149:6789/0 conn(0x7ffa4c074b70 legacy=0x7ffa4c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501754473 0 0) 0x7ffa4c15d620 con 0x7ffa4c074b70 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4aa00640 1 -- 172.21.15.116:0/299722100 >> v1:172.21.15.116:6789/0 conn(0x7ffa4c0d1d80 legacy=0x7ffa4c06edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.738+0000 7ffa4aa00640 1 -- 172.21.15.116:0/299722100 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c15dbe0 con 0x7ffa4c0af420 2024-05-14T11:20:00.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa4aa00640 1 --2- 172.21.15.116:0/299722100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c0af420 0x7ffa4c074630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489e58e305432bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0b2070 con 0x7ffa4c0af420 2024-05-14T11:20:00.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0a5a50 con 0x7ffa4c0af420 2024-05-14T11:20:00.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa4a000640 1 -- 172.21.15.116:0/299722100 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0a5e20 con 0x7ffa4c0af420 2024-05-14T11:20:00.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 -- 172.21.15.116:0/299722100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c0af420 msgr2=0x7ffa4c074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 --2- 172.21.15.116:0/299722100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c0af420 0x7ffa4c074630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c01c970 comp rx=0 tx=0).stop 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 -- 172.21.15.116:0/299722100 shutdown_connections 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 --2- 172.21.15.116:0/299722100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c0af420 0x7ffa4c074630 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 -- 172.21.15.116:0/299722100 >> 172.21.15.116:0/299722100 conn(0x7ffa4c0b4620 msgr2=0x7ffa4c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 -- 172.21.15.116:0/299722100 shutdown_connections 2024-05-14T11:20:00.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.742+0000 7ffa51800640 1 -- 172.21.15.116:0/299722100 wait complete. 2024-05-14T11:20:00.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 Processor -- start 2024-05-14T11:20:00.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 -- start start 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c074b70 0x7ffa4c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa4c0af420 0x7ffa4c09a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c16b430 con 0x7ffa4c09b280 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c16b2b0 con 0x7ffa4c074b70 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa51800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c16b5b0 con 0x7ffa4c0af420 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58412/0 (socket says 172.21.15.116:58412) 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa4b400640 1 -- 172.21.15.116:0/3067167849 learned_addr learned my addr 172.21.15.116:0/3067167849 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.746+0000 7ffa4aa00640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c074b70 0x7ffa4c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa42000640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa4c0af420 0x7ffa4c09a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 -- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa4c0af420 msgr2=0x7ffa4c09a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa4c0af420 0x7ffa4c09a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 -- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c074b70 msgr2=0x7ffa4c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c074b70 0x7ffa4c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 -- 172.21.15.116:0/3067167849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c09bbd0 con 0x7ffa4c09b280 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa4b400640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffa44001230 tx=0x7ffa4400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44017070 con 0x7ffa4c09b280 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa44005cf0 con 0x7ffa4c09b280 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c15f4b0 con 0x7ffa4c09b280 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440072f0 con 0x7ffa4c09b280 2024-05-14T11:20:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c13f560 con 0x7ffa4c09b280 2024-05-14T11:20:00.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c06a9c0 con 0x7ffa4c09b280 2024-05-14T11:20:00.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.750+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffa4401a070 con 0x7ffa4c09b280 2024-05-14T11:20:00.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.754+0000 7ffa43400640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 0x7ffa28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:00.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.754+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5249+0+0 (secure 0 0 0) 0x7ffa4408e460 con 0x7ffa4c09b280 2024-05-14T11:20:00.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.754+0000 7ffa4aa00640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 0x7ffa28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:00.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.754+0000 7ffa4aa00640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 0x7ffa28071910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0af000 tx=0x7ffa3c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:00.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.758+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffa4c09bbd0 con 0x7ffa4c09b280 2024-05-14T11:20:00.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.914+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa4c06fe00 con 0x7ffa4c09b280 2024-05-14T11:20:00.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.914+0000 7ffa43400640 1 -- 172.21.15.116:0/3067167849 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7ffa4405ae20 con 0x7ffa4c09b280 2024-05-14T11:20:00.915 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:00.915 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:20:00.915 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:20:00.916 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:20:00.916 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:20:00.916 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:00.916 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:20:00.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 msgr2=0x7ffa28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 0x7ffa28071910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0af000 tx=0x7ffa3c0a7070 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 msgr2=0x7ffa4c09b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffa44001230 tx=0x7ffa4400c890 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 shutdown_connections 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa2806f400 0x7ffa28071910 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa4c09b280 0x7ffa4c09b690 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa4c0af420 0x7ffa4c09a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 --2- 172.21.15.116:0/3067167849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa4c074b70 0x7ffa4c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 >> 172.21.15.116:0/3067167849 conn(0x7ffa4c0b4620 msgr2=0x7ffa4c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 shutdown_connections 2024-05-14T11:20:00.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:00.918+0000 7ffa51800640 1 -- 172.21.15.116:0/3067167849 wait complete. 2024-05-14T11:20:00.937 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:20:00.938 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:20:01.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.086+0000 7f84ece00640 1 Processor -- start 2024-05-14T11:20:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.086+0000 7f84ece00640 1 -- start start 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070640 0x7f84e806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84e8070a50 0x7f84e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f84e81421f0 0x7f84e8142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f84e815dc90 con 0x7f84e8070a50 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f84e815d360 con 0x7f84e81421f0 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84ece00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f84e815d640 con 0x7f84e8070640 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84e8070a50 0x7f84e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84e8070a50 0x7f84e806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58426/0 (socket says 172.21.15.116:58426) 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 -- 172.21.15.116:0/1669544994 learned_addr learned my addr 172.21.15.116:0/1669544994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84e5e00640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070640 0x7f84e806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84e6800640 1 --2- 172.21.15.116:0/1669544994 >> v2:172.21.15.149:3300/0 conn(0x7f84e81421f0 0x7f84e8142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 -- 172.21.15.116:0/1669544994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070640 msgr2=0x7f84e806e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070640 0x7f84e806e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 -- 172.21.15.116:0/1669544994 >> v2:172.21.15.149:3300/0 conn(0x7f84e81421f0 msgr2=0x7f84e8142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 --2- 172.21.15.116:0/1669544994 >> v2:172.21.15.149:3300/0 conn(0x7f84e81421f0 0x7f84e8142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.090+0000 7f84dd400640 1 -- 172.21.15.116:0/1669544994 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e815d920 con 0x7f84e8070a50 2024-05-14T11:20:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84dd400640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070a50 0x7f84e806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f84e80d0ff0 tx=0x7f84d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5677aea93cfa98ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84e5400640 1 -- 172.21.15.116:0/1669544994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00ab600 con 0x7f84e8070a50 2024-05-14T11:20:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84e5400640 1 -- 172.21.15.116:0/1669544994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d00b0070 con 0x7f84e8070a50 2024-05-14T11:20:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84e5400640 1 -- 172.21.15.116:0/1669544994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00ab9c0 con 0x7f84e8070a50 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 -- 172.21.15.116:0/1669544994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070a50 msgr2=0x7f84e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070a50 0x7f84e806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f84e80d0ff0 tx=0x7f84d001c520 comp rx=0 tx=0).stop 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 -- 172.21.15.116:0/1669544994 shutdown_connections 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070a50 0x7f84e806edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 --2- 172.21.15.116:0/1669544994 >> v2:172.21.15.149:3300/0 conn(0x7f84e81421f0 0x7f84e8142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 --2- 172.21.15.116:0/1669544994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070640 0x7f84e806e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 -- 172.21.15.116:0/1669544994 >> 172.21.15.116:0/1669544994 conn(0x7f84e80cde80 msgr2=0x7f84e80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 -- 172.21.15.116:0/1669544994 shutdown_connections 2024-05-14T11:20:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.094+0000 7f84ece00640 1 -- 172.21.15.116:0/1669544994 wait complete. 2024-05-14T11:20:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 Processor -- start 2024-05-14T11:20:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- start start 2024-05-14T11:20:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070a50 0x7f84e8143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84e81421f0 0x7f84e8139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80cd980 con 0x7f84e8070640 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80cd800 con 0x7f84e81421f0 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f84e806d7a0 con 0x7f84e8070a50 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58438/0 (socket says 172.21.15.116:58438) 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 -- 172.21.15.116:0/1667295541 learned_addr learned my addr 172.21.15.116:0/1667295541 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84dd400640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070a50 0x7f84e8143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e6800640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84e81421f0 0x7f84e8139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 -- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070a50 msgr2=0x7f84e8143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070a50 0x7f84e8143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 -- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84e81421f0 msgr2=0x7f84e8139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84e81421f0 0x7f84e8139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 -- 172.21.15.116:0/1667295541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8157fc0 con 0x7f84e8070640 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84e5e00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e8a0 tx=0x7f84d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8017070 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d800ee90 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e8138c70 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800cb40 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.098+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e8139230 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e806aa30 con 0x7f84e8070640 2024-05-14T11:20:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f84d801a070 con 0x7f84e8070640 2024-05-14T11:20:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84dea00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 0x7f84c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5249+0+0 (secure 0 0 0) 0x7f84d808f330 con 0x7f84e8070640 2024-05-14T11:20:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84dd400640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 0x7f84c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.102+0000 7f84dd400640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 0x7f84c4071910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af000 tx=0x7f84d00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:01.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.106+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f84d805bcf0 con 0x7f84e8070640 2024-05-14T11:20:01.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:01.270+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f84e806cd30 con 0x7f84e8070640 2024-05-14T11:20:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.166+0000 7f84dea00640 1 -- 172.21.15.116:0/1667295541 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7f84d805fff0 con 0x7f84e8070640 2024-05-14T11:20:02.168 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:20:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 msgr2=0x7f84c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 0x7f84c4071910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af000 tx=0x7f84d00a7000 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 msgr2=0x7f84e81fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e8a0 tx=0x7f84d800ed60 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 shutdown_connections 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f84c406f400 0x7f84c4071910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84e81421f0 0x7f84e8139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f84e8070a50 0x7f84e8143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 --2- 172.21.15.116:0/1667295541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84e8070640 0x7f84e81fbd80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.170+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 >> 172.21.15.116:0/1667295541 conn(0x7f84e80cde80 msgr2=0x7f84e80b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:02.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.174+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 shutdown_connections 2024-05-14T11:20:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.174+0000 7f84ece00640 1 -- 172.21.15.116:0/1667295541 wait complete. 2024-05-14T11:20:02.201 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:20:02.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.358+0000 7f545a600640 1 Processor -- start 2024-05-14T11:20:02.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.358+0000 7f545a600640 1 -- start start 2024-05-14T11:20:02.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f545a600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f545406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f545a600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545406f860 con 0x7f54540cde70 2024-05-14T11:20:02.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f545a600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545406fae0 con 0x7f545406f450 2024-05-14T11:20:02.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f545a600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f545406db80 con 0x7f54540d1e40 2024-05-14T11:20:02.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f54540cde70 0x7f54541675f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59166/0 (socket says 172.21.15.116:59166) 2024-05-14T11:20:02.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453e00640 1 -- 172.21.15.116:0/2725255796 learned_addr learned my addr 172.21.15.116:0/2725255796 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:02.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f544be00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f545406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977058105 0 0) 0x7f544c0014a0 con 0x7f54540cde70 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5438003530 con 0x7f54540cde70 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140267445 0 0) 0x7f545406fae0 con 0x7f545406f450 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f544c0014a0 con 0x7f545406f450 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 996759314 0 0) 0x7f5438003530 con 0x7f54540cde70 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 msgr2=0x7f545406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f545406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 >> v1:172.21.15.149:6789/0 conn(0x7f545406f450 legacy=0x7f54540d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545406fdc0 con 0x7f54540cde70 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f544c001c50 con 0x7f54540cde70 2024-05-14T11:20:02.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f544be00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f545406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 >> v1:172.21.15.116:6789/0 conn(0x7f54540cde70 legacy=0x7f54541675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5438003c20 0x7f5438004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438004810 0x7f5438004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f5438005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f545406db80 con 0x7f5438003c20 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f543800bb90 con 0x7f5438004810 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f543800bd90 con 0x7f54540d1e40 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5438003c20 0x7f5438004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f544be00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438004810 0x7f5438004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453e00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f5438005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 -- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 msgr2=0x7f5438005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f5438005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 -- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438004810 msgr2=0x7f5438004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438004810 0x7f5438004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453400640 1 -- 172.21.15.116:0/2725255796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5438003530 con 0x7f5438003c20 2024-05-14T11:20:02.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.362+0000 7f5453e00640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54540d1e40 0x7f5438005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:02.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f5453400640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5438003c20 0x7f5438004070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f54540c83b0 tx=0x7f544401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25636b90aef302f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:02.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444001550 con 0x7f5438003c20 2024-05-14T11:20:02.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54440052d0 con 0x7f5438003c20 2024-05-14T11:20:02.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f5452a00640 1 -- 172.21.15.116:0/2725255796 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444007260 con 0x7f5438003c20 2024-05-14T11:20:02.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 -- 172.21.15.116:0/2725255796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5438003c20 msgr2=0x7f5438004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 --2- 172.21.15.116:0/2725255796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5438003c20 0x7f5438004070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f54540c83b0 tx=0x7f544401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f5453e00640 1 -- 172.21.15.116:0/2725255796 reap_dead start 2024-05-14T11:20:02.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 -- 172.21.15.116:0/2725255796 shutdown_connections 2024-05-14T11:20:02.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 -- 172.21.15.116:0/2725255796 >> 172.21.15.116:0/2725255796 conn(0x7f54540b4720 msgr2=0x7f54540b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:02.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 -- 172.21.15.116:0/2725255796 shutdown_connections 2024-05-14T11:20:02.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.366+0000 7f545a600640 1 -- 172.21.15.116:0/2725255796 wait complete. 2024-05-14T11:20:02.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 Processor -- start 2024-05-14T11:20:02.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 -- start start 2024-05-14T11:20:02.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 0x7f5454158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54541591f0 0x7f54540b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f54541696a0 con 0x7f5454158ca0 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5454169520 con 0x7f5438003c20 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5454169820 con 0x7f54541591f0 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 0x7f5454158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f544be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54541591f0 0x7f54540b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58460/0 (socket says 172.21.15.116:58460) 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 0x7f5454158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:50858/0 (socket says 172.21.15.116:50858) 2024-05-14T11:20:02.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 -- 172.21.15.116:0/1145995584 learned_addr learned my addr 172.21.15.116:0/1145995584 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 -- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54541591f0 msgr2=0x7f54540b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54541591f0 0x7f54540b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 -- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 msgr2=0x7f5454158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 0x7f5454158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 -- 172.21.15.116:0/1145995584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545409ab30 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f5453e00640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f54540d2280 tx=0x7f544c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c007ae0 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c019070 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5454166300 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.370+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c013420 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.374+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54540d0260 con 0x7f5454158ca0 2024-05-14T11:20:02.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.374+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54540d04c0 con 0x7f5454158ca0 2024-05-14T11:20:02.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.374+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f544c0135c0 con 0x7f5454158ca0 2024-05-14T11:20:02.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.374+0000 7f544b400640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 0x7f5430071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:02.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.374+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5619+0+0 (secure 0 0 0) 0x7f544c08ff10 con 0x7f5454158ca0 2024-05-14T11:20:02.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.378+0000 7f5453400640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 0x7f5430071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:02.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.378+0000 7f5453400640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 0x7f5430071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54380057f0 tx=0x7f5444001370 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:02.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.382+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f545406f860 con 0x7f5454158ca0 2024-05-14T11:20:02.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:02.538+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f545406a9c0 con 0x7f5454158ca0 2024-05-14T11:20:03.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.174+0000 7f544b400640 1 -- 172.21.15.116:0/1145995584 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) ==== 170+0+0 (secure 0 0 0) 0x7f544c05c970 con 0x7f5454158ca0 2024-05-14T11:20:03.176 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 msgr2=0x7f5430071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 0x7f5430071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f54380057f0 tx=0x7f5444001370 comp rx=0 tx=0).stop 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 msgr2=0x7f54540b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f54540d2280 tx=0x7f544c00e530 comp rx=0 tx=0).stop 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 shutdown_connections 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f543006f400 0x7f5430071910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5454158ca0 0x7f54540b1f50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54541591f0 0x7f54540b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 --2- 172.21.15.116:0/1145995584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5438003c20 0x7f5454158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 >> 172.21.15.116:0/1145995584 conn(0x7f54540b4720 msgr2=0x7f545415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:03.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 shutdown_connections 2024-05-14T11:20:03.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.178+0000 7f545a600640 1 -- 172.21.15.116:0/1145995584 wait complete. 2024-05-14T11:20:03.217 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:20:03.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.370+0000 7fdf08600640 1 Processor -- start 2024-05-14T11:20:03.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.370+0000 7fdf08600640 1 -- start start 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.370+0000 7fdf08600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdf000ce450 0x7fdf000d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf08600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0006cc10 con 0x7fdf0015c1d0 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf08600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0006cef0 con 0x7fdf000cbd60 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf08600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0006d790 con 0x7fdf000ce450 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdf0015c1d0 0x7fdf000cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59170/0 (socket says 172.21.15.116:59170) 2024-05-14T11:20:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 -- 172.21.15.116:0/3936322207 learned_addr learned my addr 172.21.15.116:0/3936322207 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281978650 0 0) 0x7fdf0006cc10 con 0x7fdf0015c1d0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05200640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdf000ce450 0x7fdf000d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdee8003450 con 0x7fdf0015c1d0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1619092610 0 0) 0x7fdee8003450 con 0x7fdf0015c1d0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdf000ce450 msgr2=0x7fdf000d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdf000ce450 0x7fdf000d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 >> v1:172.21.15.149:6789/0 conn(0x7fdf000cbd60 legacy=0x7fdf000cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0006d1d0 con 0x7fdf0015c1d0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fdef4001700 con 0x7fdf0015c1d0 2024-05-14T11:20:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 >> v1:172.21.15.116:6789/0 conn(0x7fdf0015c1d0 legacy=0x7fdf000cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdee8003b30 0x7fdee8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdee8008920 0x7fdee800ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee800b6a0 0x7fdee800db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0006d790 con 0x7fdee8003b30 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800e260 con 0x7fdee8008920 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdee800e460 con 0x7fdee800b6a0 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdee8003b30 0x7fdee8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05200640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdee8008920 0x7fdee800ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf06600640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee800b6a0 0x7fdee800db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 -- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee800b6a0 msgr2=0x7fdee800db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee800b6a0 0x7fdee800db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 -- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdee8008920 msgr2=0x7fdee800ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdee8008920 0x7fdee800ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05c00640 1 -- 172.21.15.116:0/3936322207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee8003450 con 0x7fdee8003b30 2024-05-14T11:20:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.374+0000 7fdf05200640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdee8008920 0x7fdee800ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf05c00640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdee8003b30 0x7fdee8003fa0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c240 tx=0x7fdef401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e54ac9c98d64ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4007d80 con 0x7fdee8003b30 2024-05-14T11:20:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef40a9020 con 0x7fdee8003b30 2024-05-14T11:20:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdeffe00640 1 -- 172.21.15.116:0/3936322207 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4005b20 con 0x7fdee8003b30 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 -- 172.21.15.116:0/3936322207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdee8003b30 msgr2=0x7fdee8003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 --2- 172.21.15.116:0/3936322207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdee8003b30 0x7fdee8003fa0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c240 tx=0x7fdef401cbc0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf06600640 1 -- 172.21.15.116:0/3936322207 reap_dead start 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 -- 172.21.15.116:0/3936322207 shutdown_connections 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 -- 172.21.15.116:0/3936322207 >> 172.21.15.116:0/3936322207 conn(0x7fdf000b4720 msgr2=0x7fdf000b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 -- 172.21.15.116:0/3936322207 shutdown_connections 2024-05-14T11:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.378+0000 7fdf08600640 1 -- 172.21.15.116:0/3936322207 wait complete. 2024-05-14T11:20:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 Processor -- start 2024-05-14T11:20:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- start start 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 0x7fdf001f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 0x7fdf000b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf000d0c50 con 0x7fdf000c9110 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf000ce660 con 0x7fdf000ca290 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdf000d0dd0 con 0x7fdee8003b30 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58478/0 (socket says 172.21.15.116:58478) 2024-05-14T11:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 -- 172.21.15.116:0/846138891 learned_addr learned my addr 172.21.15.116:0/846138891 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05c00640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 0x7fdf001f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 -- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 msgr2=0x7fdf001f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 0x7fdf001f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 -- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 msgr2=0x7fdf000b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf06600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 0x7fdf000b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 0x7fdf000b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 -- 172.21.15.116:0/846138891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf000c9520 con 0x7fdf000c9110 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05c00640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 0x7fdf001f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf06600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 0x7fdf000b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf05200640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdeec011ca0 tx=0x7fdeec00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00fd20 con 0x7fdf000c9110 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeec004020 con 0x7fdf000c9110 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec004410 con 0x7fdf000c9110 2024-05-14T11:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001f9d90 con 0x7fdf000c9110 2024-05-14T11:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.382+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf000c9a90 con 0x7fdf000c9110 2024-05-14T11:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0006a9c0 con 0x7fdf000c9110 2024-05-14T11:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdeec012040 con 0x7fdf000c9110 2024-05-14T11:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdefd600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 0x7fdee0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6001+0+0 (secure 0 0 0) 0x7fdeec08f730 con 0x7fdf000c9110 2024-05-14T11:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdf05c00640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 0x7fdee0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.386+0000 7fdf05c00640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 0x7fdee0071910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdef40aa000 tx=0x7fdef4001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.394+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdf000c9520 con 0x7fdf000c9110 2024-05-14T11:20:03.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:03.582+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdf00069de0 con 0x7fdf000c9110 2024-05-14T11:20:04.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.178+0000 7fdefd600640 1 -- 172.21.15.116:0/846138891 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v41) ==== 174+0+0 (secure 0 0 0) 0x7fdeec019030 con 0x7fdf000c9110 2024-05-14T11:20:04.178 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 11 and data pool 12 2024-05-14T11:20:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 msgr2=0x7fdee0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 0x7fdee0071910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdef40aa000 tx=0x7fdef4001a00 comp rx=0 tx=0).stop 2024-05-14T11:20:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 msgr2=0x7fdf001fa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdeec011ca0 tx=0x7fdeec00f610 comp rx=0 tx=0).stop 2024-05-14T11:20:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 shutdown_connections 2024-05-14T11:20:04.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdee006f400 0x7fdee0071910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf000ca290 0x7fdf000b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf000c9110 0x7fdf001fa3f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.182+0000 7fdf08600640 1 --2- 172.21.15.116:0/846138891 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdee8003b30 0x7fdf001f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.186+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 >> 172.21.15.116:0/846138891 conn(0x7fdf000b4720 msgr2=0x7fdf001590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.186+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 shutdown_connections 2024-05-14T11:20:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.186+0000 7fdf08600640 1 -- 172.21.15.116:0/846138891 wait complete. 2024-05-14T11:20:04.236 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:20:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 Processor -- start 2024-05-14T11:20:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 -- start start 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 0x7fa97c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 0x7fa97c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97c06ff20 con 0x7fa97c0d1670 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa97c0ae100 con 0x7fa97c06c700 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.382+0000 7fa983600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa97c0ae870 con 0x7fa97c15db30 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 0x7fa97c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa981600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 0x7fa97c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa973e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fa97c0d1670 0x7fa97c0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:44748/0 (socket says 172.21.15.116:44748) 2024-05-14T11:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa973e00640 1 -- 172.21.15.116:0/3019962599 learned_addr learned my addr 172.21.15.116:0/3019962599 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa97be00640 1 -- 172.21.15.116:0/3019962599 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 238238450 0 0) 0x7fa97c06ff20 con 0x7fa97c0d1670 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa97be00640 1 -- 172.21.15.116:0/3019962599 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa960003530 con 0x7fa97c0d1670 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 -- 172.21.15.116:0/3019962599 >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 msgr2=0x7fa97c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 --2- 172.21.15.116:0/3019962599 >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 0x7fa97c06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 -- 172.21.15.116:0/3019962599 >> v1:172.21.15.149:6789/0 conn(0x7fa97c0d1670 legacy=0x7fa97c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 -- 172.21.15.116:0/3019962599 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c070470 con 0x7fa97c15db30 2024-05-14T11:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa981600640 1 --2- 172.21.15.116:0/3019962599 >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 0x7fa97c06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:04.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa980c00640 1 --2- 172.21.15.116:0/3019962599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 0x7fa97c0aeaa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097c0 tx=0x7fa96c01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08c6b9de3acbe8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa97be00640 1 -- 172.21.15.116:0/3019962599 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b0070 con 0x7fa97c15db30 2024-05-14T11:20:04.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa97be00640 1 -- 172.21.15.116:0/3019962599 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c0ab600 con 0x7fa97c15db30 2024-05-14T11:20:04.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.386+0000 7fa97be00640 1 -- 172.21.15.116:0/3019962599 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0ab9d0 con 0x7fa97c15db30 2024-05-14T11:20:04.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- 172.21.15.116:0/3019962599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 msgr2=0x7fa97c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 --2- 172.21.15.116:0/3019962599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 0x7fa97c0aeaa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097c0 tx=0x7fa96c01c3d0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- 172.21.15.116:0/3019962599 shutdown_connections 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 --2- 172.21.15.116:0/3019962599 >> v2:172.21.15.149:3300/0 conn(0x7fa97c06c700 0x7fa97c06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 --2- 172.21.15.116:0/3019962599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c15db30 0x7fa97c0aeaa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- 172.21.15.116:0/3019962599 >> 172.21.15.116:0/3019962599 conn(0x7fa97c0b4620 msgr2=0x7fa97c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- 172.21.15.116:0/3019962599 shutdown_connections 2024-05-14T11:20:04.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- 172.21.15.116:0/3019962599 wait complete. 2024-05-14T11:20:04.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 Processor -- start 2024-05-14T11:20:04.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.390+0000 7fa983600640 1 -- start start 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c0d1670 0x7fa97c074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c16a770 con 0x7fa97c15db30 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c15df40 con 0x7fa97c06c700 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa97c16a8f0 con 0x7fa97c0d1670 2024-05-14T11:20:04.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa981600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:50882/0 (socket says 172.21.15.116:50882) 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa981600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58490/0 (socket says 172.21.15.116:58490) 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 -- 172.21.15.116:0/692963750 learned_addr learned my addr 172.21.15.116:0/692963750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa973e00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c0d1670 0x7fa97c074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 -- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c0d1670 msgr2=0x7fa97c074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c0d1670 0x7fa97c074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 -- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 msgr2=0x7fa97c1fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 -- 172.21.15.116:0/692963750 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1fd5e0 con 0x7fa97c06c700 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa981600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa980c00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01a000 tx=0x7fa96c019170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b0040 con 0x7fa97c06c700 2024-05-14T11:20:04.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c0bc070 con 0x7fa97c06c700 2024-05-14T11:20:04.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c073790 con 0x7fa97c06c700 2024-05-14T11:20:04.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b56c0 con 0x7fa97c06c700 2024-05-14T11:20:04.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c155240 con 0x7fa97c06c700 2024-05-14T11:20:04.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.394+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c06a9c0 con 0x7fa97c06c700 2024-05-14T11:20:04.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.398+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa96c0b3070 con 0x7fa97c06c700 2024-05-14T11:20:04.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.398+0000 7fa979600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.398+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fa96c129f80 con 0x7fa97c06c700 2024-05-14T11:20:04.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.398+0000 7fa973e00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.398+0000 7fa973e00640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 0x7fa958071910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa97c068c30 tx=0x7fa9640079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.402+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa96c016050 con 0x7fa97c06c700 2024-05-14T11:20:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.562+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa97c151b20 con 0x7fa97c06c700 2024-05-14T11:20:04.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.562+0000 7fa979600640 1 -- 172.21.15.116:0/692963750 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17006 (secure 0 0 0) 0x7fa96c0f6670 con 0x7fa97c06c700 2024-05-14T11:20:04.563 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:04.563 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:20:03.586011+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:20:01.282761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-14T11:20:02.552345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:20:04.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 msgr2=0x7fa958071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 0x7fa958071910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa97c068c30 tx=0x7fa9640079c0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 msgr2=0x7fa97c1515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01a000 tx=0x7fa96c019170 comp rx=0 tx=0).stop 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 shutdown_connections 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa97c15db30 0x7fa97c1fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa97c0d1670 0x7fa97c074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 --2- 172.21.15.116:0/692963750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa97c06c700 0x7fa97c1515e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 >> 172.21.15.116:0/692963750 conn(0x7fa97c0b4620 msgr2=0x7fa97c1520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 shutdown_connections 2024-05-14T11:20:04.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.566+0000 7fa983600640 1 -- 172.21.15.116:0/692963750 wait complete. 2024-05-14T11:20:04.585 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:20:04.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 Processor -- start 2024-05-14T11:20:04.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 -- start start 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87900aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87900d1670 0x7f87900d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879006ff20 con 0x7f8790161600 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f879006d710 con 0x7f87900d1670 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8795200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f87900ae100 con 0x7f879015db30 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87900d1670 0x7f87900d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87900d1670 0x7f87900d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58498/0 (socket says 172.21.15.116:58498) 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 -- 172.21.15.116:0/763279267 learned_addr learned my addr 172.21.15.116:0/763279267 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:04.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f878de00640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87900aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 -- 172.21.15.116:0/763279267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 msgr2=0x7f87900aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:04.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87900aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 -- 172.21.15.116:0/763279267 >> v1:172.21.15.149:6789/0 conn(0x7f8790161600 legacy=0x7f8790161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f8785400640 1 -- 172.21.15.116:0/763279267 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900d1fc0 con 0x7f87900d1670 2024-05-14T11:20:04.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.734+0000 7f878de00640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87900aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:04.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8785400640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87900d1670 0x7f87900d1a80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8778009870 tx=0x7f877801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390142331f3e3b29 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f878d400640 1 -- 172.21.15.116:0/763279267 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780b0070 con 0x7f87900d1670 2024-05-14T11:20:04.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f878d400640 1 -- 172.21.15.116:0/763279267 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87780ab600 con 0x7f87900d1670 2024-05-14T11:20:04.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f878d400640 1 -- 172.21.15.116:0/763279267 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780ab9d0 con 0x7f87900d1670 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 -- 172.21.15.116:0/763279267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87900d1670 msgr2=0x7f87900d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87900d1670 0x7f87900d1a80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8778009870 tx=0x7f877801c4d0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 -- 172.21.15.116:0/763279267 shutdown_connections 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87900d1670 0x7f87900d1a80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 --2- 172.21.15.116:0/763279267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87900aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 -- 172.21.15.116:0/763279267 >> 172.21.15.116:0/763279267 conn(0x7f87900b4620 msgr2=0x7f87900b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 -- 172.21.15.116:0/763279267 shutdown_connections 2024-05-14T11:20:04.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.738+0000 7f8795200640 1 -- 172.21.15.116:0/763279267 wait complete. 2024-05-14T11:20:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 Processor -- start 2024-05-14T11:20:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- start start 2024-05-14T11:20:04.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 0x7f879015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87901fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f879015df40 con 0x7f8790161600 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87900aefe0 con 0x7f87900d1670 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f879016af80 con 0x7f879015db30 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58510/0 (socket says 172.21.15.116:58510) 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 -- 172.21.15.116:0/2051089874 learned_addr learned my addr 172.21.15.116:0/2051089874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8785400640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87901fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878de00640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 0x7f879015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 -- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 msgr2=0x7f87901fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87901fa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 -- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 msgr2=0x7f879015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 0x7f879015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 -- 172.21.15.116:0/2051089874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901fa760 con 0x7f8790161600 2024-05-14T11:20:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8785400640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87901fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878de00640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 0x7f879015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f878e800640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f878800c6b0 tx=0x7f878800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788003b80 con 0x7f8790161600 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788003d20 con 0x7f8790161600 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880052f0 con 0x7f8790161600 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879015a7e0 con 0x7f8790161600 2024-05-14T11:20:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.742+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879015ada0 con 0x7f8790161600 2024-05-14T11:20:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.746+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879006a9c0 con 0x7f8790161600 2024-05-14T11:20:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.746+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8788005490 con 0x7f8790161600 2024-05-14T11:20:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.746+0000 7f8786a00640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 0x7f876c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.746+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7f878808f370 con 0x7f8790161600 2024-05-14T11:20:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.746+0000 7f878de00640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 0x7f876c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:04.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.750+0000 7f878de00640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 0x7f876c071910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87800097a0 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:04.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.754+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f87901fa760 con 0x7f8790161600 2024-05-14T11:20:04.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.910+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f879015a570 con 0x7f8790161600 2024-05-14T11:20:04.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.910+0000 7f8786a00640 1 -- 172.21.15.116:0/2051089874 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17006 (secure 0 0 0) 0x7f878805c030 con 0x7f8790161600 2024-05-14T11:20:04.912 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:04.912 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:20:03.586011+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:20:01.282761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-14T11:20:02.552345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 msgr2=0x7f876c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 0x7f876c071910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87800097a0 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 msgr2=0x7f8790160520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f878800c6b0 tx=0x7f878800cb70 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 shutdown_connections 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f876c06f400 0x7f876c071910 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8790161600 0x7f8790160520 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f879015db30 0x7f87901fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 --2- 172.21.15.116:0/2051089874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87900d1670 0x7f879015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 >> 172.21.15.116:0/2051089874 conn(0x7f87900b4620 msgr2=0x7f8790150360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:04.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 shutdown_connections 2024-05-14T11:20:04.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:04.914+0000 7f8795200640 1 -- 172.21.15.116:0/2051089874 wait complete. 2024-05-14T11:20:04.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:20:05.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.082+0000 7fb43d000640 1 Processor -- start 2024-05-14T11:20:05.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.082+0000 7fb43d000640 1 -- start start 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb43d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4380d1e40 0x7fb4380cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb43d000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb43806c700 0x7fb43806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb43d000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb43806f0a0 con 0x7fb4380cd1c0 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb43d000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb4380cbb60 con 0x7fb4380d1e40 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb43d000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb43806f420 con 0x7fb43806c700 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4380d1e40 0x7fb4380cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4380d1e40 0x7fb4380cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58522/0 (socket says 172.21.15.116:58522) 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 -- 172.21.15.116:0/109587297 learned_addr learned my addr 172.21.15.116:0/109587297 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb436800640 1 --2- 172.21.15.116:0/109587297 >> v2:172.21.15.149:3300/0 conn(0x7fb43806c700 0x7fb43806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 -- 172.21.15.116:0/109587297 >> v2:172.21.15.149:3300/0 conn(0x7fb43806c700 msgr2=0x7fb43806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 --2- 172.21.15.116:0/109587297 >> v2:172.21.15.149:3300/0 conn(0x7fb43806c700 0x7fb43806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 -- 172.21.15.116:0/109587297 >> v1:172.21.15.149:6789/0 conn(0x7fb4380cd1c0 legacy=0x7fb4380d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:05.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.086+0000 7fb42d400640 1 -- 172.21.15.116:0/109587297 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43806f700 con 0x7fb4380d1e40 2024-05-14T11:20:05.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb42d400640 1 --2- 172.21.15.116:0/109587297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4380d1e40 0x7fb4380cd5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb43806e2c0 tx=0x7fb42001ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5427e496bdae4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:05.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb435400640 1 -- 172.21.15.116:0/109587297 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200b4070 con 0x7fb4380d1e40 2024-05-14T11:20:05.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb435400640 1 -- 172.21.15.116:0/109587297 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4200a7aa0 con 0x7fb4380d1e40 2024-05-14T11:20:05.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb435400640 1 -- 172.21.15.116:0/109587297 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200af440 con 0x7fb4380d1e40 2024-05-14T11:20:05.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 -- 172.21.15.116:0/109587297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4380d1e40 msgr2=0x7fb4380cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:05.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 --2- 172.21.15.116:0/109587297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4380d1e40 0x7fb4380cd5d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb43806e2c0 tx=0x7fb42001ea10 comp rx=0 tx=0).stop 2024-05-14T11:20:05.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 -- 172.21.15.116:0/109587297 shutdown_connections 2024-05-14T11:20:05.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 --2- 172.21.15.116:0/109587297 >> v2:172.21.15.149:3300/0 conn(0x7fb43806c700 0x7fb43806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:05.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 --2- 172.21.15.116:0/109587297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4380d1e40 0x7fb4380cd5d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:05.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 -- 172.21.15.116:0/109587297 >> 172.21.15.116:0/109587297 conn(0x7fb4380b4720 msgr2=0x7fb4380b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:05.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 -- 172.21.15.116:0/109587297 shutdown_connections 2024-05-14T11:20:05.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 -- 172.21.15.116:0/109587297 wait complete. 2024-05-14T11:20:05.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.090+0000 7fb43d000640 1 Processor -- start 2024-05-14T11:20:05.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- start start 2024-05-14T11:20:05.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 0x7fb438157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb43815fae0 0x7fb4380c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380cddb0 con 0x7fb43806c700 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb43815cf10 con 0x7fb4380cd1c0 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380cdf30 con 0x7fb43815fae0 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58530/0 (socket says 172.21.15.116:58530) 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 -- 172.21.15.116:0/2796269774 learned_addr learned my addr 172.21.15.116:0/2796269774 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb436800640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb43815fae0 0x7fb4380c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb42d400640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 0x7fb438157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 -- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb43815fae0 msgr2=0x7fb4380c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb43815fae0 0x7fb4380c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 -- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 msgr2=0x7fb438157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 0x7fb438157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 -- 172.21.15.116:0/2796269774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb438158c50 con 0x7fb43806c700 2024-05-14T11:20:05.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb42d400640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 0x7fb438157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb435e00640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb42800ba50 tx=0x7fb42800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428019070 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42800cc20 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb438158f30 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428014440 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4381594f0 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.094+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb438159730 con 0x7fb43806c700 2024-05-14T11:20:05.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.098+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb42801c070 con 0x7fb43806c700 2024-05-14T11:20:05.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.098+0000 7fb42ea00640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 0x7fb414071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:05.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.098+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6101+0+0 (secure 0 0 0) 0x7fb42808f200 con 0x7fb43806c700 2024-05-14T11:20:05.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.098+0000 7fb42d400640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 0x7fb414071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:05.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.098+0000 7fb42d400640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 0x7fb414071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4200b1000 tx=0x7fb4200a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:05.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.102+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb42805b870 con 0x7fb43806c700 2024-05-14T11:20:05.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:05.298+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb43806a9c0 con 0x7fb43806c700 2024-05-14T11:20:06.158 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:20:06.158+0000 7fcc74e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:20:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.214+0000 7fb42ea00640 1 -- 172.21.15.116:0/2796269774 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7fb42805fb70 con 0x7fb43806c700 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 msgr2=0x7fb414071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 0x7fb414071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb4200b1000 tx=0x7fb4200a9070 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 msgr2=0x7fb438158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb42800ba50 tx=0x7fb42800bf10 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 shutdown_connections 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb41406f400 0x7fb414071910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4380cd1c0 0x7fb438157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb43815fae0 0x7fb4380c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 --2- 172.21.15.116:0/2796269774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43806c700 0x7fb438158400 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 >> 172.21.15.116:0/2796269774 conn(0x7fb4380b4720 msgr2=0x7fb438160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 shutdown_connections 2024-05-14T11:20:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.218+0000 7fb43d000640 1 -- 172.21.15.116:0/2796269774 wait complete. 2024-05-14T11:20:06.242 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:20:06.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:20:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.394+0000 7fbebd600640 1 Processor -- start 2024-05-14T11:20:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.394+0000 7fbebd600640 1 -- start start 2024-05-14T11:20:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbebd600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbeb80d1e40 0x7fbeb806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbebd600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80cde70 0x7fbeb81675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbebd600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb806f860 con 0x7fbeb806f450 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbebd600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb806db80 con 0x7fbeb80d1e40 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbebd600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb806fae0 con 0x7fbeb80cde70 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeae000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbeb80d1e40 0x7fbeb806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeb6a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbeb806f450 0x7fbeb80d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59180/0 (socket says 172.21.15.116:59180) 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeb6a00640 1 -- 172.21.15.116:0/1895480187 learned_addr learned my addr 172.21.15.116:0/1895480187 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeb7400640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80cde70 0x7fbeb81675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeae000640 1 -- 172.21.15.116:0/1895480187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80cde70 msgr2=0x7fbeb81675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeae000640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80cde70 0x7fbeb81675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeae000640 1 -- 172.21.15.116:0/1895480187 >> v1:172.21.15.116:6789/0 conn(0x7fbeb806f450 legacy=0x7fbeb80d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeae000640 1 -- 172.21.15.116:0/1895480187 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb806fdc0 con 0x7fbeb80d1e40 2024-05-14T11:20:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.398+0000 7fbeb6000640 1 -- 172.21.15.116:0/1895480187 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432358945 0 0) 0x7fbea8001120 con 0x7fbeb806f450 2024-05-14T11:20:06.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbeae000640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80d1e40 0x7fbeb806d640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097c0 tx=0x7fbea001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201e9ef7f4333e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:06.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbeb6000640 1 -- 172.21.15.116:0/1895480187 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00b2070 con 0x7fbeb80d1e40 2024-05-14T11:20:06.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbeb6000640 1 -- 172.21.15.116:0/1895480187 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea00ad640 con 0x7fbeb80d1e40 2024-05-14T11:20:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbeb6000640 1 -- 172.21.15.116:0/1895480187 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00ada10 con 0x7fbeb80d1e40 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 -- 172.21.15.116:0/1895480187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80d1e40 msgr2=0x7fbeb806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80d1e40 0x7fbeb806d640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097c0 tx=0x7fbea001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 -- 172.21.15.116:0/1895480187 shutdown_connections 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80cde70 0x7fbeb81675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 --2- 172.21.15.116:0/1895480187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80d1e40 0x7fbeb806d640 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 -- 172.21.15.116:0/1895480187 >> 172.21.15.116:0/1895480187 conn(0x7fbeb80b4720 msgr2=0x7fbeb80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 -- 172.21.15.116:0/1895480187 shutdown_connections 2024-05-14T11:20:06.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.402+0000 7fbebd600640 1 -- 172.21.15.116:0/1895480187 wait complete. 2024-05-14T11:20:06.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 Processor -- start 2024-05-14T11:20:06.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- start start 2024-05-14T11:20:06.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb806f450 0x7fbeb8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80d1e40 0x7fbeb80746f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb816a4d0 con 0x7fbeb80cde70 2024-05-14T11:20:06.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb816a350 con 0x7fbeb806f450 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb816a650 con 0x7fbeb80d1e40 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58546/0 (socket says 172.21.15.116:58546) 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 -- 172.21.15.116:0/312718432 learned_addr learned my addr 172.21.15.116:0/312718432 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeb7400640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80d1e40 0x7fbeb80746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeb6a00640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb806f450 0x7fbeb8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 -- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80d1e40 msgr2=0x7fbeb80746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80d1e40 0x7fbeb80746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 -- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb806f450 msgr2=0x7fbeb8159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb806f450 0x7fbeb8159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 -- 172.21.15.116:0/312718432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb8159e70 con 0x7fbeb80cde70 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeae000640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbea001a0d0 tx=0x7fbea0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00b2040 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea00bc070 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb80d0720 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00b6630 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.406+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb80d0c20 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb806a9c0 con 0x7fbeb80cde70 2024-05-14T11:20:06.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbea00b67d0 con 0x7fbeb80cde70 2024-05-14T11:20:06.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbeaf400640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 0x7fbe94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:06.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fbea0129420 con 0x7fbeb80cde70 2024-05-14T11:20:06.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbeb6a00640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 0x7fbe94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:06.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.410+0000 7fbeb6a00640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 0x7fbe94071940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000f70 tx=0x7fbea8007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.414+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbea0016090 con 0x7fbeb80cde70 2024-05-14T11:20:06.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:06.610+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbeb80d09b0 con 0x7fbeb80cde70 2024-05-14T11:20:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.222+0000 7fbeaf400640 1 -- 172.21.15.116:0/312718432 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fbea00f60f0 con 0x7fbeb80cde70 2024-05-14T11:20:07.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 msgr2=0x7fbe94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 0x7fbe94071940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbea8000f70 tx=0x7fbea8007c10 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 msgr2=0x7fbeb80741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbea001a0d0 tx=0x7fbea0019240 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 shutdown_connections 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbe9406f430 0x7fbe94071940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbeb80d1e40 0x7fbeb80746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbeb80cde70 0x7fbeb80741b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 --2- 172.21.15.116:0/312718432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb806f450 0x7fbeb8159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 >> 172.21.15.116:0/312718432 conn(0x7fbeb80b4720 msgr2=0x7fbeb81563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 shutdown_connections 2024-05-14T11:20:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.226+0000 7fbebd600640 1 -- 172.21.15.116:0/312718432 wait complete. 2024-05-14T11:20:07.245 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:20:07.246 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 Processor -- start 2024-05-14T11:20:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 -- start start 2024-05-14T11:20:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00a41432c0 0x7f00a41436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 0x7f00a40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a4143c10 con 0x7f00a415eb60 2024-05-14T11:20:07.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f00a4140f90 con 0x7f00a41432c0 2024-05-14T11:20:07.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00ab000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00a40cf5f0 con 0x7f00a40cdac0 2024-05-14T11:20:07.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00a41432c0 0x7f00a41436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00a9000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 0x7f00a40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00a41432c0 0x7f00a41436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:50920/0 (socket says 172.21.15.116:50920) 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 -- 172.21.15.116:0/3941527931 learned_addr learned my addr 172.21.15.116:0/3941527931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00a9000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 0x7f00a40cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:49400/0 (socket says 172.21.15.116:49400) 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f00a3e00640 1 --1- 172.21.15.116:0/3941527931 >> v1:172.21.15.149:6789/0 conn(0x7f00a415eb60 0x7f00a40b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:44772/0 (socket says 172.21.15.116:44772) 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 -- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 msgr2=0x7f00a40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 --2- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 0x7f00a40cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 -- 172.21.15.116:0/3941527931 >> v1:172.21.15.149:6789/0 conn(0x7f00a415eb60 legacy=0x7f00a40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.394+0000 7f009b400640 1 -- 172.21.15.116:0/3941527931 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40b0da0 con 0x7f00a41432c0 2024-05-14T11:20:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f009b400640 1 --2- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a41432c0 0x7f00a41436d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f008c0098a0 tx=0x7f008c01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b23ba4880bc46c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00a3400640 1 -- 172.21.15.116:0/3941527931 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c0b0070 con 0x7f00a41432c0 2024-05-14T11:20:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00a3400640 1 -- 172.21.15.116:0/3941527931 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008c0ab600 con 0x7f00a41432c0 2024-05-14T11:20:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00a3400640 1 -- 172.21.15.116:0/3941527931 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c0ab9d0 con 0x7f00a41432c0 2024-05-14T11:20:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 -- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a41432c0 msgr2=0x7f00a41436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 --2- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a41432c0 0x7f00a41436d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f008c0098a0 tx=0x7f008c01c4b0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 -- 172.21.15.116:0/3941527931 shutdown_connections 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 --2- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a40cdac0 0x7f00a40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 --2- 172.21.15.116:0/3941527931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a41432c0 0x7f00a41436d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 -- 172.21.15.116:0/3941527931 >> 172.21.15.116:0/3941527931 conn(0x7f00a40ce6b0 msgr2=0x7f00a40ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 -- 172.21.15.116:0/3941527931 shutdown_connections 2024-05-14T11:20:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.398+0000 7f00ab000640 1 -- 172.21.15.116:0/3941527931 wait complete. 2024-05-14T11:20:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 Processor -- start 2024-05-14T11:20:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 -- start start 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 0x7f00a4142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 0x7f00a413e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40ce410 con 0x7f00a415eb60 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f00a406e7c0 con 0x7f00a40cdac0 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00ab000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00a415ef70 con 0x7f00a41432c0 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00a9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00a9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58562/0 (socket says 172.21.15.116:58562) 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.402+0000 7f00a9000640 1 -- 172.21.15.116:0/1953990784 learned_addr learned my addr 172.21.15.116:0/1953990784 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a3e00640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 0x7f00a4142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f009b400640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 0x7f00a413e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 -- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 msgr2=0x7f00a413e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 0x7f00a413e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 -- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 msgr2=0x7f00a4142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 0x7f00a4142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 -- 172.21.15.116:0/1953990784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a413eac0 con 0x7f00a415eb60 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f009b400640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 0x7f00a413e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a3e00640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 0x7f00a4142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a9000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f009c00efc0 tx=0x7f009c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c018020 con 0x7f00a415eb60 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c014480 con 0x7f00a415eb60 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c009e60 con 0x7f00a415eb60 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a4134b80 con 0x7f00a415eb60 2024-05-14T11:20:07.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a4135140 con 0x7f00a415eb60 2024-05-14T11:20:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a406a9c0 con 0x7f00a415eb60 2024-05-14T11:20:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f009c01b020 con 0x7f00a415eb60 2024-05-14T11:20:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.406+0000 7f00a0c00640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 0x7f0080071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.410+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f009c0901e0 con 0x7f00a415eb60 2024-05-14T11:20:07.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.410+0000 7f00a3e00640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 0x7f0080071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.410+0000 7f00a3e00640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 0x7f0080071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0094000f70 tx=0x7f0094007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.414+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f009c022030 con 0x7f00a415eb60 2024-05-14T11:20:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.602+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00a4134910 con 0x7f00a415eb60 2024-05-14T11:20:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.602+0000 7f00a0c00640 1 -- 172.21.15.116:0/1953990784 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7f009c010040 con 0x7f00a415eb60 2024-05-14T11:20:07.603 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:07.603 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:06.615674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2430466110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2430466110},{"type":"v1","addr":"172.21.15.116:6836","nonce":2430466110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6213,"qdb_cluster":[6213]},"id":5}]} 2024-05-14T11:20:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 45 2024-05-14T11:20:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 msgr2=0x7f0080071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 0x7f0080071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0094000f70 tx=0x7f0094007d50 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 msgr2=0x7f00a413bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f009c00efc0 tx=0x7f009c00c490 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 shutdown_connections 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f008006f400 0x7f0080071910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00a415eb60 0x7f00a413bc90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00a41432c0 0x7f00a413e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 --2- 172.21.15.116:0/1953990784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00a40cdac0 0x7f00a4142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 >> 172.21.15.116:0/1953990784 conn(0x7f00a40ce6b0 msgr2=0x7f00a415d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 shutdown_connections 2024-05-14T11:20:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.606+0000 7f00ab000640 1 -- 172.21.15.116:0/1953990784 wait complete. 2024-05-14T11:20:07.625 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:20:07.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 Processor -- start 2024-05-14T11:20:07.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 -- start start 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f39d4166750 0x7f39d4168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d406ff20 con 0x7f39d40d1670 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d4070470 con 0x7f39d415db30 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39dbc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f39d406d710 con 0x7f39d4166750 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39cbe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f39d40d1670 0x7f39d40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36504/0 (socket says 172.21.15.116:36504) 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39cbe00640 1 -- 172.21.15.116:0/86207971 learned_addr learned my addr 172.21.15.116:0/86207971 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d9c00640 1 --2- 172.21.15.116:0/86207971 >> v2:172.21.15.116:3300/0 conn(0x7f39d4166750 0x7f39d4168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d3e00640 1 -- 172.21.15.116:0/86207971 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555135025 0 0) 0x7f39d406ff20 con 0x7f39d40d1670 2024-05-14T11:20:07.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d3e00640 1 -- 172.21.15.116:0/86207971 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b8003530 con 0x7f39d40d1670 2024-05-14T11:20:07.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d9c00640 1 -- 172.21.15.116:0/86207971 >> v1:172.21.15.149:6789/0 conn(0x7f39d415db30 legacy=0x7f39d40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d9c00640 1 -- 172.21.15.116:0/86207971 >> v1:172.21.15.116:6789/0 conn(0x7f39d40d1670 legacy=0x7f39d40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.774+0000 7f39d9c00640 1 -- 172.21.15.116:0/86207971 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4070750 con 0x7f39d4166750 2024-05-14T11:20:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39d9c00640 1 --2- 172.21.15.116:0/86207971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d4168b70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009d90 tx=0x7f39cc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfa27d4ccb23508 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39d3e00640 1 -- 172.21.15.116:0/86207971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0b2070 con 0x7f39d4166750 2024-05-14T11:20:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39d3e00640 1 -- 172.21.15.116:0/86207971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0a5710 con 0x7f39d4166750 2024-05-14T11:20:07.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39d3e00640 1 -- 172.21.15.116:0/86207971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0a5ae0 con 0x7f39d4166750 2024-05-14T11:20:07.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 -- 172.21.15.116:0/86207971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 msgr2=0x7f39d4168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 --2- 172.21.15.116:0/86207971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d4168b70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009d90 tx=0x7f39cc01c850 comp rx=0 tx=0).stop 2024-05-14T11:20:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 -- 172.21.15.116:0/86207971 shutdown_connections 2024-05-14T11:20:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 --2- 172.21.15.116:0/86207971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d4168b70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 -- 172.21.15.116:0/86207971 >> 172.21.15.116:0/86207971 conn(0x7f39d40b4620 msgr2=0x7f39d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 -- 172.21.15.116:0/86207971 shutdown_connections 2024-05-14T11:20:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.778+0000 7f39dbc00640 1 -- 172.21.15.116:0/86207971 wait complete. 2024-05-14T11:20:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 Processor -- start 2024-05-14T11:20:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 -- start start 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f39d40d1670 0x7f39d40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d415db30 0x7f39d41602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f39d416bae0 con 0x7f39d4166750 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40ae870 con 0x7f39d415db30 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39dbc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f39d416bc60 con 0x7f39d40d1670 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49706/0 (socket says 172.21.15.116:49706) 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 -- 172.21.15.116:0/2911934098 learned_addr learned my addr 172.21.15.116:0/2911934098 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:07.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9200640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f39d40d1670 0x7f39d40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39cbe00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d415db30 0x7f39d41602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 -- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f39d40d1670 msgr2=0x7f39d40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f39d40d1670 0x7f39d40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 -- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d415db30 msgr2=0x7f39d41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d415db30 0x7f39d41602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 -- 172.21.15.116:0/2911934098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40b2100 con 0x7f39d4166750 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.782+0000 7f39d9c00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01a480 tx=0x7f39cc0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0bd020 con 0x7f39d4166750 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0b8470 con 0x7f39d4166750 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0b2040 con 0x7f39d4166750 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d40b23e0 con 0x7f39d4166750 2024-05-14T11:20:07.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d40b29a0 con 0x7f39d4166750 2024-05-14T11:20:07.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d40cc5e0 con 0x7f39d4166750 2024-05-14T11:20:07.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39cc0c0020 con 0x7f39d4166750 2024-05-14T11:20:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 0x7f39b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f39cc12a4b0 con 0x7f39d4166750 2024-05-14T11:20:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.786+0000 7f39d9200640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 0x7f39b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.790+0000 7f39d9200640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 0x7f39b0071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001650 tx=0x7f39c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:07.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.794+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f39cc0c7030 con 0x7f39d4166750 2024-05-14T11:20:07.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.950+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39d406a9c0 con 0x7f39d4166750 2024-05-14T11:20:07.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.950+0000 7f39d1600640 1 -- 172.21.15.116:0/2911934098 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16993 (secure 0 0 0) 0x7f39cc016560 con 0x7f39d4166750 2024-05-14T11:20:07.951 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:07.952 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":72,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:20:05.194898+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:20:01.282761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-14T11:20:02.552345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 msgr2=0x7f39b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 0x7f39b0071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001650 tx=0x7f39c4009040 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 msgr2=0x7f39d40b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f39cc01a480 tx=0x7f39cc0195f0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 shutdown_connections 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39b006f400 0x7f39b0071910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39d415db30 0x7f39d41602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39d4166750 0x7f39d40b2ec0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 --2- 172.21.15.116:0/2911934098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f39d40d1670 0x7f39d40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:07.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 >> 172.21.15.116:0/2911934098 conn(0x7f39d40b4620 msgr2=0x7f39d40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 shutdown_connections 2024-05-14T11:20:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:07.954+0000 7f39dbc00640 1 -- 172.21.15.116:0/2911934098 wait complete. 2024-05-14T11:20:07.974 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:20:07.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:20:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 Processor -- start 2024-05-14T11:20:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 -- start start 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7faf580d15e0 0x7faf580d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faf5806ead0 0x7faf5806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf580d1f30 con 0x7faf5806c700 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7faf5815ce20 con 0x7faf5806ead0 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ec00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7faf5806f850 con 0x7faf580d15e0 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faf5806ead0 0x7faf5806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faf5806ead0 0x7faf5806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49714/0 (socket says 172.21.15.116:49714) 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 -- 172.21.15.116:0/2290547409 learned_addr learned my addr 172.21.15.116:0/2290547409 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5d800640 1 --2- 172.21.15.116:0/2290547409 >> v2:172.21.15.149:3300/0 conn(0x7faf580d15e0 0x7faf580d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 -- 172.21.15.116:0/2290547409 >> v2:172.21.15.149:3300/0 conn(0x7faf580d15e0 msgr2=0x7faf580d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:08.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 --2- 172.21.15.116:0/2290547409 >> v2:172.21.15.149:3300/0 conn(0x7faf580d15e0 0x7faf580d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 -- 172.21.15.116:0/2290547409 >> v1:172.21.15.149:6789/0 conn(0x7faf5806c700 legacy=0x7faf5806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.122+0000 7faf5ce00640 1 -- 172.21.15.116:0/2290547409 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5806fb30 con 0x7faf5806ead0 2024-05-14T11:20:08.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ce00640 1 --2- 172.21.15.116:0/2290547409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf5806ead0 0x7faf5806eee0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faf580d22b0 tx=0x7faf4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02d788e91d97cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf57e00640 1 -- 172.21.15.116:0/2290547409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440aa680 con 0x7faf5806ead0 2024-05-14T11:20:08.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf57e00640 1 -- 172.21.15.116:0/2290547409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf440af070 con 0x7faf5806ead0 2024-05-14T11:20:08.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf57e00640 1 -- 172.21.15.116:0/2290547409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440aaa40 con 0x7faf5806ead0 2024-05-14T11:20:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 -- 172.21.15.116:0/2290547409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf5806ead0 msgr2=0x7faf5806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 --2- 172.21.15.116:0/2290547409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf5806ead0 0x7faf5806eee0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faf580d22b0 tx=0x7faf4401c520 comp rx=0 tx=0).stop 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 -- 172.21.15.116:0/2290547409 shutdown_connections 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 --2- 172.21.15.116:0/2290547409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf5806ead0 0x7faf5806eee0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 --2- 172.21.15.116:0/2290547409 >> v2:172.21.15.149:3300/0 conn(0x7faf580d15e0 0x7faf580d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 -- 172.21.15.116:0/2290547409 >> 172.21.15.116:0/2290547409 conn(0x7faf580b4530 msgr2=0x7faf580b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 -- 172.21.15.116:0/2290547409 shutdown_connections 2024-05-14T11:20:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.126+0000 7faf5ec00640 1 -- 172.21.15.116:0/2290547409 wait complete. 2024-05-14T11:20:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 Processor -- start 2024-05-14T11:20:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 -- start start 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faf5806c700 0x7faf580c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf5806ead0 0x7faf581648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faf5806f5a0 con 0x7faf580d15e0 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf5806f420 con 0x7faf5806ead0 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7faf58167d70 con 0x7faf5806c700 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49722/0 (socket says 172.21.15.116:49722) 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 -- 172.21.15.116:0/1388055521 learned_addr learned my addr 172.21.15.116:0/1388055521 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ce00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf5806ead0 0x7faf581648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5d800640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faf5806c700 0x7faf580c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 -- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faf5806c700 msgr2=0x7faf580c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faf5806c700 0x7faf580c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 -- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf5806ead0 msgr2=0x7faf581648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf5806ead0 0x7faf581648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 -- 172.21.15.116:0/1388055521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf58164e00 con 0x7faf580d15e0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5e200640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c3c0 tx=0x7faf5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50017070 con 0x7faf580d15e0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf500052d0 con 0x7faf580d15e0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf58158580 con 0x7faf580d15e0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.130+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50003800 con 0x7faf580d15e0 2024-05-14T11:20:08.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf58168f80 con 0x7faf580d15e0 2024-05-14T11:20:08.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf58050880 con 0x7faf580d15e0 2024-05-14T11:20:08.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faf5001a070 con 0x7faf580d15e0 2024-05-14T11:20:08.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf55600640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 0x7faf38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7faf5008e6b0 con 0x7faf580d15e0 2024-05-14T11:20:08.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf5d800640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 0x7faf38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.134+0000 7faf5d800640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 0x7faf38071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011340 tx=0x7faf4c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.138+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7faf580d1f30 con 0x7faf580d15e0 2024-05-14T11:20:08.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.298+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf580cdf40 con 0x7faf3806f400 2024-05-14T11:20:08.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.306+0000 7faf55600640 1 -- 172.21.15.116:0/1388055521 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252024 (secure 0 0 0) 0x7faf580cdf40 con 0x7faf3806f400 2024-05-14T11:20:08.307 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:08.309 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:20:08.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 msgr2=0x7faf38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 0x7faf38071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011340 tx=0x7faf4c00b040 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 msgr2=0x7faf58158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c3c0 tx=0x7faf5000c880 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 shutdown_connections 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7faf3806f400 0x7faf38071910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf5806ead0 0x7faf581648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faf580d15e0 0x7faf58158040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 --2- 172.21.15.116:0/1388055521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faf5806c700 0x7faf580c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 >> 172.21.15.116:0/1388055521 conn(0x7faf580b4530 msgr2=0x7faf580b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 shutdown_connections 2024-05-14T11:20:08.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.310+0000 7faf5ec00640 1 -- 172.21.15.116:0/1388055521 wait complete. 2024-05-14T11:20:08.328 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-05-14T11:20:06.416145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253328,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721392,"statfs":{"total":773094113280,"available":772834705408,"internally_reserved":0,"allocated":8200192,"data_stored":4327940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.243974"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.207918+0000","last_change":"2024-05-14T11:20:04.202773+0000","last_active":"2024-05-14T11:20:04.207918+0000","last_peered":"2024-05-14T11:20:04.207918+0000","last_clean":"2024-05-14T11:20:04.207918+0000","last_became_active":"2024-05-14T11:20:04.201262+0000","last_became_peered":"2024-05-14T11:20:04.201262+0000","last_unstale":"2024-05-14T11:20:04.207918+0000","last_undegraded":"2024-05-14T11:20:04.207918+0000","last_fullsized":"2024-05-14T11:20:04.207918+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.229556+0000","last_change":"2024-05-14T11:20:04.224117+0000","last_active":"2024-05-14T11:20:04.229556+0000","last_peered":"2024-05-14T11:20:04.229556+0000","last_clean":"2024-05-14T11:20:04.229556+0000","last_became_active":"2024-05-14T11:20:04.223446+0000","last_became_peered":"2024-05-14T11:20:04.223446+0000","last_unstale":"2024-05-14T11:20:04.229556+0000","last_undegraded":"2024-05-14T11:20:04.229556+0000","last_fullsized":"2024-05-14T11:20:04.229556+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207957+0000","last_change":"2024-05-14T11:20:04.203925+0000","last_active":"2024-05-14T11:20:05.207957+0000","last_peered":"2024-05-14T11:20:05.207957+0000","last_clean":"2024-05-14T11:20:05.207957+0000","last_became_active":"2024-05-14T11:20:04.203367+0000","last_became_peered":"2024-05-14T11:20:04.203367+0000","last_unstale":"2024-05-14T11:20:05.207957+0000","last_undegraded":"2024-05-14T11:20:05.207957+0000","last_fullsized":"2024-05-14T11:20:05.207957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:32:38.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275109+0000","last_change":"2024-05-14T11:20:04.219364+0000","last_active":"2024-05-14T11:20:05.275109+0000","last_peered":"2024-05-14T11:20:05.275109+0000","last_clean":"2024-05-14T11:20:05.275109+0000","last_became_active":"2024-05-14T11:20:04.218960+0000","last_became_peered":"2024-05-14T11:20:04.218960+0000","last_unstale":"2024-05-14T11:20:05.275109+0000","last_undegraded":"2024-05-14T11:20:05.275109+0000","last_fullsized":"2024-05-14T11:20:05.275109+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222024+0000","last_change":"2024-05-14T11:20:04.209951+0000","last_active":"2024-05-14T11:20:04.222024+0000","last_peered":"2024-05-14T11:20:04.222024+0000","last_clean":"2024-05-14T11:20:04.222024+0000","last_became_active":"2024-05-14T11:20:04.209604+0000","last_became_peered":"2024-05-14T11:20:04.209604+0000","last_unstale":"2024-05-14T11:20:04.222024+0000","last_undegraded":"2024-05-14T11:20:04.222024+0000","last_fullsized":"2024-05-14T11:20:04.222024+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:02:57.705222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223990+0000","last_change":"2024-05-14T11:20:04.217499+0000","last_active":"2024-05-14T11:20:04.223990+0000","last_peered":"2024-05-14T11:20:04.223990+0000","last_clean":"2024-05-14T11:20:04.223990+0000","last_became_active":"2024-05-14T11:20:04.216910+0000","last_became_peered":"2024-05-14T11:20:04.216910+0000","last_unstale":"2024-05-14T11:20:04.223990+0000","last_undegraded":"2024-05-14T11:20:04.223990+0000","last_fullsized":"2024-05-14T11:20:04.223990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211416+0000","last_change":"2024-05-14T11:20:04.222869+0000","last_active":"2024-05-14T11:20:05.211416+0000","last_peered":"2024-05-14T11:20:05.211416+0000","last_clean":"2024-05-14T11:20:05.211416+0000","last_became_active":"2024-05-14T11:20:04.222494+0000","last_became_peered":"2024-05-14T11:20:04.222494+0000","last_unstale":"2024-05-14T11:20:05.211416+0000","last_undegraded":"2024-05-14T11:20:05.211416+0000","last_fullsized":"2024-05-14T11:20:05.211416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:00.460929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.224869+0000","last_change":"2024-05-14T11:20:04.210629+0000","last_active":"2024-05-14T11:20:04.224869+0000","last_peered":"2024-05-14T11:20:04.224869+0000","last_clean":"2024-05-14T11:20:04.224869+0000","last_became_active":"2024-05-14T11:20:04.210354+0000","last_became_peered":"2024-05-14T11:20:04.210354+0000","last_unstale":"2024-05-14T11:20:04.224869+0000","last_undegraded":"2024-05-14T11:20:04.224869+0000","last_fullsized":"2024-05-14T11:20:04.224869+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:46.148240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213229+0000","last_change":"2024-05-14T11:20:04.220330+0000","last_active":"2024-05-14T11:20:05.213229+0000","last_peered":"2024-05-14T11:20:05.213229+0000","last_clean":"2024-05-14T11:20:05.213229+0000","last_became_active":"2024-05-14T11:20:04.219965+0000","last_became_peered":"2024-05-14T11:20:04.219965+0000","last_unstale":"2024-05-14T11:20:05.213229+0000","last_undegraded":"2024-05-14T11:20:05.213229+0000","last_fullsized":"2024-05-14T11:20:05.213229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:07:42.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275000+0000","last_change":"2024-05-14T11:20:04.209043+0000","last_active":"2024-05-14T11:20:05.275000+0000","last_peered":"2024-05-14T11:20:05.275000+0000","last_clean":"2024-05-14T11:20:05.275000+0000","last_became_active":"2024-05-14T11:20:04.208717+0000","last_became_peered":"2024-05-14T11:20:04.208717+0000","last_unstale":"2024-05-14T11:20:05.275000+0000","last_undegraded":"2024-05-14T11:20:05.275000+0000","last_fullsized":"2024-05-14T11:20:05.275000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228260+0000","last_change":"2024-05-14T11:20:04.224912+0000","last_active":"2024-05-14T11:20:04.228260+0000","last_peered":"2024-05-14T11:20:04.228260+0000","last_clean":"2024-05-14T11:20:04.228260+0000","last_became_active":"2024-05-14T11:20:04.224364+0000","last_became_peered":"2024-05-14T11:20:04.224364+0000","last_unstale":"2024-05-14T11:20:04.228260+0000","last_undegraded":"2024-05-14T11:20:04.228260+0000","last_fullsized":"2024-05-14T11:20:04.228260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:21:43.903087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275091+0000","last_change":"2024-05-14T11:20:04.213592+0000","last_active":"2024-05-14T11:20:05.275091+0000","last_peered":"2024-05-14T11:20:05.275091+0000","last_clean":"2024-05-14T11:20:05.275091+0000","last_became_active":"2024-05-14T11:20:04.213249+0000","last_became_peered":"2024-05-14T11:20:04.213249+0000","last_unstale":"2024-05-14T11:20:05.275091+0000","last_undegraded":"2024-05-14T11:20:05.275091+0000","last_fullsized":"2024-05-14T11:20:05.275091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225695+0000","last_change":"2024-05-14T11:20:04.219496+0000","last_active":"2024-05-14T11:20:04.225695+0000","last_peered":"2024-05-14T11:20:04.225695+0000","last_clean":"2024-05-14T11:20:04.225695+0000","last_became_active":"2024-05-14T11:20:04.219114+0000","last_became_peered":"2024-05-14T11:20:04.219114+0000","last_unstale":"2024-05-14T11:20:04.225695+0000","last_undegraded":"2024-05-14T11:20:04.225695+0000","last_fullsized":"2024-05-14T11:20:04.225695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:08.006324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223967+0000","last_change":"2024-05-14T11:20:04.217319+0000","last_active":"2024-05-14T11:20:04.223967+0000","last_peered":"2024-05-14T11:20:04.223967+0000","last_clean":"2024-05-14T11:20:04.223967+0000","last_became_active":"2024-05-14T11:20:04.216623+0000","last_became_peered":"2024-05-14T11:20:04.216623+0000","last_unstale":"2024-05-14T11:20:04.223967+0000","last_undegraded":"2024-05-14T11:20:04.223967+0000","last_fullsized":"2024-05-14T11:20:04.223967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223089+0000","last_change":"2024-05-14T11:20:04.214903+0000","last_active":"2024-05-14T11:20:04.223089+0000","last_peered":"2024-05-14T11:20:04.223089+0000","last_clean":"2024-05-14T11:20:04.223089+0000","last_became_active":"2024-05-14T11:20:04.214524+0000","last_became_peered":"2024-05-14T11:20:04.214524+0000","last_unstale":"2024-05-14T11:20:04.223089+0000","last_undegraded":"2024-05-14T11:20:04.223089+0000","last_fullsized":"2024-05-14T11:20:04.223089+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.205905+0000","last_change":"2024-05-14T11:20:04.203981+0000","last_active":"2024-05-14T11:20:04.205905+0000","last_peered":"2024-05-14T11:20:04.205905+0000","last_clean":"2024-05-14T11:20:04.205905+0000","last_became_active":"2024-05-14T11:20:04.203651+0000","last_became_peered":"2024-05-14T11:20:04.203651+0000","last_unstale":"2024-05-14T11:20:04.205905+0000","last_undegraded":"2024-05-14T11:20:04.205905+0000","last_fullsized":"2024-05-14T11:20:04.205905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225599+0000","last_change":"2024-05-14T11:20:04.212080+0000","last_active":"2024-05-14T11:20:04.225599+0000","last_peered":"2024-05-14T11:20:04.225599+0000","last_clean":"2024-05-14T11:20:04.225599+0000","last_became_active":"2024-05-14T11:20:04.211749+0000","last_became_peered":"2024-05-14T11:20:04.211749+0000","last_unstale":"2024-05-14T11:20:04.225599+0000","last_undegraded":"2024-05-14T11:20:04.225599+0000","last_fullsized":"2024-05-14T11:20:04.225599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:09.681689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207985+0000","last_change":"2024-05-14T11:20:04.194987+0000","last_active":"2024-05-14T11:20:05.207985+0000","last_peered":"2024-05-14T11:20:05.207985+0000","last_clean":"2024-05-14T11:20:05.207985+0000","last_became_active":"2024-05-14T11:20:04.194648+0000","last_became_peered":"2024-05-14T11:20:04.194648+0000","last_unstale":"2024-05-14T11:20:05.207985+0000","last_undegraded":"2024-05-14T11:20:05.207985+0000","last_fullsized":"2024-05-14T11:20:05.207985+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:47:06.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.230503+0000","last_change":"2024-05-14T11:20:04.228326+0000","last_active":"2024-05-14T11:20:04.230503+0000","last_peered":"2024-05-14T11:20:04.230503+0000","last_clean":"2024-05-14T11:20:04.230503+0000","last_became_active":"2024-05-14T11:20:04.227647+0000","last_became_peered":"2024-05-14T11:20:04.227647+0000","last_unstale":"2024-05-14T11:20:04.230503+0000","last_undegraded":"2024-05-14T11:20:04.230503+0000","last_fullsized":"2024-05-14T11:20:04.230503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222054+0000","last_change":"2024-05-14T11:20:04.207574+0000","last_active":"2024-05-14T11:20:04.222054+0000","last_peered":"2024-05-14T11:20:04.222054+0000","last_clean":"2024-05-14T11:20:04.222054+0000","last_became_active":"2024-05-14T11:20:04.207255+0000","last_became_peered":"2024-05-14T11:20:04.207255+0000","last_unstale":"2024-05-14T11:20:04.222054+0000","last_undegraded":"2024-05-14T11:20:04.222054+0000","last_fullsized":"2024-05-14T11:20:04.222054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:02:05.964643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211699+0000","last_change":"2024-05-14T11:20:04.209017+0000","last_active":"2024-05-14T11:20:05.211699+0000","last_peered":"2024-05-14T11:20:05.211699+0000","last_clean":"2024-05-14T11:20:05.211699+0000","last_became_active":"2024-05-14T11:20:04.208734+0000","last_became_peered":"2024-05-14T11:20:04.208734+0000","last_unstale":"2024-05-14T11:20:05.211699+0000","last_undegraded":"2024-05-14T11:20:05.211699+0000","last_fullsized":"2024-05-14T11:20:05.211699+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:24.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.231371+0000","last_change":"2024-05-14T11:20:04.228904+0000","last_active":"2024-05-14T11:20:04.231371+0000","last_peered":"2024-05-14T11:20:04.231371+0000","last_clean":"2024-05-14T11:20:04.231371+0000","last_became_active":"2024-05-14T11:20:04.228327+0000","last_became_peered":"2024-05-14T11:20:04.228327+0000","last_unstale":"2024-05-14T11:20:04.231371+0000","last_undegraded":"2024-05-14T11:20:04.231371+0000","last_fullsized":"2024-05-14T11:20:04.231371+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:31.257252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212121+0000","last_change":"2024-05-14T11:20:04.203730+0000","last_active":"2024-05-14T11:20:05.212121+0000","last_peered":"2024-05-14T11:20:05.212121+0000","last_clean":"2024-05-14T11:20:05.212121+0000","last_became_active":"2024-05-14T11:20:04.203355+0000","last_became_peered":"2024-05-14T11:20:04.203355+0000","last_unstale":"2024-05-14T11:20:05.212121+0000","last_undegraded":"2024-05-14T11:20:05.212121+0000","last_fullsized":"2024-05-14T11:20:05.212121+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:28.617457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.226372+0000","last_change":"2024-05-14T11:20:04.223004+0000","last_active":"2024-05-14T11:20:04.226372+0000","last_peered":"2024-05-14T11:20:04.226372+0000","last_clean":"2024-05-14T11:20:04.226372+0000","last_became_active":"2024-05-14T11:20:04.222020+0000","last_became_peered":"2024-05-14T11:20:04.222020+0000","last_unstale":"2024-05-14T11:20:04.226372+0000","last_undegraded":"2024-05-14T11:20:04.226372+0000","last_fullsized":"2024-05-14T11:20:04.226372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228146+0000","last_change":"2024-05-14T11:20:04.224261+0000","last_active":"2024-05-14T11:20:04.228146+0000","last_peered":"2024-05-14T11:20:04.228146+0000","last_clean":"2024-05-14T11:20:04.228146+0000","last_became_active":"2024-05-14T11:20:04.223552+0000","last_became_peered":"2024-05-14T11:20:04.223552+0000","last_unstale":"2024-05-14T11:20:04.228146+0000","last_undegraded":"2024-05-14T11:20:04.228146+0000","last_fullsized":"2024-05-14T11:20:04.228146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:00.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275108+0000","last_change":"2024-05-14T11:20:04.214671+0000","last_active":"2024-05-14T11:20:05.275108+0000","last_peered":"2024-05-14T11:20:05.275108+0000","last_clean":"2024-05-14T11:20:05.275108+0000","last_became_active":"2024-05-14T11:20:04.214377+0000","last_became_peered":"2024-05-14T11:20:04.214377+0000","last_unstale":"2024-05-14T11:20:05.275108+0000","last_undegraded":"2024-05-14T11:20:05.275108+0000","last_fullsized":"2024-05-14T11:20:05.275108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.217052+0000","last_change":"2024-05-14T11:20:04.213156+0000","last_active":"2024-05-14T11:20:04.217052+0000","last_peered":"2024-05-14T11:20:04.217052+0000","last_clean":"2024-05-14T11:20:04.217052+0000","last_became_active":"2024-05-14T11:20:04.212860+0000","last_became_peered":"2024-05-14T11:20:04.212860+0000","last_unstale":"2024-05-14T11:20:04.217052+0000","last_undegraded":"2024-05-14T11:20:04.217052+0000","last_fullsized":"2024-05-14T11:20:04.217052+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225647+0000","last_change":"2024-05-14T11:20:04.214006+0000","last_active":"2024-05-14T11:20:04.225647+0000","last_peered":"2024-05-14T11:20:04.225647+0000","last_clean":"2024-05-14T11:20:04.225647+0000","last_became_active":"2024-05-14T11:20:04.213675+0000","last_became_peered":"2024-05-14T11:20:04.213675+0000","last_unstale":"2024-05-14T11:20:04.225647+0000","last_undegraded":"2024-05-14T11:20:04.225647+0000","last_fullsized":"2024-05-14T11:20:04.225647+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:10.356205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222846+0000","last_change":"2024-05-14T11:20:04.213649+0000","last_active":"2024-05-14T11:20:04.222846+0000","last_peered":"2024-05-14T11:20:04.222846+0000","last_clean":"2024-05-14T11:20:04.222846+0000","last_became_active":"2024-05-14T11:20:04.213282+0000","last_became_peered":"2024-05-14T11:20:04.213282+0000","last_unstale":"2024-05-14T11:20:04.222846+0000","last_undegraded":"2024-05-14T11:20:04.222846+0000","last_fullsized":"2024-05-14T11:20:04.222846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.211211+0000","last_change":"2024-05-14T11:20:04.207797+0000","last_active":"2024-05-14T11:20:04.211211+0000","last_peered":"2024-05-14T11:20:04.211211+0000","last_clean":"2024-05-14T11:20:04.211211+0000","last_became_active":"2024-05-14T11:20:04.207410+0000","last_became_peered":"2024-05-14T11:20:04.207410+0000","last_unstale":"2024-05-14T11:20:04.211211+0000","last_undegraded":"2024-05-14T11:20:04.211211+0000","last_fullsized":"2024-05-14T11:20:04.211211+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207634+0000","last_change":"2024-05-14T11:20:04.194301+0000","last_active":"2024-05-14T11:20:05.207634+0000","last_peered":"2024-05-14T11:20:05.207634+0000","last_clean":"2024-05-14T11:20:05.207634+0000","last_became_active":"2024-05-14T11:20:04.193967+0000","last_became_peered":"2024-05-14T11:20:04.193967+0000","last_unstale":"2024-05-14T11:20:05.207634+0000","last_undegraded":"2024-05-14T11:20:05.207634+0000","last_fullsized":"2024-05-14T11:20:05.207634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:07:42.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.204730+0000","last_change":"2024-05-14T11:20:04.201818+0000","last_active":"2024-05-14T11:20:04.204730+0000","last_peered":"2024-05-14T11:20:04.204730+0000","last_clean":"2024-05-14T11:20:04.204730+0000","last_became_active":"2024-05-14T11:20:04.201433+0000","last_became_peered":"2024-05-14T11:20:04.201433+0000","last_unstale":"2024-05-14T11:20:04.204730+0000","last_undegraded":"2024-05-14T11:20:04.204730+0000","last_fullsized":"2024-05-14T11:20:04.204730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207693+0000","last_change":"2024-05-14T11:20:04.196149+0000","last_active":"2024-05-14T11:20:05.207693+0000","last_peered":"2024-05-14T11:20:05.207693+0000","last_clean":"2024-05-14T11:20:05.207693+0000","last_became_active":"2024-05-14T11:20:04.195832+0000","last_became_peered":"2024-05-14T11:20:04.195832+0000","last_unstale":"2024-05-14T11:20:05.207693+0000","last_undegraded":"2024-05-14T11:20:05.207693+0000","last_fullsized":"2024-05-14T11:20:05.207693+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:00.460929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.227386+0000","last_change":"2024-05-14T11:20:03.207601+0000","last_active":"2024-05-14T11:20:04.227386+0000","last_peered":"2024-05-14T11:20:04.227386+0000","last_clean":"2024-05-14T11:20:04.227386+0000","last_became_active":"2024-05-14T11:20:03.207243+0000","last_became_peered":"2024-05-14T11:20:03.207243+0000","last_unstale":"2024-05-14T11:20:04.227386+0000","last_undegraded":"2024-05-14T11:20:04.227386+0000","last_fullsized":"2024-05-14T11:20:04.227386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:35.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223116+0000","last_change":"2024-05-14T11:20:04.217480+0000","last_active":"2024-05-14T11:20:04.223116+0000","last_peered":"2024-05-14T11:20:04.223116+0000","last_clean":"2024-05-14T11:20:04.223116+0000","last_became_active":"2024-05-14T11:20:04.217139+0000","last_became_peered":"2024-05-14T11:20:04.217139+0000","last_unstale":"2024-05-14T11:20:04.223116+0000","last_undegraded":"2024-05-14T11:20:04.223116+0000","last_fullsized":"2024-05-14T11:20:04.223116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.214538+0000","last_change":"2024-05-14T11:20:03.206104+0000","last_active":"2024-05-14T11:20:04.214538+0000","last_peered":"2024-05-14T11:20:04.214538+0000","last_clean":"2024-05-14T11:20:04.214538+0000","last_became_active":"2024-05-14T11:20:03.205810+0000","last_became_peered":"2024-05-14T11:20:03.205810+0000","last_unstale":"2024-05-14T11:20:04.214538+0000","last_undegraded":"2024-05-14T11:20:04.214538+0000","last_fullsized":"2024-05-14T11:20:04.214538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:21.616464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213098+0000","last_change":"2024-05-14T11:20:04.220756+0000","last_active":"2024-05-14T11:20:05.213098+0000","last_peered":"2024-05-14T11:20:05.213098+0000","last_clean":"2024-05-14T11:20:05.213098+0000","last_became_active":"2024-05-14T11:20:04.219980+0000","last_became_peered":"2024-05-14T11:20:04.219980+0000","last_unstale":"2024-05-14T11:20:05.213098+0000","last_undegraded":"2024-05-14T11:20:05.213098+0000","last_fullsized":"2024-05-14T11:20:05.213098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:52.277990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.215759+0000","last_change":"2024-05-14T11:20:03.207642+0000","last_active":"2024-05-14T11:20:04.215759+0000","last_peered":"2024-05-14T11:20:04.215759+0000","last_clean":"2024-05-14T11:20:04.215759+0000","last_became_active":"2024-05-14T11:20:03.207327+0000","last_became_peered":"2024-05-14T11:20:03.207327+0000","last_unstale":"2024-05-14T11:20:04.215759+0000","last_undegraded":"2024-05-14T11:20:04.215759+0000","last_fullsized":"2024-05-14T11:20:04.215759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212077+0000","last_change":"2024-05-14T11:20:04.206542+0000","last_active":"2024-05-14T11:20:05.212077+0000","last_peered":"2024-05-14T11:20:05.212077+0000","last_clean":"2024-05-14T11:20:05.212077+0000","last_became_active":"2024-05-14T11:20:04.206220+0000","last_became_peered":"2024-05-14T11:20:04.206220+0000","last_unstale":"2024-05-14T11:20:05.212077+0000","last_undegraded":"2024-05-14T11:20:05.212077+0000","last_fullsized":"2024-05-14T11:20:05.212077+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:43.692745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212645+0000","last_change":"2024-05-14T11:20:03.202155+0000","last_active":"2024-05-14T11:20:05.212645+0000","last_peered":"2024-05-14T11:20:05.212645+0000","last_clean":"2024-05-14T11:20:05.212645+0000","last_became_active":"2024-05-14T11:20:03.201797+0000","last_became_peered":"2024-05-14T11:20:03.201797+0000","last_unstale":"2024-05-14T11:20:05.212645+0000","last_undegraded":"2024-05-14T11:20:05.212645+0000","last_fullsized":"2024-05-14T11:20:05.212645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:38:58.772273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228190+0000","last_change":"2024-05-14T11:20:04.225577+0000","last_active":"2024-05-14T11:20:04.228190+0000","last_peered":"2024-05-14T11:20:04.228190+0000","last_clean":"2024-05-14T11:20:04.228190+0000","last_became_active":"2024-05-14T11:20:04.225145+0000","last_became_peered":"2024-05-14T11:20:04.225145+0000","last_unstale":"2024-05-14T11:20:04.228190+0000","last_undegraded":"2024-05-14T11:20:04.228190+0000","last_fullsized":"2024-05-14T11:20:04.228190+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:49.990226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211845+0000","last_change":"2024-05-14T11:20:03.200524+0000","last_active":"2024-05-14T11:20:05.211845+0000","last_peered":"2024-05-14T11:20:05.211845+0000","last_clean":"2024-05-14T11:20:05.211845+0000","last_became_active":"2024-05-14T11:20:03.200138+0000","last_became_peered":"2024-05-14T11:20:03.200138+0000","last_unstale":"2024-05-14T11:20:05.211845+0000","last_undegraded":"2024-05-14T11:20:05.211845+0000","last_fullsized":"2024-05-14T11:20:05.211845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:42:49.328097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228236+0000","last_change":"2024-05-14T11:20:04.224657+0000","last_active":"2024-05-14T11:20:04.228236+0000","last_peered":"2024-05-14T11:20:04.228236+0000","last_clean":"2024-05-14T11:20:04.228236+0000","last_became_active":"2024-05-14T11:20:04.223962+0000","last_became_peered":"2024-05-14T11:20:04.223962+0000","last_unstale":"2024-05-14T11:20:04.228236+0000","last_undegraded":"2024-05-14T11:20:04.228236+0000","last_fullsized":"2024-05-14T11:20:04.228236+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:57:16.632330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.274792+0000","last_change":"2024-05-14T11:20:03.211912+0000","last_active":"2024-05-14T11:20:05.274792+0000","last_peered":"2024-05-14T11:20:05.274792+0000","last_clean":"2024-05-14T11:20:05.274792+0000","last_became_active":"2024-05-14T11:20:03.211585+0000","last_became_peered":"2024-05-14T11:20:03.211585+0000","last_unstale":"2024-05-14T11:20:05.274792+0000","last_undegraded":"2024-05-14T11:20:05.274792+0000","last_fullsized":"2024-05-14T11:20:05.274792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:18.898658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213208+0000","last_change":"2024-05-14T11:20:04.220351+0000","last_active":"2024-05-14T11:20:05.213208+0000","last_peered":"2024-05-14T11:20:05.213208+0000","last_clean":"2024-05-14T11:20:05.213208+0000","last_became_active":"2024-05-14T11:20:04.219770+0000","last_became_peered":"2024-05-14T11:20:04.219770+0000","last_unstale":"2024-05-14T11:20:05.213208+0000","last_undegraded":"2024-05-14T11:20:05.213208+0000","last_fullsized":"2024-05-14T11:20:05.213208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:24.324434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.193246+0000","last_change":"2024-05-14T11:20:03.203723+0000","last_active":"2024-05-14T11:20:04.193246+0000","last_peered":"2024-05-14T11:20:04.193246+0000","last_clean":"2024-05-14T11:20:04.193246+0000","last_became_active":"2024-05-14T11:20:03.203393+0000","last_became_peered":"2024-05-14T11:20:03.203393+0000","last_unstale":"2024-05-14T11:20:04.193246+0000","last_undegraded":"2024-05-14T11:20:04.193246+0000","last_fullsized":"2024-05-14T11:20:04.193246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212915+0000","last_change":"2024-05-14T11:20:04.210575+0000","last_active":"2024-05-14T11:20:05.212915+0000","last_peered":"2024-05-14T11:20:05.212915+0000","last_clean":"2024-05-14T11:20:05.212915+0000","last_became_active":"2024-05-14T11:20:04.209508+0000","last_became_peered":"2024-05-14T11:20:04.209508+0000","last_unstale":"2024-05-14T11:20:05.212915+0000","last_undegraded":"2024-05-14T11:20:05.212915+0000","last_fullsized":"2024-05-14T11:20:05.212915+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:04.327882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.197882+0000","last_change":"2024-05-14T11:20:03.201338+0000","last_active":"2024-05-14T11:20:04.197882+0000","last_peered":"2024-05-14T11:20:04.197882+0000","last_clean":"2024-05-14T11:20:04.197882+0000","last_became_active":"2024-05-14T11:20:03.201025+0000","last_became_peered":"2024-05-14T11:20:03.201025+0000","last_unstale":"2024-05-14T11:20:04.197882+0000","last_undegraded":"2024-05-14T11:20:04.197882+0000","last_fullsized":"2024-05-14T11:20:04.197882+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:12.193652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.208688+0000","last_change":"2024-05-14T11:20:04.204618+0000","last_active":"2024-05-14T11:20:04.208688+0000","last_peered":"2024-05-14T11:20:04.208688+0000","last_clean":"2024-05-14T11:20:04.208688+0000","last_became_active":"2024-05-14T11:20:04.204311+0000","last_became_peered":"2024-05-14T11:20:04.204311+0000","last_unstale":"2024-05-14T11:20:04.208688+0000","last_undegraded":"2024-05-14T11:20:04.208688+0000","last_fullsized":"2024-05-14T11:20:04.208688+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:41.886312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.193264+0000","last_change":"2024-05-14T11:20:03.206041+0000","last_active":"2024-05-14T11:20:04.193264+0000","last_peered":"2024-05-14T11:20:04.193264+0000","last_clean":"2024-05-14T11:20:04.193264+0000","last_became_active":"2024-05-14T11:20:03.205724+0000","last_became_peered":"2024-05-14T11:20:03.205724+0000","last_unstale":"2024-05-14T11:20:04.193264+0000","last_undegraded":"2024-05-14T11:20:04.193264+0000","last_fullsized":"2024-05-14T11:20:04.193264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:31.846994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275046+0000","last_change":"2024-05-14T11:20:04.212077+0000","last_active":"2024-05-14T11:20:05.275046+0000","last_peered":"2024-05-14T11:20:05.275046+0000","last_clean":"2024-05-14T11:20:05.275046+0000","last_became_active":"2024-05-14T11:20:04.211758+0000","last_became_peered":"2024-05-14T11:20:04.211758+0000","last_unstale":"2024-05-14T11:20:05.275046+0000","last_undegraded":"2024-05-14T11:20:05.275046+0000","last_fullsized":"2024-05-14T11:20:05.275046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275123+0000","last_change":"2024-05-14T11:20:03.202985+0000","last_active":"2024-05-14T11:20:05.275123+0000","last_peered":"2024-05-14T11:20:05.275123+0000","last_clean":"2024-05-14T11:20:05.275123+0000","last_became_active":"2024-05-14T11:20:03.202678+0000","last_became_peered":"2024-05-14T11:20:03.202678+0000","last_unstale":"2024-05-14T11:20:05.275123+0000","last_undegraded":"2024-05-14T11:20:05.275123+0000","last_fullsized":"2024-05-14T11:20:05.275123+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:08.571014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223942+0000","last_change":"2024-05-14T11:20:04.217236+0000","last_active":"2024-05-14T11:20:04.223942+0000","last_peered":"2024-05-14T11:20:04.223942+0000","last_clean":"2024-05-14T11:20:04.223942+0000","last_became_active":"2024-05-14T11:20:04.216285+0000","last_became_peered":"2024-05-14T11:20:04.216285+0000","last_unstale":"2024-05-14T11:20:04.223942+0000","last_undegraded":"2024-05-14T11:20:04.223942+0000","last_fullsized":"2024-05-14T11:20:04.223942+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:32.849960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225765+0000","last_change":"2024-05-14T11:20:03.209170+0000","last_active":"2024-05-14T11:20:04.225765+0000","last_peered":"2024-05-14T11:20:04.225765+0000","last_clean":"2024-05-14T11:20:04.225765+0000","last_became_active":"2024-05-14T11:20:03.208826+0000","last_became_peered":"2024-05-14T11:20:03.208826+0000","last_unstale":"2024-05-14T11:20:04.225765+0000","last_undegraded":"2024-05-14T11:20:04.225765+0000","last_fullsized":"2024-05-14T11:20:04.225765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:55.303200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207745+0000","last_change":"2024-05-14T11:20:04.202809+0000","last_active":"2024-05-14T11:20:05.207745+0000","last_peered":"2024-05-14T11:20:05.207745+0000","last_clean":"2024-05-14T11:20:05.207745+0000","last_became_active":"2024-05-14T11:20:04.202520+0000","last_became_peered":"2024-05-14T11:20:04.202520+0000","last_unstale":"2024-05-14T11:20:05.207745+0000","last_undegraded":"2024-05-14T11:20:05.207745+0000","last_fullsized":"2024-05-14T11:20:05.207745+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:49:25.840325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212724+0000","last_change":"2024-05-14T11:20:03.200622+0000","last_active":"2024-05-14T11:20:05.212724+0000","last_peered":"2024-05-14T11:20:05.212724+0000","last_clean":"2024-05-14T11:20:05.212724+0000","last_became_active":"2024-05-14T11:20:03.200328+0000","last_became_peered":"2024-05-14T11:20:03.200328+0000","last_unstale":"2024-05-14T11:20:05.212724+0000","last_undegraded":"2024-05-14T11:20:05.212724+0000","last_fullsized":"2024-05-14T11:20:05.212724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:52.776153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275284+0000","last_change":"2024-05-14T11:20:04.223251+0000","last_active":"2024-05-14T11:20:05.275284+0000","last_peered":"2024-05-14T11:20:05.275284+0000","last_clean":"2024-05-14T11:20:05.275284+0000","last_became_active":"2024-05-14T11:20:04.222982+0000","last_became_peered":"2024-05-14T11:20:04.222982+0000","last_unstale":"2024-05-14T11:20:05.275284+0000","last_undegraded":"2024-05-14T11:20:05.275284+0000","last_fullsized":"2024-05-14T11:20:05.275284+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:00.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.274832+0000","last_change":"2024-05-14T11:20:03.205900+0000","last_active":"2024-05-14T11:20:05.274832+0000","last_peered":"2024-05-14T11:20:05.274832+0000","last_clean":"2024-05-14T11:20:05.274832+0000","last_became_active":"2024-05-14T11:20:03.204983+0000","last_became_peered":"2024-05-14T11:20:03.204983+0000","last_unstale":"2024-05-14T11:20:05.274832+0000","last_undegraded":"2024-05-14T11:20:05.274832+0000","last_fullsized":"2024-05-14T11:20:05.274832+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:10.782361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212336+0000","last_change":"2024-05-14T11:20:04.200713+0000","last_active":"2024-05-14T11:20:05.212336+0000","last_peered":"2024-05-14T11:20:05.212336+0000","last_clean":"2024-05-14T11:20:05.212336+0000","last_became_active":"2024-05-14T11:20:04.200396+0000","last_became_peered":"2024-05-14T11:20:04.200396+0000","last_unstale":"2024-05-14T11:20:05.212336+0000","last_undegraded":"2024-05-14T11:20:05.212336+0000","last_fullsized":"2024-05-14T11:20:05.212336+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:32:38.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191803+0000","last_change":"2024-05-14T11:20:03.209105+0000","last_active":"2024-05-14T11:20:04.191803+0000","last_peered":"2024-05-14T11:20:04.191803+0000","last_clean":"2024-05-14T11:20:04.191803+0000","last_became_active":"2024-05-14T11:20:03.208813+0000","last_became_peered":"2024-05-14T11:20:03.208813+0000","last_unstale":"2024-05-14T11:20:04.191803+0000","last_undegraded":"2024-05-14T11:20:04.191803+0000","last_fullsized":"2024-05-14T11:20:04.191803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212590+0000","last_change":"2024-05-14T11:20:04.197781+0000","last_active":"2024-05-14T11:20:05.212590+0000","last_peered":"2024-05-14T11:20:05.212590+0000","last_clean":"2024-05-14T11:20:05.212590+0000","last_became_active":"2024-05-14T11:20:04.196851+0000","last_became_peered":"2024-05-14T11:20:04.196851+0000","last_unstale":"2024-05-14T11:20:05.212590+0000","last_undegraded":"2024-05-14T11:20:05.212590+0000","last_fullsized":"2024-05-14T11:20:05.212590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:35.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198020+0000","last_change":"2024-05-14T11:20:03.209831+0000","last_active":"2024-05-14T11:20:04.198020+0000","last_peered":"2024-05-14T11:20:04.198020+0000","last_clean":"2024-05-14T11:20:04.198020+0000","last_became_active":"2024-05-14T11:20:03.209141+0000","last_became_peered":"2024-05-14T11:20:03.209141+0000","last_unstale":"2024-05-14T11:20:04.198020+0000","last_undegraded":"2024-05-14T11:20:04.198020+0000","last_fullsized":"2024-05-14T11:20:04.198020+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:33.206842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212946+0000","last_change":"2024-05-14T11:20:04.218744+0000","last_active":"2024-05-14T11:20:05.212946+0000","last_peered":"2024-05-14T11:20:05.212946+0000","last_clean":"2024-05-14T11:20:05.212946+0000","last_became_active":"2024-05-14T11:20:04.218390+0000","last_became_peered":"2024-05-14T11:20:04.218390+0000","last_unstale":"2024-05-14T11:20:05.212946+0000","last_undegraded":"2024-05-14T11:20:05.212946+0000","last_fullsized":"2024-05-14T11:20:05.212946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:25.876727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.226344+0000","last_change":"2024-05-14T11:20:03.197088+0000","last_active":"2024-05-14T11:20:04.226344+0000","last_peered":"2024-05-14T11:20:04.226344+0000","last_clean":"2024-05-14T11:20:04.226344+0000","last_became_active":"2024-05-14T11:20:03.196017+0000","last_became_peered":"2024-05-14T11:20:03.196017+0000","last_unstale":"2024-05-14T11:20:04.226344+0000","last_undegraded":"2024-05-14T11:20:04.226344+0000","last_fullsized":"2024-05-14T11:20:04.226344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:06.057974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.235947+0000","last_change":"2024-05-14T11:20:04.230080+0000","last_active":"2024-05-14T11:20:04.235947+0000","last_peered":"2024-05-14T11:20:04.235947+0000","last_clean":"2024-05-14T11:20:04.235947+0000","last_became_active":"2024-05-14T11:20:04.224638+0000","last_became_peered":"2024-05-14T11:20:04.224638+0000","last_unstale":"2024-05-14T11:20:04.235947+0000","last_undegraded":"2024-05-14T11:20:04.235947+0000","last_fullsized":"2024-05-14T11:20:04.235947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.211138+0000","last_change":"2024-05-14T11:20:03.210778+0000","last_active":"2024-05-14T11:20:04.211138+0000","last_peered":"2024-05-14T11:20:04.211138+0000","last_clean":"2024-05-14T11:20:04.211138+0000","last_became_active":"2024-05-14T11:20:03.210403+0000","last_became_peered":"2024-05-14T11:20:03.210403+0000","last_unstale":"2024-05-14T11:20:04.211138+0000","last_undegraded":"2024-05-14T11:20:04.211138+0000","last_fullsized":"2024-05-14T11:20:04.211138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:40.883346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228232+0000","last_change":"2024-05-14T11:20:04.222687+0000","last_active":"2024-05-14T11:20:04.228232+0000","last_peered":"2024-05-14T11:20:04.228232+0000","last_clean":"2024-05-14T11:20:04.228232+0000","last_became_active":"2024-05-14T11:20:04.221813+0000","last_became_peered":"2024-05-14T11:20:04.221813+0000","last_unstale":"2024-05-14T11:20:04.228232+0000","last_undegraded":"2024-05-14T11:20:04.228232+0000","last_fullsized":"2024-05-14T11:20:04.228232+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:54.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212250+0000","last_change":"2024-05-14T11:20:03.206067+0000","last_active":"2024-05-14T11:20:05.212250+0000","last_peered":"2024-05-14T11:20:05.212250+0000","last_clean":"2024-05-14T11:20:05.212250+0000","last_became_active":"2024-05-14T11:20:03.205778+0000","last_became_peered":"2024-05-14T11:20:03.205778+0000","last_unstale":"2024-05-14T11:20:05.212250+0000","last_undegraded":"2024-05-14T11:20:05.212250+0000","last_fullsized":"2024-05-14T11:20:05.212250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:51.058200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.221990+0000","last_change":"2024-05-14T11:20:04.207600+0000","last_active":"2024-05-14T11:20:04.221990+0000","last_peered":"2024-05-14T11:20:04.221990+0000","last_clean":"2024-05-14T11:20:04.221990+0000","last_became_active":"2024-05-14T11:20:04.207211+0000","last_became_peered":"2024-05-14T11:20:04.207211+0000","last_unstale":"2024-05-14T11:20:04.221990+0000","last_undegraded":"2024-05-14T11:20:04.221990+0000","last_fullsized":"2024-05-14T11:20:04.221990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:58.918256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225285+0000","last_change":"2024-05-14T11:20:03.202064+0000","last_active":"2024-05-14T11:20:04.225285+0000","last_peered":"2024-05-14T11:20:04.225285+0000","last_clean":"2024-05-14T11:20:04.225285+0000","last_became_active":"2024-05-14T11:20:03.201766+0000","last_became_peered":"2024-05-14T11:20:03.201766+0000","last_unstale":"2024-05-14T11:20:04.225285+0000","last_undegraded":"2024-05-14T11:20:04.225285+0000","last_fullsized":"2024-05-14T11:20:04.225285+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:17.496082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.230539+0000","last_change":"2024-05-14T11:20:04.229610+0000","last_active":"2024-05-14T11:20:04.230539+0000","last_peered":"2024-05-14T11:20:04.230539+0000","last_clean":"2024-05-14T11:20:04.230539+0000","last_became_active":"2024-05-14T11:20:04.224620+0000","last_became_peered":"2024-05-14T11:20:04.224620+0000","last_unstale":"2024-05-14T11:20:04.230539+0000","last_undegraded":"2024-05-14T11:20:04.230539+0000","last_fullsized":"2024-05-14T11:20:04.230539+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207836+0000","last_change":"2024-05-14T11:20:03.213245+0000","last_active":"2024-05-14T11:20:05.207836+0000","last_peered":"2024-05-14T11:20:05.207836+0000","last_clean":"2024-05-14T11:20:05.207836+0000","last_became_active":"2024-05-14T11:20:03.212890+0000","last_became_peered":"2024-05-14T11:20:03.212890+0000","last_unstale":"2024-05-14T11:20:05.207836+0000","last_undegraded":"2024-05-14T11:20:05.207836+0000","last_fullsized":"2024-05-14T11:20:05.207836+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:52.776153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.219824+0000","last_change":"2024-05-14T11:20:04.213655+0000","last_active":"2024-05-14T11:20:04.219824+0000","last_peered":"2024-05-14T11:20:04.219824+0000","last_clean":"2024-05-14T11:20:04.219824+0000","last_became_active":"2024-05-14T11:20:04.213368+0000","last_became_peered":"2024-05-14T11:20:04.213368+0000","last_unstale":"2024-05-14T11:20:04.219824+0000","last_undegraded":"2024-05-14T11:20:04.219824+0000","last_fullsized":"2024-05-14T11:20:04.219824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.224131+0000","last_change":"2024-05-14T11:20:03.207110+0000","last_active":"2024-05-14T11:20:04.224131+0000","last_peered":"2024-05-14T11:20:04.224131+0000","last_clean":"2024-05-14T11:20:04.224131+0000","last_became_active":"2024-05-14T11:20:03.206731+0000","last_became_peered":"2024-05-14T11:20:03.206731+0000","last_unstale":"2024-05-14T11:20:04.224131+0000","last_undegraded":"2024-05-14T11:20:04.224131+0000","last_fullsized":"2024-05-14T11:20:04.224131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.209143+0000","last_change":"2024-05-14T11:20:04.204062+0000","last_active":"2024-05-14T11:20:04.209143+0000","last_peered":"2024-05-14T11:20:04.209143+0000","last_clean":"2024-05-14T11:20:04.209143+0000","last_became_active":"2024-05-14T11:20:04.203759+0000","last_became_peered":"2024-05-14T11:20:04.203759+0000","last_unstale":"2024-05-14T11:20:04.209143+0000","last_undegraded":"2024-05-14T11:20:04.209143+0000","last_fullsized":"2024-05-14T11:20:04.209143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:49.966784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.208389+0000","last_change":"2024-05-14T11:20:03.199866+0000","last_active":"2024-05-14T11:20:04.208389+0000","last_peered":"2024-05-14T11:20:04.208389+0000","last_clean":"2024-05-14T11:20:04.208389+0000","last_became_active":"2024-05-14T11:20:03.199571+0000","last_became_peered":"2024-05-14T11:20:03.199571+0000","last_unstale":"2024-05-14T11:20:04.208389+0000","last_undegraded":"2024-05-14T11:20:04.208389+0000","last_fullsized":"2024-05-14T11:20:04.208389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:40.883346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213021+0000","last_change":"2024-05-14T11:20:04.212434+0000","last_active":"2024-05-14T11:20:05.213021+0000","last_peered":"2024-05-14T11:20:05.213021+0000","last_clean":"2024-05-14T11:20:05.213021+0000","last_became_active":"2024-05-14T11:20:04.211306+0000","last_became_peered":"2024-05-14T11:20:04.211306+0000","last_unstale":"2024-05-14T11:20:05.213021+0000","last_undegraded":"2024-05-14T11:20:05.213021+0000","last_fullsized":"2024-05-14T11:20:05.213021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:36.011861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198386+0000","last_change":"2024-05-14T11:20:03.200502+0000","last_active":"2024-05-14T11:20:04.198386+0000","last_peered":"2024-05-14T11:20:04.198386+0000","last_clean":"2024-05-14T11:20:04.198386+0000","last_became_active":"2024-05-14T11:20:03.200218+0000","last_became_peered":"2024-05-14T11:20:03.200218+0000","last_unstale":"2024-05-14T11:20:04.198386+0000","last_undegraded":"2024-05-14T11:20:04.198386+0000","last_fullsized":"2024-05-14T11:20:04.198386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211459+0000","last_change":"2024-05-14T11:20:04.193627+0000","last_active":"2024-05-14T11:20:05.211459+0000","last_peered":"2024-05-14T11:20:05.211459+0000","last_clean":"2024-05-14T11:20:05.211459+0000","last_became_active":"2024-05-14T11:20:04.193168+0000","last_became_peered":"2024-05-14T11:20:04.193168+0000","last_unstale":"2024-05-14T11:20:05.211459+0000","last_undegraded":"2024-05-14T11:20:05.211459+0000","last_fullsized":"2024-05-14T11:20:05.211459+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:16:21.864624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275237+0000","last_change":"2024-05-14T11:20:03.210614+0000","last_active":"2024-05-14T11:20:05.275237+0000","last_peered":"2024-05-14T11:20:05.275237+0000","last_clean":"2024-05-14T11:20:05.275237+0000","last_became_active":"2024-05-14T11:20:03.210304+0000","last_became_peered":"2024-05-14T11:20:03.210304+0000","last_unstale":"2024-05-14T11:20:05.275237+0000","last_undegraded":"2024-05-14T11:20:05.275237+0000","last_fullsized":"2024-05-14T11:20:05.275237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:21.616464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211785+0000","last_change":"2024-05-14T11:20:04.211434+0000","last_active":"2024-05-14T11:20:05.211785+0000","last_peered":"2024-05-14T11:20:05.211785+0000","last_clean":"2024-05-14T11:20:05.211785+0000","last_became_active":"2024-05-14T11:20:04.211082+0000","last_became_peered":"2024-05-14T11:20:04.211082+0000","last_unstale":"2024-05-14T11:20:05.211785+0000","last_undegraded":"2024-05-14T11:20:05.211785+0000","last_fullsized":"2024-05-14T11:20:05.211785+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:49:25.840325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.194159+0000","last_change":"2024-05-14T11:20:03.199010+0000","last_active":"2024-05-14T11:20:04.194159+0000","last_peered":"2024-05-14T11:20:04.194159+0000","last_clean":"2024-05-14T11:20:04.194159+0000","last_became_active":"2024-05-14T11:20:03.198711+0000","last_became_peered":"2024-05-14T11:20:03.198711+0000","last_unstale":"2024-05-14T11:20:04.194159+0000","last_undegraded":"2024-05-14T11:20:04.194159+0000","last_fullsized":"2024-05-14T11:20:04.194159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212861+0000","last_change":"2024-05-14T11:20:04.219615+0000","last_active":"2024-05-14T11:20:05.212861+0000","last_peered":"2024-05-14T11:20:05.212861+0000","last_clean":"2024-05-14T11:20:05.212861+0000","last_became_active":"2024-05-14T11:20:04.219278+0000","last_became_peered":"2024-05-14T11:20:04.219278+0000","last_unstale":"2024-05-14T11:20:05.212861+0000","last_undegraded":"2024-05-14T11:20:05.212861+0000","last_fullsized":"2024-05-14T11:20:05.212861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:51.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.192555+0000","last_change":"2024-05-14T11:20:03.205481+0000","last_active":"2024-05-14T11:20:04.192555+0000","last_peered":"2024-05-14T11:20:04.192555+0000","last_clean":"2024-05-14T11:20:04.192555+0000","last_became_active":"2024-05-14T11:20:03.205141+0000","last_became_peered":"2024-05-14T11:20:03.205141+0000","last_unstale":"2024-05-14T11:20:04.192555+0000","last_undegraded":"2024-05-14T11:20:04.192555+0000","last_fullsized":"2024-05-14T11:20:04.192555+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:48.963819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222832+0000","last_change":"2024-05-14T11:20:04.211305+0000","last_active":"2024-05-14T11:20:04.222832+0000","last_peered":"2024-05-14T11:20:04.222832+0000","last_clean":"2024-05-14T11:20:04.222832+0000","last_became_active":"2024-05-14T11:20:04.211002+0000","last_became_peered":"2024-05-14T11:20:04.211002+0000","last_unstale":"2024-05-14T11:20:04.222832+0000","last_undegraded":"2024-05-14T11:20:04.222832+0000","last_fullsized":"2024-05-14T11:20:04.222832+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:32.849960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191783+0000","last_change":"2024-05-14T11:20:03.204552+0000","last_active":"2024-05-14T11:20:04.191783+0000","last_peered":"2024-05-14T11:20:04.191783+0000","last_clean":"2024-05-14T11:20:04.191783+0000","last_became_active":"2024-05-14T11:20:03.204251+0000","last_became_peered":"2024-05-14T11:20:03.204251+0000","last_unstale":"2024-05-14T11:20:04.191783+0000","last_undegraded":"2024-05-14T11:20:04.191783+0000","last_fullsized":"2024-05-14T11:20:04.191783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:06.057974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212514+0000","last_change":"2024-05-14T11:20:04.217189+0000","last_active":"2024-05-14T11:20:05.212514+0000","last_peered":"2024-05-14T11:20:05.212514+0000","last_clean":"2024-05-14T11:20:05.212514+0000","last_became_active":"2024-05-14T11:20:04.216636+0000","last_became_peered":"2024-05-14T11:20:04.216636+0000","last_unstale":"2024-05-14T11:20:05.212514+0000","last_undegraded":"2024-05-14T11:20:05.212514+0000","last_fullsized":"2024-05-14T11:20:05.212514+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:55:02.889890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212110+0000","last_change":"2024-05-14T11:20:03.202016+0000","last_active":"2024-05-14T11:20:05.212110+0000","last_peered":"2024-05-14T11:20:05.212110+0000","last_clean":"2024-05-14T11:20:05.212110+0000","last_became_active":"2024-05-14T11:20:03.201696+0000","last_became_peered":"2024-05-14T11:20:03.201696+0000","last_unstale":"2024-05-14T11:20:05.212110+0000","last_undegraded":"2024-05-14T11:20:05.212110+0000","last_fullsized":"2024-05-14T11:20:05.212110+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:45:45.591695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212498+0000","last_change":"2024-05-14T11:20:04.215582+0000","last_active":"2024-05-14T11:20:05.212498+0000","last_peered":"2024-05-14T11:20:05.212498+0000","last_clean":"2024-05-14T11:20:05.212498+0000","last_became_active":"2024-05-14T11:20:04.215243+0000","last_became_peered":"2024-05-14T11:20:04.215243+0000","last_unstale":"2024-05-14T11:20:05.212498+0000","last_undegraded":"2024-05-14T11:20:05.212498+0000","last_fullsized":"2024-05-14T11:20:05.212498+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:47:06.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198464+0000","last_change":"2024-05-14T11:20:03.213477+0000","last_active":"2024-05-14T11:20:04.198464+0000","last_peered":"2024-05-14T11:20:04.198464+0000","last_clean":"2024-05-14T11:20:04.198464+0000","last_became_active":"2024-05-14T11:20:03.212174+0000","last_became_peered":"2024-05-14T11:20:03.212174+0000","last_unstale":"2024-05-14T11:20:04.198464+0000","last_undegraded":"2024-05-14T11:20:04.198464+0000","last_fullsized":"2024-05-14T11:20:04.198464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:05.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.208009+0000","last_change":"2024-05-14T11:20:04.203849+0000","last_active":"2024-05-14T11:20:05.208009+0000","last_peered":"2024-05-14T11:20:05.208009+0000","last_clean":"2024-05-14T11:20:05.208009+0000","last_became_active":"2024-05-14T11:20:04.203553+0000","last_became_peered":"2024-05-14T11:20:04.203553+0000","last_unstale":"2024-05-14T11:20:05.208009+0000","last_undegraded":"2024-05-14T11:20:05.208009+0000","last_fullsized":"2024-05-14T11:20:05.208009+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:55:02.889890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211532+0000","last_change":"2024-05-14T11:20:03.200988+0000","last_active":"2024-05-14T11:20:05.211532+0000","last_peered":"2024-05-14T11:20:05.211532+0000","last_clean":"2024-05-14T11:20:05.211532+0000","last_became_active":"2024-05-14T11:20:03.199743+0000","last_became_peered":"2024-05-14T11:20:03.199743+0000","last_unstale":"2024-05-14T11:20:05.211532+0000","last_undegraded":"2024-05-14T11:20:05.211532+0000","last_fullsized":"2024-05-14T11:20:05.211532+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:51.584569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.208144+0000","last_change":"2024-05-14T11:20:04.201338+0000","last_active":"2024-05-14T11:20:05.208144+0000","last_peered":"2024-05-14T11:20:05.208144+0000","last_clean":"2024-05-14T11:20:05.208144+0000","last_became_active":"2024-05-14T11:20:04.200861+0000","last_became_peered":"2024-05-14T11:20:04.200861+0000","last_unstale":"2024-05-14T11:20:05.208144+0000","last_undegraded":"2024-05-14T11:20:05.208144+0000","last_fullsized":"2024-05-14T11:20:05.208144+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:35.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207497+0000","last_change":"2024-05-14T11:20:03.213018+0000","last_active":"2024-05-14T11:20:05.207497+0000","last_peered":"2024-05-14T11:20:05.207497+0000","last_clean":"2024-05-14T11:20:05.207497+0000","last_became_active":"2024-05-14T11:20:03.212685+0000","last_became_peered":"2024-05-14T11:20:03.212685+0000","last_unstale":"2024-05-14T11:20:05.207497+0000","last_undegraded":"2024-05-14T11:20:05.207497+0000","last_fullsized":"2024-05-14T11:20:05.207497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:02:03.606870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.213039+0000","last_change":"2024-05-14T11:20:04.209316+0000","last_active":"2024-05-14T11:20:04.213039+0000","last_peered":"2024-05-14T11:20:04.213039+0000","last_clean":"2024-05-14T11:20:04.213039+0000","last_became_active":"2024-05-14T11:20:04.207163+0000","last_became_peered":"2024-05-14T11:20:04.207163+0000","last_unstale":"2024-05-14T11:20:04.213039+0000","last_undegraded":"2024-05-14T11:20:04.213039+0000","last_fullsized":"2024-05-14T11:20:04.213039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198618+0000","last_change":"2024-05-14T11:20:03.204942+0000","last_active":"2024-05-14T11:20:04.198618+0000","last_peered":"2024-05-14T11:20:04.198618+0000","last_clean":"2024-05-14T11:20:04.198618+0000","last_became_active":"2024-05-14T11:20:03.204239+0000","last_became_peered":"2024-05-14T11:20:03.204239+0000","last_unstale":"2024-05-14T11:20:04.198618+0000","last_undegraded":"2024-05-14T11:20:04.198618+0000","last_fullsized":"2024-05-14T11:20:04.198618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:48.963819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":202,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191928+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:20:04.191928+0000","last_peered":"2024-05-14T11:20:04.191928+0000","last_clean":"2024-05-14T11:20:04.191928+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:20:04.191928+0000","last_undegraded":"2024-05-14T11:20:04.191928+0000","last_fullsized":"2024-05-14T11:20:04.191928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":983188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":394410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":392832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":888832,"data_stored":394514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":393380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":393540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":392910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":983166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:20:08.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:20:08.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.518+0000 7fe321400640 1 Processor -- start 2024-05-14T11:20:08.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.518+0000 7fe321400640 1 -- start start 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe321400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe31c06f730 0x7fe31c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe321400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31c070080 con 0x7fe31c072a30 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe321400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31c06cf10 con 0x7fe31c0b0770 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe321400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe31c06db20 con 0x7fe31c06f730 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe31c06f730 0x7fe31c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe31b200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe31c072a30 0x7fe31c072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36512/0 (socket says 172.21.15.116:36512) 2024-05-14T11:20:08.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe31b200640 1 -- 172.21.15.116:0/3192452972 learned_addr learned my addr 172.21.15.116:0/3192452972 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:08.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319400640 1 -- 172.21.15.116:0/3192452972 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264432570 0 0) 0x7fe3140014a0 con 0x7fe31c072a30 2024-05-14T11:20:08.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319400640 1 -- 172.21.15.116:0/3192452972 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe300003530 con 0x7fe31c072a30 2024-05-14T11:20:08.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319e00640 1 -- 172.21.15.116:0/3192452972 >> v1:172.21.15.149:6789/0 conn(0x7fe31c0b0770 legacy=0x7fe31c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:08.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319e00640 1 -- 172.21.15.116:0/3192452972 >> v1:172.21.15.116:6789/0 conn(0x7fe31c072a30 legacy=0x7fe31c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.522+0000 7fe319e00640 1 -- 172.21.15.116:0/3192452972 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c06d1f0 con 0x7fe31c06f730 2024-05-14T11:20:08.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe319e00640 1 --2- 172.21.15.116:0/3192452972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c06f730 0x7fe31c06fb40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe3080097c0 tx=0x7fe30801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1de81b4c6c929d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe319400640 1 -- 172.21.15.116:0/3192452972 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080b1070 con 0x7fe31c06f730 2024-05-14T11:20:08.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe319400640 1 -- 172.21.15.116:0/3192452972 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080a8050 con 0x7fe31c06f730 2024-05-14T11:20:08.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe319400640 1 -- 172.21.15.116:0/3192452972 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080ac860 con 0x7fe31c06f730 2024-05-14T11:20:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 -- 172.21.15.116:0/3192452972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c06f730 msgr2=0x7fe31c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 --2- 172.21.15.116:0/3192452972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c06f730 0x7fe31c06fb40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe3080097c0 tx=0x7fe30801c3e0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 -- 172.21.15.116:0/3192452972 shutdown_connections 2024-05-14T11:20:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 --2- 172.21.15.116:0/3192452972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c06f730 0x7fe31c06fb40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 -- 172.21.15.116:0/3192452972 >> 172.21.15.116:0/3192452972 conn(0x7fe31c0b4620 msgr2=0x7fe31c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:08.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 -- 172.21.15.116:0/3192452972 shutdown_connections 2024-05-14T11:20:08.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 -- 172.21.15.116:0/3192452972 wait complete. 2024-05-14T11:20:08.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.526+0000 7fe321400640 1 Processor -- start 2024-05-14T11:20:08.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- start start 2024-05-14T11:20:08.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe31c06f730 0x7fe31c09bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe31c0b0770 0x7fe31c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c073ae0 con 0x7fe31c072a30 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c073960 con 0x7fe31c0b0770 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe31c0d1670 con 0x7fe31c06f730 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49734/0 (socket says 172.21.15.116:49734) 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 -- 172.21.15.116:0/1026776575 learned_addr learned my addr 172.21.15.116:0/1026776575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe31a800640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe31c06f730 0x7fe31c09bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe31b200640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe31c0b0770 0x7fe31c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 -- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe31c06f730 msgr2=0x7fe31c09bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe31c06f730 0x7fe31c09bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 -- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe31c0b0770 msgr2=0x7fe31c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe31c0b0770 0x7fe31c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 -- 172.21.15.116:0/1026776575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c1fc0a0 con 0x7fe31c072a30 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe319e00640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe30801a010 tx=0x7fe30801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080b1040 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080bc070 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080b7690 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c157950 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c157f10 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.530+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31c09f150 con 0x7fe31c072a30 2024-05-14T11:20:08.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.534+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe3080b7830 con 0x7fe31c072a30 2024-05-14T11:20:08.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.534+0000 7fe312a00640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 0x7fe2fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.534+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fe30812a2d0 con 0x7fe31c072a30 2024-05-14T11:20:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.534+0000 7fe31a800640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 0x7fe2fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.534+0000 7fe31a800640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 0x7fe2fc071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0d1240 tx=0x7fe30c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.538+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe308016090 con 0x7fe31c072a30 2024-05-14T11:20:08.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.690+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe31c1fc380 con 0x7fe2fc06f400 2024-05-14T11:20:08.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.698+0000 7fe312a00640 1 -- 172.21.15.116:0/1026776575 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252024 (secure 0 0 0) 0x7fe31c1fc380 con 0x7fe2fc06f400 2024-05-14T11:20:08.699 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:08.700 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 msgr2=0x7fe2fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 0x7fe2fc071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0d1240 tx=0x7fe30c008040 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 msgr2=0x7fe31c158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe30801a010 tx=0x7fe30801ac20 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 shutdown_connections 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe2fc06f400 0x7fe2fc071910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe31c0b0770 0x7fe31c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe31c072a30 0x7fe31c158430 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 --2- 172.21.15.116:0/1026776575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe31c06f730 0x7fe31c09bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 >> 172.21.15.116:0/1026776575 conn(0x7fe31c0b4620 msgr2=0x7fe31c15fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 shutdown_connections 2024-05-14T11:20:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.702+0000 7fe321400640 1 -- 172.21.15.116:0/1026776575 wait complete. 2024-05-14T11:20:08.720 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-05-14T11:20:08.416630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253328,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721392,"statfs":{"total":773094113280,"available":772834705408,"internally_reserved":0,"allocated":8200192,"data_stored":4327940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001104"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.207918+0000","last_change":"2024-05-14T11:20:04.202773+0000","last_active":"2024-05-14T11:20:04.207918+0000","last_peered":"2024-05-14T11:20:04.207918+0000","last_clean":"2024-05-14T11:20:04.207918+0000","last_became_active":"2024-05-14T11:20:04.201262+0000","last_became_peered":"2024-05-14T11:20:04.201262+0000","last_unstale":"2024-05-14T11:20:04.207918+0000","last_undegraded":"2024-05-14T11:20:04.207918+0000","last_fullsized":"2024-05-14T11:20:04.207918+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.229556+0000","last_change":"2024-05-14T11:20:04.224117+0000","last_active":"2024-05-14T11:20:04.229556+0000","last_peered":"2024-05-14T11:20:04.229556+0000","last_clean":"2024-05-14T11:20:04.229556+0000","last_became_active":"2024-05-14T11:20:04.223446+0000","last_became_peered":"2024-05-14T11:20:04.223446+0000","last_unstale":"2024-05-14T11:20:04.229556+0000","last_undegraded":"2024-05-14T11:20:04.229556+0000","last_fullsized":"2024-05-14T11:20:04.229556+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207957+0000","last_change":"2024-05-14T11:20:04.203925+0000","last_active":"2024-05-14T11:20:05.207957+0000","last_peered":"2024-05-14T11:20:05.207957+0000","last_clean":"2024-05-14T11:20:05.207957+0000","last_became_active":"2024-05-14T11:20:04.203367+0000","last_became_peered":"2024-05-14T11:20:04.203367+0000","last_unstale":"2024-05-14T11:20:05.207957+0000","last_undegraded":"2024-05-14T11:20:05.207957+0000","last_fullsized":"2024-05-14T11:20:05.207957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:32:38.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275109+0000","last_change":"2024-05-14T11:20:04.219364+0000","last_active":"2024-05-14T11:20:05.275109+0000","last_peered":"2024-05-14T11:20:05.275109+0000","last_clean":"2024-05-14T11:20:05.275109+0000","last_became_active":"2024-05-14T11:20:04.218960+0000","last_became_peered":"2024-05-14T11:20:04.218960+0000","last_unstale":"2024-05-14T11:20:05.275109+0000","last_undegraded":"2024-05-14T11:20:05.275109+0000","last_fullsized":"2024-05-14T11:20:05.275109+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222024+0000","last_change":"2024-05-14T11:20:04.209951+0000","last_active":"2024-05-14T11:20:04.222024+0000","last_peered":"2024-05-14T11:20:04.222024+0000","last_clean":"2024-05-14T11:20:04.222024+0000","last_became_active":"2024-05-14T11:20:04.209604+0000","last_became_peered":"2024-05-14T11:20:04.209604+0000","last_unstale":"2024-05-14T11:20:04.222024+0000","last_undegraded":"2024-05-14T11:20:04.222024+0000","last_fullsized":"2024-05-14T11:20:04.222024+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:02:57.705222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223990+0000","last_change":"2024-05-14T11:20:04.217499+0000","last_active":"2024-05-14T11:20:04.223990+0000","last_peered":"2024-05-14T11:20:04.223990+0000","last_clean":"2024-05-14T11:20:04.223990+0000","last_became_active":"2024-05-14T11:20:04.216910+0000","last_became_peered":"2024-05-14T11:20:04.216910+0000","last_unstale":"2024-05-14T11:20:04.223990+0000","last_undegraded":"2024-05-14T11:20:04.223990+0000","last_fullsized":"2024-05-14T11:20:04.223990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211416+0000","last_change":"2024-05-14T11:20:04.222869+0000","last_active":"2024-05-14T11:20:05.211416+0000","last_peered":"2024-05-14T11:20:05.211416+0000","last_clean":"2024-05-14T11:20:05.211416+0000","last_became_active":"2024-05-14T11:20:04.222494+0000","last_became_peered":"2024-05-14T11:20:04.222494+0000","last_unstale":"2024-05-14T11:20:05.211416+0000","last_undegraded":"2024-05-14T11:20:05.211416+0000","last_fullsized":"2024-05-14T11:20:05.211416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:00.460929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.224869+0000","last_change":"2024-05-14T11:20:04.210629+0000","last_active":"2024-05-14T11:20:04.224869+0000","last_peered":"2024-05-14T11:20:04.224869+0000","last_clean":"2024-05-14T11:20:04.224869+0000","last_became_active":"2024-05-14T11:20:04.210354+0000","last_became_peered":"2024-05-14T11:20:04.210354+0000","last_unstale":"2024-05-14T11:20:04.224869+0000","last_undegraded":"2024-05-14T11:20:04.224869+0000","last_fullsized":"2024-05-14T11:20:04.224869+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:46.148240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213229+0000","last_change":"2024-05-14T11:20:04.220330+0000","last_active":"2024-05-14T11:20:05.213229+0000","last_peered":"2024-05-14T11:20:05.213229+0000","last_clean":"2024-05-14T11:20:05.213229+0000","last_became_active":"2024-05-14T11:20:04.219965+0000","last_became_peered":"2024-05-14T11:20:04.219965+0000","last_unstale":"2024-05-14T11:20:05.213229+0000","last_undegraded":"2024-05-14T11:20:05.213229+0000","last_fullsized":"2024-05-14T11:20:05.213229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:07:42.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275000+0000","last_change":"2024-05-14T11:20:04.209043+0000","last_active":"2024-05-14T11:20:05.275000+0000","last_peered":"2024-05-14T11:20:05.275000+0000","last_clean":"2024-05-14T11:20:05.275000+0000","last_became_active":"2024-05-14T11:20:04.208717+0000","last_became_peered":"2024-05-14T11:20:04.208717+0000","last_unstale":"2024-05-14T11:20:05.275000+0000","last_undegraded":"2024-05-14T11:20:05.275000+0000","last_fullsized":"2024-05-14T11:20:05.275000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228260+0000","last_change":"2024-05-14T11:20:04.224912+0000","last_active":"2024-05-14T11:20:04.228260+0000","last_peered":"2024-05-14T11:20:04.228260+0000","last_clean":"2024-05-14T11:20:04.228260+0000","last_became_active":"2024-05-14T11:20:04.224364+0000","last_became_peered":"2024-05-14T11:20:04.224364+0000","last_unstale":"2024-05-14T11:20:04.228260+0000","last_undegraded":"2024-05-14T11:20:04.228260+0000","last_fullsized":"2024-05-14T11:20:04.228260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:21:43.903087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275091+0000","last_change":"2024-05-14T11:20:04.213592+0000","last_active":"2024-05-14T11:20:05.275091+0000","last_peered":"2024-05-14T11:20:05.275091+0000","last_clean":"2024-05-14T11:20:05.275091+0000","last_became_active":"2024-05-14T11:20:04.213249+0000","last_became_peered":"2024-05-14T11:20:04.213249+0000","last_unstale":"2024-05-14T11:20:05.275091+0000","last_undegraded":"2024-05-14T11:20:05.275091+0000","last_fullsized":"2024-05-14T11:20:05.275091+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225695+0000","last_change":"2024-05-14T11:20:04.219496+0000","last_active":"2024-05-14T11:20:04.225695+0000","last_peered":"2024-05-14T11:20:04.225695+0000","last_clean":"2024-05-14T11:20:04.225695+0000","last_became_active":"2024-05-14T11:20:04.219114+0000","last_became_peered":"2024-05-14T11:20:04.219114+0000","last_unstale":"2024-05-14T11:20:04.225695+0000","last_undegraded":"2024-05-14T11:20:04.225695+0000","last_fullsized":"2024-05-14T11:20:04.225695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:08.006324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223967+0000","last_change":"2024-05-14T11:20:04.217319+0000","last_active":"2024-05-14T11:20:04.223967+0000","last_peered":"2024-05-14T11:20:04.223967+0000","last_clean":"2024-05-14T11:20:04.223967+0000","last_became_active":"2024-05-14T11:20:04.216623+0000","last_became_peered":"2024-05-14T11:20:04.216623+0000","last_unstale":"2024-05-14T11:20:04.223967+0000","last_undegraded":"2024-05-14T11:20:04.223967+0000","last_fullsized":"2024-05-14T11:20:04.223967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223089+0000","last_change":"2024-05-14T11:20:04.214903+0000","last_active":"2024-05-14T11:20:04.223089+0000","last_peered":"2024-05-14T11:20:04.223089+0000","last_clean":"2024-05-14T11:20:04.223089+0000","last_became_active":"2024-05-14T11:20:04.214524+0000","last_became_peered":"2024-05-14T11:20:04.214524+0000","last_unstale":"2024-05-14T11:20:04.223089+0000","last_undegraded":"2024-05-14T11:20:04.223089+0000","last_fullsized":"2024-05-14T11:20:04.223089+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.205905+0000","last_change":"2024-05-14T11:20:04.203981+0000","last_active":"2024-05-14T11:20:04.205905+0000","last_peered":"2024-05-14T11:20:04.205905+0000","last_clean":"2024-05-14T11:20:04.205905+0000","last_became_active":"2024-05-14T11:20:04.203651+0000","last_became_peered":"2024-05-14T11:20:04.203651+0000","last_unstale":"2024-05-14T11:20:04.205905+0000","last_undegraded":"2024-05-14T11:20:04.205905+0000","last_fullsized":"2024-05-14T11:20:04.205905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225599+0000","last_change":"2024-05-14T11:20:04.212080+0000","last_active":"2024-05-14T11:20:04.225599+0000","last_peered":"2024-05-14T11:20:04.225599+0000","last_clean":"2024-05-14T11:20:04.225599+0000","last_became_active":"2024-05-14T11:20:04.211749+0000","last_became_peered":"2024-05-14T11:20:04.211749+0000","last_unstale":"2024-05-14T11:20:04.225599+0000","last_undegraded":"2024-05-14T11:20:04.225599+0000","last_fullsized":"2024-05-14T11:20:04.225599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:09.681689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207985+0000","last_change":"2024-05-14T11:20:04.194987+0000","last_active":"2024-05-14T11:20:05.207985+0000","last_peered":"2024-05-14T11:20:05.207985+0000","last_clean":"2024-05-14T11:20:05.207985+0000","last_became_active":"2024-05-14T11:20:04.194648+0000","last_became_peered":"2024-05-14T11:20:04.194648+0000","last_unstale":"2024-05-14T11:20:05.207985+0000","last_undegraded":"2024-05-14T11:20:05.207985+0000","last_fullsized":"2024-05-14T11:20:05.207985+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:47:06.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.230503+0000","last_change":"2024-05-14T11:20:04.228326+0000","last_active":"2024-05-14T11:20:04.230503+0000","last_peered":"2024-05-14T11:20:04.230503+0000","last_clean":"2024-05-14T11:20:04.230503+0000","last_became_active":"2024-05-14T11:20:04.227647+0000","last_became_peered":"2024-05-14T11:20:04.227647+0000","last_unstale":"2024-05-14T11:20:04.230503+0000","last_undegraded":"2024-05-14T11:20:04.230503+0000","last_fullsized":"2024-05-14T11:20:04.230503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222054+0000","last_change":"2024-05-14T11:20:04.207574+0000","last_active":"2024-05-14T11:20:04.222054+0000","last_peered":"2024-05-14T11:20:04.222054+0000","last_clean":"2024-05-14T11:20:04.222054+0000","last_became_active":"2024-05-14T11:20:04.207255+0000","last_became_peered":"2024-05-14T11:20:04.207255+0000","last_unstale":"2024-05-14T11:20:04.222054+0000","last_undegraded":"2024-05-14T11:20:04.222054+0000","last_fullsized":"2024-05-14T11:20:04.222054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:02:05.964643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211699+0000","last_change":"2024-05-14T11:20:04.209017+0000","last_active":"2024-05-14T11:20:05.211699+0000","last_peered":"2024-05-14T11:20:05.211699+0000","last_clean":"2024-05-14T11:20:05.211699+0000","last_became_active":"2024-05-14T11:20:04.208734+0000","last_became_peered":"2024-05-14T11:20:04.208734+0000","last_unstale":"2024-05-14T11:20:05.211699+0000","last_undegraded":"2024-05-14T11:20:05.211699+0000","last_fullsized":"2024-05-14T11:20:05.211699+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:24.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.231371+0000","last_change":"2024-05-14T11:20:04.228904+0000","last_active":"2024-05-14T11:20:04.231371+0000","last_peered":"2024-05-14T11:20:04.231371+0000","last_clean":"2024-05-14T11:20:04.231371+0000","last_became_active":"2024-05-14T11:20:04.228327+0000","last_became_peered":"2024-05-14T11:20:04.228327+0000","last_unstale":"2024-05-14T11:20:04.231371+0000","last_undegraded":"2024-05-14T11:20:04.231371+0000","last_fullsized":"2024-05-14T11:20:04.231371+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:31.257252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212121+0000","last_change":"2024-05-14T11:20:04.203730+0000","last_active":"2024-05-14T11:20:05.212121+0000","last_peered":"2024-05-14T11:20:05.212121+0000","last_clean":"2024-05-14T11:20:05.212121+0000","last_became_active":"2024-05-14T11:20:04.203355+0000","last_became_peered":"2024-05-14T11:20:04.203355+0000","last_unstale":"2024-05-14T11:20:05.212121+0000","last_undegraded":"2024-05-14T11:20:05.212121+0000","last_fullsized":"2024-05-14T11:20:05.212121+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:28.617457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.226372+0000","last_change":"2024-05-14T11:20:04.223004+0000","last_active":"2024-05-14T11:20:04.226372+0000","last_peered":"2024-05-14T11:20:04.226372+0000","last_clean":"2024-05-14T11:20:04.226372+0000","last_became_active":"2024-05-14T11:20:04.222020+0000","last_became_peered":"2024-05-14T11:20:04.222020+0000","last_unstale":"2024-05-14T11:20:04.226372+0000","last_undegraded":"2024-05-14T11:20:04.226372+0000","last_fullsized":"2024-05-14T11:20:04.226372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228146+0000","last_change":"2024-05-14T11:20:04.224261+0000","last_active":"2024-05-14T11:20:04.228146+0000","last_peered":"2024-05-14T11:20:04.228146+0000","last_clean":"2024-05-14T11:20:04.228146+0000","last_became_active":"2024-05-14T11:20:04.223552+0000","last_became_peered":"2024-05-14T11:20:04.223552+0000","last_unstale":"2024-05-14T11:20:04.228146+0000","last_undegraded":"2024-05-14T11:20:04.228146+0000","last_fullsized":"2024-05-14T11:20:04.228146+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:00.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275108+0000","last_change":"2024-05-14T11:20:04.214671+0000","last_active":"2024-05-14T11:20:05.275108+0000","last_peered":"2024-05-14T11:20:05.275108+0000","last_clean":"2024-05-14T11:20:05.275108+0000","last_became_active":"2024-05-14T11:20:04.214377+0000","last_became_peered":"2024-05-14T11:20:04.214377+0000","last_unstale":"2024-05-14T11:20:05.275108+0000","last_undegraded":"2024-05-14T11:20:05.275108+0000","last_fullsized":"2024-05-14T11:20:05.275108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.217052+0000","last_change":"2024-05-14T11:20:04.213156+0000","last_active":"2024-05-14T11:20:04.217052+0000","last_peered":"2024-05-14T11:20:04.217052+0000","last_clean":"2024-05-14T11:20:04.217052+0000","last_became_active":"2024-05-14T11:20:04.212860+0000","last_became_peered":"2024-05-14T11:20:04.212860+0000","last_unstale":"2024-05-14T11:20:04.217052+0000","last_undegraded":"2024-05-14T11:20:04.217052+0000","last_fullsized":"2024-05-14T11:20:04.217052+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:55.945105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225647+0000","last_change":"2024-05-14T11:20:04.214006+0000","last_active":"2024-05-14T11:20:04.225647+0000","last_peered":"2024-05-14T11:20:04.225647+0000","last_clean":"2024-05-14T11:20:04.225647+0000","last_became_active":"2024-05-14T11:20:04.213675+0000","last_became_peered":"2024-05-14T11:20:04.213675+0000","last_unstale":"2024-05-14T11:20:04.225647+0000","last_undegraded":"2024-05-14T11:20:04.225647+0000","last_fullsized":"2024-05-14T11:20:04.225647+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:10.356205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222846+0000","last_change":"2024-05-14T11:20:04.213649+0000","last_active":"2024-05-14T11:20:04.222846+0000","last_peered":"2024-05-14T11:20:04.222846+0000","last_clean":"2024-05-14T11:20:04.222846+0000","last_became_active":"2024-05-14T11:20:04.213282+0000","last_became_peered":"2024-05-14T11:20:04.213282+0000","last_unstale":"2024-05-14T11:20:04.222846+0000","last_undegraded":"2024-05-14T11:20:04.222846+0000","last_fullsized":"2024-05-14T11:20:04.222846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:32.493683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.211211+0000","last_change":"2024-05-14T11:20:04.207797+0000","last_active":"2024-05-14T11:20:04.211211+0000","last_peered":"2024-05-14T11:20:04.211211+0000","last_clean":"2024-05-14T11:20:04.211211+0000","last_became_active":"2024-05-14T11:20:04.207410+0000","last_became_peered":"2024-05-14T11:20:04.207410+0000","last_unstale":"2024-05-14T11:20:04.211211+0000","last_undegraded":"2024-05-14T11:20:04.211211+0000","last_fullsized":"2024-05-14T11:20:04.211211+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207634+0000","last_change":"2024-05-14T11:20:04.194301+0000","last_active":"2024-05-14T11:20:05.207634+0000","last_peered":"2024-05-14T11:20:05.207634+0000","last_clean":"2024-05-14T11:20:05.207634+0000","last_became_active":"2024-05-14T11:20:04.193967+0000","last_became_peered":"2024-05-14T11:20:04.193967+0000","last_unstale":"2024-05-14T11:20:05.207634+0000","last_undegraded":"2024-05-14T11:20:05.207634+0000","last_fullsized":"2024-05-14T11:20:05.207634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:07:42.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.204730+0000","last_change":"2024-05-14T11:20:04.201818+0000","last_active":"2024-05-14T11:20:04.204730+0000","last_peered":"2024-05-14T11:20:04.204730+0000","last_clean":"2024-05-14T11:20:04.204730+0000","last_became_active":"2024-05-14T11:20:04.201433+0000","last_became_peered":"2024-05-14T11:20:04.201433+0000","last_unstale":"2024-05-14T11:20:04.204730+0000","last_undegraded":"2024-05-14T11:20:04.204730+0000","last_fullsized":"2024-05-14T11:20:04.204730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207693+0000","last_change":"2024-05-14T11:20:04.196149+0000","last_active":"2024-05-14T11:20:05.207693+0000","last_peered":"2024-05-14T11:20:05.207693+0000","last_clean":"2024-05-14T11:20:05.207693+0000","last_became_active":"2024-05-14T11:20:04.195832+0000","last_became_peered":"2024-05-14T11:20:04.195832+0000","last_unstale":"2024-05-14T11:20:05.207693+0000","last_undegraded":"2024-05-14T11:20:05.207693+0000","last_fullsized":"2024-05-14T11:20:05.207693+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:36:00.460929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.227386+0000","last_change":"2024-05-14T11:20:03.207601+0000","last_active":"2024-05-14T11:20:04.227386+0000","last_peered":"2024-05-14T11:20:04.227386+0000","last_clean":"2024-05-14T11:20:04.227386+0000","last_became_active":"2024-05-14T11:20:03.207243+0000","last_became_peered":"2024-05-14T11:20:03.207243+0000","last_unstale":"2024-05-14T11:20:04.227386+0000","last_undegraded":"2024-05-14T11:20:04.227386+0000","last_fullsized":"2024-05-14T11:20:04.227386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:51:35.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223116+0000","last_change":"2024-05-14T11:20:04.217480+0000","last_active":"2024-05-14T11:20:04.223116+0000","last_peered":"2024-05-14T11:20:04.223116+0000","last_clean":"2024-05-14T11:20:04.223116+0000","last_became_active":"2024-05-14T11:20:04.217139+0000","last_became_peered":"2024-05-14T11:20:04.217139+0000","last_unstale":"2024-05-14T11:20:04.223116+0000","last_undegraded":"2024-05-14T11:20:04.223116+0000","last_fullsized":"2024-05-14T11:20:04.223116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.214538+0000","last_change":"2024-05-14T11:20:03.206104+0000","last_active":"2024-05-14T11:20:04.214538+0000","last_peered":"2024-05-14T11:20:04.214538+0000","last_clean":"2024-05-14T11:20:04.214538+0000","last_became_active":"2024-05-14T11:20:03.205810+0000","last_became_peered":"2024-05-14T11:20:03.205810+0000","last_unstale":"2024-05-14T11:20:04.214538+0000","last_undegraded":"2024-05-14T11:20:04.214538+0000","last_fullsized":"2024-05-14T11:20:04.214538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:21.616464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213098+0000","last_change":"2024-05-14T11:20:04.220756+0000","last_active":"2024-05-14T11:20:05.213098+0000","last_peered":"2024-05-14T11:20:05.213098+0000","last_clean":"2024-05-14T11:20:05.213098+0000","last_became_active":"2024-05-14T11:20:04.219980+0000","last_became_peered":"2024-05-14T11:20:04.219980+0000","last_unstale":"2024-05-14T11:20:05.213098+0000","last_undegraded":"2024-05-14T11:20:05.213098+0000","last_fullsized":"2024-05-14T11:20:05.213098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:21:52.277990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.215759+0000","last_change":"2024-05-14T11:20:03.207642+0000","last_active":"2024-05-14T11:20:04.215759+0000","last_peered":"2024-05-14T11:20:04.215759+0000","last_clean":"2024-05-14T11:20:04.215759+0000","last_became_active":"2024-05-14T11:20:03.207327+0000","last_became_peered":"2024-05-14T11:20:03.207327+0000","last_unstale":"2024-05-14T11:20:04.215759+0000","last_undegraded":"2024-05-14T11:20:04.215759+0000","last_fullsized":"2024-05-14T11:20:04.215759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212077+0000","last_change":"2024-05-14T11:20:04.206542+0000","last_active":"2024-05-14T11:20:05.212077+0000","last_peered":"2024-05-14T11:20:05.212077+0000","last_clean":"2024-05-14T11:20:05.212077+0000","last_became_active":"2024-05-14T11:20:04.206220+0000","last_became_peered":"2024-05-14T11:20:04.206220+0000","last_unstale":"2024-05-14T11:20:05.212077+0000","last_undegraded":"2024-05-14T11:20:05.212077+0000","last_fullsized":"2024-05-14T11:20:05.212077+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:47:43.692745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212645+0000","last_change":"2024-05-14T11:20:03.202155+0000","last_active":"2024-05-14T11:20:05.212645+0000","last_peered":"2024-05-14T11:20:05.212645+0000","last_clean":"2024-05-14T11:20:05.212645+0000","last_became_active":"2024-05-14T11:20:03.201797+0000","last_became_peered":"2024-05-14T11:20:03.201797+0000","last_unstale":"2024-05-14T11:20:05.212645+0000","last_undegraded":"2024-05-14T11:20:05.212645+0000","last_fullsized":"2024-05-14T11:20:05.212645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:38:58.772273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228190+0000","last_change":"2024-05-14T11:20:04.225577+0000","last_active":"2024-05-14T11:20:04.228190+0000","last_peered":"2024-05-14T11:20:04.228190+0000","last_clean":"2024-05-14T11:20:04.228190+0000","last_became_active":"2024-05-14T11:20:04.225145+0000","last_became_peered":"2024-05-14T11:20:04.225145+0000","last_unstale":"2024-05-14T11:20:04.228190+0000","last_undegraded":"2024-05-14T11:20:04.228190+0000","last_fullsized":"2024-05-14T11:20:04.228190+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:49.990226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211845+0000","last_change":"2024-05-14T11:20:03.200524+0000","last_active":"2024-05-14T11:20:05.211845+0000","last_peered":"2024-05-14T11:20:05.211845+0000","last_clean":"2024-05-14T11:20:05.211845+0000","last_became_active":"2024-05-14T11:20:03.200138+0000","last_became_peered":"2024-05-14T11:20:03.200138+0000","last_unstale":"2024-05-14T11:20:05.211845+0000","last_undegraded":"2024-05-14T11:20:05.211845+0000","last_fullsized":"2024-05-14T11:20:05.211845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:42:49.328097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228236+0000","last_change":"2024-05-14T11:20:04.224657+0000","last_active":"2024-05-14T11:20:04.228236+0000","last_peered":"2024-05-14T11:20:04.228236+0000","last_clean":"2024-05-14T11:20:04.228236+0000","last_became_active":"2024-05-14T11:20:04.223962+0000","last_became_peered":"2024-05-14T11:20:04.223962+0000","last_unstale":"2024-05-14T11:20:04.228236+0000","last_undegraded":"2024-05-14T11:20:04.228236+0000","last_fullsized":"2024-05-14T11:20:04.228236+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:57:16.632330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.274792+0000","last_change":"2024-05-14T11:20:03.211912+0000","last_active":"2024-05-14T11:20:05.274792+0000","last_peered":"2024-05-14T11:20:05.274792+0000","last_clean":"2024-05-14T11:20:05.274792+0000","last_became_active":"2024-05-14T11:20:03.211585+0000","last_became_peered":"2024-05-14T11:20:03.211585+0000","last_unstale":"2024-05-14T11:20:05.274792+0000","last_undegraded":"2024-05-14T11:20:05.274792+0000","last_fullsized":"2024-05-14T11:20:05.274792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:18.898658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213208+0000","last_change":"2024-05-14T11:20:04.220351+0000","last_active":"2024-05-14T11:20:05.213208+0000","last_peered":"2024-05-14T11:20:05.213208+0000","last_clean":"2024-05-14T11:20:05.213208+0000","last_became_active":"2024-05-14T11:20:04.219770+0000","last_became_peered":"2024-05-14T11:20:04.219770+0000","last_unstale":"2024-05-14T11:20:05.213208+0000","last_undegraded":"2024-05-14T11:20:05.213208+0000","last_fullsized":"2024-05-14T11:20:05.213208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:19:24.324434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.193246+0000","last_change":"2024-05-14T11:20:03.203723+0000","last_active":"2024-05-14T11:20:04.193246+0000","last_peered":"2024-05-14T11:20:04.193246+0000","last_clean":"2024-05-14T11:20:04.193246+0000","last_became_active":"2024-05-14T11:20:03.203393+0000","last_became_peered":"2024-05-14T11:20:03.203393+0000","last_unstale":"2024-05-14T11:20:04.193246+0000","last_undegraded":"2024-05-14T11:20:04.193246+0000","last_fullsized":"2024-05-14T11:20:04.193246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212915+0000","last_change":"2024-05-14T11:20:04.210575+0000","last_active":"2024-05-14T11:20:05.212915+0000","last_peered":"2024-05-14T11:20:05.212915+0000","last_clean":"2024-05-14T11:20:05.212915+0000","last_became_active":"2024-05-14T11:20:04.209508+0000","last_became_peered":"2024-05-14T11:20:04.209508+0000","last_unstale":"2024-05-14T11:20:05.212915+0000","last_undegraded":"2024-05-14T11:20:05.212915+0000","last_fullsized":"2024-05-14T11:20:05.212915+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:04.327882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.197882+0000","last_change":"2024-05-14T11:20:03.201338+0000","last_active":"2024-05-14T11:20:04.197882+0000","last_peered":"2024-05-14T11:20:04.197882+0000","last_clean":"2024-05-14T11:20:04.197882+0000","last_became_active":"2024-05-14T11:20:03.201025+0000","last_became_peered":"2024-05-14T11:20:03.201025+0000","last_unstale":"2024-05-14T11:20:04.197882+0000","last_undegraded":"2024-05-14T11:20:04.197882+0000","last_fullsized":"2024-05-14T11:20:04.197882+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:12.193652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.208688+0000","last_change":"2024-05-14T11:20:04.204618+0000","last_active":"2024-05-14T11:20:04.208688+0000","last_peered":"2024-05-14T11:20:04.208688+0000","last_clean":"2024-05-14T11:20:04.208688+0000","last_became_active":"2024-05-14T11:20:04.204311+0000","last_became_peered":"2024-05-14T11:20:04.204311+0000","last_unstale":"2024-05-14T11:20:04.208688+0000","last_undegraded":"2024-05-14T11:20:04.208688+0000","last_fullsized":"2024-05-14T11:20:04.208688+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:41.886312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.193264+0000","last_change":"2024-05-14T11:20:03.206041+0000","last_active":"2024-05-14T11:20:04.193264+0000","last_peered":"2024-05-14T11:20:04.193264+0000","last_clean":"2024-05-14T11:20:04.193264+0000","last_became_active":"2024-05-14T11:20:03.205724+0000","last_became_peered":"2024-05-14T11:20:03.205724+0000","last_unstale":"2024-05-14T11:20:04.193264+0000","last_undegraded":"2024-05-14T11:20:04.193264+0000","last_fullsized":"2024-05-14T11:20:04.193264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:31.846994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275046+0000","last_change":"2024-05-14T11:20:04.212077+0000","last_active":"2024-05-14T11:20:05.275046+0000","last_peered":"2024-05-14T11:20:05.275046+0000","last_clean":"2024-05-14T11:20:05.275046+0000","last_became_active":"2024-05-14T11:20:04.211758+0000","last_became_peered":"2024-05-14T11:20:04.211758+0000","last_unstale":"2024-05-14T11:20:05.275046+0000","last_undegraded":"2024-05-14T11:20:05.275046+0000","last_fullsized":"2024-05-14T11:20:05.275046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:25:34.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275123+0000","last_change":"2024-05-14T11:20:03.202985+0000","last_active":"2024-05-14T11:20:05.275123+0000","last_peered":"2024-05-14T11:20:05.275123+0000","last_clean":"2024-05-14T11:20:05.275123+0000","last_became_active":"2024-05-14T11:20:03.202678+0000","last_became_peered":"2024-05-14T11:20:03.202678+0000","last_unstale":"2024-05-14T11:20:05.275123+0000","last_undegraded":"2024-05-14T11:20:05.275123+0000","last_fullsized":"2024-05-14T11:20:05.275123+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:08:08.571014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.223942+0000","last_change":"2024-05-14T11:20:04.217236+0000","last_active":"2024-05-14T11:20:04.223942+0000","last_peered":"2024-05-14T11:20:04.223942+0000","last_clean":"2024-05-14T11:20:04.223942+0000","last_became_active":"2024-05-14T11:20:04.216285+0000","last_became_peered":"2024-05-14T11:20:04.216285+0000","last_unstale":"2024-05-14T11:20:04.223942+0000","last_undegraded":"2024-05-14T11:20:04.223942+0000","last_fullsized":"2024-05-14T11:20:04.223942+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:32.849960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225765+0000","last_change":"2024-05-14T11:20:03.209170+0000","last_active":"2024-05-14T11:20:04.225765+0000","last_peered":"2024-05-14T11:20:04.225765+0000","last_clean":"2024-05-14T11:20:04.225765+0000","last_became_active":"2024-05-14T11:20:03.208826+0000","last_became_peered":"2024-05-14T11:20:03.208826+0000","last_unstale":"2024-05-14T11:20:04.225765+0000","last_undegraded":"2024-05-14T11:20:04.225765+0000","last_fullsized":"2024-05-14T11:20:04.225765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:55.303200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207745+0000","last_change":"2024-05-14T11:20:04.202809+0000","last_active":"2024-05-14T11:20:05.207745+0000","last_peered":"2024-05-14T11:20:05.207745+0000","last_clean":"2024-05-14T11:20:05.207745+0000","last_became_active":"2024-05-14T11:20:04.202520+0000","last_became_peered":"2024-05-14T11:20:04.202520+0000","last_unstale":"2024-05-14T11:20:05.207745+0000","last_undegraded":"2024-05-14T11:20:05.207745+0000","last_fullsized":"2024-05-14T11:20:05.207745+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:49:25.840325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212724+0000","last_change":"2024-05-14T11:20:03.200622+0000","last_active":"2024-05-14T11:20:05.212724+0000","last_peered":"2024-05-14T11:20:05.212724+0000","last_clean":"2024-05-14T11:20:05.212724+0000","last_became_active":"2024-05-14T11:20:03.200328+0000","last_became_peered":"2024-05-14T11:20:03.200328+0000","last_unstale":"2024-05-14T11:20:05.212724+0000","last_undegraded":"2024-05-14T11:20:05.212724+0000","last_fullsized":"2024-05-14T11:20:05.212724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:52.776153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275284+0000","last_change":"2024-05-14T11:20:04.223251+0000","last_active":"2024-05-14T11:20:05.275284+0000","last_peered":"2024-05-14T11:20:05.275284+0000","last_clean":"2024-05-14T11:20:05.275284+0000","last_became_active":"2024-05-14T11:20:04.222982+0000","last_became_peered":"2024-05-14T11:20:04.222982+0000","last_unstale":"2024-05-14T11:20:05.275284+0000","last_undegraded":"2024-05-14T11:20:05.275284+0000","last_fullsized":"2024-05-14T11:20:05.275284+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:00.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.274832+0000","last_change":"2024-05-14T11:20:03.205900+0000","last_active":"2024-05-14T11:20:05.274832+0000","last_peered":"2024-05-14T11:20:05.274832+0000","last_clean":"2024-05-14T11:20:05.274832+0000","last_became_active":"2024-05-14T11:20:03.204983+0000","last_became_peered":"2024-05-14T11:20:03.204983+0000","last_unstale":"2024-05-14T11:20:05.274832+0000","last_undegraded":"2024-05-14T11:20:05.274832+0000","last_fullsized":"2024-05-14T11:20:05.274832+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:10.782361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212336+0000","last_change":"2024-05-14T11:20:04.200713+0000","last_active":"2024-05-14T11:20:05.212336+0000","last_peered":"2024-05-14T11:20:05.212336+0000","last_clean":"2024-05-14T11:20:05.212336+0000","last_became_active":"2024-05-14T11:20:04.200396+0000","last_became_peered":"2024-05-14T11:20:04.200396+0000","last_unstale":"2024-05-14T11:20:05.212336+0000","last_undegraded":"2024-05-14T11:20:05.212336+0000","last_fullsized":"2024-05-14T11:20:05.212336+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:32:38.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191803+0000","last_change":"2024-05-14T11:20:03.209105+0000","last_active":"2024-05-14T11:20:04.191803+0000","last_peered":"2024-05-14T11:20:04.191803+0000","last_clean":"2024-05-14T11:20:04.191803+0000","last_became_active":"2024-05-14T11:20:03.208813+0000","last_became_peered":"2024-05-14T11:20:03.208813+0000","last_unstale":"2024-05-14T11:20:04.191803+0000","last_undegraded":"2024-05-14T11:20:04.191803+0000","last_fullsized":"2024-05-14T11:20:04.191803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212590+0000","last_change":"2024-05-14T11:20:04.197781+0000","last_active":"2024-05-14T11:20:05.212590+0000","last_peered":"2024-05-14T11:20:05.212590+0000","last_clean":"2024-05-14T11:20:05.212590+0000","last_became_active":"2024-05-14T11:20:04.196851+0000","last_became_peered":"2024-05-14T11:20:04.196851+0000","last_unstale":"2024-05-14T11:20:05.212590+0000","last_undegraded":"2024-05-14T11:20:05.212590+0000","last_fullsized":"2024-05-14T11:20:05.212590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:35.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198020+0000","last_change":"2024-05-14T11:20:03.209831+0000","last_active":"2024-05-14T11:20:04.198020+0000","last_peered":"2024-05-14T11:20:04.198020+0000","last_clean":"2024-05-14T11:20:04.198020+0000","last_became_active":"2024-05-14T11:20:03.209141+0000","last_became_peered":"2024-05-14T11:20:03.209141+0000","last_unstale":"2024-05-14T11:20:04.198020+0000","last_undegraded":"2024-05-14T11:20:04.198020+0000","last_fullsized":"2024-05-14T11:20:04.198020+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:33.206842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212946+0000","last_change":"2024-05-14T11:20:04.218744+0000","last_active":"2024-05-14T11:20:05.212946+0000","last_peered":"2024-05-14T11:20:05.212946+0000","last_clean":"2024-05-14T11:20:05.212946+0000","last_became_active":"2024-05-14T11:20:04.218390+0000","last_became_peered":"2024-05-14T11:20:04.218390+0000","last_unstale":"2024-05-14T11:20:05.212946+0000","last_undegraded":"2024-05-14T11:20:05.212946+0000","last_fullsized":"2024-05-14T11:20:05.212946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:25.876727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.226344+0000","last_change":"2024-05-14T11:20:03.197088+0000","last_active":"2024-05-14T11:20:04.226344+0000","last_peered":"2024-05-14T11:20:04.226344+0000","last_clean":"2024-05-14T11:20:04.226344+0000","last_became_active":"2024-05-14T11:20:03.196017+0000","last_became_peered":"2024-05-14T11:20:03.196017+0000","last_unstale":"2024-05-14T11:20:04.226344+0000","last_undegraded":"2024-05-14T11:20:04.226344+0000","last_fullsized":"2024-05-14T11:20:04.226344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:06.057974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.235947+0000","last_change":"2024-05-14T11:20:04.230080+0000","last_active":"2024-05-14T11:20:04.235947+0000","last_peered":"2024-05-14T11:20:04.235947+0000","last_clean":"2024-05-14T11:20:04.235947+0000","last_became_active":"2024-05-14T11:20:04.224638+0000","last_became_peered":"2024-05-14T11:20:04.224638+0000","last_unstale":"2024-05-14T11:20:04.235947+0000","last_undegraded":"2024-05-14T11:20:04.235947+0000","last_fullsized":"2024-05-14T11:20:04.235947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.211138+0000","last_change":"2024-05-14T11:20:03.210778+0000","last_active":"2024-05-14T11:20:04.211138+0000","last_peered":"2024-05-14T11:20:04.211138+0000","last_clean":"2024-05-14T11:20:04.211138+0000","last_became_active":"2024-05-14T11:20:03.210403+0000","last_became_peered":"2024-05-14T11:20:03.210403+0000","last_unstale":"2024-05-14T11:20:04.211138+0000","last_undegraded":"2024-05-14T11:20:04.211138+0000","last_fullsized":"2024-05-14T11:20:04.211138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:40.883346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.228232+0000","last_change":"2024-05-14T11:20:04.222687+0000","last_active":"2024-05-14T11:20:04.228232+0000","last_peered":"2024-05-14T11:20:04.228232+0000","last_clean":"2024-05-14T11:20:04.228232+0000","last_became_active":"2024-05-14T11:20:04.221813+0000","last_became_peered":"2024-05-14T11:20:04.221813+0000","last_unstale":"2024-05-14T11:20:04.228232+0000","last_undegraded":"2024-05-14T11:20:04.228232+0000","last_fullsized":"2024-05-14T11:20:04.228232+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:54.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212250+0000","last_change":"2024-05-14T11:20:03.206067+0000","last_active":"2024-05-14T11:20:05.212250+0000","last_peered":"2024-05-14T11:20:05.212250+0000","last_clean":"2024-05-14T11:20:05.212250+0000","last_became_active":"2024-05-14T11:20:03.205778+0000","last_became_peered":"2024-05-14T11:20:03.205778+0000","last_unstale":"2024-05-14T11:20:05.212250+0000","last_undegraded":"2024-05-14T11:20:05.212250+0000","last_fullsized":"2024-05-14T11:20:05.212250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:51.058200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.221990+0000","last_change":"2024-05-14T11:20:04.207600+0000","last_active":"2024-05-14T11:20:04.221990+0000","last_peered":"2024-05-14T11:20:04.221990+0000","last_clean":"2024-05-14T11:20:04.221990+0000","last_became_active":"2024-05-14T11:20:04.207211+0000","last_became_peered":"2024-05-14T11:20:04.207211+0000","last_unstale":"2024-05-14T11:20:04.221990+0000","last_undegraded":"2024-05-14T11:20:04.221990+0000","last_fullsized":"2024-05-14T11:20:04.221990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:02:58.918256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.225285+0000","last_change":"2024-05-14T11:20:03.202064+0000","last_active":"2024-05-14T11:20:04.225285+0000","last_peered":"2024-05-14T11:20:04.225285+0000","last_clean":"2024-05-14T11:20:04.225285+0000","last_became_active":"2024-05-14T11:20:03.201766+0000","last_became_peered":"2024-05-14T11:20:03.201766+0000","last_unstale":"2024-05-14T11:20:04.225285+0000","last_undegraded":"2024-05-14T11:20:04.225285+0000","last_fullsized":"2024-05-14T11:20:04.225285+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:17.496082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.230539+0000","last_change":"2024-05-14T11:20:04.229610+0000","last_active":"2024-05-14T11:20:04.230539+0000","last_peered":"2024-05-14T11:20:04.230539+0000","last_clean":"2024-05-14T11:20:04.230539+0000","last_became_active":"2024-05-14T11:20:04.224620+0000","last_became_peered":"2024-05-14T11:20:04.224620+0000","last_unstale":"2024-05-14T11:20:04.230539+0000","last_undegraded":"2024-05-14T11:20:04.230539+0000","last_fullsized":"2024-05-14T11:20:04.230539+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:20.362945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207836+0000","last_change":"2024-05-14T11:20:03.213245+0000","last_active":"2024-05-14T11:20:05.207836+0000","last_peered":"2024-05-14T11:20:05.207836+0000","last_clean":"2024-05-14T11:20:05.207836+0000","last_became_active":"2024-05-14T11:20:03.212890+0000","last_became_peered":"2024-05-14T11:20:03.212890+0000","last_unstale":"2024-05-14T11:20:05.207836+0000","last_undegraded":"2024-05-14T11:20:05.207836+0000","last_fullsized":"2024-05-14T11:20:05.207836+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:52.776153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.219824+0000","last_change":"2024-05-14T11:20:04.213655+0000","last_active":"2024-05-14T11:20:04.219824+0000","last_peered":"2024-05-14T11:20:04.219824+0000","last_clean":"2024-05-14T11:20:04.219824+0000","last_became_active":"2024-05-14T11:20:04.213368+0000","last_became_peered":"2024-05-14T11:20:04.213368+0000","last_unstale":"2024-05-14T11:20:04.219824+0000","last_undegraded":"2024-05-14T11:20:04.219824+0000","last_fullsized":"2024-05-14T11:20:04.219824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:29:39.382571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.224131+0000","last_change":"2024-05-14T11:20:03.207110+0000","last_active":"2024-05-14T11:20:04.224131+0000","last_peered":"2024-05-14T11:20:04.224131+0000","last_clean":"2024-05-14T11:20:04.224131+0000","last_became_active":"2024-05-14T11:20:03.206731+0000","last_became_peered":"2024-05-14T11:20:03.206731+0000","last_unstale":"2024-05-14T11:20:04.224131+0000","last_undegraded":"2024-05-14T11:20:04.224131+0000","last_fullsized":"2024-05-14T11:20:04.224131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:57.460432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.209143+0000","last_change":"2024-05-14T11:20:04.204062+0000","last_active":"2024-05-14T11:20:04.209143+0000","last_peered":"2024-05-14T11:20:04.209143+0000","last_clean":"2024-05-14T11:20:04.209143+0000","last_became_active":"2024-05-14T11:20:04.203759+0000","last_became_peered":"2024-05-14T11:20:04.203759+0000","last_unstale":"2024-05-14T11:20:04.209143+0000","last_undegraded":"2024-05-14T11:20:04.209143+0000","last_fullsized":"2024-05-14T11:20:04.209143+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:49.966784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.208389+0000","last_change":"2024-05-14T11:20:03.199866+0000","last_active":"2024-05-14T11:20:04.208389+0000","last_peered":"2024-05-14T11:20:04.208389+0000","last_clean":"2024-05-14T11:20:04.208389+0000","last_became_active":"2024-05-14T11:20:03.199571+0000","last_became_peered":"2024-05-14T11:20:03.199571+0000","last_unstale":"2024-05-14T11:20:04.208389+0000","last_undegraded":"2024-05-14T11:20:04.208389+0000","last_fullsized":"2024-05-14T11:20:04.208389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:40.883346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.213021+0000","last_change":"2024-05-14T11:20:04.212434+0000","last_active":"2024-05-14T11:20:05.213021+0000","last_peered":"2024-05-14T11:20:05.213021+0000","last_clean":"2024-05-14T11:20:05.213021+0000","last_became_active":"2024-05-14T11:20:04.211306+0000","last_became_peered":"2024-05-14T11:20:04.211306+0000","last_unstale":"2024-05-14T11:20:05.213021+0000","last_undegraded":"2024-05-14T11:20:05.213021+0000","last_fullsized":"2024-05-14T11:20:05.213021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:45:36.011861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198386+0000","last_change":"2024-05-14T11:20:03.200502+0000","last_active":"2024-05-14T11:20:04.198386+0000","last_peered":"2024-05-14T11:20:04.198386+0000","last_clean":"2024-05-14T11:20:04.198386+0000","last_became_active":"2024-05-14T11:20:03.200218+0000","last_became_peered":"2024-05-14T11:20:03.200218+0000","last_unstale":"2024-05-14T11:20:04.198386+0000","last_undegraded":"2024-05-14T11:20:04.198386+0000","last_fullsized":"2024-05-14T11:20:04.198386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211459+0000","last_change":"2024-05-14T11:20:04.193627+0000","last_active":"2024-05-14T11:20:05.211459+0000","last_peered":"2024-05-14T11:20:05.211459+0000","last_clean":"2024-05-14T11:20:05.211459+0000","last_became_active":"2024-05-14T11:20:04.193168+0000","last_became_peered":"2024-05-14T11:20:04.193168+0000","last_unstale":"2024-05-14T11:20:05.211459+0000","last_undegraded":"2024-05-14T11:20:05.211459+0000","last_fullsized":"2024-05-14T11:20:05.211459+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:16:21.864624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.275237+0000","last_change":"2024-05-14T11:20:03.210614+0000","last_active":"2024-05-14T11:20:05.275237+0000","last_peered":"2024-05-14T11:20:05.275237+0000","last_clean":"2024-05-14T11:20:05.275237+0000","last_became_active":"2024-05-14T11:20:03.210304+0000","last_became_peered":"2024-05-14T11:20:03.210304+0000","last_unstale":"2024-05-14T11:20:05.275237+0000","last_undegraded":"2024-05-14T11:20:05.275237+0000","last_fullsized":"2024-05-14T11:20:05.275237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:26:21.616464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211785+0000","last_change":"2024-05-14T11:20:04.211434+0000","last_active":"2024-05-14T11:20:05.211785+0000","last_peered":"2024-05-14T11:20:05.211785+0000","last_clean":"2024-05-14T11:20:05.211785+0000","last_became_active":"2024-05-14T11:20:04.211082+0000","last_became_peered":"2024-05-14T11:20:04.211082+0000","last_unstale":"2024-05-14T11:20:05.211785+0000","last_undegraded":"2024-05-14T11:20:05.211785+0000","last_fullsized":"2024-05-14T11:20:05.211785+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:49:25.840325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.194159+0000","last_change":"2024-05-14T11:20:03.199010+0000","last_active":"2024-05-14T11:20:04.194159+0000","last_peered":"2024-05-14T11:20:04.194159+0000","last_clean":"2024-05-14T11:20:04.194159+0000","last_became_active":"2024-05-14T11:20:03.198711+0000","last_became_peered":"2024-05-14T11:20:03.198711+0000","last_unstale":"2024-05-14T11:20:04.194159+0000","last_undegraded":"2024-05-14T11:20:04.194159+0000","last_fullsized":"2024-05-14T11:20:04.194159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:24:35.243462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212861+0000","last_change":"2024-05-14T11:20:04.219615+0000","last_active":"2024-05-14T11:20:05.212861+0000","last_peered":"2024-05-14T11:20:05.212861+0000","last_clean":"2024-05-14T11:20:05.212861+0000","last_became_active":"2024-05-14T11:20:04.219278+0000","last_became_peered":"2024-05-14T11:20:04.219278+0000","last_unstale":"2024-05-14T11:20:05.212861+0000","last_undegraded":"2024-05-14T11:20:05.212861+0000","last_fullsized":"2024-05-14T11:20:05.212861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:58:51.129729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.192555+0000","last_change":"2024-05-14T11:20:03.205481+0000","last_active":"2024-05-14T11:20:04.192555+0000","last_peered":"2024-05-14T11:20:04.192555+0000","last_clean":"2024-05-14T11:20:04.192555+0000","last_became_active":"2024-05-14T11:20:03.205141+0000","last_became_peered":"2024-05-14T11:20:03.205141+0000","last_unstale":"2024-05-14T11:20:04.192555+0000","last_undegraded":"2024-05-14T11:20:04.192555+0000","last_fullsized":"2024-05-14T11:20:04.192555+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:48.963819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.222832+0000","last_change":"2024-05-14T11:20:04.211305+0000","last_active":"2024-05-14T11:20:04.222832+0000","last_peered":"2024-05-14T11:20:04.222832+0000","last_clean":"2024-05-14T11:20:04.222832+0000","last_became_active":"2024-05-14T11:20:04.211002+0000","last_became_peered":"2024-05-14T11:20:04.211002+0000","last_unstale":"2024-05-14T11:20:04.222832+0000","last_undegraded":"2024-05-14T11:20:04.222832+0000","last_fullsized":"2024-05-14T11:20:04.222832+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:43:32.849960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191783+0000","last_change":"2024-05-14T11:20:03.204552+0000","last_active":"2024-05-14T11:20:04.191783+0000","last_peered":"2024-05-14T11:20:04.191783+0000","last_clean":"2024-05-14T11:20:04.191783+0000","last_became_active":"2024-05-14T11:20:03.204251+0000","last_became_peered":"2024-05-14T11:20:03.204251+0000","last_unstale":"2024-05-14T11:20:04.191783+0000","last_undegraded":"2024-05-14T11:20:04.191783+0000","last_fullsized":"2024-05-14T11:20:04.191783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:06.057974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212514+0000","last_change":"2024-05-14T11:20:04.217189+0000","last_active":"2024-05-14T11:20:05.212514+0000","last_peered":"2024-05-14T11:20:05.212514+0000","last_clean":"2024-05-14T11:20:05.212514+0000","last_became_active":"2024-05-14T11:20:04.216636+0000","last_became_peered":"2024-05-14T11:20:04.216636+0000","last_unstale":"2024-05-14T11:20:05.212514+0000","last_undegraded":"2024-05-14T11:20:05.212514+0000","last_fullsized":"2024-05-14T11:20:05.212514+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:55:02.889890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212110+0000","last_change":"2024-05-14T11:20:03.202016+0000","last_active":"2024-05-14T11:20:05.212110+0000","last_peered":"2024-05-14T11:20:05.212110+0000","last_clean":"2024-05-14T11:20:05.212110+0000","last_became_active":"2024-05-14T11:20:03.201696+0000","last_became_peered":"2024-05-14T11:20:03.201696+0000","last_unstale":"2024-05-14T11:20:05.212110+0000","last_undegraded":"2024-05-14T11:20:05.212110+0000","last_fullsized":"2024-05-14T11:20:05.212110+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:45:45.591695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.212498+0000","last_change":"2024-05-14T11:20:04.215582+0000","last_active":"2024-05-14T11:20:05.212498+0000","last_peered":"2024-05-14T11:20:05.212498+0000","last_clean":"2024-05-14T11:20:05.212498+0000","last_became_active":"2024-05-14T11:20:04.215243+0000","last_became_peered":"2024-05-14T11:20:04.215243+0000","last_unstale":"2024-05-14T11:20:05.212498+0000","last_undegraded":"2024-05-14T11:20:05.212498+0000","last_fullsized":"2024-05-14T11:20:05.212498+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:47:06.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198464+0000","last_change":"2024-05-14T11:20:03.213477+0000","last_active":"2024-05-14T11:20:04.198464+0000","last_peered":"2024-05-14T11:20:04.198464+0000","last_clean":"2024-05-14T11:20:04.198464+0000","last_became_active":"2024-05-14T11:20:03.212174+0000","last_became_peered":"2024-05-14T11:20:03.212174+0000","last_unstale":"2024-05-14T11:20:04.198464+0000","last_undegraded":"2024-05-14T11:20:04.198464+0000","last_fullsized":"2024-05-14T11:20:04.198464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:05.545440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.208009+0000","last_change":"2024-05-14T11:20:04.203849+0000","last_active":"2024-05-14T11:20:05.208009+0000","last_peered":"2024-05-14T11:20:05.208009+0000","last_clean":"2024-05-14T11:20:05.208009+0000","last_became_active":"2024-05-14T11:20:04.203553+0000","last_became_peered":"2024-05-14T11:20:04.203553+0000","last_unstale":"2024-05-14T11:20:05.208009+0000","last_undegraded":"2024-05-14T11:20:05.208009+0000","last_fullsized":"2024-05-14T11:20:05.208009+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:55:02.889890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.211532+0000","last_change":"2024-05-14T11:20:03.200988+0000","last_active":"2024-05-14T11:20:05.211532+0000","last_peered":"2024-05-14T11:20:05.211532+0000","last_clean":"2024-05-14T11:20:05.211532+0000","last_became_active":"2024-05-14T11:20:03.199743+0000","last_became_peered":"2024-05-14T11:20:03.199743+0000","last_unstale":"2024-05-14T11:20:05.211532+0000","last_undegraded":"2024-05-14T11:20:05.211532+0000","last_fullsized":"2024-05-14T11:20:05.211532+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:44:51.584569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.208144+0000","last_change":"2024-05-14T11:20:04.201338+0000","last_active":"2024-05-14T11:20:05.208144+0000","last_peered":"2024-05-14T11:20:05.208144+0000","last_clean":"2024-05-14T11:20:05.208144+0000","last_became_active":"2024-05-14T11:20:04.200861+0000","last_became_peered":"2024-05-14T11:20:04.200861+0000","last_unstale":"2024-05-14T11:20:05.208144+0000","last_undegraded":"2024-05-14T11:20:05.208144+0000","last_fullsized":"2024-05-14T11:20:05.208144+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:35.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-05-14T11:20:05.207497+0000","last_change":"2024-05-14T11:20:03.213018+0000","last_active":"2024-05-14T11:20:05.207497+0000","last_peered":"2024-05-14T11:20:05.207497+0000","last_clean":"2024-05-14T11:20:05.207497+0000","last_became_active":"2024-05-14T11:20:03.212685+0000","last_became_peered":"2024-05-14T11:20:03.212685+0000","last_unstale":"2024-05-14T11:20:05.207497+0000","last_undegraded":"2024-05-14T11:20:05.207497+0000","last_fullsized":"2024-05-14T11:20:05.207497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:02:03.606870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.213039+0000","last_change":"2024-05-14T11:20:04.209316+0000","last_active":"2024-05-14T11:20:04.213039+0000","last_peered":"2024-05-14T11:20:04.213039+0000","last_clean":"2024-05-14T11:20:04.213039+0000","last_became_active":"2024-05-14T11:20:04.207163+0000","last_became_peered":"2024-05-14T11:20:04.207163+0000","last_unstale":"2024-05-14T11:20:04.213039+0000","last_undegraded":"2024-05-14T11:20:04.213039+0000","last_fullsized":"2024-05-14T11:20:04.213039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:03.171727+0000","last_clean_scrub_stamp":"2024-05-14T11:20:03.171727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:51.186601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.198618+0000","last_change":"2024-05-14T11:20:03.204942+0000","last_active":"2024-05-14T11:20:04.198618+0000","last_peered":"2024-05-14T11:20:04.198618+0000","last_clean":"2024-05-14T11:20:04.198618+0000","last_became_active":"2024-05-14T11:20:03.204239+0000","last_became_peered":"2024-05-14T11:20:03.204239+0000","last_unstale":"2024-05-14T11:20:04.198618+0000","last_undegraded":"2024-05-14T11:20:04.198618+0000","last_fullsized":"2024-05-14T11:20:04.198618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:20:02.168761+0000","last_clean_scrub_stamp":"2024-05-14T11:20:02.168761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:04:48.963819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":202,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-05-14T11:20:04.191928+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:20:04.191928+0000","last_peered":"2024-05-14T11:20:04.191928+0000","last_clean":"2024-05-14T11:20:04.191928+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:20:04.191928+0000","last_undegraded":"2024-05-14T11:20:04.191928+0000","last_fullsized":"2024-05-14T11:20:04.191928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":983188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":394410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":392832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":888832,"data_stored":394514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":393380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":393540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":392910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":983166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:20:08.724 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:20:08.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:20:08.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.914+0000 7f5456e00640 1 Processor -- start 2024-05-14T11:20:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.914+0000 7f5456e00640 1 -- start start 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5456e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f54500d1e40 0x7f545006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5456e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545006f860 con 0x7f545006f450 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5456e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545006fae0 con 0x7f54500cde70 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5456e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f545006db80 con 0x7f54500d1e40 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f545006f450 0x7f54500d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36520/0 (socket says 172.21.15.116:36520) 2024-05-14T11:20:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544fe00640 1 -- 172.21.15.116:0/4246145804 learned_addr learned my addr 172.21.15.116:0/4246145804 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 --2- 172.21.15.116:0/4246145804 >> v2:172.21.15.149:3300/0 conn(0x7f54500d1e40 0x7f545006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353005476 0 0) 0x7f5440001120 con 0x7f545006f450 2024-05-14T11:20:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5434003530 con 0x7f545006f450 2024-05-14T11:20:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 57727550 0 0) 0x7f5434003530 con 0x7f545006f450 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 >> v2:172.21.15.149:3300/0 conn(0x7f54500d1e40 msgr2=0x7f545006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 --2- 172.21.15.116:0/4246145804 >> v2:172.21.15.149:3300/0 conn(0x7f54500d1e40 0x7f545006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 >> v1:172.21.15.149:6789/0 conn(0x7f54500cde70 legacy=0x7f54501675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545006fdc0 con 0x7f545006f450 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f54400018d0 con 0x7f545006f450 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 >> v1:172.21.15.116:6789/0 conn(0x7f545006f450 legacy=0x7f54500d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5434003a60 0x7f5434003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5434005260 0x7f543400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f545006db80 con 0x7f5434004810 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f543400c1f0 con 0x7f5434003a60 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f543400c3f0 con 0x7f5434005260 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544fe00640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5434003a60 0x7f5434003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5454e00640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5434005260 0x7f543400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 -- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5434005260 msgr2=0x7f543400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5434005260 0x7f543400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 -- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5434003a60 msgr2=0x7f5434003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5434003a60 0x7f5434003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5447400640 1 -- 172.21.15.116:0/4246145804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5434003530 con 0x7f5434004810 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f5454e00640 1 -- 172.21.15.116:0/4246145804 reap_dead start 2024-05-14T11:20:08.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.918+0000 7f544fe00640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5434003a60 0x7f5434003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5447400640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f5434004c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f54500d1180 tx=0x7f543801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a723779f781688 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380b2070 con 0x7f5434004810 2024-05-14T11:20:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54380a5510 con 0x7f5434004810 2024-05-14T11:20:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f544f400640 1 -- 172.21.15.116:0/4246145804 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380096b0 con 0x7f5434004810 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 -- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 msgr2=0x7f5434004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f5434004c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f54500d1180 tx=0x7f543801ca30 comp rx=0 tx=0).stop 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 -- 172.21.15.116:0/4246145804 shutdown_connections 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 --2- 172.21.15.116:0/4246145804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 -- 172.21.15.116:0/4246145804 >> 172.21.15.116:0/4246145804 conn(0x7f54500b4720 msgr2=0x7f54500b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 -- 172.21.15.116:0/4246145804 shutdown_connections 2024-05-14T11:20:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.922+0000 7f5456e00640 1 -- 172.21.15.116:0/4246145804 wait complete. 2024-05-14T11:20:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 Processor -- start 2024-05-14T11:20:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 -- start start 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 0x7f54500cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 0x7f545015b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5450070660 con 0x7f5434004810 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5450070380 con 0x7f545015ae50 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5456e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f545006d640 con 0x7f54500ca780 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49746/0 (socket says 172.21.15.116:49746) 2024-05-14T11:20:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 -- 172.21.15.116:0/2409644922 learned_addr learned my addr 172.21.15.116:0/2409644922 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5447400640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 0x7f54500cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5454e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 0x7f545015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 -- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 msgr2=0x7f54500cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 0x7f54500cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 -- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 msgr2=0x7f545015b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 0x7f545015b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f544fe00640 1 -- 172.21.15.116:0/2409644922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545015b7a0 con 0x7f5434004810 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5447400640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 0x7f54500cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.926+0000 7f5454e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 0x7f545015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544fe00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f54400117c0 tx=0x7f5440011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440019070 con 0x7f5434004810 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544000f8d0 con 0x7f5434004810 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54500c8510 con 0x7f5434004810 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440010040 con 0x7f5434004810 2024-05-14T11:20:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54500c8ad0 con 0x7f5434004810 2024-05-14T11:20:08.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545006a9c0 con 0x7f5434004810 2024-05-14T11:20:08.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f544001c070 con 0x7f5434004810 2024-05-14T11:20:08.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 0x7f542c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:08.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.930+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f5440091e80 con 0x7f5434004810 2024-05-14T11:20:08.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.934+0000 7f5447400640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 0x7f542c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:08.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.934+0000 7f5447400640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 0x7f542c071910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54380af000 tx=0x7f5438008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:08.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:08.938+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f545006f860 con 0x7f5434004810 2024-05-14T11:20:09.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.146+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5450069de0 con 0x7f5434004810 2024-05-14T11:20:09.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.150+0000 7f544cc00640 1 -- 172.21.15.116:0/2409644922 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f544005eb40 con 0x7f5434004810 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:20:09.151 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:20:09.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 msgr2=0x7f542c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 0x7f542c071910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54380af000 tx=0x7f5438008040 comp rx=0 tx=0).stop 2024-05-14T11:20:09.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 msgr2=0x7f54500c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f54400117c0 tx=0x7f5440011c80 comp rx=0 tx=0).stop 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 shutdown_connections 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f542c06f400 0x7f542c071910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f545015ae50 0x7f545015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f54500ca780 0x7f54500cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 --2- 172.21.15.116:0/2409644922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5434004810 0x7f54500c7630 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 >> 172.21.15.116:0/2409644922 conn(0x7f54500b4720 msgr2=0x7f545015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 shutdown_connections 2024-05-14T11:20:09.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.154+0000 7f5456e00640 1 -- 172.21.15.116:0/2409644922 wait complete. 2024-05-14T11:20:09.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.318+0000 7f2fbbe00640 1 Processor -- start 2024-05-14T11:20:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.318+0000 7f2fbbe00640 1 -- start start 2024-05-14T11:20:09.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fbbe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2fb40af420 0x7f2fb4074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fbbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2fb40d1d80 0x7f2fb406edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fbbe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb415d340 con 0x7f2fb4074b70 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fbbe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb415d620 con 0x7f2fb40d1d80 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fbbe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb415d900 con 0x7f2fb40af420 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2fb40d1d80 0x7f2fb406edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2fb40d1d80 0x7f2fb406edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49754/0 (socket says 172.21.15.116:49754) 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 -- 172.21.15.116:0/502854249 learned_addr learned my addr 172.21.15.116:0/502854249 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9400640 1 --2- 172.21.15.116:0/502854249 >> v2:172.21.15.149:3300/0 conn(0x7f2fb40af420 0x7f2fb4074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 -- 172.21.15.116:0/502854249 >> v2:172.21.15.149:3300/0 conn(0x7f2fb40af420 msgr2=0x7f2fb4074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 --2- 172.21.15.116:0/502854249 >> v2:172.21.15.149:3300/0 conn(0x7f2fb40af420 0x7f2fb4074630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 -- 172.21.15.116:0/502854249 >> v1:172.21.15.149:6789/0 conn(0x7f2fb4074b70 legacy=0x7f2fb40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.322+0000 7f2fb9e00640 1 -- 172.21.15.116:0/502854249 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb415dbe0 con 0x7f2fb40d1d80 2024-05-14T11:20:09.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fb9e00640 1 --2- 172.21.15.116:0/502854249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb40d1d80 0x7f2fb406edc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4071f20 tx=0x7f2fac01e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e164a85d2943fb37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:09.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fb3e00640 1 -- 172.21.15.116:0/502854249 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a86f0 con 0x7f2fb40d1d80 2024-05-14T11:20:09.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fb3e00640 1 -- 172.21.15.116:0/502854249 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0a8890 con 0x7f2fb40d1d80 2024-05-14T11:20:09.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fb3e00640 1 -- 172.21.15.116:0/502854249 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a8c60 con 0x7f2fb40d1d80 2024-05-14T11:20:09.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 -- 172.21.15.116:0/502854249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb40d1d80 msgr2=0x7f2fb406edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/502854249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb40d1d80 0x7f2fb406edc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4071f20 tx=0x7f2fac01e730 comp rx=0 tx=0).stop 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 -- 172.21.15.116:0/502854249 shutdown_connections 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/502854249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb40d1d80 0x7f2fb406edc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/502854249 >> v2:172.21.15.149:3300/0 conn(0x7f2fb40af420 0x7f2fb4074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 -- 172.21.15.116:0/502854249 >> 172.21.15.116:0/502854249 conn(0x7f2fb40b4620 msgr2=0x7f2fb40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 -- 172.21.15.116:0/502854249 shutdown_connections 2024-05-14T11:20:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.326+0000 7f2fbbe00640 1 -- 172.21.15.116:0/502854249 wait complete. 2024-05-14T11:20:09.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 Processor -- start 2024-05-14T11:20:09.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- start start 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb4074b70 0x7f2fb414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 0x7f2fb4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb406ca90 con 0x7f2fb414e3b0 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb406c910 con 0x7f2fb4074b70 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb406cc10 con 0x7f2fb40af420 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49768/0 (socket says 172.21.15.116:49768) 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 -- 172.21.15.116:0/1070964350 learned_addr learned my addr 172.21.15.116:0/1070964350 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9400640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb4074b70 0x7f2fb414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb8a00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 0x7f2fb4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 -- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 msgr2=0x7f2fb4154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 0x7f2fb4154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 -- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb4074b70 msgr2=0x7f2fb414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb4074b70 0x7f2fb414c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 -- 172.21.15.116:0/1070964350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb409edf0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb8a00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 0x7f2fb4154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:09.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb9e00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b2040 tx=0x7f2fac01cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0b0be0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac0c1070 con 0x7f2fb414e3b0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0a86d0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb409f0d0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.330+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb40b3560 con 0x7f2fb414e3b0 2024-05-14T11:20:09.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb406a9c0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2fac0bb470 con 0x7f2fb414e3b0 2024-05-14T11:20:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fb1600640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 0x7f2f94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f2fac12c220 con 0x7f2fb414e3b0 2024-05-14T11:20:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fb9400640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 0x7f2f94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.334+0000 7f2fb9400640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 0x7f2f94071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4070ac0 tx=0x7f2fa8009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.338+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2fac0ca020 con 0x7f2fb414e3b0 2024-05-14T11:20:09.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.534+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fb40b3cd0 con 0x7f2fb414e3b0 2024-05-14T11:20:09.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.534+0000 7f2fb1600640 1 -- 172.21.15.116:0/1070964350 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7f2fac018320 con 0x7f2fb414e3b0 2024-05-14T11:20:09.535 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:09.535 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:06.615674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2430466110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2430466110},{"type":"v1","addr":"172.21.15.116:6836","nonce":2430466110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6213,"qdb_cluster":[6213]},"id":5}]} 2024-05-14T11:20:09.535 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 45 2024-05-14T11:20:09.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 msgr2=0x7f2f94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 0x7f2f94071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4070ac0 tx=0x7f2fa8009830 comp rx=0 tx=0).stop 2024-05-14T11:20:09.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 msgr2=0x7f2fb4155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0b2040 tx=0x7f2fac01cf70 comp rx=0 tx=0).stop 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 shutdown_connections 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2f9406f400 0x7f2f94071910 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fb414e3b0 0x7f2fb4155210 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2fb40af420 0x7f2fb4154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 --2- 172.21.15.116:0/1070964350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb4074b70 0x7f2fb414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 >> 172.21.15.116:0/1070964350 conn(0x7f2fb40b4620 msgr2=0x7f2fb415f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 shutdown_connections 2024-05-14T11:20:09.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:09.538+0000 7f2fbbe00640 1 -- 172.21.15.116:0/1070964350 wait complete. 2024-05-14T11:20:09.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:20:03.585967+0000', 'modified': '2024-05-14T11:20:06.615674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6213}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6213': {'gid': 6213, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/2430466110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2430466110}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2430466110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6213, 'qdb_cluster': [6213]} 2024-05-14T11:20:09.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:20:09.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:20:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 Processor -- start 2024-05-14T11:20:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 -- start start 2024-05-14T11:20:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa9d40d1670 0x7fa9d40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 0x7fa9d406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d406ff20 con 0x7fa9d415db30 2024-05-14T11:20:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d40ae100 con 0x7fa9d40d1670 2024-05-14T11:20:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9dc400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d40ae870 con 0x7fa9d406c700 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa9d40d1670 0x7fa9d40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9da400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 0x7fa9d406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9a00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fa9d415db30 0x7fa9d40aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:54872/0 (socket says 172.21.15.116:54872) 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9a00640 1 -- 172.21.15.116:0/3219089952 learned_addr learned my addr 172.21.15.116:0/3219089952 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9000640 1 -- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 msgr2=0x7fa9d406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 0x7fa9d406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9000640 1 -- 172.21.15.116:0/3219089952 >> v1:172.21.15.149:6789/0 conn(0x7fa9d415db30 legacy=0x7fa9d40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9d9000640 1 -- 172.21.15.116:0/3219089952 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d40d1fc0 con 0x7fa9d40d1670 2024-05-14T11:20:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.110+0000 7fa9da400640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 0x7fa9d406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d40d1a80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00097c0 tx=0x7fa9c001c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48865051b221c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9d3e00640 1 -- 172.21.15.116:0/3219089952 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00b0070 con 0x7fa9d40d1670 2024-05-14T11:20:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9d3e00640 1 -- 172.21.15.116:0/3219089952 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c00ab600 con 0x7fa9d40d1670 2024-05-14T11:20:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9d3e00640 1 -- 172.21.15.116:0/3219089952 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00ab9d0 con 0x7fa9d40d1670 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 -- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 msgr2=0x7fa9d40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d40d1a80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00097c0 tx=0x7fa9c001c340 comp rx=0 tx=0).stop 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 -- 172.21.15.116:0/3219089952 shutdown_connections 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d406c700 0x7fa9d406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 --2- 172.21.15.116:0/3219089952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d40d1a80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 -- 172.21.15.116:0/3219089952 >> 172.21.15.116:0/3219089952 conn(0x7fa9d40b4620 msgr2=0x7fa9d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 -- 172.21.15.116:0/3219089952 shutdown_connections 2024-05-14T11:20:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.114+0000 7fa9dc400640 1 -- 172.21.15.116:0/3219089952 wait complete. 2024-05-14T11:20:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 Processor -- start 2024-05-14T11:20:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 -- start start 2024-05-14T11:20:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d41486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d415db30 0x7fa9d4148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d416a770 con 0x7fa9d406c700 2024-05-14T11:20:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d415df40 con 0x7fa9d40d1670 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9dc400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d416a8f0 con 0x7fa9d415db30 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49776/0 (socket says 172.21.15.116:49776) 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 -- 172.21.15.116:0/46545019 learned_addr learned my addr 172.21.15.116:0/46545019 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9da400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d415db30 0x7fa9d4148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9000640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d41486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 -- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d415db30 msgr2=0x7fa9d4148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d415db30 0x7fa9d4148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 -- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 msgr2=0x7fa9d41486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.118+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d41486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d9a00640 1 -- 172.21.15.116:0/46545019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d4074450 con 0x7fa9d406c700 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c460 tx=0x7fa9c800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 -- 172.21.15.116:0/46545019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8017070 con 0x7fa9d406c700 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 -- 172.21.15.116:0/46545019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c8005d10 con 0x7fa9d406c700 2024-05-14T11:20:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d4073690 con 0x7fa9d406c700 2024-05-14T11:20:10.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 -- 172.21.15.116:0/46545019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80072b0 con 0x7fa9d406c700 2024-05-14T11:20:10.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d4073c50 con 0x7fa9d406c700 2024-05-14T11:20:10.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 -- 172.21.15.116:0/46545019 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9c801a070 con 0x7fa9d406c700 2024-05-14T11:20:10.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 0x7fa9b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.122+0000 7fa9d1600640 1 -- 172.21.15.116:0/46545019 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fa9c808e6f0 con 0x7fa9d406c700 2024-05-14T11:20:10.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9000640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 0x7fa9b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 Processor -- start 2024-05-14T11:20:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9000640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 0x7fa9b4071940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ad000 tx=0x7fa9c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 -- start start 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b168ff80 0x55b3b17e1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b16796b0 0x55b3b17e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3b167e620 con 0x55b3b12a77c0 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b3b1599740 con 0x55b3b168ff80 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b3b15e7090 con 0x55b3b16796b0 2024-05-14T11:20:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49784/0 (socket says 172.21.15.116:49784) 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 -- 172.21.15.116:0/1316178049 learned_addr learned my addr 172.21.15.116:0/1316178049 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9000640 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b168ff80 0x55b3b17e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9da400640 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b16796b0 0x55b3b17e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 -- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b16796b0 msgr2=0x55b3b17e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b16796b0 0x55b3b17e1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 -- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b168ff80 msgr2=0x55b3b17e1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b168ff80 0x55b3b17e1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 -- 172.21.15.116:0/1316178049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b177b0a0 con 0x55b3b12a77c0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa9c805ace0 tx=0x7fa9c805a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae485e736a221 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9c6000640 1 -- 172.21.15.116:0/1316178049 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80742b0 con 0x55b3b12a77c0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9c6000640 1 -- 172.21.15.116:0/1316178049 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c808ce50 con 0x55b3b12a77c0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.126+0000 7fa9c6000640 1 -- 172.21.15.116:0/1316178049 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c80638b0 con 0x55b3b12a77c0 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1316178049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 msgr2=0x55b3b17e1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa9c805ace0 tx=0x7fa9c805a860 comp rx=0 tx=0).stop 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1316178049 shutdown_connections 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b168ff80 0x55b3b17e1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b16796b0 0x55b3b17e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1316178049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b17e1320 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1316178049 >> 172.21.15.116:0/1316178049 conn(0x55b3b1766340 msgr2=0x55b3b1795d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1316178049 shutdown_connections 2024-05-14T11:20:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1316178049 wait complete. 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- start start 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 Processor -- start 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b12a77c0 0x55b3b179c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b17c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3b15e3050 con 0x55b3b16796b0 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b3b159fec0 con 0x55b3b12a77c0 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b3b1586c90 con 0x55b3b168ff80 2024-05-14T11:20:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9e3b88000 10 client.0 _ll_register_callbacks cb 0x55b3b1782e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49796/0 (socket says 172.21.15.116:49796) 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 -- 172.21.15.116:0/1103504377 learned_addr learned my addr 172.21.15.116:0/1103504377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9da400640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b17c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b12a77c0 0x55b3b179c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 -- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 msgr2=0x55b3b17c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b17c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 -- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b12a77c0 msgr2=0x55b3b179c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b12a77c0 0x55b3b179c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.130+0000 7fa9d9000640 1 -- 172.21.15.116:0/1103504377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b16916f0 con 0x55b3b16796b0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9d9000640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa9c001a9f0 tx=0x7fa9c001ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b52f4dc794ef7e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00b0040 con 0x55b3b16796b0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fa9c00b67a0 con 0x55b3b16796b0 2024-05-14T11:20:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c808ce50 con 0x55b3b16796b0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa9c00cf050 con 0x55b3b16796b0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 10 client.6333 fetch_fsmap learned FSMap version 45 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55b3b168d6a0 con 0x55b3b16796b0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c808ce50 con 0x55b3b16796b0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fa9c00d2020 con 0x55b3b16796b0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 10 client.6333 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 10 client.6333 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 20 client.6333 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b128c7d0 0x55b3b17f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 4 client.6333 mds_command: new command op to 6213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 --> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3b157fbe0 con 0x55b3b128c7d0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b128c7d0 0x55b3b17f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b128c7d0 0x55b3b17f28e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb00f12e89b4568b server_cookie=d4fb55acad81b5f7 in_seq=0 out_seq=0 2024-05-14T11:20:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 10 client.6333 ms_handle_connect on v2:172.21.15.116:6834/2430466110 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 1 -- 172.21.15.116:0/1103504377 <== mds.0 v2:172.21.15.116:6834/2430466110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3b157fbe0 con 0x55b3b128c7d0 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 10 client.6333 handle_command_reply: tid=0 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 client.6333 shutdown 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 msgr2=0x55b3b179c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa9c001a9f0 tx=0x7fa9c001ab80 comp rx=0 tx=0).stop 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 shutdown_connections 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b128c7d0 0x55b3b17f28e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b17c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b16796b0 0x55b3b179c8e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9baa00640 0 client.6333 ms_handle_reset on v2:172.21.15.116:6834/2430466110 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/1103504377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b12a77c0 0x55b3b179c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 >> 172.21.15.116:0/1103504377 conn(0x55b3b1766340 msgr2=0x55b3b1796490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 shutdown_connections 2024-05-14T11:20:10.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 1 -- 172.21.15.116:0/1103504377 wait complete. 2024-05-14T11:20:10.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.134+0000 7fa9e3b88000 20 client.6333 trim_cache size 0 max 16384 2024-05-14T11:20:10.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 Processor -- start 2024-05-14T11:20:10.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 -- start start 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b12a77c0 0x55b3b17c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b17c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d414c5e0 con 0x55b3b168ff80 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4167a80 con 0x55b3b16796b0 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d414bf90 con 0x55b3b12a77c0 2024-05-14T11:20:10.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49806/0 (socket says 172.21.15.116:49806) 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 -- 172.21.15.116:0/3320197153 learned_addr learned my addr 172.21.15.116:0/3320197153 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b12a77c0 0x55b3b17c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b17c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 -- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b12a77c0 msgr2=0x55b3b17c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b12a77c0 0x55b3b17c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 -- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 msgr2=0x55b3b17c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b17c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 -- 172.21.15.116:0/3320197153 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b15c0af0 con 0x55b3b168ff80 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9da400640 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4072800 tx=0x7fa9cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32342458b1ac81e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9c5600640 1 -- 172.21.15.116:0/3320197153 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc017820 con 0x55b3b168ff80 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9c5600640 1 -- 172.21.15.116:0/3320197153 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0179c0 con 0x55b3b168ff80 2024-05-14T11:20:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.154+0000 7fa9c5600640 1 -- 172.21.15.116:0/3320197153 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc017d70 con 0x55b3b168ff80 2024-05-14T11:20:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3320197153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 msgr2=0x55b3b17962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4072800 tx=0x7fa9cc009750 comp rx=0 tx=0).stop 2024-05-14T11:20:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3320197153 shutdown_connections 2024-05-14T11:20:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b17c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b168ff80 0x55b3b17962e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3320197153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b12a77c0 0x55b3b17c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3320197153 >> 172.21.15.116:0/3320197153 conn(0x55b3b1766340 msgr2=0x55b3b17c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3320197153 shutdown_connections 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3320197153 wait complete. 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- start start 2024-05-14T11:20:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 Processor -- start 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b18540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b180e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3b15f0c30 con 0x55b3b12a77c0 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b3b158a7d0 con 0x55b3b16796b0 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b3b16a1d80 con 0x55b3b168ff80 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9e3b88000 10 client.0 _ll_register_callbacks cb 0x7fa9c00ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49816/0 (socket says 172.21.15.116:49816) 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 -- 172.21.15.116:0/3306882070 learned_addr learned my addr 172.21.15.116:0/3306882070 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b18540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9da400640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b180e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 -- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 msgr2=0x55b3b180e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b180e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 -- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 msgr2=0x55b3b18540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b18540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.158+0000 7fa9d9a00640 1 -- 172.21.15.116:0/3306882070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b15bfe40 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9d9a00640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800ce20 tx=0x7fa9c8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4505e6f3f9120e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8080060 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fa9c8063690 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc0179c0 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa9c80742b0 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 10 client.6339 fetch_fsmap learned FSMap version 45 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55b3b16674e0 con 0x55b3b12a77c0 2024-05-14T11:20:10.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0179c0 con 0x55b3b12a77c0 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fa9c807bb60 con 0x55b3b12a77c0 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 10 client.6339 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 10 client.6339 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 20 client.6339 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b127a830 0x55b3b180f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 4 client.6339 mds_command: new command op to 6213 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 --> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] -- command(tid 0: {"prefix": "status"}) -- 0x55b3b15f4ca0 con 0x55b3b127a830 2024-05-14T11:20:10.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b127a830 0x55b3b180f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 10 client.6339 ms_handle_connect on v2:172.21.15.116:6834/2430466110 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9d9000640 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b127a830 0x55b3b180f100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea5a7f9e0d83fd3 server_cookie=49cb5b156465a702 in_seq=0 out_seq=0 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 1 -- 172.21.15.116:0/3306882070 <== mds.0 v2:172.21.15.116:6834/2430466110 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b3b15f4ca0 con 0x55b3b127a830 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 10 client.6339 handle_command_reply: tid=0 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 client.6339 shutdown 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 msgr2=0x55b3b1853b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800ce20 tx=0x7fa9c8000bd0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 shutdown_connections 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:6834/2430466110,v1:172.21.15.116:6836/2430466110] conn(0x55b3b127a830 0x55b3b180f100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3b16796b0 0x55b3b18540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b3b168ff80 0x55b3b180e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 --2- 172.21.15.116:0/3306882070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3b12a77c0 0x55b3b1853b70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9baa00640 0 client.6339 ms_handle_reset on v2:172.21.15.116:6834/2430466110 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 >> 172.21.15.116:0/3306882070 conn(0x55b3b1766340 msgr2=0x55b3b17edd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 shutdown_connections 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 1 -- 172.21.15.116:0/3306882070 wait complete. 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9e3b88000 20 client.6339 trim_cache size 0 max 16384 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:20:10.165 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6213, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 5.9828781019999999, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 45, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 72, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 72, 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 10.846034092 2024-05-14T11:20:10.166 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 msgr2=0x7fa9b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 0x7fa9b4071940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ad000 tx=0x7fa9c0008040 comp rx=0 tx=0).stop 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 msgr2=0x7fa9d409a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c460 tx=0x7fa9c800c920 comp rx=0 tx=0).stop 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 shutdown_connections 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9b406f430 0x7fa9b4071940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9d40d1670 0x7fa9d41486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9d415db30 0x7fa9d4148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 --2- 172.21.15.116:0/46545019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d406c700 0x7fa9d409a460 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 >> 172.21.15.116:0/46545019 conn(0x7fa9d40b4620 msgr2=0x7fa9d4150770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 shutdown_connections 2024-05-14T11:20:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.162+0000 7fa9dc400640 1 -- 172.21.15.116:0/46545019 wait complete. 2024-05-14T11:20:10.182 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:20:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.330+0000 7fe627a00640 1 Processor -- start 2024-05-14T11:20:10.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.330+0000 7fe627a00640 1 -- start start 2024-05-14T11:20:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe627a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe620156fa0 0x7fe62006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe627a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6200d1670 0x7fe6200d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe627a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6200d1fc0 con 0x7fe62006e9e0 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe627a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe6201573b0 con 0x7fe620156fa0 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe627a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe620161120 con 0x7fe6200d1670 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe620156fa0 0x7fe62006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe620156fa0 0x7fe62006ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49822/0 (socket says 172.21.15.116:49822) 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 -- 172.21.15.116:0/338165948 learned_addr learned my addr 172.21.15.116:0/338165948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe61fe00640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6200d1670 0x7fe6200d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 -- 172.21.15.116:0/338165948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6200d1670 msgr2=0x7fe6200d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:10.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6200d1670 0x7fe6200d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 -- 172.21.15.116:0/338165948 >> v1:172.21.15.149:6789/0 conn(0x7fe62006e9e0 legacy=0x7fe62006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.334+0000 7fe625000640 1 -- 172.21.15.116:0/338165948 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe620160510 con 0x7fe620156fa0 2024-05-14T11:20:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe625000640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe620156fa0 0x7fe62006ffe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe614009870 tx=0x7fe61401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07802d76c28e197 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe61f400640 1 -- 172.21.15.116:0/338165948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140a95f0 con 0x7fe620156fa0 2024-05-14T11:20:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe61f400640 1 -- 172.21.15.116:0/338165948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140a9790 con 0x7fe620156fa0 2024-05-14T11:20:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe61f400640 1 -- 172.21.15.116:0/338165948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140a9b60 con 0x7fe620156fa0 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- 172.21.15.116:0/338165948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe620156fa0 msgr2=0x7fe62006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe620156fa0 0x7fe62006ffe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe614009870 tx=0x7fe61401c530 comp rx=0 tx=0).stop 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- 172.21.15.116:0/338165948 shutdown_connections 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6200d1670 0x7fe6200d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 --2- 172.21.15.116:0/338165948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe620156fa0 0x7fe62006ffe0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- 172.21.15.116:0/338165948 >> 172.21.15.116:0/338165948 conn(0x7fe6200b4620 msgr2=0x7fe6200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- 172.21.15.116:0/338165948 shutdown_connections 2024-05-14T11:20:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- 172.21.15.116:0/338165948 wait complete. 2024-05-14T11:20:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 Processor -- start 2024-05-14T11:20:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.338+0000 7fe627a00640 1 -- start start 2024-05-14T11:20:10.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe62006e9e0 0x7fe620165810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 0x7fe620073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe620070520 con 0x7fe6200d1670 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200b07a0 con 0x7fe62006e9e0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200706a0 con 0x7fe620156fa0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49830/0 (socket says 172.21.15.116:49830) 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 -- 172.21.15.116:0/3254413423 learned_addr learned my addr 172.21.15.116:0/3254413423 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe625a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 0x7fe620073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe625000640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe62006e9e0 0x7fe620165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 -- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 msgr2=0x7fe620073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 0x7fe620073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 -- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe62006e9e0 msgr2=0x7fe620165810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe62006e9e0 0x7fe620165810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 -- 172.21.15.116:0/3254413423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6201662b0 con 0x7fe6200d1670 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe625a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 0x7fe620073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61fe00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e870 tx=0x7fe60c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c017070 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c00c940 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00ccf0 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe620166590 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201672b0 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.342+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62006a9c0 con 0x7fe6200d1670 2024-05-14T11:20:10.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.346+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe60c01a070 con 0x7fe6200d1670 2024-05-14T11:20:10.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.346+0000 7fe61cc00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 0x7fe600071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:10.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.346+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7fe60c08f5f0 con 0x7fe6200d1670 2024-05-14T11:20:10.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.346+0000 7fe625000640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 0x7fe600071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:10.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.346+0000 7fe625000640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 0x7fe600071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ae000 tx=0x7fe6140b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:10.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.350+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe60c05bd00 con 0x7fe6200d1670 2024-05-14T11:20:10.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.546+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe620154630 con 0x7fe6200d1670 2024-05-14T11:20:10.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.546+0000 7fe61cc00640 1 -- 172.21.15.116:0/3254413423 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fe60c060000 con 0x7fe6200d1670 2024-05-14T11:20:10.546 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:20:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 msgr2=0x7fe600071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 0x7fe600071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ae000 tx=0x7fe6140b3040 comp rx=0 tx=0).stop 2024-05-14T11:20:10.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 msgr2=0x7fe6201fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e870 tx=0x7fe60c00ec70 comp rx=0 tx=0).stop 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 shutdown_connections 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60006f400 0x7fe600071910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe620156fa0 0x7fe620073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6200d1670 0x7fe6201fcd50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 --2- 172.21.15.116:0/3254413423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe62006e9e0 0x7fe620165810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 >> 172.21.15.116:0/3254413423 conn(0x7fe6200b4620 msgr2=0x7fe62015f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 shutdown_connections 2024-05-14T11:20:10.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10.550+0000 7fe627a00640 1 -- 172.21.15.116:0/3254413423 wait complete. 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:20:10.568 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:10.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:10.569 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:20:10.577 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:20:10.577 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:20:10.577 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:20:10.577 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:20:10.578 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42192sec preferred_lft 42192sec 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:10.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:20:10.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.580 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:20:10.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:10.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:20:10.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:10.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:10.581 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:20:10.646 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:20:10.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:10.648 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:20:10.706 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:20:10.706 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:20:10.706 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:20:10.706 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:20:10.706 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:10.707 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:20:10.707 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:20:10.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:20:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:20:10.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:10.841 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:20:10.850 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:20:10.850 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:20:10.851 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:10.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:10.851 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:20:10.910 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:20:10.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:20:10.911 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:10.959 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:10.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:20:10.959 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:11.014 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:20:11.074 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:20:11.124 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:20:11.125 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:20:11.126 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:11.127 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:11.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.128 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:20:11.183 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:20:11.183 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:20:11.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:20:11.247 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:20:11.297 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:20:11.298 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:11.299 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.300 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:20:11.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:20:11.330+0000 7f3673561580 -1 init, newargv = 0x7f3658005d30 newargc=15 2024-05-14T11:20:11.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[41792]: starting ceph client 2024-05-14T11:20:11.339 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:20:11.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[41792]: starting fuse 2024-05-14T11:20:11.340 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:20:11.340 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:20:11.341 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:11.342 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:20:11.342 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:20:11.463 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.41792.asok 2024-05-14T11:20:11.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:20:11.474 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41792.asok 2024-05-14T11:20:11.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41792.asok status 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:20:11.651 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "41792", 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6351, 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:20:11.653 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 6351 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2316958597 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2316958597 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.6351 192.168.0.1:0/2316958597", 2024-05-14T11:20:11.654 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/2316958597", 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 45, 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 72, 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:20:11.655 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:11.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.668 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:20:11.668 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:11.677 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:20:11.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:11.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:11.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.739 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-14T11:20:11.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.799 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-14T11:20:11.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:11.851 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-14T11:20:11.902 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:20:11.908 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:20:11.947 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:20:12.008 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:20:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:20:12.017 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:20:12.017 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:20:12.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:20:12.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.266+0000 7f180f200640 1 Processor -- start 2024-05-14T11:20:12.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.266+0000 7f180f200640 1 -- start start 2024-05-14T11:20:12.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 0x7f18080aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f18080d1670 0x7f18080d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1808161600 0x7f1808161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f18080ae100 con 0x7f1808161600 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f18080ae870 con 0x7f18080d1670 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180f200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f180806ff20 con 0x7f180815db30 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1808161600 0x7f1808161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1808161600 0x7f1808161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49842/0 (socket says 172.21.15.116:49842) 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 -- 172.21.15.116:0/4010290782 learned_addr learned my addr 172.21.15.116:0/4010290782 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f17ff400640 1 --2- 172.21.15.116:0/4010290782 >> v2:172.21.15.149:3300/0 conn(0x7f18080d1670 0x7f18080d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f1807e00640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 0x7f18080aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 -- 172.21.15.116:0/4010290782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 msgr2=0x7f18080aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 0x7f18080aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 -- 172.21.15.116:0/4010290782 >> v2:172.21.15.149:3300/0 conn(0x7f18080d1670 msgr2=0x7f18080d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 --2- 172.21.15.116:0/4010290782 >> v2:172.21.15.149:3300/0 conn(0x7f18080d1670 0x7f18080d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f180d200640 1 -- 172.21.15.116:0/4010290782 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18080d1fc0 con 0x7f1808161600 2024-05-14T11:20:12.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.270+0000 7f1807e00640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 0x7f18080aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:12.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180d200640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1808161600 0x7f1808161a10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f18080d2280 tx=0x7f180001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3aa7a798c2dee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f1807400640 1 -- 172.21.15.116:0/4010290782 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000a56d0 con 0x7f1808161600 2024-05-14T11:20:12.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f1807400640 1 -- 172.21.15.116:0/4010290782 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18000a5870 con 0x7f1808161600 2024-05-14T11:20:12.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f1807400640 1 -- 172.21.15.116:0/4010290782 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000a5c40 con 0x7f1808161600 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 -- 172.21.15.116:0/4010290782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1808161600 msgr2=0x7f1808161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1808161600 0x7f1808161a10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f18080d2280 tx=0x7f180001c960 comp rx=0 tx=0).stop 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 -- 172.21.15.116:0/4010290782 shutdown_connections 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1808161600 0x7f1808161a10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 --2- 172.21.15.116:0/4010290782 >> v2:172.21.15.149:3300/0 conn(0x7f18080d1670 0x7f18080d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 --2- 172.21.15.116:0/4010290782 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f180815db30 0x7f18080aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 -- 172.21.15.116:0/4010290782 >> 172.21.15.116:0/4010290782 conn(0x7f18080b4620 msgr2=0x7f18080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 -- 172.21.15.116:0/4010290782 shutdown_connections 2024-05-14T11:20:12.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.274+0000 7f180f200640 1 -- 172.21.15.116:0/4010290782 wait complete. 2024-05-14T11:20:12.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 Processor -- start 2024-05-14T11:20:12.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- start start 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18080d1670 0x7f180814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1808161600 0x7f1808072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1808162360 con 0x7f180815db30 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18080aefe0 con 0x7f1808161600 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f180815df40 con 0x7f18080d1670 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49854/0 (socket says 172.21.15.116:49854) 2024-05-14T11:20:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 -- 172.21.15.116:0/776568520 learned_addr learned my addr 172.21.15.116:0/776568520 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f1807e00640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18080d1670 0x7f180814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180d200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1808161600 0x7f1808072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 -- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18080d1670 msgr2=0x7f180814dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18080d1670 0x7f180814dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 -- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1808161600 msgr2=0x7f1808072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1808161600 0x7f1808072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 -- 172.21.15.116:0/776568520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081613a0 con 0x7f180815db30 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f17ff400640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e8a0 tx=0x7f17f000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0017070 con 0x7f180815db30 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f000c940 con 0x7f180815db30 2024-05-14T11:20:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000cd10 con 0x7f180815db30 2024-05-14T11:20:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18080743b0 con 0x7f180815db30 2024-05-14T11:20:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.278+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180815f6e0 con 0x7f180815db30 2024-05-14T11:20:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180815d050 con 0x7f180815db30 2024-05-14T11:20:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17f001a070 con 0x7f180815db30 2024-05-14T11:20:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f1804c00640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 0x7f17e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f17f008f2b0 con 0x7f180815db30 2024-05-14T11:20:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f1807e00640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 0x7f17e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.282+0000 7f1807e00640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 0x7f17e4071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d8d0 tx=0x7f17f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.286+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f17f005bff0 con 0x7f180815db30 2024-05-14T11:20:12.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.458+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f180806a9c0 con 0x7f180815db30 2024-05-14T11:20:12.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f1804c00640 1 -- 172.21.15.116:0/776568520 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7f17f005b8f0 con 0x7f180815db30 2024-05-14T11:20:12.460 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:20:12.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 msgr2=0x7f17e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 0x7f17e4071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d8d0 tx=0x7f17f8008040 comp rx=0 tx=0).stop 2024-05-14T11:20:12.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 msgr2=0x7f1808073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e8a0 tx=0x7f17f000ed60 comp rx=0 tx=0).stop 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 shutdown_connections 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17e406f400 0x7f17e4071910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1808161600 0x7f1808072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f180815db30 0x7f1808073e70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 --2- 172.21.15.116:0/776568520 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18080d1670 0x7f180814dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 >> 172.21.15.116:0/776568520 conn(0x7f18080b4620 msgr2=0x7f1808068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 shutdown_connections 2024-05-14T11:20:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.462+0000 7f180f200640 1 -- 172.21.15.116:0/776568520 wait complete. 2024-05-14T11:20:12.481 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:20:12.481 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:20:12.490 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.678+0000 7f32f0600640 1 Processor -- start 2024-05-14T11:20:12.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.678+0000 7f32f0600640 1 -- start start 2024-05-14T11:20:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32f0600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32e8156fa0 0x7f32e806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32f0600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 0x7f32e80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32f0600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e8160510 con 0x7f32e80d1670 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32f0600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f32e81573b0 con 0x7f32e8156fa0 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32f0600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f32e8161120 con 0x7f32e80af380 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32e8156fa0 0x7f32e806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32ed200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f32e80d1670 0x7f32e80d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36536/0 (socket says 172.21.15.116:36536) 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32ed200640 1 -- 172.21.15.116:0/3801615240 learned_addr learned my addr 172.21.15.116:0/3801615240 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32ee600640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 0x7f32e80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 -- 172.21.15.116:0/3801615240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 msgr2=0x7f32e80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 0x7f32e80af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 -- 172.21.15.116:0/3801615240 >> v1:172.21.15.116:6789/0 conn(0x7f32e80d1670 legacy=0x7f32e80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 -- 172.21.15.116:0/3801615240 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e81607f0 con 0x7f32e8156fa0 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32e7e00640 1 -- 172.21.15.116:0/3801615240 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452780924 0 0) 0x7f32e8160510 con 0x7f32e80d1670 2024-05-14T11:20:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32ee600640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 0x7f32e80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:12.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.682+0000 7f32edc00640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e8156fa0 0x7f32e806ffe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097c0 tx=0x7f32dc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9ea5b4caa8fa4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32e7e00640 1 -- 172.21.15.116:0/3801615240 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a7070 con 0x7f32e8156fa0 2024-05-14T11:20:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32e7e00640 1 -- 172.21.15.116:0/3801615240 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc0b0070 con 0x7f32e8156fa0 2024-05-14T11:20:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32e7e00640 1 -- 172.21.15.116:0/3801615240 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0ab890 con 0x7f32e8156fa0 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- 172.21.15.116:0/3801615240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e8156fa0 msgr2=0x7f32e806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e8156fa0 0x7f32e806ffe0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097c0 tx=0x7f32dc01c470 comp rx=0 tx=0).stop 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- 172.21.15.116:0/3801615240 shutdown_connections 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e80af380 0x7f32e80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 --2- 172.21.15.116:0/3801615240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e8156fa0 0x7f32e806ffe0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- 172.21.15.116:0/3801615240 >> 172.21.15.116:0/3801615240 conn(0x7f32e80b4620 msgr2=0x7f32e80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- 172.21.15.116:0/3801615240 shutdown_connections 2024-05-14T11:20:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- 172.21.15.116:0/3801615240 wait complete. 2024-05-14T11:20:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 Processor -- start 2024-05-14T11:20:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.686+0000 7f32f0600640 1 -- start start 2024-05-14T11:20:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 0x7f32e81fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 0x7f32e8150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32e80b0660 con 0x7f32e80d1670 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32e80b04e0 con 0x7f32e80af380 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f32e8169810 con 0x7f32e8156fa0 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49862/0 (socket says 172.21.15.116:49862) 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 -- 172.21.15.116:0/2179551177 learned_addr learned my addr 172.21.15.116:0/2179551177 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ee600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 0x7f32e8150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32edc00640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 0x7f32e81fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 -- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 msgr2=0x7f32e8150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 0x7f32e8150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 -- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 msgr2=0x7f32e81fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 0x7f32e81fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 -- 172.21.15.116:0/2179551177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e80b1410 con 0x7f32e80d1670 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ee600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 0x7f32e8150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32edc00640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 0x7f32e81fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32ed200640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c460 tx=0x7f32d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4018070 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d4007500 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4004a30 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e8159710 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e80cf740 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e806a9c0 con 0x7f32e80d1670 2024-05-14T11:20:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.690+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f32d401b070 con 0x7f32e80d1670 2024-05-14T11:20:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.694+0000 7f32e5600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.694+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f32d408e6f0 con 0x7f32e80d1670 2024-05-14T11:20:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.694+0000 7f32edc00640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:12.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.694+0000 7f32edc00640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 0x7f32c8071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0af000 tx=0x7f32dc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:12.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.698+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f32d405b440 con 0x7f32e80d1670 2024-05-14T11:20:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.862+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32e80cf1f0 con 0x7f32e80d1670 2024-05-14T11:20:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32e5600640 1 -- 172.21.15.116:0/2179551177 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7f32d405ad40 con 0x7f32e80d1670 2024-05-14T11:20:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 msgr2=0x7f32c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 0x7f32c8071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0af000 tx=0x7f32dc0a7000 comp rx=0 tx=0).stop 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 msgr2=0x7f32e80b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c460 tx=0x7f32d400c920 comp rx=0 tx=0).stop 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 shutdown_connections 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32c806f400 0x7f32c8071910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32e8156fa0 0x7f32e8150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32e80d1670 0x7f32e80b3860 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 --2- 172.21.15.116:0/2179551177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80af380 0x7f32e81fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 >> 172.21.15.116:0/2179551177 conn(0x7f32e80b4620 msgr2=0x7f32e815b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 shutdown_connections 2024-05-14T11:20:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:12.866+0000 7f32f0600640 1 -- 172.21.15.116:0/2179551177 wait complete. 2024-05-14T11:20:12.886 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:20:12.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:12.887 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:12.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[41792]: fuse finished with error 0 and tester_r 0 2024-05-14T11:20:12.908 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:20:19.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:20:19.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:19.009 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:20:19.020 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:19.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.222+0000 7f2282400640 1 Processor -- start 2024-05-14T11:20:19.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.222+0000 7f2282400640 1 -- start start 2024-05-14T11:20:19.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 0x7f227c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 0x7f227c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227c161600 0x7f227c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f227c0ae100 con 0x7f227c161600 2024-05-14T11:20:19.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f227c0ae870 con 0x7f227c0d1670 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f2282400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f227c06ff20 con 0x7f227c15db30 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227c161600 0x7f227c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f227c161600 0x7f227c161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55800/0 (socket says 172.21.15.116:55800) 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 -- 172.21.15.116:0/485382111 learned_addr learned my addr 172.21.15.116:0/485382111 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227aa00640 1 --2- 172.21.15.116:0/485382111 >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 0x7f227c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227b400640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 0x7f227c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 -- 172.21.15.116:0/485382111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 msgr2=0x7f227c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 0x7f227c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 -- 172.21.15.116:0/485382111 >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 msgr2=0x7f227c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 --2- 172.21.15.116:0/485382111 >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 0x7f227c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227be00640 1 -- 172.21.15.116:0/485382111 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c0d1fc0 con 0x7f227c161600 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227b400640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 0x7f227c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:19.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.226+0000 7f227aa00640 1 --2- 172.21.15.116:0/485382111 >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 0x7f227c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:19.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f227be00640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c161a10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f227c0d2280 tx=0x7f227401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64108f6ee630b053 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f227a000640 1 -- 172.21.15.116:0/485382111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740a56d0 con 0x7f227c161600 2024-05-14T11:20:19.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f227a000640 1 -- 172.21.15.116:0/485382111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22740a5870 con 0x7f227c161600 2024-05-14T11:20:19.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f227a000640 1 -- 172.21.15.116:0/485382111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740a5c40 con 0x7f227c161600 2024-05-14T11:20:19.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 -- 172.21.15.116:0/485382111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 msgr2=0x7f227c161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c161a10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f227c0d2280 tx=0x7f227401c960 comp rx=0 tx=0).stop 2024-05-14T11:20:19.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 -- 172.21.15.116:0/485382111 shutdown_connections 2024-05-14T11:20:19.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c161a10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 --2- 172.21.15.116:0/485382111 >> v2:172.21.15.149:3300/0 conn(0x7f227c0d1670 0x7f227c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 --2- 172.21.15.116:0/485382111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c15db30 0x7f227c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 -- 172.21.15.116:0/485382111 >> 172.21.15.116:0/485382111 conn(0x7f227c0b4620 msgr2=0x7f227c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:19.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 -- 172.21.15.116:0/485382111 shutdown_connections 2024-05-14T11:20:19.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.230+0000 7f2282400640 1 -- 172.21.15.116:0/485382111 wait complete. 2024-05-14T11:20:19.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 Processor -- start 2024-05-14T11:20:19.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- start start 2024-05-14T11:20:19.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c0d1670 0x7f227c09aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c15db30 0x7f227c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f227c162360 con 0x7f227c161600 2024-05-14T11:20:19.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f227c0aefe0 con 0x7f227c15db30 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f227c15df40 con 0x7f227c0d1670 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55816/0 (socket says 172.21.15.116:55816) 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 -- 172.21.15.116:0/1843936278 learned_addr learned my addr 172.21.15.116:0/1843936278 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227aa00640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c15db30 0x7f227c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227b400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c0d1670 0x7f227c09aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 -- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c0d1670 msgr2=0x7f227c09aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c0d1670 0x7f227c09aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 -- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c15db30 msgr2=0x7f227c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c15db30 0x7f227c072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 -- 172.21.15.116:0/1843936278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c14e060 con 0x7f227c161600 2024-05-14T11:20:19.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f227be00640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f22740af040 tx=0x7f2274019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740bc020 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22740b8690 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274009510 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227c14e340 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.234+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227c0734b0 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227c06a9c0 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22740bf020 con 0x7f227c161600 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f2273400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 0x7f225c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f2274129f80 con 0x7f227c161600 2024-05-14T11:20:19.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f227b400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 0x7f225c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.238+0000 7f227b400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 0x7f225c071940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f226c006e90 tx=0x7f226c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.242+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f22740c6070 con 0x7f227c161600 2024-05-14T11:20:19.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.438+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f227c14e510 con 0x7f227c161600 2024-05-14T11:20:19.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.438+0000 7f2273400640 1 -- 172.21.15.116:0/1843936278 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7f227c0d1fc0 con 0x7f227c161600 2024-05-14T11:20:19.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:19.440 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:06.615674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/2430466110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2430466110},{"type":"v1","addr":"172.21.15.116:6836","nonce":2430466110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6213,"qdb_cluster":[6213]},"id":5}]} 2024-05-14T11:20:19.442 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 45 2024-05-14T11:20:19.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 msgr2=0x7f225c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 0x7f225c071940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f226c006e90 tx=0x7f226c008040 comp rx=0 tx=0).stop 2024-05-14T11:20:19.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 msgr2=0x7f227c167eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f22740af040 tx=0x7f2274019700 comp rx=0 tx=0).stop 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 shutdown_connections 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f225c06f430 0x7f225c071940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c15db30 0x7f227c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f227c161600 0x7f227c167eb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 --2- 172.21.15.116:0/1843936278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f227c0d1670 0x7f227c09aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 >> 172.21.15.116:0/1843936278 conn(0x7f227c0b4620 msgr2=0x7f227c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 shutdown_connections 2024-05-14T11:20:19.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.442+0000 7f2282400640 1 -- 172.21.15.116:0/1843936278 wait complete. 2024-05-14T11:20:19.461 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-14T11:20:19.637 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:20:19.637 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:20:19.638 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:20:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.782+0000 7f238c200640 1 Processor -- start 2024-05-14T11:20:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238c200640 1 -- start start 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238c200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f2384168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238c200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f238406ff20 con 0x7f238415db30 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238c200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2384070470 con 0x7f23840d1670 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238c200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f238406d710 con 0x7f2384166750 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f238415db30 0x7f23840aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48554/0 (socket says 172.21.15.116:48554) 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 -- 172.21.15.116:0/2333251086 learned_addr learned my addr 172.21.15.116:0/2333251086 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238a200640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f2384168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112560543 0 0) 0x7f238406ff20 con 0x7f238415db30 2024-05-14T11:20:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f236c003530 con 0x7f238415db30 2024-05-14T11:20:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1940963523 0 0) 0x7f236c003530 con 0x7f238415db30 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 msgr2=0x7f2384168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f2384168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 >> v1:172.21.15.149:6789/0 conn(0x7f23840d1670 legacy=0x7f23840d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2384070750 con 0x7f238415db30 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f2378001700 con 0x7f238415db30 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 >> v1:172.21.15.116:6789/0 conn(0x7f238415db30 legacy=0x7f23840aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c003a40 0x7f236c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f236c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c004e70 0x7f236c0052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f236c00bca0 con 0x7f236c003a40 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f236c00be50 con 0x7f236c004e70 2024-05-14T11:20:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f236c00c050 con 0x7f2384166750 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c003a40 0x7f236c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2388e00640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f236c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f238a200640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c004e70 0x7f236c0052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 -- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 msgr2=0x7f236c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384166750 0x7f236c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 -- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c004e70 msgr2=0x7f236c0052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c004e70 0x7f236c0052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.786+0000 7f2389800640 1 -- 172.21.15.116:0/2333251086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c003530 con 0x7f236c003a40 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f2389800640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c003a40 0x7f236c003eb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f237800c240 tx=0x7f237801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9005f5992e68a880 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378007d70 con 0x7f236c003a40 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23780a9020 con 0x7f236c003a40 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f2383e00640 1 -- 172.21.15.116:0/2333251086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378005b20 con 0x7f236c003a40 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 -- 172.21.15.116:0/2333251086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c003a40 msgr2=0x7f236c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 --2- 172.21.15.116:0/2333251086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c003a40 0x7f236c003eb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f237800c240 tx=0x7f237801cbc0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238a200640 1 -- 172.21.15.116:0/2333251086 reap_dead start 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 -- 172.21.15.116:0/2333251086 shutdown_connections 2024-05-14T11:20:19.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 -- 172.21.15.116:0/2333251086 >> 172.21.15.116:0/2333251086 conn(0x7f23840b4620 msgr2=0x7f23840b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:19.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 -- 172.21.15.116:0/2333251086 shutdown_connections 2024-05-14T11:20:19.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.790+0000 7f238c200640 1 -- 172.21.15.116:0/2333251086 wait complete. 2024-05-14T11:20:19.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 Processor -- start 2024-05-14T11:20:19.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 -- start start 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c003a40 0x7f238414e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384159ed0 0x7f238415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2384169590 con 0x7f238415c210 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23840ae870 con 0x7f236c003a40 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238c200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2384169710 con 0x7f2384159ed0 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238a200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238a200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55834/0 (socket says 172.21.15.116:55834) 2024-05-14T11:20:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f2389800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c003a40 0x7f238414e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.794+0000 7f238a200640 1 -- 172.21.15.116:0/2860717117 learned_addr learned my addr 172.21.15.116:0/2860717117 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f2388e00640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384159ed0 0x7f238415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 -- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384159ed0 msgr2=0x7f238415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384159ed0 0x7f238415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 -- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c003a40 msgr2=0x7f238414e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c003a40 0x7f238414e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 -- 172.21.15.116:0/2860717117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238415a820 con 0x7f238415c210 2024-05-14T11:20:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238a200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23841626c0 tx=0x7f237c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c011e90 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c01c070 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c003be0 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238414fbf0 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238414f630 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238414f0a0 con 0x7f238415c210 2024-05-14T11:20:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.798+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f237c004030 con 0x7f238415c210 2024-05-14T11:20:19.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.802+0000 7f2381600640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 0x7f2364071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:19.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.802+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6101+0+0 (secure 0 0 0) 0x7f237c08fc50 con 0x7f238415c210 2024-05-14T11:20:19.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.802+0000 7f2389800640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 0x7f2364071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:19.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.802+0000 7f2389800640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 0x7f2364071ab0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23780aa000 tx=0x7f23780b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:19.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.806+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f238415a820 con 0x7f238415c210 2024-05-14T11:20:19.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.994+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f238406a9c0 con 0x7f238415c210 2024-05-14T11:20:19.995 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:20:19.994+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:20:19.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:19.998+0000 7f2381600640 1 -- 172.21.15.116:0/2860717117 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f237c010040 con 0x7f238415c210 2024-05-14T11:20:19.997 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:20:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 msgr2=0x7f2364071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 0x7f2364071ab0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23780aa000 tx=0x7f23780b1040 comp rx=0 tx=0).stop 2024-05-14T11:20:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 msgr2=0x7f238415c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f23841626c0 tx=0x7f237c00f580 comp rx=0 tx=0).stop 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 shutdown_connections 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f236406f5a0 0x7f2364071ab0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c210 0x7f238415c620 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2384159ed0 0x7f238415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 --2- 172.21.15.116:0/2860717117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f236c003a40 0x7f238414e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 >> 172.21.15.116:0/2860717117 conn(0x7f23840b4620 msgr2=0x7f23840b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 shutdown_connections 2024-05-14T11:20:20.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.002+0000 7f238c200640 1 -- 172.21.15.116:0/2860717117 wait complete. 2024-05-14T11:20:20.008 INFO:tasks.ceph.mds.c.smithi116.stderr: -3525> 2024-05-14T11:20:06.158+0000 7fcc74e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:20:20.023 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-14T11:20:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.082+0000 7f8a17cd3080 1 Processor -- start 2024-05-14T11:20:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- start start 2024-05-14T11:20:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558ebaa24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558ebaa248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558eba997f60 con 0x558eba951100 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558eba8f4aa0 con 0x558eba997570 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558eba998200 con 0x558ebaa08560 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55848/0 (socket says 172.21.15.116:55848) 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 -- 172.21.15.116:0/780129001 learned_addr learned my addr 172.21.15.116:0/780129001 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a0be00640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558ebaa24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a11800640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558ebaa248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 -- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 msgr2=0x558ebaa248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558ebaa248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 -- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 msgr2=0x558ebaa24390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558ebaa24390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 -- 172.21.15.116:0/780129001 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eba979ac0 con 0x558eba951100 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a11800640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558ebaa248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a10e00640 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b790 tx=0x7f8a04003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f079072f733272 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a0aa00640 1 -- 172.21.15.116:0/780129001 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b4070 con 0x558eba951100 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a0aa00640 1 -- 172.21.15.116:0/780129001 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a040be070 con 0x558eba951100 2024-05-14T11:20:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a0aa00640 1 -- 172.21.15.116:0/780129001 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b9440 con 0x558eba951100 2024-05-14T11:20:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- 172.21.15.116:0/780129001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 msgr2=0x558eba978850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b790 tx=0x7f8a04003030 comp rx=0 tx=0).stop 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- 172.21.15.116:0/780129001 shutdown_connections 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558ebaa24390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558ebaa248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/780129001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba978850 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- 172.21.15.116:0/780129001 >> 172.21.15.116:0/780129001 conn(0x558eba939e10 msgr2=0x558ebaa24f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- 172.21.15.116:0/780129001 shutdown_connections 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.086+0000 7f8a17cd3080 1 -- 172.21.15.116:0/780129001 wait complete. 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 Processor -- start 2024-05-14T11:20:20.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 -- start start 2024-05-14T11:20:20.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba996ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558eba976780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558eba976cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558eba979d60 con 0x558eba951100 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558eba97d960 con 0x558eba997570 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558eba9712c0 con 0x558ebaa08560 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba996ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba996ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55850/0 (socket says 172.21.15.116:55850) 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 -- 172.21.15.116:0/2921035242 learned_addr learned my addr 172.21.15.116:0/2921035242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a11800640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558eba976cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a0be00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558eba976780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 -- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 msgr2=0x558eba976cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ebaa08560 0x558eba976cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 -- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 msgr2=0x558eba976780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558eba997570 0x558eba976780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 -- 172.21.15.116:0/2921035242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eba971560 con 0x558eba951100 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a10e00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba996ce0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8a040bb040 tx=0x7f8a0401a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a09600640 1 -- 172.21.15.116:0/2921035242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b4040 con 0x558eba951100 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a09600640 1 -- 172.21.15.116:0/2921035242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a040c8070 con 0x558eba951100 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.090+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558eba971800 con 0x558eba951100 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a09600640 1 -- 172.21.15.116:0/2921035242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040afcb0 con 0x558eba951100 2024-05-14T11:20:20.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebaa076b0 con 0x558eba951100 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a09600640 1 -- 172.21.15.116:0/2921035242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8a040cd460 con 0x558eba951100 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a09600640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89f006dde0 0x7f89f00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a09600640 1 -- 172.21.15.116:0/2921035242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8a0412d730 con 0x558eba951100 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a0be00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89f006dde0 0x7f89f00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x558eba9731f0 0x558ebaac67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73) -- 0x558ebaac6ce0 con 0x558eba9731f0 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.094+0000 7f8a11800640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x558eba9731f0 0x558ebaac67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.098+0000 7f8a0be00640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89f006dde0 0x7f89f00702f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89fc000c10 tx=0x7f89fc001730 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.098+0000 7f8a11800640 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x558eba9731f0 0x558ebaac67a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a11800640 1 -- 172.21.15.116:0/2921035242 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v73'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8a0c00e9a0 con 0x558eba9731f0 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x558eba9731f0 msgr2=0x558ebaac67a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x558eba9731f0 0x558ebaac67a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89f006dde0 msgr2=0x7f89f00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f89f006dde0 0x7f89f00702f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89fc000c10 tx=0x7f89fc001730 comp rx=0 tx=0).stop 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 msgr2=0x558eba996ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 --2- 172.21.15.116:0/2921035242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558eba951100 0x558eba996ce0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8a040bb040 tx=0x7f8a0401a900 comp rx=0 tx=0).stop 2024-05-14T11:20:20.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a11800640 1 -- 172.21.15.116:0/2921035242 reap_dead start 2024-05-14T11:20:20.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 shutdown_connections 2024-05-14T11:20:20.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 >> 172.21.15.116:0/2921035242 conn(0x558eba939e10 msgr2=0x558eba97ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:20.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 shutdown_connections 2024-05-14T11:20:20.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.102+0000 7f8a17cd3080 1 -- 172.21.15.116:0/2921035242 wait complete. 2024-05-14T11:20:20.106 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:20:20.133 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:20:20.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 Processor -- start 2024-05-14T11:20:20.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 -- start start 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f38bc15c1d0 0x7f38bc0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc06cc10 con 0x7f38bc0cbd60 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc06cef0 con 0x7f38bc0cdd40 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38c1200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc06d790 con 0x7f38bc15c1d0 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38b9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f38bc15c1d0 0x7f38bc0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38ba800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f38bc0cbd60 0x7f38bc0cc170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48584/0 (socket says 172.21.15.116:48584) 2024-05-14T11:20:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38b9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f38bc15c1d0 0x7f38bc0d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55860/0 (socket says 172.21.15.116:55860) 2024-05-14T11:20:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38ba800640 1 -- 172.21.15.116:0/3442382548 learned_addr learned my addr 172.21.15.116:0/3442382548 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38b9e00640 1 -- 172.21.15.116:0/3442382548 >> v1:172.21.15.149:6789/0 conn(0x7f38bc0cdd40 legacy=0x7f38bc0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38b9e00640 1 -- 172.21.15.116:0/3442382548 >> v1:172.21.15.116:6789/0 conn(0x7f38bc0cbd60 legacy=0x7f38bc0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.266+0000 7f38b9e00640 1 -- 172.21.15.116:0/3442382548 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc06d1d0 con 0x7f38bc15c1d0 2024-05-14T11:20:20.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38b9e00640 1 --2- 172.21.15.116:0/3442382548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc15c1d0 0x7f38bc0d0c50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0097c0 tx=0x7f38ac01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722cee5c7ef7f999 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38b8a00640 1 -- 172.21.15.116:0/3442382548 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0b0070 con 0x7f38bc15c1d0 2024-05-14T11:20:20.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38b8a00640 1 -- 172.21.15.116:0/3442382548 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac0ab600 con 0x7f38bc15c1d0 2024-05-14T11:20:20.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38b8a00640 1 -- 172.21.15.116:0/3442382548 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0ab9d0 con 0x7f38bc15c1d0 2024-05-14T11:20:20.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 -- 172.21.15.116:0/3442382548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc15c1d0 msgr2=0x7f38bc0d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 --2- 172.21.15.116:0/3442382548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc15c1d0 0x7f38bc0d0c50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0097c0 tx=0x7f38ac01c380 comp rx=0 tx=0).stop 2024-05-14T11:20:20.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 -- 172.21.15.116:0/3442382548 shutdown_connections 2024-05-14T11:20:20.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 --2- 172.21.15.116:0/3442382548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc15c1d0 0x7f38bc0d0c50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 -- 172.21.15.116:0/3442382548 >> 172.21.15.116:0/3442382548 conn(0x7f38bc0b4720 msgr2=0x7f38bc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:20.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 -- 172.21.15.116:0/3442382548 shutdown_connections 2024-05-14T11:20:20.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.270+0000 7f38c1200640 1 -- 172.21.15.116:0/3442382548 wait complete. 2024-05-14T11:20:20.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 Processor -- start 2024-05-14T11:20:20.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 -- start start 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38bc0cdd40 0x7f38bc15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f38bc15c1d0 0x7f38bc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0ccc10 con 0x7f38bc0cbd60 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0cca90 con 0x7f38bc0cdd40 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38c1200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc0ccd90 con 0x7f38bc15c1d0 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55864/0 (socket says 172.21.15.116:55864) 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38b9e00640 1 -- 172.21.15.116:0/3909745580 learned_addr learned my addr 172.21.15.116:0/3909745580 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38ba800640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f38bc15c1d0 0x7f38bc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.274+0000 7f38b9400640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38bc0cdd40 0x7f38bc15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 -- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f38bc15c1d0 msgr2=0x7f38bc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f38bc15c1d0 0x7f38bc0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 -- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38bc0cdd40 msgr2=0x7f38bc15f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38bc0cdd40 0x7f38bc15f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 -- 172.21.15.116:0/3909745580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1671a0 con 0x7f38bc0cbd60 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b9e00640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0ae040 tx=0x7f38ac01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0b0050 con 0x7f38bc0cbd60 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac0bc070 con 0x7f38bc0cbd60 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc074d50 con 0x7f38bc0cbd60 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0b5a70 con 0x7f38bc0cbd60 2024-05-14T11:20:20.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc073cb0 con 0x7f38bc0cbd60 2024-05-14T11:20:20.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc06a9c0 con 0x7f38bc0cbd60 2024-05-14T11:20:20.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.278+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38ac0ab4c0 con 0x7f38bc0cbd60 2024-05-14T11:20:20.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.282+0000 7f38b2000640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 0x7f389c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:20.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.282+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f38ac129a90 con 0x7f38bc0cbd60 2024-05-14T11:20:20.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.282+0000 7f38b9400640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 0x7f389c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:20.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.282+0000 7f38b9400640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 0x7f389c071940 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f38bc14d990 tx=0x7f38a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:20.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.286+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f38ac0c6020 con 0x7f38bc0cbd60 2024-05-14T11:20:20.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.478+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f38bc069de0 con 0x7f38bc0cbd60 2024-05-14T11:20:20.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:20.998+0000 7f38b2000640 1 -- 172.21.15.116:0/3909745580 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7f38ac016090 con 0x7f38bc0cbd60 2024-05-14T11:20:20.999 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 msgr2=0x7f389c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 0x7f389c071940 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f38bc14d990 tx=0x7f38a8009040 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 msgr2=0x7f38bc0732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0ae040 tx=0x7f38ac01abc0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 shutdown_connections 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f389c06f430 0x7f389c071940 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38bc0cdd40 0x7f38bc15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f38bc15c1d0 0x7f38bc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 --2- 172.21.15.116:0/3909745580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38bc0cbd60 0x7f38bc0732f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 >> 172.21.15.116:0/3909745580 conn(0x7f38bc0b4720 msgr2=0x7f38bc06ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 shutdown_connections 2024-05-14T11:20:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.002+0000 7f38c1200640 1 -- 172.21.15.116:0/3909745580 wait complete. 2024-05-14T11:20:21.028 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 Processor -- start 2024-05-14T11:20:21.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 -- start start 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f00d1d80 0x7f30f0168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f0161f30 con 0x7f30f0161b20 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f0073550 con 0x7f30f006f620 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30f0164ad0 con 0x7f30f00d1d80 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30efe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f30f0161b20 0x7f30f00d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48592/0 (socket says 172.21.15.116:48592) 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30efe00640 1 -- 172.21.15.116:0/1859710071 learned_addr learned my addr 172.21.15.116:0/1859710071 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:21.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30ef400640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f00d1d80 0x7f30f0168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419605729 0 0) 0x7f30e0001120 con 0x7f30f0161b20 2024-05-14T11:20:21.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30d4003530 con 0x7f30f0161b20 2024-05-14T11:20:21.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670920490 0 0) 0x7f30f0073550 con 0x7f30f006f620 2024-05-14T11:20:21.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30e0001120 con 0x7f30f006f620 2024-05-14T11:20:21.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1694083240 0 0) 0x7f30d4003530 con 0x7f30f0161b20 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f00d1d80 msgr2=0x7f30f0168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f00d1d80 0x7f30f0168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 >> v1:172.21.15.149:6789/0 conn(0x7f30f006f620 legacy=0x7f30f006fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f0164db0 con 0x7f30f0161b20 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f30e00018d0 con 0x7f30f0161b20 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 >> v1:172.21.15.116:6789/0 conn(0x7f30f0161b20 legacy=0x7f30f00d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d4003c20 0x7f30d4004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30d4004810 0x7f30d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0164ad0 con 0x7f30d40052c0 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30d400c240 con 0x7f30d4003c20 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30d400c440 con 0x7f30d4004810 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30ef400640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f00d1d80 0x7f30f0168580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:20:21.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30f5000640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30efe00640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d4003c20 0x7f30d4004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.222+0000 7f30ef400640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30d4004810 0x7f30d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 -- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30d4004810 msgr2=0x7f30d4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30d4004810 0x7f30d4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 -- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d4003c20 msgr2=0x7f30d4004070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d4003c20 0x7f30d4004070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 -- 172.21.15.116:0/1859710071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d4003530 con 0x7f30d40052c0 2024-05-14T11:20:21.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30ef400640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30d4004810 0x7f30d4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:21.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30d400b9d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f30f0070c70 tx=0x7f30e801f030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a8bbd3a6fb839 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:21.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8001c50 con 0x7f30d40052c0 2024-05-14T11:20:21.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8006d60 con 0x7f30d40052c0 2024-05-14T11:20:21.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30eea00640 1 -- 172.21.15.116:0/1859710071 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e80a8ca0 con 0x7f30d40052c0 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f7000640 1 -- 172.21.15.116:0/1859710071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 msgr2=0x7f30d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f7000640 1 --2- 172.21.15.116:0/1859710071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30d400b9d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f30f0070c70 tx=0x7f30e801f030 comp rx=0 tx=0).stop 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.226+0000 7f30f5000640 1 -- 172.21.15.116:0/1859710071 reap_dead start 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- 172.21.15.116:0/1859710071 shutdown_connections 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- 172.21.15.116:0/1859710071 >> 172.21.15.116:0/1859710071 conn(0x7f30f00b4620 msgr2=0x7f30f00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- 172.21.15.116:0/1859710071 shutdown_connections 2024-05-14T11:20:21.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- 172.21.15.116:0/1859710071 wait complete. 2024-05-14T11:20:21.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 Processor -- start 2024-05-14T11:20:21.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- start start 2024-05-14T11:20:21.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30f01587e0 0x7f30f009cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f009d470 0x7f30f00b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00d1d80 con 0x7f30d40052c0 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.230+0000 7f30f7000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0165370 con 0x7f30f01587e0 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30f7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00d1f00 con 0x7f30f009d470 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55888/0 (socket says 172.21.15.116:55888) 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 -- 172.21.15.116:0/162445490 learned_addr learned my addr 172.21.15.116:0/162445490 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30f5000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f009d470 0x7f30f00b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30ef400640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30f01587e0 0x7f30f009cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 -- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f009d470 msgr2=0x7f30f00b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f009d470 0x7f30f00b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 -- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30f01587e0 msgr2=0x7f30f009cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30f01587e0 0x7f30f009cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 -- 172.21.15.116:0/162445490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f009b710 con 0x7f30d40052c0 2024-05-14T11:20:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30efe00640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30e00072c0 tx=0x7f30e00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000f940 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e0018070 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f009dc20 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000fd60 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01eed80 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f01eefe0 con 0x7f30d40052c0 2024-05-14T11:20:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.234+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f30e001c410 con 0x7f30d40052c0 2024-05-14T11:20:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.238+0000 7f30e7e00640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 0x7f30d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.238+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f30e008f1d0 con 0x7f30d40052c0 2024-05-14T11:20:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.238+0000 7f30ef400640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 0x7f30d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:21.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.238+0000 7f30ef400640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 0x7f30d0071940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30f0159820 tx=0x7f30dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:21.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.242+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f30f0161f30 con 0x7f30d40052c0 2024-05-14T11:20:21.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.430+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f006a9c0 con 0x7f30d40052c0 2024-05-14T11:20:21.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.430+0000 7f30e7e00640 1 -- 172.21.15.116:0/162445490 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5186 (secure 0 0 0) 0x7f30e005be40 con 0x7f30d40052c0 2024-05-14T11:20:21.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:21.432 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:21.005899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-14T11:20:21.433 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 48 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 msgr2=0x7f30d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 0x7f30d0071940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30f0159820 tx=0x7f30dc008040 comp rx=0 tx=0).stop 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 msgr2=0x7f30f009ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30e00072c0 tx=0x7f30e00076c0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 shutdown_connections 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f30d006f430 0x7f30d0071940 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30f01587e0 0x7f30f009cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30f009d470 0x7f30f00b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 --2- 172.21.15.116:0/162445490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d40052c0 0x7f30f009ecf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:21.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 >> 172.21.15.116:0/162445490 conn(0x7f30f00b4620 msgr2=0x7f30f0162110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:21.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 shutdown_connections 2024-05-14T11:20:21.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:21.434+0000 7f30f7000640 1 -- 172.21.15.116:0/162445490 wait complete. 2024-05-14T11:20:21.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:20:03.585967+0000', 'modified': '2024-05-14T11:20:21.005899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6396': {'gid': 6396, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.116:6836/2154548589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2154548589}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2154548589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:20:21.856 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6396:up:replay 2024-05-14T11:20:22.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:23.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 Processor -- start 2024-05-14T11:20:23.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 -- start start 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3570070140 0x7f35700d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3570070550 0x7f35700d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3570162500 con 0x7f35700cda30 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3570161980 con 0x7f3570070140 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3575800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3570161c60 con 0x7f3570070550 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3570070140 0x7f35700d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3574e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f35700cda30 0x7f35700cde40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48600/0 (socket says 172.21.15.116:48600) 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3574e00640 1 -- 172.21.15.116:0/3722118229 learned_addr learned my addr 172.21.15.116:0/3722118229 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:23.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f3567400640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3570070550 0x7f35700d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356fe00640 1 -- 172.21.15.116:0/3722118229 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3570070550 msgr2=0x7f35700d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:23.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356fe00640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3570070550 0x7f35700d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356fe00640 1 -- 172.21.15.116:0/3722118229 >> v1:172.21.15.116:6789/0 conn(0x7f35700cda30 legacy=0x7f35700cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356fe00640 1 -- 172.21.15.116:0/3722118229 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3570161f40 con 0x7f3570070140 2024-05-14T11:20:23.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.018+0000 7f356f400640 1 -- 172.21.15.116:0/3722118229 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080134873 0 0) 0x7f3570162500 con 0x7f35700cda30 2024-05-14T11:20:23.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f356fe00640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070140 0x7f35700d15e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f35600097c0 tx=0x7f356001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2025933e85649b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:23.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f356f400640 1 -- 172.21.15.116:0/3722118229 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600b0070 con 0x7f3570070140 2024-05-14T11:20:23.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f356f400640 1 -- 172.21.15.116:0/3722118229 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35600ab600 con 0x7f3570070140 2024-05-14T11:20:23.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f356f400640 1 -- 172.21.15.116:0/3722118229 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600ab9d0 con 0x7f3570070140 2024-05-14T11:20:23.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 -- 172.21.15.116:0/3722118229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070140 msgr2=0x7f35700d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070140 0x7f35700d15e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f35600097c0 tx=0x7f356001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 -- 172.21.15.116:0/3722118229 shutdown_connections 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3570070550 0x7f35700d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 --2- 172.21.15.116:0/3722118229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070140 0x7f35700d15e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 -- 172.21.15.116:0/3722118229 >> 172.21.15.116:0/3722118229 conn(0x7f35700b4530 msgr2=0x7f35700b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 -- 172.21.15.116:0/3722118229 shutdown_connections 2024-05-14T11:20:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.022+0000 7f3575800640 1 -- 172.21.15.116:0/3722118229 wait complete. 2024-05-14T11:20:23.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 Processor -- start 2024-05-14T11:20:23.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 -- start start 2024-05-14T11:20:23.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 0x7f35700c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 0x7f357015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3570161610 con 0x7f3570070550 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35700ce960 con 0x7f3570070140 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3575800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f357016b160 con 0x7f35700cda30 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3567400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3567400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55918/0 (socket says 172.21.15.116:55918) 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.026+0000 7f3567400640 1 -- 172.21.15.116:0/1249548398 learned_addr learned my addr 172.21.15.116:0/1249548398 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356fe00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 0x7f35700c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 -- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 msgr2=0x7f357015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3574e00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 0x7f357015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 0x7f357015c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 -- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 msgr2=0x7f35700c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 0x7f35700c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 -- 172.21.15.116:0/1249548398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35700c8dd0 con 0x7f3570070550 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356fe00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 0x7f35700c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3574e00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 0x7f357015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3567400640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f355800d620 tx=0x7f355800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355800b620 con 0x7f3570070550 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3558019070 con 0x7f3570070550 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355800b9e0 con 0x7f3570070550 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3570153d10 con 0x7f3570070550 2024-05-14T11:20:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3570153470 con 0x7f3570070550 2024-05-14T11:20:23.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3570050880 con 0x7f3570070550 2024-05-14T11:20:23.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f355800bb80 con 0x7f3570070550 2024-05-14T11:20:23.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 0x7f354c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:23.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f355808f880 con 0x7f3570070550 2024-05-14T11:20:23.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.030+0000 7f356fe00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 0x7f354c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:23.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.034+0000 7f356fe00640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 0x7f354c071940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35600ad000 tx=0x7f3560008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:23.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.038+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f355805c590 con 0x7f3570070550 2024-05-14T11:20:23.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.226+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35700b38b0 con 0x7f3570070550 2024-05-14T11:20:23.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.226+0000 7f356cc00640 1 -- 172.21.15.116:0/1249548398 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5186 (secure 0 0 0) 0x7f355805be90 con 0x7f3570070550 2024-05-14T11:20:23.226 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:23.227 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:22.014580+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-14T11:20:23.227 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 50 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 msgr2=0x7f354c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 0x7f354c071940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35600ad000 tx=0x7f3560008040 comp rx=0 tx=0).stop 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 msgr2=0x7f3570152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f355800d620 tx=0x7f355800dae0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 shutdown_connections 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f354c06f430 0x7f354c071940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35700cda30 0x7f357015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3570070550 0x7f3570152460 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 --2- 172.21.15.116:0/1249548398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3570070140 0x7f35700c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:23.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 >> 172.21.15.116:0/1249548398 conn(0x7f35700b4530 msgr2=0x7f35700b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:23.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 shutdown_connections 2024-05-14T11:20:23.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:23.230+0000 7f3575800640 1 -- 172.21.15.116:0/1249548398 wait complete. 2024-05-14T11:20:23.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:20:03.585967+0000', 'modified': '2024-05-14T11:20:22.014580+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6396': {'gid': 6396, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.116:6836/2154548589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2154548589}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2154548589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:20:23.651 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6396:up:rejoin 2024-05-14T11:20:24.652 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:20:24.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.810+0000 7f4e2a400640 1 Processor -- start 2024-05-14T11:20:24.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.810+0000 7f4e2a400640 1 -- start start 2024-05-14T11:20:24.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e2a400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4e2406f730 0x7f4e2406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e2a400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e24070080 con 0x7f4e24072a30 2024-05-14T11:20:24.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e2a400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2406cf10 con 0x7f4e240b0770 2024-05-14T11:20:24.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e2a400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2406db20 con 0x7f4e2406f730 2024-05-14T11:20:24.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4e24072a30 0x7f4e24072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48602/0 (socket says 172.21.15.116:48602) 2024-05-14T11:20:24.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 -- 172.21.15.116:0/4030988355 learned_addr learned my addr 172.21.15.116:0/4030988355 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e1aa00640 1 --2- 172.21.15.116:0/4030988355 >> v2:172.21.15.149:3300/0 conn(0x7f4e2406f730 0x7f4e2406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299923214 0 0) 0x7f4e1c0014a0 con 0x7f4e24072a30 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e08003530 con 0x7f4e24072a30 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447911178 0 0) 0x7f4e2406cf10 con 0x7f4e240b0770 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e1c0014a0 con 0x7f4e240b0770 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1675070263 0 0) 0x7f4e08003530 con 0x7f4e24072a30 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 >> v2:172.21.15.149:3300/0 conn(0x7f4e2406f730 msgr2=0x7f4e2406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 --2- 172.21.15.116:0/4030988355 >> v2:172.21.15.149:3300/0 conn(0x7f4e2406f730 0x7f4e2406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 >> v1:172.21.15.149:6789/0 conn(0x7f4e240b0770 legacy=0x7f4e2406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2406d1f0 con 0x7f4e24072a30 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f4e1c001c70 con 0x7f4e24072a30 2024-05-14T11:20:24.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 >> v1:172.21.15.116:6789/0 conn(0x7f4e24072a30 legacy=0x7f4e24072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08003c40 0x7f4e08004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e08004810 0x7f4e08004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 0x7f4e0800b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406db20 con 0x7f4e08005330 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0800c2d0 con 0x7f4e08004810 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0800c4d0 con 0x7f4e08003c40 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 0x7f4e0800b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23400640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08003c40 0x7f4e08004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e1aa00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e08004810 0x7f4e08004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 -- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08003c40 msgr2=0x7f4e08004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08003c40 0x7f4e08004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 -- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e08004810 msgr2=0x7f4e08004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e08004810 0x7f4e08004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.814+0000 7f4e23e00640 1 -- 172.21.15.116:0/4030988355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e08003530 con 0x7f4e08005330 2024-05-14T11:20:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e23e00640 1 -- 172.21.15.116:0/4030988355 reap_dead start 2024-05-14T11:20:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e23e00640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 0x7f4e0800b9f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4e24070740 tx=0x7f4e1c01d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c217874225288d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00a5e0 con 0x7f4e08005330 2024-05-14T11:20:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c0ac620 con 0x7f4e08005330 2024-05-14T11:20:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e22a00640 1 -- 172.21.15.116:0/4030988355 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0a5e60 con 0x7f4e08005330 2024-05-14T11:20:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 -- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 msgr2=0x7f4e0800b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 0x7f4e0800b9f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4e24070740 tx=0x7f4e1c01d1e0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 -- 172.21.15.116:0/4030988355 shutdown_connections 2024-05-14T11:20:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 --2- 172.21.15.116:0/4030988355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e08005330 0x7f4e0800b9f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 -- 172.21.15.116:0/4030988355 >> 172.21.15.116:0/4030988355 conn(0x7f4e240b4620 msgr2=0x7f4e240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 -- 172.21.15.116:0/4030988355 shutdown_connections 2024-05-14T11:20:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.818+0000 7f4e2a400640 1 -- 172.21.15.116:0/4030988355 wait complete. 2024-05-14T11:20:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 Processor -- start 2024-05-14T11:20:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- start start 2024-05-14T11:20:24.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08005330 0x7f4e2414e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e240cc160 0x7f4e240caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406dfb0 con 0x7f4e241f61a0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406d7b0 con 0x7f4e240cc160 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2406fb40 con 0x7f4e08005330 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55934/0 (socket says 172.21.15.116:55934) 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 -- 172.21.15.116:0/3172662301 learned_addr learned my addr 172.21.15.116:0/3172662301 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e1aa00640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e240cc160 0x7f4e240caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08005330 0x7f4e2414e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 -- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08005330 msgr2=0x7f4e2414e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08005330 0x7f4e2414e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 -- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e240cc160 msgr2=0x7f4e240caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e240cc160 0x7f4e240caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 -- 172.21.15.116:0/3172662301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e240ccad0 con 0x7f4e241f61a0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e23e00640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c01b8e0 tx=0x7f4e08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0a5e60 con 0x7f4e241f61a0 2024-05-14T11:20:24.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c0ac420 con 0x7f4e241f61a0 2024-05-14T11:20:24.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004d70 con 0x7f4e241f61a0 2024-05-14T11:20:24.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e240ccdb0 con 0x7f4e241f61a0 2024-05-14T11:20:24.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.822+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e240ca9d0 con 0x7f4e241f61a0 2024-05-14T11:20:24.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.826+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e240cc820 con 0x7f4e241f61a0 2024-05-14T11:20:24.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.826+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e1c0ac5d0 con 0x7f4e241f61a0 2024-05-14T11:20:24.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.826+0000 7f4e1be00640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 0x7f4e00071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:24.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.826+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f4e1c1291d0 con 0x7f4e241f61a0 2024-05-14T11:20:24.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.826+0000 7f4e23400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 0x7f4e00071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:24.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.830+0000 7f4e23400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 0x7f4e00071940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005630 tx=0x7f4e14007c70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:24.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:24.834+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4e24070080 con 0x7f4e241f61a0 2024-05-14T11:20:25.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e241e8cd0 con 0x7f4e241f61a0 2024-05-14T11:20:25.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e1be00640 1 -- 172.21.15.116:0/3172662301 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5193 (secure 0 0 0) 0x7f4e1c0f5e40 con 0x7f4e241f61a0 2024-05-14T11:20:25.022 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:20:25.022 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:24.013609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:20:25.022 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 51 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 msgr2=0x7f4e00071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 0x7f4e00071940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005630 tx=0x7f4e14007c70 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 msgr2=0x7f4e241e69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.022+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c01b8e0 tx=0x7f4e08005840 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 shutdown_connections 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e0006f430 0x7f4e00071940 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e240cc160 0x7f4e240caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e241f61a0 0x7f4e241e69d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 --2- 172.21.15.116:0/3172662301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e08005330 0x7f4e2414e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 >> 172.21.15.116:0/3172662301 conn(0x7f4e240b4620 msgr2=0x7f4e24073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 shutdown_connections 2024-05-14T11:20:25.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.026+0000 7f4e2a400640 1 -- 172.21.15.116:0/3172662301 wait complete. 2024-05-14T11:20:25.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:20:03.585967+0000', 'modified': '2024-05-14T11:20:24.013609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6396': {'gid': 6396, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6836/2154548589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2154548589}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 2154548589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6396, 'qdb_cluster': [6396]} 2024-05-14T11:20:25.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:20:25.446 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:20:25.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.594+0000 7ff1e2600640 1 Processor -- start 2024-05-14T11:20:25.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.594+0000 7ff1e2600640 1 -- start start 2024-05-14T11:20:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1e2600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1dc0b0770 0x7ff1dc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1e2600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1e2600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1dc06cc40 con 0x7ff1dc06f730 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1e2600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc06db20 con 0x7ff1dc0b0770 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1e2600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc06cf10 con 0x7ff1dc072a30 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1dc0b0770 0x7ff1dc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1dc0b0770 0x7ff1dc06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55942/0 (socket says 172.21.15.116:55942) 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 -- 172.21.15.116:0/3569954104 learned_addr learned my addr 172.21.15.116:0/3569954104 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 -- 172.21.15.116:0/3569954104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 msgr2=0x7ff1dc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 -- 172.21.15.116:0/3569954104 >> v1:172.21.15.149:6789/0 conn(0x7ff1dc06f730 legacy=0x7ff1dc06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.598+0000 7ff1db400640 1 -- 172.21.15.116:0/3569954104 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc06d1f0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1db400640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc06c700 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d150 tx=0x7ff1cc01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676457123bd693 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1daa00640 1 -- 172.21.15.116:0/3569954104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00b2d0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1daa00640 1 -- 172.21.15.116:0/3569954104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00b470 con 0x7ff1dc0b0770 2024-05-14T11:20:25.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1daa00640 1 -- 172.21.15.116:0/3569954104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0a7be0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 -- 172.21.15.116:0/3569954104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 msgr2=0x7ff1dc06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc06c700 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d150 tx=0x7ff1cc01eb70 comp rx=0 tx=0).stop 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 -- 172.21.15.116:0/3569954104 shutdown_connections 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 --2- 172.21.15.116:0/3569954104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc06c700 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 -- 172.21.15.116:0/3569954104 >> 172.21.15.116:0/3569954104 conn(0x7ff1dc0b4620 msgr2=0x7ff1dc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 -- 172.21.15.116:0/3569954104 shutdown_connections 2024-05-14T11:20:25.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.602+0000 7ff1e2600640 1 -- 172.21.15.116:0/3569954104 wait complete. 2024-05-14T11:20:25.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 Processor -- start 2024-05-14T11:20:25.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- start start 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc06f730 0x7ff1dc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc073500 con 0x7ff1dc0b0770 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc073380 con 0x7ff1dc06f730 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc073680 con 0x7ff1dc072a30 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55952/0 (socket says 172.21.15.116:55952) 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 -- 172.21.15.116:0/898395365 learned_addr learned my addr 172.21.15.116:0/898395365 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1db400640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc06f730 0x7ff1dc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 -- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 msgr2=0x7ff1dc0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 -- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc06f730 msgr2=0x7ff1dc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc06f730 0x7ff1dc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 -- 172.21.15.116:0/898395365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc0d0330 con 0x7ff1dc0b0770 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400d500 tx=0x7ff1d400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1d3e00640 1 -- 172.21.15.116:0/898395365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40125f0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1d3e00640 1 -- 172.21.15.116:0/898395365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d4019070 con 0x7ff1dc0b0770 2024-05-14T11:20:25.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1d3e00640 1 -- 172.21.15.116:0/898395365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40129d0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc0d05d0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.606+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc0d0b90 con 0x7ff1dc0b0770 2024-05-14T11:20:25.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1d3e00640 1 -- 172.21.15.116:0/898395365 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff1d4012b70 con 0x7ff1dc0b0770 2024-05-14T11:20:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1d3e00640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 0x7ff1b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1d3e00640 1 -- 172.21.15.116:0/898395365 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff1d408f7c0 con 0x7ff1dc0b0770 2024-05-14T11:20:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1db400640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 0x7ff1b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1e9c9a000 1 Processor -- start 2024-05-14T11:20:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1e9c9a000 1 -- start start 2024-05-14T11:20:25.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.610+0000 7ff1db400640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 0x7ff1b8071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b0000 tx=0x7ff1cc0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739530860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x560739530da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5607393cd620 con 0x560738ff67c0 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5607392e8740 con 0x5607393c86b0 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x560739336090 con 0x5607393def80 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55956/0 (socket says 172.21.15.116:55956) 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 -- 172.21.15.116:0/3779661097 learned_addr learned my addr 172.21.15.116:0/3779661097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739530860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x560739530da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 -- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 msgr2=0x560739530860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739530860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 -- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 msgr2=0x560739530da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x560739530da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 -- 172.21.15.116:0/3779661097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607394ca0a0 con 0x560738ff67c0 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1db400640 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bea0 tx=0x7ff1cc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a541786847a84a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1cbe00640 1 -- 172.21.15.116:0/3779661097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00b300 con 0x560738ff67c0 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1cbe00640 1 -- 172.21.15.116:0/3779661097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc009600 con 0x560738ff67c0 2024-05-14T11:20:25.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1cbe00640 1 -- 172.21.15.116:0/3779661097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0099b0 con 0x560738ff67c0 2024-05-14T11:20:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3779661097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 msgr2=0x560739530320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bea0 tx=0x7ff1cc004ac0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3779661097 shutdown_connections 2024-05-14T11:20:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x560739530da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739530860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3779661097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739530320 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3779661097 >> 172.21.15.116:0/3779661097 conn(0x5607394b5340 msgr2=0x5607394e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3779661097 shutdown_connections 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3779661097 wait complete. 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 -- start start 2024-05-14T11:20:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.614+0000 7ff1e9c9a000 1 Processor -- start 2024-05-14T11:20:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393c86b0 0x560739536ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x560739514720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560739332050 con 0x560738ff67c0 2024-05-14T11:20:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5607392eeec0 con 0x5607393def80 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5607392d5c90 con 0x5607393c86b0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 10 client.0 _ll_register_callbacks cb 0x5607394d1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55964/0 (socket says 172.21.15.116:55964) 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 -- 172.21.15.116:0/4218294500 learned_addr learned my addr 172.21.15.116:0/4218294500 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x560739514720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393c86b0 0x560739536ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 -- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393c86b0 msgr2=0x560739536ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393c86b0 0x560739536ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 -- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 msgr2=0x560739514720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x560739514720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 -- 172.21.15.116:0/4218294500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607393e06f0 con 0x560738ff67c0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x560739514720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1db400640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b040 tx=0x7ff1cc01d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ce944285973471 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0d3060 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc009600 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff1cc0c1040 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc0c8030 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 10 client.6435 fetch_fsmap learned FSMap version 51 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5607393dc6a0 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc009600 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7ff1cc008040 con 0x560738ff67c0 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 10 client.6435 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 10 client.6435 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 20 client.6435 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fdb7d0 0x5607395154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 4 client.6435 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:20:25.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607392cebe0 con 0x560738fdb7d0 2024-05-14T11:20:25.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.618+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fdb7d0 0x5607395154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fdb7d0 0x5607395154c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670eb4bfd0bfefcf server_cookie=461038fc8ddafcf4 in_seq=0 out_seq=0 2024-05-14T11:20:25.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1c8c00640 10 client.6435 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1c8c00640 1 -- 172.21.15.116:0/4218294500 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607392cebe0 con 0x560738fdb7d0 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1c8c00640 10 client.6435 handle_command_reply: tid=0 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 client.6435 shutdown 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 msgr2=0x560739536980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b040 tx=0x7ff1cc01d3b0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 shutdown_connections 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fdb7d0 0x5607395154c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x560739514720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393c86b0 0x560739536ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1c8c00640 0 client.6435 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/4218294500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x560739536980 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 >> 172.21.15.116:0/4218294500 conn(0x5607394b5340 msgr2=0x5607394e5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 shutdown_connections 2024-05-14T11:20:25.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/4218294500 wait complete. 2024-05-14T11:20:25.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.622+0000 7ff1e9c9a000 20 client.6435 trim_cache size 0 max 16384 2024-05-14T11:20:25.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.638+0000 7ff1e9c9a000 1 Processor -- start 2024-05-14T11:20:25.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- start start 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560738ff67c0 0x56073959e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x56073959ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc205170 con 0x5607393c86b0 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc204500 con 0x5607393def80 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc204b50 con 0x560738ff67c0 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55968/0 (socket says 172.21.15.116:55968) 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 -- 172.21.15.116:0/3262501458 learned_addr learned my addr 172.21.15.116:0/3262501458 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 -- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560738ff67c0 msgr2=0x56073959e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560738ff67c0 0x56073959e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 -- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 msgr2=0x56073959ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x56073959ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 -- 172.21.15.116:0/3262501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073930faf0 con 0x5607393c86b0 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4005630 tx=0x7ff1c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48727ea9881f3c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1cb400640 1 -- 172.21.15.116:0/3262501458 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4007030 con 0x5607393c86b0 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1cb400640 1 -- 172.21.15.116:0/3262501458 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c40071d0 con 0x5607393c86b0 2024-05-14T11:20:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1cb400640 1 -- 172.21.15.116:0/3262501458 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4007380 con 0x5607393c86b0 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3262501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 msgr2=0x56073959ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4005630 tx=0x7ff1c4004a40 comp rx=0 tx=0).stop 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3262501458 shutdown_connections 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393def80 0x56073959ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5607393c86b0 0x56073959ea00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3262501458 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560738ff67c0 0x56073959e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3262501458 >> 172.21.15.116:0/3262501458 conn(0x5607394b5340 msgr2=0x56073953c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3262501458 shutdown_connections 2024-05-14T11:20:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.642+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3262501458 wait complete. 2024-05-14T11:20:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- start start 2024-05-14T11:20:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 Processor -- start 2024-05-14T11:20:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x5607395a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739533e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560739332050 con 0x560738ff67c0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56073933a6c0 con 0x5607393c86b0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5607392d0220 con 0x5607393def80 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 10 client.0 _ll_register_callbacks cb 0x7ff1cc0a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55974/0 (socket says 172.21.15.116:55974) 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 -- 172.21.15.116:0/3769652323 learned_addr learned my addr 172.21.15.116:0/3769652323 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x5607395a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1dbe00640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739533e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 -- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 msgr2=0x560739533e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739533e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 -- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 msgr2=0x5607395a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x5607395a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 -- 172.21.15.116:0/3769652323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073930ee40 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1db400640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bea0 tx=0x7ff1cc01b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59efa45fa064069a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc009600 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff1cc0c9040 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c40071d0 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff1cc0097a0 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 10 client.6441 fetch_fsmap learned FSMap version 51 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5607393b64e0 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c40071d0 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7ff1cc0db030 con 0x560738ff67c0 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 10 client.6441 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 10 client.6441 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 20 client.6441 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fc9830 0x5607395347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 4 client.6441 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:20:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "status"}) -- 0x560739343ca0 con 0x560738fc9830 2024-05-14T11:20:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.646+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fc9830 0x5607395347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1c8c00640 10 client.6441 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1d2a00640 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fc9830 0x5607395347e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2309a29e0f3d8b4 server_cookie=550533a9bcf24af3 in_seq=0 out_seq=0 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1c8c00640 1 -- 172.21.15.116:0/3769652323 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560739343ca0 con 0x560738fc9830 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1c8c00640 10 client.6441 handle_command_reply: tid=0 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 client.6441 shutdown 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 msgr2=0x5607395a28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bea0 tx=0x7ff1cc01b910 comp rx=0 tx=0).stop 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 shutdown_connections 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560738fc9830 0x5607395347e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607393c86b0 0x5607395a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5607393def80 0x560739533e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1c8c00640 0 client.6441 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 --2- 172.21.15.116:0/3769652323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560738ff67c0 0x5607395a28f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 >> 172.21.15.116:0/3769652323 conn(0x5607394b5340 msgr2=0x56073953cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 shutdown_connections 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 1 -- 172.21.15.116:0/3769652323 wait complete. 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e9c9a000 20 client.6441 trim_cache size 0 max 16384 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6396, 2024-05-14T11:20:25.652 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.6400894959999999, 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 51, 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 73, 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 73, 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 5.5119549399999999 2024-05-14T11:20:25.653 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 msgr2=0x7ff1b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 0x7ff1b8071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b0000 tx=0x7ff1cc0a9070 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 msgr2=0x7ff1dc0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400d500 tx=0x7ff1d400d9c0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 shutdown_connections 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff1b806f400 0x7ff1b8071910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1dc0b0770 0x7ff1dc0b3ab0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc072a30 0x7ff1dc0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 --2- 172.21.15.116:0/898395365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc06f730 0x7ff1dc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 >> 172.21.15.116:0/898395365 conn(0x7ff1dc0b4620 msgr2=0x7ff1dc0b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 shutdown_connections 2024-05-14T11:20:25.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.650+0000 7ff1e2600640 1 -- 172.21.15.116:0/898395365 wait complete. 2024-05-14T11:20:25.669 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:20:25.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.814+0000 7f9210400640 1 Processor -- start 2024-05-14T11:20:25.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.814+0000 7f9210400640 1 -- start start 2024-05-14T11:20:25.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92080c70e0 0x7f920815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9208163980 0x7f9208163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 0x7f920806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f92080cfe50 con 0x7f9208163980 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f92080c74f0 con 0x7f92081642d0 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f9210400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f920806e260 con 0x7f92080c70e0 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9208163980 0x7f9208163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9208163980 0x7f9208163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55984/0 (socket says 172.21.15.116:55984) 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 -- 172.21.15.116:0/187022676 learned_addr learned my addr 172.21.15.116:0/187022676 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920da00640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92080c70e0 0x7f920815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920e400640 1 --2- 172.21.15.116:0/187022676 >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 0x7f920806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 -- 172.21.15.116:0/187022676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92080c70e0 msgr2=0x7f920815fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92080c70e0 0x7f920815fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 -- 172.21.15.116:0/187022676 >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 msgr2=0x7f920806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 --2- 172.21.15.116:0/187022676 >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 0x7f920806efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920d000640 1 -- 172.21.15.116:0/187022676 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920806e500 con 0x7f9208163980 2024-05-14T11:20:25.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.818+0000 7f920e400640 1 --2- 172.21.15.116:0/187022676 >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 0x7f920806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:25.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f920d000640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9208163980 0x7f9208163d90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f92080d4500 tx=0x7f91f401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21462d8a8797aef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9207e00640 1 -- 172.21.15.116:0/187022676 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f40b5070 con 0x7f9208163980 2024-05-14T11:20:25.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9207e00640 1 -- 172.21.15.116:0/187022676 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f40a89d0 con 0x7f9208163980 2024-05-14T11:20:25.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9207e00640 1 -- 172.21.15.116:0/187022676 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f40a8da0 con 0x7f9208163980 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 -- 172.21.15.116:0/187022676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9208163980 msgr2=0x7f9208163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9208163980 0x7f9208163d90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f92080d4500 tx=0x7f91f401ed10 comp rx=0 tx=0).stop 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 -- 172.21.15.116:0/187022676 shutdown_connections 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9208163980 0x7f9208163d90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 --2- 172.21.15.116:0/187022676 >> v2:172.21.15.149:3300/0 conn(0x7f92081642d0 0x7f920806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 --2- 172.21.15.116:0/187022676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92080c70e0 0x7f920815fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 -- 172.21.15.116:0/187022676 >> 172.21.15.116:0/187022676 conn(0x7f92080d0670 msgr2=0x7f92080d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 -- 172.21.15.116:0/187022676 shutdown_connections 2024-05-14T11:20:25.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.822+0000 7f9210400640 1 -- 172.21.15.116:0/187022676 wait complete. 2024-05-14T11:20:25.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 Processor -- start 2024-05-14T11:20:25.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- start start 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080c70e0 0x7f920815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 0x7f92081f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9208164870 con 0x7f92081642d0 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f920806eb00 con 0x7f92080c70e0 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9208160100 con 0x7f9208163980 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55990/0 (socket says 172.21.15.116:55990) 2024-05-14T11:20:25.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 -- 172.21.15.116:0/3915131937 learned_addr learned my addr 172.21.15.116:0/3915131937 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920d000640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 0x7f92081f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920da00640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080c70e0 0x7f920815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 -- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 msgr2=0x7f92081f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 0x7f92081f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 -- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080c70e0 msgr2=0x7f920815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080c70e0 0x7f920815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 -- 172.21.15.116:0/3915131937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920815e480 con 0x7f92081642d0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920d000640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 0x7f92081f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f920e400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9200007dc0 tx=0x7f920000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920001a070 con 0x7f92081642d0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92000039a0 con 0x7f92081642d0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200003d90 con 0x7f92081642d0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920815e760 con 0x7f92081642d0 2024-05-14T11:20:25.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.826+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081f61d0 con 0x7f92081642d0 2024-05-14T11:20:25.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920806a9c0 con 0x7f92081642d0 2024-05-14T11:20:25.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f920001d070 con 0x7f92081642d0 2024-05-14T11:20:25.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f9205600640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 0x7f91e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:25.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f920008f6c0 con 0x7f92081642d0 2024-05-14T11:20:25.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f920da00640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 0x7f91e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:25.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.830+0000 7f920da00640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 0x7f91e8071910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92080d2510 tx=0x7f91fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:25.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:25.834+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9200010070 con 0x7f92081642d0 2024-05-14T11:20:26.011 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:20:26.010+0000 7fe335000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:20:26.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f92080b9510 con 0x7f92081642d0 2024-05-14T11:20:26.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9205600640 1 -- 172.21.15.116:0/3915131937 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f920005c450 con 0x7f92081642d0 2024-05-14T11:20:26.025 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 msgr2=0x7f91e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 0x7f91e8071910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92080d2510 tx=0x7f91fc008040 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 msgr2=0x7f920815d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.026+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9200007dc0 tx=0x7f920000d4a0 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 shutdown_connections 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f91e806f400 0x7f91e8071910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92081642d0 0x7f920815d8b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9208163980 0x7f92081f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 --2- 172.21.15.116:0/3915131937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080c70e0 0x7f920815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 >> 172.21.15.116:0/3915131937 conn(0x7f92080d0670 msgr2=0x7f92080c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 shutdown_connections 2024-05-14T11:20:26.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26.030+0000 7f9210400640 1 -- 172.21.15.116:0/3915131937 wait complete. 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:20:26.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:26.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.048 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:20:26.056 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:20:26.057 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42176sec preferred_lft 42176sec 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:20:26.058 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:20:26.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.059 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:20:26.116 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:20:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.118 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:20:26.170 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:20:26.170 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:20:26.171 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:20:26.171 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:20:26.171 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:20:26.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:20:26.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:20:26.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:20:26.172 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:20:26.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:20:26.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:20:26.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.304 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:20:26.313 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:20:26.313 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:20:26.313 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:26.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.314 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:20:26.374 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:20:26.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:20:26.374 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:26.424 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:20:26.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:20:26.424 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:26.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:20:26.540 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:20:26.597 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:20:26.598 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:26.599 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:26.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.600 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:20:26.655 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:20:26.655 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:20:26.701 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:20:26.721 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:20:26.773 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:20:26.774 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:26.775 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:26.776 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:26.776 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:26.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:26.777 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:20:26.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:20:26.806+0000 7fa0fc9ff580 -1 init, newargv = 0x7fa0e4001090 newargc=15 2024-05-14T11:20:26.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42230]: starting ceph client 2024-05-14T11:20:26.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42230]: starting fuse 2024-05-14T11:20:27.916 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:20:27.939 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:20:27.993 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:20:27.993 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:20:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:27.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:20:27.996 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:27.996 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:20:27.996 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:27.997 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:20:28.052 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:20:28.052 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:20:28.052 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.053 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:20:28.054 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:20:28.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.42230.asok 2024-05-14T11:20:28.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:20:28.197 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42230.asok 2024-05-14T11:20:28.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:28.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42230.asok status 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "42230", 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:20:28.376 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6456, 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 6456 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1972035653 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:20:28.377 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1972035653 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.6456 192.168.0.1:0/1972035653", 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/1972035653", 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 51, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 73, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:20:28.378 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:28.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:28.391 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:20:28.391 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:28.400 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:20:28.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:28.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:28.401 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:28.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:28.462 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-14T11:20:28.516 INFO:teuthology.orchestra.run.smithi116.stdout:file_undamaged 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> import json 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> import sys 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.517 DEBUG:teuthology.orchestra.run.smithi116:> try: 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> except OSError as e: 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> sys.exit(e.errno) 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> print(json.dumps( 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> indent=2)) 2024-05-14T11:20:28.518 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:20:28.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-14T11:20:28.636 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-14T11:20:28.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:28.637 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-14T11:20:28.694 INFO:teuthology.orchestra.run.smithi116.stdout:2 2024-05-14T11:20:28.695 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:20:28.695 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:20:28.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:20:28.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 Processor -- start 2024-05-14T11:20:28.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 -- start start 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13680d24d0 0x7f13680b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13680d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13680d28e0 con 0x7f13680d07f0 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f13680d2040 con 0x7f13680d24d0 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1367200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13680d0c00 con 0x7f13680cdcf0 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1365e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13680d24d0 0x7f13680b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1366800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f13680d07f0 0x7f136815f2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50114/0 (socket says 172.21.15.116:50114) 2024-05-14T11:20:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f1366800640 1 -- 172.21.15.116:0/3997834261 learned_addr learned my addr 172.21.15.116:0/3997834261 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.938+0000 7f135d400640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13680d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365e00640 1 -- 172.21.15.116:0/3997834261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 msgr2=0x7f13680d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365e00640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13680d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365e00640 1 -- 172.21.15.116:0/3997834261 >> v1:172.21.15.116:6789/0 conn(0x7f13680d07f0 legacy=0x7f136815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365e00640 1 -- 172.21.15.116:0/3997834261 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680d1430 con 0x7f13680d24d0 2024-05-14T11:20:28.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f135d400640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13680d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:28.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365400640 1 -- 172.21.15.116:0/3997834261 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569004340 0 0) 0x7f13600014a0 con 0x7f13680d07f0 2024-05-14T11:20:28.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365e00640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d24d0 0x7f13680b9020 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f13500097c0 tx=0x7f135001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f9e03801fae530 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:28.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365400640 1 -- 172.21.15.116:0/3997834261 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500b0070 con 0x7f13680d24d0 2024-05-14T11:20:28.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365400640 1 -- 172.21.15.116:0/3997834261 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13500ab600 con 0x7f13680d24d0 2024-05-14T11:20:28.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1365400640 1 -- 172.21.15.116:0/3997834261 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500ab9d0 con 0x7f13680d24d0 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1367200640 1 -- 172.21.15.116:0/3997834261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d24d0 msgr2=0x7f13680b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.942+0000 7f1367200640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d24d0 0x7f13680b9020 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f13500097c0 tx=0x7f135001c440 comp rx=0 tx=0).stop 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- 172.21.15.116:0/3997834261 shutdown_connections 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13680d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 --2- 172.21.15.116:0/3997834261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d24d0 0x7f13680b9020 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- 172.21.15.116:0/3997834261 >> 172.21.15.116:0/3997834261 conn(0x7f13680b51f0 msgr2=0x7f13680b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- 172.21.15.116:0/3997834261 shutdown_connections 2024-05-14T11:20:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- 172.21.15.116:0/3997834261 wait complete. 2024-05-14T11:20:28.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 Processor -- start 2024-05-14T11:20:28.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- start start 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13681f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13680d24d0 0x7f136806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13680ce280 con 0x7f13680d07f0 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13680ce100 con 0x7f13680d24d0 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.946+0000 7f1367200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f136815fdc0 con 0x7f13680cdcf0 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59794/0 (socket says 172.21.15.116:59794) 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 -- 172.21.15.116:0/3336307009 learned_addr learned my addr 172.21.15.116:0/3336307009 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f1365e00640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13681f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f1366800640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13680d24d0 0x7f136806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 -- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 msgr2=0x7f13681f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13681f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 -- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13680d24d0 msgr2=0x7f136806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13680d24d0 0x7f136806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 -- 172.21.15.116:0/3336307009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1368168270 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135d400640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f135800d800 tx=0x7f135800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358018020 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13580096c0 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358009ab0 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1368167cd0 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136806e1b0 con 0x7f13680d07f0 2024-05-14T11:20:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368050880 con 0x7f13680d07f0 2024-05-14T11:20:28.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.950+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f135801b020 con 0x7f13680d07f0 2024-05-14T11:20:28.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.954+0000 7f135ea00640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:28.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.954+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f135808fa60 con 0x7f13680d07f0 2024-05-14T11:20:28.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.954+0000 7f1365e00640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:28.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.954+0000 7f1365e00640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 0x7f1344071910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13500ad000 tx=0x7f13500a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:28.958+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1358022020 con 0x7f13680d07f0 2024-05-14T11:20:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.126+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1368168440 con 0x7f13680d07f0 2024-05-14T11:20:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.126+0000 7f135ea00640 1 -- 172.21.15.116:0/3336307009 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) ==== 80+0+124 (secure 0 0 0) 0x7f135805c770 con 0x7f13680d07f0 2024-05-14T11:20:29.126 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:20:29.126 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6834/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:20:29.127 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:20:29.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 msgr2=0x7f1344071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 0x7f1344071910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13500ad000 tx=0x7f13500a6000 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 msgr2=0x7f1368154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f135800d800 tx=0x7f135800dcc0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 shutdown_connections 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13680d24d0 0x7f136806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13680d07f0 0x7f1368154b80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 --2- 172.21.15.116:0/3336307009 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13680cdcf0 0x7f13681f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 >> 172.21.15.116:0/3336307009 conn(0x7f13680b51f0 msgr2=0x7f136815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 shutdown_connections 2024-05-14T11:20:29.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.130+0000 7f1367200640 1 -- 172.21.15.116:0/3336307009 wait complete. 2024-05-14T11:20:29.148 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:20:29.148 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:20:29.157 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:20:29.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 Processor -- start 2024-05-14T11:20:29.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 -- start start 2024-05-14T11:20:29.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c156fa0 0x7fc98c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc98c0d1670 0x7fc98c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc98c06e9e0 0x7fc98c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc98c1573b0 con 0x7fc98c0d1670 2024-05-14T11:20:29.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc98c161120 con 0x7fc98c06e9e0 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc991a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc98c0d1fc0 con 0x7fc98c156fa0 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc98c0d1670 0x7fc98c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc98c0d1670 0x7fc98c0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59808/0 (socket says 172.21.15.116:59808) 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 -- 172.21.15.116:0/783457931 learned_addr learned my addr 172.21.15.116:0/783457931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc98b400640 1 --2- 172.21.15.116:0/783457931 >> v2:172.21.15.149:3300/0 conn(0x7fc98c06e9e0 0x7fc98c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc98aa00640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c156fa0 0x7fc98c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 -- 172.21.15.116:0/783457931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c156fa0 msgr2=0x7fc98c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c156fa0 0x7fc98c06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 -- 172.21.15.116:0/783457931 >> v2:172.21.15.149:3300/0 conn(0x7fc98c06e9e0 msgr2=0x7fc98c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 --2- 172.21.15.116:0/783457931 >> v2:172.21.15.149:3300/0 conn(0x7fc98c06e9e0 0x7fc98c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.346+0000 7fc982000640 1 -- 172.21.15.116:0/783457931 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c160510 con 0x7fc98c0d1670 2024-05-14T11:20:29.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc982000640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c0d1670 0x7fc98c0d1a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0d2170 tx=0x7fc97401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992c53cbf5b405a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:29.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc98a000640 1 -- 172.21.15.116:0/783457931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740aa610 con 0x7fc98c0d1670 2024-05-14T11:20:29.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc98a000640 1 -- 172.21.15.116:0/783457931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9740af070 con 0x7fc98c0d1670 2024-05-14T11:20:29.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc98a000640 1 -- 172.21.15.116:0/783457931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740aa9d0 con 0x7fc98c0d1670 2024-05-14T11:20:29.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 -- 172.21.15.116:0/783457931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c0d1670 msgr2=0x7fc98c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c0d1670 0x7fc98c0d1a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0d2170 tx=0x7fc97401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 -- 172.21.15.116:0/783457931 shutdown_connections 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c0d1670 0x7fc98c0d1a80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 --2- 172.21.15.116:0/783457931 >> v2:172.21.15.149:3300/0 conn(0x7fc98c06e9e0 0x7fc98c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 --2- 172.21.15.116:0/783457931 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c156fa0 0x7fc98c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 -- 172.21.15.116:0/783457931 >> 172.21.15.116:0/783457931 conn(0x7fc98c0b4620 msgr2=0x7fc98c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 -- 172.21.15.116:0/783457931 shutdown_connections 2024-05-14T11:20:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.350+0000 7fc991a00640 1 -- 172.21.15.116:0/783457931 wait complete. 2024-05-14T11:20:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 Processor -- start 2024-05-14T11:20:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 -- start start 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c06e9e0 0x7fc98c09aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc98c0d1670 0x7fc98c0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c070520 con 0x7fc98c156fa0 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c06f740 con 0x7fc98c0d1670 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc991a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc98c0706a0 con 0x7fc98c06e9e0 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc98b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc98b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59820/0 (socket says 172.21.15.116:59820) 2024-05-14T11:20:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.354+0000 7fc98b400640 1 -- 172.21.15.116:0/1339823197 learned_addr learned my addr 172.21.15.116:0/1339823197 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc982000640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc98c0d1670 0x7fc98c0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98aa00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c06e9e0 0x7fc98c09aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 -- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c06e9e0 msgr2=0x7fc98c09aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c06e9e0 0x7fc98c09aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 -- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc98c0d1670 msgr2=0x7fc98c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc98c0d1670 0x7fc98c0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 -- 172.21.15.116:0/1339823197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c159580 con 0x7fc98c156fa0 2024-05-14T11:20:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc98b400640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc98400efc0 tx=0x7fc98400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984018020 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc984014480 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98c159860 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984009e60 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98c14ec40 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98c06a9c0 con 0x7fc98c156fa0 2024-05-14T11:20:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.358+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc98401b020 con 0x7fc98c156fa0 2024-05-14T11:20:29.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.362+0000 7fc983400640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 0x7fc968071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:29.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.362+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fc9840900f0 con 0x7fc98c156fa0 2024-05-14T11:20:29.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.362+0000 7fc98aa00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 0x7fc968071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:29.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.362+0000 7fc98aa00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 0x7fc968071910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0b0d60 tx=0x7fc97c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.366+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc984022030 con 0x7fc98c156fa0 2024-05-14T11:20:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc98c06dfc0 con 0x7fc98c156fa0 2024-05-14T11:20:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc983400640 1 -- 172.21.15.116:0/1339823197 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) ==== 80+0+124 (secure 0 0 0) 0x7fc984010040 con 0x7fc98c156fa0 2024-05-14T11:20:29.532 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:20:29.532 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6834/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:20:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:20:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 msgr2=0x7fc968071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 0x7fc968071910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0b0d60 tx=0x7fc97c009190 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 msgr2=0x7fc98c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc98400efc0 tx=0x7fc98400c490 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 shutdown_connections 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc96806f400 0x7fc968071910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc98c0d1670 0x7fc98c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc98c156fa0 0x7fc98c0cdb90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 --2- 172.21.15.116:0/1339823197 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc98c06e9e0 0x7fc98c09aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 >> 172.21.15.116:0/1339823197 conn(0x7fc98c0b4620 msgr2=0x7fc98c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 shutdown_connections 2024-05-14T11:20:29.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:29.534+0000 7fc991a00640 1 -- 172.21.15.116:0/1339823197 wait complete. 2024-05-14T11:20:29.553 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:20:29.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:29.554 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:20:29.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42230]: fuse finished with error 0 and tester_r 0 2024-05-14T11:20:29.573 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:20:31.039 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:20:31.038+0000 7f7ed0800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T11:20:35.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:20:35.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:35.676 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:20:35.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:35.687 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-14T11:20:35.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.882+0000 7fa12a400640 1 Processor -- start 2024-05-14T11:20:35.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.882+0000 7fa12a400640 1 -- start start 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa12a400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa12415d5a0 0x7fa1240d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa12a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa1240b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa12a400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1240d2a20 con 0x7fa1240d2610 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa12a400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa1240cfbe0 con 0x7fa12415d5a0 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa12a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa124160db0 con 0x7fa1241608b0 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa123e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa1240d2610 0x7fa1241601a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50118/0 (socket says 172.21.15.116:50118) 2024-05-14T11:20:35.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa123e00640 1 -- 172.21.15.116:0/2365767990 learned_addr learned my addr 172.21.15.116:0/2365767990 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:20:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129000640 1 --2- 172.21.15.116:0/2365767990 >> v2:172.21.15.149:3300/0 conn(0x7fa12415d5a0 0x7fa1240d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129a00640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa1240b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa123400640 1 -- 172.21.15.116:0/2365767990 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966726709 0 0) 0x7fa110001120 con 0x7fa1240d2610 2024-05-14T11:20:35.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa123400640 1 -- 172.21.15.116:0/2365767990 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa10c003530 con 0x7fa1240d2610 2024-05-14T11:20:35.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129000640 1 -- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 msgr2=0x7fa1240b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129000640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa1240b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129000640 1 -- 172.21.15.116:0/2365767990 >> v1:172.21.15.116:6789/0 conn(0x7fa1240d2610 legacy=0x7fa1241601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.886+0000 7fa129000640 1 -- 172.21.15.116:0/2365767990 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa124161090 con 0x7fa12415d5a0 2024-05-14T11:20:35.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa129000640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa1240d20d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097c0 tx=0x7fa11801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0fa199b037eb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa123400640 1 -- 172.21.15.116:0/2365767990 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b0070 con 0x7fa12415d5a0 2024-05-14T11:20:35.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa123400640 1 -- 172.21.15.116:0/2365767990 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1180ab600 con 0x7fa12415d5a0 2024-05-14T11:20:35.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa123400640 1 -- 172.21.15.116:0/2365767990 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180ab9b0 con 0x7fa12415d5a0 2024-05-14T11:20:35.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 -- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 msgr2=0x7fa1240d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa1240d20d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097c0 tx=0x7fa11801c410 comp rx=0 tx=0).stop 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 -- 172.21.15.116:0/2365767990 shutdown_connections 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa1240b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 --2- 172.21.15.116:0/2365767990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa1240d20d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 -- 172.21.15.116:0/2365767990 >> 172.21.15.116:0/2365767990 conn(0x7fa1240b4fe0 msgr2=0x7fa1240b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 -- 172.21.15.116:0/2365767990 shutdown_connections 2024-05-14T11:20:35.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.890+0000 7fa12a400640 1 -- 172.21.15.116:0/2365767990 wait complete. 2024-05-14T11:20:35.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 Processor -- start 2024-05-14T11:20:35.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 -- start start 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa124161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa124162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa12416ca60 con 0x7fa1240d2610 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa12416c8e0 con 0x7fa12415d5a0 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa12a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa12416cbe0 con 0x7fa1241608b0 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59824/0 (socket says 172.21.15.116:59824) 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 -- 172.21.15.116:0/500279923 learned_addr learned my addr 172.21.15.116:0/500279923 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa123e00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa124161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129a00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa124162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 -- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 msgr2=0x7fa124162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa124162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 -- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 msgr2=0x7fa124161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa124161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 -- 172.21.15.116:0/500279923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa124150340 con 0x7fa1240d2610 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa123e00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa124161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:35.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.894+0000 7fa129000640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa1180ae040 tx=0x7fa11801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa120c00640 1 -- 172.21.15.116:0/500279923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b0040 con 0x7fa1240d2610 2024-05-14T11:20:35.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa120c00640 1 -- 172.21.15.116:0/500279923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1180abe50 con 0x7fa1240d2610 2024-05-14T11:20:35.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa124156630 con 0x7fa1240d2610 2024-05-14T11:20:35.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa120c00640 1 -- 172.21.15.116:0/500279923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118006e20 con 0x7fa1240d2610 2024-05-14T11:20:35.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa124156bf0 con 0x7fa1240d2610 2024-05-14T11:20:35.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.898+0000 7fa120c00640 1 -- 172.21.15.116:0/500279923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa118004230 con 0x7fa1240d2610 2024-05-14T11:20:35.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 Processor -- start 2024-05-14T11:20:35.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa120c00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 0x7fa104071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 -- start start 2024-05-14T11:20:35.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa120c00640 1 -- 172.21.15.116:0/500279923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fa118129bf0 con 0x7fa1240d2610 2024-05-14T11:20:35.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa129a00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 0x7fa104071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa129a00640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 0x7fa104071910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1240b2ea0 tx=0x7fa11c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x5625730d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x5625730d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562572f6f350 con 0x562572f81070 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562572ed6f60 con 0x562572f6a840 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562572e8fb00 con 0x562572fa3c30 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59834/0 (socket says 172.21.15.116:59834) 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 -- 172.21.15.116:0/1733223850 learned_addr learned my addr 172.21.15.116:0/1733223850 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:35.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 -- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 msgr2=0x5625730d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x5625730d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 -- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 msgr2=0x5625730d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa129a00640 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x5625730d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x5625730d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 -- 172.21.15.116:0/1733223850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562572eafe40 con 0x562572f81070 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa129a00640 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x5625730d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa123e00640 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa110001640 tx=0x7fa110001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7423ae3fd47fb01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa115600640 1 -- 172.21.15.116:0/1733223850 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110019070 con 0x562572f81070 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa115600640 1 -- 172.21.15.116:0/1733223850 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110001cf0 con 0x562572f81070 2024-05-14T11:20:35.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.902+0000 7fa115600640 1 -- 172.21.15.116:0/1733223850 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110005cd0 con 0x562572f81070 2024-05-14T11:20:35.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- 172.21.15.116:0/1733223850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 msgr2=0x5625730d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa110001640 tx=0x7fa110001b00 comp rx=0 tx=0).stop 2024-05-14T11:20:35.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- 172.21.15.116:0/1733223850 shutdown_connections 2024-05-14T11:20:35.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x5625730d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d26d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- 172.21.15.116:0/1733223850 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x5625730d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- 172.21.15.116:0/1733223850 >> 172.21.15.116:0/1733223850 conn(0x562573057090 msgr2=0x562573086950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- 172.21.15.116:0/1733223850 shutdown_connections 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- 172.21.15.116:0/1733223850 wait complete. 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- start start 2024-05-14T11:20:35.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 Processor -- start 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730d9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562572e76be0 con 0x562572f81070 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562572ed2f20 con 0x562572fa3c30 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562572e92ee0 con 0x562572f6a840 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa133076000 10 client.0 _ll_register_callbacks cb 0x562572f89e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59838/0 (socket says 172.21.15.116:59838) 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 -- 172.21.15.116:0/700356032 learned_addr learned my addr 172.21.15.116:0/700356032 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa129000640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730d9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa129a00640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 -- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 msgr2=0x5625730d9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730d9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 -- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 msgr2=0x5625730b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa123e00640 1 -- 172.21.15.116:0/700356032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562572f1c7a0 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.906+0000 7fa129a00640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa123e00640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa110005890 tx=0x7fa110001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d677e7715ad6dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110023060 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fa1100072e0 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa110001cf0 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa110019070 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 10 client.6483 fetch_fsmap learned FSMap version 52 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x562572e8fd50 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110001cf0 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fa1100074e0 con 0x562572f81070 2024-05-14T11:20:35.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 10 client.6483 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 10 client.6483 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 20 client.6483 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b7cb10 0x5625730b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 4 client.6483 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562572e6fe10 con 0x562572b7cb10 2024-05-14T11:20:35.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa129000640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b7cb10 0x5625730b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa129000640 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b7cb10 0x5625730b7120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b72b49981c7a04 server_cookie=c48df14af3847ac9 in_seq=0 out_seq=0 2024-05-14T11:20:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 10 client.6483 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 1 -- 172.21.15.116:0/700356032 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562572e6fe10 con 0x562572b7cb10 2024-05-14T11:20:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 10 client.6483 handle_command_reply: tid=0 2024-05-14T11:20:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 client.6483 shutdown 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 msgr2=0x5625730d9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa110005890 tx=0x7fa110001360 comp rx=0 tx=0).stop 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 shutdown_connections 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b7cb10 0x5625730b7120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730d9d40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 --2- 172.21.15.116:0/700356032 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730d9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa10a000640 0 client.6483 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 >> 172.21.15.116:0/700356032 conn(0x562573057090 msgr2=0x562573085f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 shutdown_connections 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 1 -- 172.21.15.116:0/700356032 wait complete. 2024-05-14T11:20:35.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.910+0000 7fa133076000 20 client.6483 trim_cache size 0 max 16384 2024-05-14T11:20:35.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 Processor -- start 2024-05-14T11:20:35.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 -- start start 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730e8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1242062a0 con 0x562572f81070 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa124205280 con 0x562572fa3c30 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa1241531a0 con 0x562572f6a840 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59840/0 (socket says 172.21.15.116:59840) 2024-05-14T11:20:35.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 -- 172.21.15.116:0/3108728901 learned_addr learned my addr 172.21.15.116:0/3108728901 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:35.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 -- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 msgr2=0x5625730e8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 -- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 msgr2=0x5625730e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 -- 172.21.15.116:0/3108728901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562572f53520 con 0x562572f81070 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa123e00640 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa110007c20 tx=0x7fa110019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7fb5628cbfaa15 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa114c00640 1 -- 172.21.15.116:0/3108728901 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110031070 con 0x562572f81070 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa114c00640 1 -- 172.21.15.116:0/3108728901 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11001a040 con 0x562572f81070 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.930+0000 7fa114c00640 1 -- 172.21.15.116:0/3108728901 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100072e0 con 0x562572f81070 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- 172.21.15.116:0/3108728901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 msgr2=0x5625730e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa110007c20 tx=0x7fa110019000 comp rx=0 tx=0).stop 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- 172.21.15.116:0/3108728901 shutdown_connections 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572fa3c30 0x5625730e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x5625730e8ed0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- 172.21.15.116:0/3108728901 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572f6a840 0x5625730e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- 172.21.15.116:0/3108728901 >> 172.21.15.116:0/3108728901 conn(0x562573057090 msgr2=0x5625730d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- 172.21.15.116:0/3108728901 shutdown_connections 2024-05-14T11:20:35.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- 172.21.15.116:0/3108728901 wait complete. 2024-05-14T11:20:35.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- start start 2024-05-14T11:20:35.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 Processor -- start 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x562573088550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x562573102300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562572ee4360 con 0x562572f81070 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241537f0 con 0x562572f6a840 2024-05-14T11:20:35.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562572e8bea0 con 0x562572fa3c30 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa133076000 10 client.0 _ll_register_callbacks cb 0x7fa110017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59842/0 (socket says 172.21.15.116:59842) 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 -- 172.21.15.116:0/3318845807 learned_addr learned my addr 172.21.15.116:0/3318845807 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa129000640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x562573088550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa129a00640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x562573102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 -- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 msgr2=0x562573102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x562573102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 -- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 msgr2=0x562573088550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x562573088550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 -- 172.21.15.116:0/3318845807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562572f2f710 con 0x562572f81070 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa123e00640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa110007f00 tx=0x7fa110009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da792c872a7e1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:35.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.934+0000 7fa10a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110031070 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fa110001cf0 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11001a040 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa110042050 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 10 client.6489 fetch_fsmap learned FSMap version 52 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x562572b90400 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11001a040 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fa110045020 con 0x562572f81070 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 10 client.6489 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 10 client.6489 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 20 client.6489 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b6ab70 0x562573102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:35.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 4 client.6489 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x562572ee4b70 con 0x562572b6ab70 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa129000640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b6ab70 0x562573102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa129000640 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b6ab70 0x562573102cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee448995f264975 server_cookie=f27ce27c0001f1d in_seq=0 out_seq=0 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 10 client.6489 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 1 -- 172.21.15.116:0/3318845807 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562572ee4b70 con 0x562572b6ab70 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 10 client.6489 handle_command_reply: tid=0 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 client.6489 shutdown 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 msgr2=0x562573088a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa110007f00 tx=0x7fa110009040 comp rx=0 tx=0).stop 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 shutdown_connections 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562572b6ab70 0x562573102cc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562572fa3c30 0x562573102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562572f81070 0x562573088a90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 --2- 172.21.15.116:0/3318845807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562572f6a840 0x562573088550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 >> 172.21.15.116:0/3318845807 conn(0x562573057090 msgr2=0x5625730e93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa10a000640 0 client.6489 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 shutdown_connections 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 1 -- 172.21.15.116:0/3318845807 wait complete. 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa133076000 20 client.6489 trim_cache size 0 max 16384 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "0d674080-e235-49a6-88a5-57b7b65ed680", 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:20:35.941 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 msgr2=0x7fa104071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 0x7fa104071910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1240b2ea0 tx=0x7fa11c00b040 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 msgr2=0x7fa12414fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa1180ae040 tx=0x7fa11801ac50 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 shutdown_connections 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa10406f400 0x7fa104071910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12415d5a0 0x7fa124161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa1241608b0 0x7fa124162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 --2- 172.21.15.116:0/500279923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1240d2610 0x7fa12414fe00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:35.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 >> 172.21.15.116:0/500279923 conn(0x7fa1240b4fe0 msgr2=0x7fa124068750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:35.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 shutdown_connections 2024-05-14T11:20:35.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:35.938+0000 7fa12a400640 1 -- 172.21.15.116:0/500279923 wait complete. 2024-05-14T11:20:35.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:20:35.958 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-14T11:20:36.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.106+0000 7ff7b1e00640 1 Processor -- start 2024-05-14T11:20:36.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.106+0000 7ff7b1e00640 1 -- start start 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac160080 0x7ff7ac0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff7ac0d1a50 0x7ff7ac0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0d1e60 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0b2f40 con 0x7ff7ac0d1a50 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7b1e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0b3220 con 0x7ff7ac160080 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59856/0 (socket says 172.21.15.116:59856) 2024-05-14T11:20:36.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 -- 172.21.15.116:0/3295120627 learned_addr learned my addr 172.21.15.116:0/3295120627 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac160080 0x7ff7ac0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7ab400640 1 --2- 172.21.15.116:0/3295120627 >> v2:172.21.15.149:3300/0 conn(0x7ff7ac0d1a50 0x7ff7ac0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 -- 172.21.15.116:0/3295120627 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac160080 msgr2=0x7ff7ac0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac160080 0x7ff7ac0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 -- 172.21.15.116:0/3295120627 >> v2:172.21.15.149:3300/0 conn(0x7ff7ac0d1a50 msgr2=0x7ff7ac0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 --2- 172.21.15.116:0/3295120627 >> v2:172.21.15.149:3300/0 conn(0x7ff7ac0d1a50 0x7ff7ac0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.110+0000 7ff7aa000640 1 -- 172.21.15.116:0/3295120627 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac0b3500 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7aa000640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0d2590 tx=0x7ff79801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798001519faf1c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7a9600640 1 -- 172.21.15.116:0/3295120627 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b2070 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7a9600640 1 -- 172.21.15.116:0/3295120627 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980ad640 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7a9600640 1 -- 172.21.15.116:0/3295120627 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980ada10 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 -- 172.21.15.116:0/3295120627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 msgr2=0x7ff7ac0d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0d2590 tx=0x7ff79801c3d0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 -- 172.21.15.116:0/3295120627 shutdown_connections 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0d1510 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/3295120627 >> v2:172.21.15.149:3300/0 conn(0x7ff7ac0d1a50 0x7ff7ac0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/3295120627 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac160080 0x7ff7ac0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 -- 172.21.15.116:0/3295120627 >> 172.21.15.116:0/3295120627 conn(0x7ff7ac0d0770 msgr2=0x7ff7ac0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 -- 172.21.15.116:0/3295120627 shutdown_connections 2024-05-14T11:20:36.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.114+0000 7ff7b1e00640 1 -- 172.21.15.116:0/3295120627 wait complete. 2024-05-14T11:20:36.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 Processor -- start 2024-05-14T11:20:36.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- start start 2024-05-14T11:20:36.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac0d1a50 0x7ff7ac159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac160080 0x7ff7ac15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac06ff50 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac06fdd0 con 0x7ff7ac160080 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac1639e0 con 0x7ff7ac0d1a50 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59870/0 (socket says 172.21.15.116:59870) 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1603306243 learned_addr learned my addr 172.21.15.116:0/1603306243 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7ab400640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac160080 0x7ff7ac15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac0d1a50 0x7ff7ac159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac0d1a50 msgr2=0x7ff7ac159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac0d1a50 0x7ff7ac159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac160080 msgr2=0x7ff7ac15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac160080 0x7ff7ac15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1603306243 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac0caa80 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e840 tx=0x7ff79c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7a3400640 1 -- 172.21.15.116:0/1603306243 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c017070 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7a3400640 1 -- 172.21.15.116:0/1603306243 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c00ee90 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7a3400640 1 -- 172.21.15.116:0/1603306243 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00cb60 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac0cad60 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.118+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac0cb320 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7a3400640 1 -- 172.21.15.116:0/1603306243 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff79c01a070 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7a3400640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 0x7ff78c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7b943c000 1 Processor -- start 2024-05-14T11:20:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7a3400640 1 -- 172.21.15.116:0/1603306243 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff79c08f770 con 0x7ff7ac0ce4c0 2024-05-14T11:20:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7b943c000 1 -- start start 2024-05-14T11:20:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 0x7ff78c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.122+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 0x7ff78c071940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7980af000 tx=0x7ff798008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e1c1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56130dfc6100 con 0x56130e047f00 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56130df7eca0 con 0x56130dc473e0 2024-05-14T11:20:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56130dfc20c0 con 0x56130e05a120 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59876/0 (socket says 172.21.15.116:59876) 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 -- 172.21.15.116:0/1545900973 learned_addr learned my addr 172.21.15.116:0/1545900973 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e1c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7ab400640 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 -- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 msgr2=0x56130e1c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e1c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 -- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 msgr2=0x56130e1c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 -- 172.21.15.116:0/1545900973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130df83800 con 0x56130e047f00 2024-05-14T11:20:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b0040 tx=0x7ff798004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb880f3e155860a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff797400640 1 -- 172.21.15.116:0/1545900973 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980c8070 con 0x56130e047f00 2024-05-14T11:20:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff797400640 1 -- 172.21.15.116:0/1545900973 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980adce0 con 0x56130e047f00 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff797400640 1 -- 172.21.15.116:0/1545900973 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b2040 con 0x56130e047f00 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- 172.21.15.116:0/1545900973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 msgr2=0x56130e1c1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b0040 tx=0x7ff798004740 comp rx=0 tx=0).stop 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- 172.21.15.116:0/1545900973 shutdown_connections 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c1890 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1545900973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e1c1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- 172.21.15.116:0/1545900973 >> 172.21.15.116:0/1545900973 conn(0x56130dc87400 msgr2=0x56130e1a4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- 172.21.15.116:0/1545900973 shutdown_connections 2024-05-14T11:20:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- 172.21.15.116:0/1545900973 wait complete. 2024-05-14T11:20:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 -- start start 2024-05-14T11:20:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.126+0000 7ff7b943c000 1 Processor -- start 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e17afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56130e0912a0 con 0x56130e047f00 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56130e081f40 con 0x56130dc473e0 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56130df82080 con 0x56130e05a120 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 10 client.0 _ll_register_callbacks cb 0x56130e162c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59878/0 (socket says 172.21.15.116:59878) 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 -- 172.21.15.116:0/4227587293 learned_addr learned my addr 172.21.15.116:0/4227587293 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7ab400640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e17afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 -- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 msgr2=0x56130e17afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e17afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 -- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 msgr2=0x56130e1c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 -- 172.21.15.116:0/4227587293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130df70860 con 0x56130e047f00 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aa000640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff798000c10 tx=0x7ff79801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84da2d000c1a55d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980d1060 con 0x56130e047f00 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff7980be040 con 0x56130e047f00 2024-05-14T11:20:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7980adce0 con 0x56130e047f00 2024-05-14T11:20:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff7980d7050 con 0x56130e047f00 2024-05-14T11:20:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 10 client.6501 fetch_fsmap learned FSMap version 52 2024-05-14T11:20:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56130df7eef0 con 0x56130e047f00 2024-05-14T11:20:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980adce0 con 0x56130e047f00 2024-05-14T11:20:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff7980da020 con 0x56130e047f00 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 10 client.6501 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 10 client.6501 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 20 client.6501 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc6b930 0x56130e17bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 4 client.6501 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130df5edf0 con 0x56130dc6b930 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc6b930 0x56130e17bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc6b930 0x56130e17bd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93062fcf2a432130 server_cookie=814c39e1e1a96fda in_seq=0 out_seq=0 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.130+0000 7ff78be00640 10 client.6501 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff78be00640 1 -- 172.21.15.116:0/4227587293 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56130df5edf0 con 0x56130dc6b930 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff78be00640 10 client.6501 handle_command_reply: tid=0 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 client.6501 shutdown 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 msgr2=0x56130e1c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff798000c10 tx=0x7ff79801ac10 comp rx=0 tx=0).stop 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 shutdown_connections 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc6b930 0x56130e17bd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e05a120 0x56130e17afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1c8310 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff78be00640 0 client.6501 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 --2- 172.21.15.116:0/4227587293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130dc473e0 0x56130e1c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 >> 172.21.15.116:0/4227587293 conn(0x56130dc87400 msgr2=0x56130e1a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 shutdown_connections 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 1 -- 172.21.15.116:0/4227587293 wait complete. 2024-05-14T11:20:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.134+0000 7ff7b943c000 20 client.6501 trim_cache size 0 max 16384 2024-05-14T11:20:36.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.150+0000 7ff7b943c000 1 Processor -- start 2024-05-14T11:20:36.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.150+0000 7ff7b943c000 1 -- start start 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e047f00 0x56130e22f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e22fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac205790 con 0x56130dc473e0 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac1610c0 con 0x56130e05a120 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac161710 con 0x56130e047f00 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59886/0 (socket says 172.21.15.116:59886) 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1267124371 learned_addr learned my addr 172.21.15.116:0/1267124371 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aa000640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e047f00 0x56130e22f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7ab400640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e22fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e047f00 msgr2=0x56130e22f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e047f00 0x56130e22f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 msgr2=0x56130e22fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e22fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 -- 172.21.15.116:0/1267124371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130e047340 con 0x56130dc473e0 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7ab400640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e22fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff79c05b9e0 tx=0x7ff79c08f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ed96035273f36b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff796a00640 1 -- 172.21.15.116:0/1267124371 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c021040 con 0x56130dc473e0 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff796a00640 1 -- 172.21.15.116:0/1267124371 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c060090 con 0x56130dc473e0 2024-05-14T11:20:36.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff796a00640 1 -- 172.21.15.116:0/1267124371 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c064b40 con 0x56130dc473e0 2024-05-14T11:20:36.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- 172.21.15.116:0/1267124371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 msgr2=0x56130e22f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff79c05b9e0 tx=0x7ff79c08f040 comp rx=0 tx=0).stop 2024-05-14T11:20:36.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- 172.21.15.116:0/1267124371 shutdown_connections 2024-05-14T11:20:36.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e22fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130e047f00 0x56130e22f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 --2- 172.21.15.116:0/1267124371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130dc473e0 0x56130e22f2e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- 172.21.15.116:0/1267124371 >> 172.21.15.116:0/1267124371 conn(0x56130dc87400 msgr2=0x56130e1a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- 172.21.15.116:0/1267124371 shutdown_connections 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- 172.21.15.116:0/1267124371 wait complete. 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 -- start start 2024-05-14T11:20:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.154+0000 7ff7b943c000 1 Processor -- start 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130dc473e0 0x56130e1cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e1cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56130dfca730 con 0x56130e047f00 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56130dfc5410 con 0x56130e05a120 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56130dfd3500 con 0x56130dc473e0 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 10 client.0 _ll_register_callbacks cb 0x7ff7980c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59900/0 (socket says 172.21.15.116:59900) 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 -- 172.21.15.116:0/117210941 learned_addr learned my addr 172.21.15.116:0/117210941 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7ab400640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e1cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130dc473e0 0x56130e1cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 -- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130dc473e0 msgr2=0x56130e1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130dc473e0 0x56130e1cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 -- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 msgr2=0x56130e1cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e1cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 -- 172.21.15.116:0/117210941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130df628e0 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aa000640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff79801c410 tx=0x7ff7980a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475f97e90c5f7429 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980df020 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff7980c9040 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c060090 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff7980e9050 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 10 client.6507 fetch_fsmap learned FSMap version 52 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56130dc7f220 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c060090 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff7980ec020 con 0x56130e047f00 2024-05-14T11:20:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 10 client.6507 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 10 client.6507 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 20 client.6507 populate_metadata read hostname 'smithi116' 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc59990 0x56130e1cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 4 client.6507 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56130dfd3d10 con 0x56130dc59990 2024-05-14T11:20:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc59990 0x56130e1cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff7aaa00640 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc59990 0x56130e1cbc70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52428699521ccfd9 server_cookie=34e8a41c573abf4f in_seq=0 out_seq=0 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.158+0000 7ff78be00640 10 client.6507 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff78be00640 1 -- 172.21.15.116:0/117210941 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56130dfd3d10 con 0x56130dc59990 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff78be00640 10 client.6507 handle_command_reply: tid=0 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 client.6507 shutdown 2024-05-14T11:20:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 msgr2=0x56130e1cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff79801c410 tx=0x7ff7980a7070 comp rx=0 tx=0).stop 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 shutdown_connections 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56130dc59990 0x56130e1cbc70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56130e05a120 0x56130e1cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56130e047f00 0x56130e1cb1f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 --2- 172.21.15.116:0/117210941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56130dc473e0 0x56130e1cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff78be00640 0 client.6507 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 >> 172.21.15.116:0/117210941 conn(0x56130dc87400 msgr2=0x56130e179b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 shutdown_connections 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 1 -- 172.21.15.116:0/117210941 wait complete. 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b943c000 20 client.6507 trim_cache size 0 max 16384 2024-05-14T11:20:36.163 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "0d674080-e235-49a6-88a5-57b7b65ed680": { 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/subdir", 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "0d674080-e235-49a6-88a5-57b7b65ed680", 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive" 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:20:36.164 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 msgr2=0x7ff78c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 0x7ff78c071940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7980af000 tx=0x7ff798008040 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 msgr2=0x7ff7ac0cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e840 tx=0x7ff79c00ec40 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 shutdown_connections 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff78c06f430 0x7ff78c071940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac160080 0x7ff7ac15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7ac0d1a50 0x7ff7ac159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 --2- 172.21.15.116:0/1603306243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7ac0ce4c0 0x7ff7ac0cbb20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 >> 172.21.15.116:0/1603306243 conn(0x7ff7ac0d0770 msgr2=0x7ff7ac15ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:20:36.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 shutdown_connections 2024-05-14T11:20:36.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:20:36.162+0000 7ff7b1e00640 1 -- 172.21.15.116:0/1603306243 wait complete. 2024-05-14T11:20:36.180 INFO:tasks.cephfs.filesystem:scrub status for tag:0d674080-e235-49a6-88a5-57b7b65ed680 - {'path': '/subdir', 'tag': '0d674080-e235-49a6-88a5-57b7b65ed680', 'options': 'recursive'} 2024-05-14T11:20:42.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:20:42.010 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:21:06.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:06.182 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-14T11:21:06.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.342+0000 7efe2fe00640 1 Processor -- start 2024-05-14T11:21:06.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.342+0000 7efe2fe00640 1 -- start start 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7efe30160340 0x7efe300cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe300ce030 0x7efe3006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe300c79a0 con 0x7efe300c6990 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efe3006e9a0 con 0x7efe300ce030 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7efe300cf060 con 0x7efe30160340 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efe300c6990 0x7efe3015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37182/0 (socket says 172.21.15.116:37182) 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2ea00640 1 -- 172.21.15.116:0/1927932370 learned_addr learned my addr 172.21.15.116:0/1927932370 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:06.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2f400640 1 --2- 172.21.15.116:0/1927932370 >> v2:172.21.15.116:3300/0 conn(0x7efe300ce030 0x7efe3006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe26000640 1 --2- 172.21.15.116:0/1927932370 >> v2:172.21.15.149:3300/0 conn(0x7efe30160340 0x7efe300cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2f400640 1 -- 172.21.15.116:0/1927932370 >> v2:172.21.15.149:3300/0 conn(0x7efe30160340 msgr2=0x7efe300cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2f400640 1 --2- 172.21.15.116:0/1927932370 >> v2:172.21.15.149:3300/0 conn(0x7efe30160340 0x7efe300cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2f400640 1 -- 172.21.15.116:0/1927932370 >> v1:172.21.15.116:6789/0 conn(0x7efe300c6990 legacy=0x7efe3015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2f400640 1 -- 172.21.15.116:0/1927932370 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe300cf340 con 0x7efe300ce030 2024-05-14T11:21:06.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.346+0000 7efe2e000640 1 -- 172.21.15.116:0/1927932370 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2581404632 0 0) 0x7efe300c79a0 con 0x7efe300c6990 2024-05-14T11:21:06.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2f400640 1 --2- 172.21.15.116:0/1927932370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300ce030 0x7efe3006ee30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efe30071ac0 tx=0x7efe2801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ff9d39a75a99a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2e000640 1 -- 172.21.15.116:0/1927932370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280b2070 con 0x7efe300ce030 2024-05-14T11:21:06.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2e000640 1 -- 172.21.15.116:0/1927932370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280a58d0 con 0x7efe300ce030 2024-05-14T11:21:06.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2e000640 1 -- 172.21.15.116:0/1927932370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280a5ca0 con 0x7efe300ce030 2024-05-14T11:21:06.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- 172.21.15.116:0/1927932370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300ce030 msgr2=0x7efe3006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 --2- 172.21.15.116:0/1927932370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300ce030 0x7efe3006ee30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efe30071ac0 tx=0x7efe2801c900 comp rx=0 tx=0).stop 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- 172.21.15.116:0/1927932370 shutdown_connections 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 --2- 172.21.15.116:0/1927932370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300ce030 0x7efe3006ee30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 --2- 172.21.15.116:0/1927932370 >> v2:172.21.15.149:3300/0 conn(0x7efe30160340 0x7efe300cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- 172.21.15.116:0/1927932370 >> 172.21.15.116:0/1927932370 conn(0x7efe300b4510 msgr2=0x7efe300b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- 172.21.15.116:0/1927932370 shutdown_connections 2024-05-14T11:21:06.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- 172.21.15.116:0/1927932370 wait complete. 2024-05-14T11:21:06.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 Processor -- start 2024-05-14T11:21:06.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.350+0000 7efe2fe00640 1 -- start start 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe300c6990 0x7efe301534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 0x7efe300c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015c4c0 con 0x7efe300c8b80 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015c340 con 0x7efe300c6990 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe3006f370 con 0x7efe300ce030 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41876/0 (socket says 172.21.15.116:41876) 2024-05-14T11:21:06.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 -- 172.21.15.116:0/3190715421 learned_addr learned my addr 172.21.15.116:0/3190715421 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2ea00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe300c6990 0x7efe301534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe26000640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 0x7efe300c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 -- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 msgr2=0x7efe300c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 0x7efe300c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 -- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe300c6990 msgr2=0x7efe301534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe300c6990 0x7efe301534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 -- 172.21.15.116:0/3190715421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe300c8140 con 0x7efe300c8b80 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe26000640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 0x7efe300c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2f400640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efe280b0040 tx=0x7efe280196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe27400640 1 -- 172.21.15.116:0/3190715421 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280b2040 con 0x7efe300c8b80 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe27400640 1 -- 172.21.15.116:0/3190715421 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280bf070 con 0x7efe300c8b80 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3015e8a0 con 0x7efe300c8b80 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe27400640 1 -- 172.21.15.116:0/3190715421 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280b88b0 con 0x7efe300c8b80 2024-05-14T11:21:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.354+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3015ee60 con 0x7efe300c8b80 2024-05-14T11:21:06.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe27400640 1 -- 172.21.15.116:0/3190715421 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe280b8a50 con 0x7efe300c8b80 2024-05-14T11:21:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe27400640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 0x7efe0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe27400640 1 -- 172.21.15.116:0/3190715421 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7efe2812a5f0 con 0x7efe300c8b80 2024-05-14T11:21:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe2ea00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 0x7efe0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe3d392000 1 Processor -- start 2024-05-14T11:21:06.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe3d392000 1 -- start start 2024-05-14T11:21:06.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.358+0000 7efe2ea00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 0x7efe0c071910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe20000f70 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bc860ba100 con 0x55bc85d3b3e0 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bc86072ca0 con 0x55bc8613bf00 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bc860b60c0 con 0x55bc8614e120 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41878/0 (socket says 172.21.15.116:41878) 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 -- 172.21.15.116:0/1125025609 learned_addr learned my addr 172.21.15.116:0/1125025609 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2ea00640 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe26000640 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 -- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 msgr2=0x55bc862b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 -- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 msgr2=0x55bc862b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 -- 172.21.15.116:0/1125025609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc86077800 con 0x55bc85d3b3e0 2024-05-14T11:21:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe2f400640 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efe280f3270 tx=0x7efe280f2c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f1c2e2ee9fda26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe1f400640 1 -- 172.21.15.116:0/1125025609 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2810e020 con 0x55bc85d3b3e0 2024-05-14T11:21:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe1f400640 1 -- 172.21.15.116:0/1125025609 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280f7380 con 0x55bc85d3b3e0 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe1f400640 1 -- 172.21.15.116:0/1125025609 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280f6c80 con 0x55bc85d3b3e0 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- 172.21.15.116:0/1125025609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 msgr2=0x55bc862b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efe280f3270 tx=0x7efe280f2c70 comp rx=0 tx=0).stop 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- 172.21.15.116:0/1125025609 shutdown_connections 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862b5dd0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 --2- 172.21.15.116:0/1125025609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- 172.21.15.116:0/1125025609 >> 172.21.15.116:0/1125025609 conn(0x55bc85d7b400 msgr2=0x55bc86298310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- 172.21.15.116:0/1125025609 shutdown_connections 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- 172.21.15.116:0/1125025609 wait complete. 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 -- start start 2024-05-14T11:21:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.362+0000 7efe3d392000 1 Processor -- start 2024-05-14T11:21:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 0x55bc862bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc862bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bc861852a0 con 0x55bc8614e120 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bc86175f40 con 0x55bc85d3b3e0 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bc86076080 con 0x55bc8613bf00 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 10 client.0 _ll_register_callbacks cb 0x55bc86256c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 0x55bc862bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41892/0 (socket says 172.21.15.116:41892) 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 0x55bc862bbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:37636/0 (socket says 172.21.15.116:37636) 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 -- 172.21.15.116:0/3722372351 learned_addr learned my addr 172.21.15.116:0/3722372351 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe26000640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc862bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 -- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 msgr2=0x55bc862bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc862bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 -- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 msgr2=0x55bc862bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 0x55bc862bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 -- 172.21.15.116:0/3722372351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc86064860 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2f400640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efe280f6c10 tx=0x7efe28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a15186455cdc4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2810e030 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7efe280ff850 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe280f7380 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe28108050 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 10 client.6522 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55bc86072ef0 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280f7380 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7efe2810b020 con 0x55bc8614e120 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 10 client.6522 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 10 client.6522 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 20 client.6522 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d5f930 0x55bc8626fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 4 client.6522 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:06.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc86052df0 con 0x55bc85d5f930 2024-05-14T11:21:06.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2ea00640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d5f930 0x55bc8626fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe13e00640 10 client.6522 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.366+0000 7efe2ea00640 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d5f930 0x55bc8626fd30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4de008f6279654a server_cookie=7087d62189d8c5db in_seq=0 out_seq=0 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe13e00640 1 -- 172.21.15.116:0/3722372351 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc86052df0 con 0x55bc85d5f930 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe13e00640 10 client.6522 handle_command_reply: tid=0 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 client.6522 shutdown 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 msgr2=0x55bc8626ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efe280f6c10 tx=0x7efe28009750 comp rx=0 tx=0).stop 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 shutdown_connections 2024-05-14T11:21:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d5f930 0x55bc8626fd30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc8614e120 0x55bc8626ef90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc862bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe13e00640 0 client.6522 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 --2- 172.21.15.116:0/3722372351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc85d3b3e0 0x55bc862bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 >> 172.21.15.116:0/3722372351 conn(0x55bc85d7b400 msgr2=0x55bc86298200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 shutdown_connections 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 1 -- 172.21.15.116:0/3722372351 wait complete. 2024-05-14T11:21:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.370+0000 7efe3d392000 20 client.6522 trim_cache size 0 max 16384 2024-05-14T11:21:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.386+0000 7efe3d392000 1 Processor -- start 2024-05-14T11:21:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.386+0000 7efe3d392000 1 -- start start 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc86296d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8614e120 0x55bc86297270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe30204cf0 con 0x55bc85d3b3e0 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe300ca9b0 con 0x55bc8614e120 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe302046a0 con 0x55bc8613bf00 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41904/0 (socket says 172.21.15.116:41904) 2024-05-14T11:21:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 -- 172.21.15.116:0/731051439 learned_addr learned my addr 172.21.15.116:0/731051439 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2f400640 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8614e120 0x55bc86297270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe26000640 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc86296d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 -- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 msgr2=0x55bc86296d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc86296d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 -- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8614e120 msgr2=0x55bc86297270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8614e120 0x55bc86297270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 -- 172.21.15.116:0/731051439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8626d010 con 0x55bc85d3b3e0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe2ea00640 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efe2000fa90 tx=0x7efe2000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bce218444bd3e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe1ea00640 1 -- 172.21.15.116:0/731051439 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20016d50 con 0x55bc85d3b3e0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe1ea00640 1 -- 172.21.15.116:0/731051439 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2001b040 con 0x55bc85d3b3e0 2024-05-14T11:21:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe1ea00640 1 -- 172.21.15.116:0/731051439 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20016110 con 0x55bc85d3b3e0 2024-05-14T11:21:06.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- 172.21.15.116:0/731051439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 msgr2=0x55bc862c0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efe2000fa90 tx=0x7efe2000a000 comp rx=0 tx=0).stop 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- 172.21.15.116:0/731051439 shutdown_connections 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8614e120 0x55bc86297270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8613bf00 0x55bc86296d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 --2- 172.21.15.116:0/731051439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc862c0e40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- 172.21.15.116:0/731051439 >> 172.21.15.116:0/731051439 conn(0x55bc85d7b400 msgr2=0x55bc8626b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- 172.21.15.116:0/731051439 shutdown_connections 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- 172.21.15.116:0/731051439 wait complete. 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 -- start start 2024-05-14T11:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.390+0000 7efe3d392000 1 Processor -- start 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bc860be730 con 0x55bc85d3b3e0 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bc860b9410 con 0x55bc8613bf00 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bc860c7500 con 0x55bc8614e120 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 10 client.0 _ll_register_callbacks cb 0x7efe1800b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41906/0 (socket says 172.21.15.116:41906) 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 -- 172.21.15.116:0/222220799 learned_addr learned my addr 172.21.15.116:0/222220799 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe26000640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2f400640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 -- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 msgr2=0x55bc862be140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862be140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 -- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 msgr2=0x55bc862bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 -- 172.21.15.116:0/222220799 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8613b340 con 0x55bc85d3b3e0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe2ea00640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efe2000fa90 tx=0x7efe20005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b7f5eb5605f50c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2002a4e0 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7efe2002a680 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe2001b040 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7efe2003c050 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 10 client.6528 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55bc860568e0 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2001b040 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7efe2003f020 con 0x55bc85d3b3e0 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 10 client.6528 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 10 client.6528 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 20 client.6528 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d4d990 0x55bc8632ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 4 client.6528 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:21:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc860c7d10 con 0x55bc85d4d990 2024-05-14T11:21:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.394+0000 7efe26000640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d4d990 0x55bc8632ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe13e00640 10 client.6528 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe26000640 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d4d990 0x55bc8632ed20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eace89450ff5bfef server_cookie=fc80f486fa6d64c in_seq=0 out_seq=0 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe13e00640 1 -- 172.21.15.116:0/222220799 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc860c7d10 con 0x55bc85d4d990 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe13e00640 10 client.6528 handle_command_reply: tid=0 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 client.6528 shutdown 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 msgr2=0x55bc86271170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efe2000fa90 tx=0x7efe20005800 comp rx=0 tx=0).stop 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 shutdown_connections 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55bc85d4d990 0x55bc8632ed20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc8613bf00 0x55bc862bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bc8614e120 0x55bc862be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 --2- 172.21.15.116:0/222220799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc85d3b3e0 0x55bc86271170 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe13e00640 0 client.6528 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 >> 172.21.15.116:0/222220799 conn(0x55bc85d7b400 msgr2=0x55bc86296d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 shutdown_connections 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 1 -- 172.21.15.116:0/222220799 wait complete. 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe3d392000 20 client.6528 trim_cache size 0 max 16384 2024-05-14T11:21:06.400 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:06.401 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:21:06.401 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:21:06.401 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 msgr2=0x7efe0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 0x7efe0c071910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe20000f70 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 msgr2=0x7efe300c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efe280b0040 tx=0x7efe280196a0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 shutdown_connections 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efe0c06f400 0x7efe0c071910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe300c8b80 0x7efe300c8f90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe300ce030 0x7efe300c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 --2- 172.21.15.116:0/3190715421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe300c6990 0x7efe301534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 >> 172.21.15.116:0/3190715421 conn(0x7efe300b4510 msgr2=0x7efe3006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 shutdown_connections 2024-05-14T11:21:06.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.398+0000 7efe2fe00640 1 -- 172.21.15.116:0/3190715421 wait complete. 2024-05-14T11:21:06.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:21:06.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.562+0000 7f3ff9600640 1 Processor -- start 2024-05-14T11:21:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.562+0000 7f3ff9600640 1 -- start start 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff4156fa0 0x7f3ff406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3ff40d1670 0x7f3ff40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff406e9e0 0x7f3ff406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff41573b0 con 0x7f3ff406e9e0 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff4161120 con 0x7f3ff40d1670 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40d1fc0 con 0x7f3ff4156fa0 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff406e9e0 0x7f3ff406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ff406e9e0 0x7f3ff406edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41920/0 (socket says 172.21.15.116:41920) 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 -- 172.21.15.116:0/1865591621 learned_addr learned my addr 172.21.15.116:0/1865591621 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3fea000640 1 --2- 172.21.15.116:0/1865591621 >> v2:172.21.15.149:3300/0 conn(0x7f3ff40d1670 0x7f3ff40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff2a00640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff4156fa0 0x7f3ff406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 -- 172.21.15.116:0/1865591621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff4156fa0 msgr2=0x7f3ff406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff4156fa0 0x7f3ff406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 -- 172.21.15.116:0/1865591621 >> v2:172.21.15.149:3300/0 conn(0x7f3ff40d1670 msgr2=0x7f3ff40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 --2- 172.21.15.116:0/1865591621 >> v2:172.21.15.149:3300/0 conn(0x7f3ff40d1670 0x7f3ff40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.566+0000 7f3ff3400640 1 -- 172.21.15.116:0/1865591621 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff4160510 con 0x7f3ff406e9e0 2024-05-14T11:21:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff3400640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff406e9e0 0x7f3ff406edf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40d2170 tx=0x7f3fec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b508a095bec89fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff2000640 1 -- 172.21.15.116:0/1865591621 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0a5840 con 0x7f3ff406e9e0 2024-05-14T11:21:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff2000640 1 -- 172.21.15.116:0/1865591621 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec0a59e0 con 0x7f3ff406e9e0 2024-05-14T11:21:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff2000640 1 -- 172.21.15.116:0/1865591621 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0a5db0 con 0x7f3ff406e9e0 2024-05-14T11:21:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 -- 172.21.15.116:0/1865591621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff406e9e0 msgr2=0x7f3ff406edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff406e9e0 0x7f3ff406edf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40d2170 tx=0x7f3fec01c9c0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 -- 172.21.15.116:0/1865591621 shutdown_connections 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff406e9e0 0x7f3ff406edf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1865591621 >> v2:172.21.15.149:3300/0 conn(0x7f3ff40d1670 0x7f3ff40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1865591621 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff4156fa0 0x7f3ff406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 -- 172.21.15.116:0/1865591621 >> 172.21.15.116:0/1865591621 conn(0x7f3ff40b4620 msgr2=0x7f3ff40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 -- 172.21.15.116:0/1865591621 shutdown_connections 2024-05-14T11:21:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.570+0000 7f3ff9600640 1 -- 172.21.15.116:0/1865591621 wait complete. 2024-05-14T11:21:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 Processor -- start 2024-05-14T11:21:06.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- start start 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff406e9e0 0x7f3ff414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4156fa0 0x7f3ff4143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4070520 con 0x7f3ff40d1670 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff406f740 con 0x7f3ff4156fa0 2024-05-14T11:21:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40706a0 con 0x7f3ff406e9e0 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41924/0 (socket says 172.21.15.116:41924) 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 -- 172.21.15.116:0/1199156533 learned_addr learned my addr 172.21.15.116:0/1199156533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff3400640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4156fa0 0x7f3ff4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff2a00640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff406e9e0 0x7f3ff414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 -- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff406e9e0 msgr2=0x7f3ff414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff406e9e0 0x7f3ff414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 -- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4156fa0 msgr2=0x7f3ff4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4156fa0 0x7f3ff4143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 -- 172.21.15.116:0/1199156533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff415b920 con 0x7f3ff40d1670 2024-05-14T11:21:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3fea000640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e8a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x7f3ff40d1670 2024-05-14T11:21:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc00ee90 con 0x7f3ff40d1670 2024-05-14T11:21:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff40ceb10 con 0x7f3ff40d1670 2024-05-14T11:21:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00cb40 con 0x7f3ff40d1670 2024-05-14T11:21:06.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.574+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff40cf0d0 con 0x7f3ff40d1670 2024-05-14T11:21:06.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff415b690 con 0x7f3ff40d1670 2024-05-14T11:21:06.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3fdc01a070 con 0x7f3ff40d1670 2024-05-14T11:21:06.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3feb400640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 0x7f3fd0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f3fdc08f280 con 0x7f3ff40d1670 2024-05-14T11:21:06.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3ff2a00640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 0x7f3fd0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.578+0000 7f3ff2a00640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 0x7f3fd0071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40b0d60 tx=0x7f3fe4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.582+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3fdc05b930 con 0x7f3ff40d1670 2024-05-14T11:21:06.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.778+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff406a9c0 con 0x7f3ff40d1670 2024-05-14T11:21:06.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.778+0000 7f3feb400640 1 -- 172.21.15.116:0/1199156533 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5193 (secure 0 0 0) 0x7f3fdc05fc30 con 0x7f3ff40d1670 2024-05-14T11:21:06.779 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:06.779 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:20:31.015427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:06.779 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 52 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 msgr2=0x7f3fd0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 0x7f3fd0071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40b0d60 tx=0x7f3fe4009190 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 msgr2=0x7f3ff4157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e8a0 tx=0x7f3fdc00ed60 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 shutdown_connections 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3fd006f400 0x7f3fd0071910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4156fa0 0x7f3ff4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ff40d1670 0x7f3ff4157540 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 --2- 172.21.15.116:0/1199156533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3ff406e9e0 0x7f3ff414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 >> 172.21.15.116:0/1199156533 conn(0x7f3ff40b4620 msgr2=0x7f3ff41662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 shutdown_connections 2024-05-14T11:21:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.782+0000 7f3ff9600640 1 -- 172.21.15.116:0/1199156533 wait complete. 2024-05-14T11:21:06.800 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-14T11:21:06.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.946+0000 7fbf3f400640 1 Processor -- start 2024-05-14T11:21:06.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.946+0000 7fbf3f400640 1 -- start start 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf3806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3f400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3806cc40 con 0x7fbf38072a30 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3f400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3806cf10 con 0x7fbf3806f730 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3806db20 con 0x7fbf380b0770 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3d400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbf38072a30 0x7fbf38072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37188/0 (socket says 172.21.15.116:37188) 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3d400640 1 -- 172.21.15.116:0/540638521 learned_addr learned my addr 172.21.15.116:0/540638521 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/540638521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf3806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583872848 0 0) 0x7fbf3806cc40 con 0x7fbf38072a30 2024-05-14T11:21:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf1c003530 con 0x7fbf38072a30 2024-05-14T11:21:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312204178 0 0) 0x7fbf3806cf10 con 0x7fbf3806f730 2024-05-14T11:21:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf3806cc40 con 0x7fbf3806f730 2024-05-14T11:21:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3ca00640 1 -- 172.21.15.116:0/540638521 >> v1:172.21.15.149:6789/0 conn(0x7fbf3806f730 legacy=0x7fbf3806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3ca00640 1 -- 172.21.15.116:0/540638521 >> v1:172.21.15.116:6789/0 conn(0x7fbf38072a30 legacy=0x7fbf38072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.950+0000 7fbf3ca00640 1 -- 172.21.15.116:0/540638521 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3806d1f0 con 0x7fbf380b0770 2024-05-14T11:21:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/540638521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf3806c700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800a990 tx=0x7fbf2801d7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95653ad47c6023b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280b3070 con 0x7fbf380b0770 2024-05-14T11:21:06.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf280a6820 con 0x7fbf380b0770 2024-05-14T11:21:06.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf37e00640 1 -- 172.21.15.116:0/540638521 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800b040 con 0x7fbf380b0770 2024-05-14T11:21:06.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 -- 172.21.15.116:0/540638521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 msgr2=0x7fbf3806c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 --2- 172.21.15.116:0/540638521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf3806c700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800a990 tx=0x7fbf2801d7f0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 -- 172.21.15.116:0/540638521 shutdown_connections 2024-05-14T11:21:06.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 --2- 172.21.15.116:0/540638521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf3806c700 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 -- 172.21.15.116:0/540638521 >> 172.21.15.116:0/540638521 conn(0x7fbf380b4620 msgr2=0x7fbf380b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 -- 172.21.15.116:0/540638521 shutdown_connections 2024-05-14T11:21:06.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.954+0000 7fbf3f400640 1 -- 172.21.15.116:0/540638521 wait complete. 2024-05-14T11:21:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 Processor -- start 2024-05-14T11:21:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 -- start start 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3806f730 0x7fbf38152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf38167950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38073ae0 con 0x7fbf38072a30 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38073960 con 0x7fbf3806f730 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbf38169fb0 con 0x7fbf380b0770 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41926/0 (socket says 172.21.15.116:41926) 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 -- 172.21.15.116:0/1745968023 learned_addr learned my addr 172.21.15.116:0/1745968023 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3806f730 0x7fbf38152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf3d400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf38167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 -- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 msgr2=0x7fbf38167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf38167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 -- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3806f730 msgr2=0x7fbf38152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3806f730 0x7fbf38152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 -- 172.21.15.116:0/1745968023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf38159760 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c3f0 tx=0x7fbf2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.958+0000 7fbf35600640 1 -- 172.21.15.116:0/1745968023 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20017070 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf35600640 1 -- 172.21.15.116:0/1745968023 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf200052f0 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf38157600 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf35600640 1 -- 172.21.15.116:0/1745968023 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20003840 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf38157bc0 con 0x7fbf38072a30 2024-05-14T11:21:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf35600640 1 -- 172.21.15.116:0/1745968023 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbf2001a070 con 0x7fbf38072a30 2024-05-14T11:21:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf35600640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 0x7fbf14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf35600640 1 -- 172.21.15.116:0/1745968023 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fbf2008e7c0 con 0x7fbf38072a30 2024-05-14T11:21:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf46a58000 1 Processor -- start 2024-05-14T11:21:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 0x7fbf14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.962+0000 7fbf46a58000 1 -- start start 2024-05-14T11:21:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 0x7fbf14071910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf280b0000 tx=0x7fbf280a8070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913eb5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913eb68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562913cbb1a0 con 0x562913d422d0 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562913c73b80 con 0x56291393c580 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562913cb7160 con 0x562913d52bd0 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41942/0 (socket says 172.21.15.116:41942) 2024-05-14T11:21:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 -- 172.21.15.116:0/3438188343 learned_addr learned my addr 172.21.15.116:0/3438188343 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913eb5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf3d400640 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913eb68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 -- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 msgr2=0x562913eb5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913eb5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 -- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 msgr2=0x562913eb68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913eb68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 -- 172.21.15.116:0/3438188343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562913cc7630 con 0x562913d422d0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbf20056ef0 tx=0x7fbf20005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e86af92df0995 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2d200640 1 -- 172.21.15.116:0/3438188343 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2005f170 con 0x562913d422d0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2d200640 1 -- 172.21.15.116:0/3438188343 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf20071070 con 0x562913d422d0 2024-05-14T11:21:06.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf2d200640 1 -- 172.21.15.116:0/3438188343 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2005a2d0 con 0x562913d422d0 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- 172.21.15.116:0/3438188343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 msgr2=0x562913eb63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbf20056ef0 tx=0x7fbf20005560 comp rx=0 tx=0).stop 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- 172.21.15.116:0/3438188343 shutdown_connections 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913eb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913eb63b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3438188343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913eb5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- 172.21.15.116:0/3438188343 >> 172.21.15.116:0/3438188343 conn(0x56291397b760 msgr2=0x562913e98e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- 172.21.15.116:0/3438188343 shutdown_connections 2024-05-14T11:21:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.966+0000 7fbf46a58000 1 -- 172.21.15.116:0/3438188343 wait complete. 2024-05-14T11:21:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- start start 2024-05-14T11:21:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 Processor -- start 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913ebd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d422d0 0x562913ebda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562913c5aab0 con 0x562913d52bd0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562913c6ff20 con 0x56291393c580 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562913d1ada0 con 0x562913d422d0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 10 client.0 _ll_register_callbacks cb 0x562913e56f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41952/0 (socket says 172.21.15.116:41952) 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 -- 172.21.15.116:0/313726928 learned_addr learned my addr 172.21.15.116:0/313726928 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d422d0 0x562913ebda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913ebd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 -- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d422d0 msgr2=0x562913ebda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d422d0 0x562913ebda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 -- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 msgr2=0x562913ebd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913ebd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 -- 172.21.15.116:0/313726928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562913c79ce0 con 0x562913d52bd0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913ebd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf3d400640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf300074d0 tx=0x7fbf30007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d72e5140eae86b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf25600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30018020 con 0x562913d52bd0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fbf30010bd0 con 0x562913d52bd0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf20071070 con 0x562913d52bd0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fbf30022050 con 0x562913d52bd0 2024-05-14T11:21:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 10 client.6546 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x562913c65740 con 0x562913d52bd0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.970+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf20071070 con 0x562913d52bd0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fbf30025020 con 0x562913d52bd0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 10 client.6546 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 10 client.6546 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 20 client.6546 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562913960ad0 0x562913e6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 4 client.6546 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562913c53e60 con 0x562913960ad0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562913960ad0 0x562913e6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562913960ad0 0x562913e6fff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf80db1aeb92c8b server_cookie=23f5b1ac1ef107e7 in_seq=0 out_seq=0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf25600640 10 client.6546 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf25600640 1 -- 172.21.15.116:0/313726928 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562913c53e60 con 0x562913960ad0 2024-05-14T11:21:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf25600640 10 client.6546 handle_command_reply: tid=0 2024-05-14T11:21:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 client.6546 shutdown 2024-05-14T11:21:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 msgr2=0x562913e6f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf300074d0 tx=0x7fbf30007990 comp rx=0 tx=0).stop 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 shutdown_connections 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x562913960ad0 0x562913e6fff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d52bd0 0x562913e6f280 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d422d0 0x562913ebda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 --2- 172.21.15.116:0/313726928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56291393c580 0x562913ebd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 >> 172.21.15.116:0/313726928 conn(0x56291397b760 msgr2=0x562913e97640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf25600640 0 client.6546 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 shutdown_connections 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 1 -- 172.21.15.116:0/313726928 wait complete. 2024-05-14T11:21:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.974+0000 7fbf46a58000 20 client.6546 trim_cache size 0 max 16384 2024-05-14T11:21:06.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 Processor -- start 2024-05-14T11:21:06.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- start start 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913f25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38205490 con 0x56291393c580 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38204920 con 0x562913d422d0 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbf38204e70 con 0x562913d52bd0 2024-05-14T11:21:06.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41966/0 (socket says 172.21.15.116:41966) 2024-05-14T11:21:06.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:37720/0 (socket says 172.21.15.116:37720) 2024-05-14T11:21:06.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 -- 172.21.15.116:0/3677029086 learned_addr learned my addr 172.21.15.116:0/3677029086 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913f25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 -- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 msgr2=0x562913f25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913f25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 -- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 msgr2=0x562913f24d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 -- 172.21.15.116:0/3677029086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562913e6f1c0 con 0x56291393c580 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800a8e0 tx=0x7fbf280a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a05c84f1b635fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf27e00640 1 -- 172.21.15.116:0/3677029086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280b3040 con 0x56291393c580 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf27e00640 1 -- 172.21.15.116:0/3677029086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf280042e0 con 0x56291393c580 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf27e00640 1 -- 172.21.15.116:0/3677029086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28003af0 con 0x56291393c580 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- 172.21.15.116:0/3677029086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 msgr2=0x562913f247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800a8e0 tx=0x7fbf280a8000 comp rx=0 tx=0).stop 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- 172.21.15.116:0/3677029086 shutdown_connections 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d422d0 0x562913f24d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562913d52bd0 0x562913f25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 --2- 172.21.15.116:0/3677029086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56291393c580 0x562913f247f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- 172.21.15.116:0/3677029086 >> 172.21.15.116:0/3677029086 conn(0x56291397b760 msgr2=0x562913ecc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:06.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- 172.21.15.116:0/3677029086 shutdown_connections 2024-05-14T11:21:06.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.990+0000 7fbf46a58000 1 -- 172.21.15.116:0/3677029086 wait complete. 2024-05-14T11:21:06.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- start start 2024-05-14T11:21:06.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 Processor -- start 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 0x562913f28b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 0x562913ee3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562913c645f0 con 0x562913d422d0 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562913cb5e80 con 0x562913d52bd0 2024-05-14T11:21:06.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562913c76f60 con 0x56291393c580 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 10 client.0 _ll_register_callbacks cb 0x7fbf30001ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41982/0 (socket says 172.21.15.116:41982) 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 -- 172.21.15.116:0/87842892 learned_addr learned my addr 172.21.15.116:0/87842892 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf3d400640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 0x562913ee3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 0x562913f28b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 -- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 msgr2=0x562913f28b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 0x562913f28b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 -- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 msgr2=0x562913ee3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 0x562913ee3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 -- 172.21.15.116:0/87842892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562913c73dd0 con 0x562913d422d0 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf3d400640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 0x562913ee3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:06.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 0x562913f28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf2fe00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbf2005ad00 tx=0x7fbf20014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540dbed254a03d2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf25600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2008c050 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fbf20061370 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf280042e0 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fbf20063b90 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 10 client.6552 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x562913d3bc40 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.994+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf280042e0 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fbf200567e0 con 0x562913d422d0 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 10 client.6552 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:06.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 10 client.6552 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 20 client.6552 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56291394eb30 0x562913ee4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 4 client.6552 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562913cc8db0 con 0x56291394eb30 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56291394eb30 0x562913ee4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf25600640 10 client.6552 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3ca00640 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56291394eb30 0x562913ee4900 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a636a4938205c4d2 server_cookie=8c7ab6b566fa0c9b in_seq=0 out_seq=0 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf25600640 1 -- 172.21.15.116:0/87842892 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x562913cc8db0 con 0x56291394eb30 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf25600640 10 client.6552 handle_command_reply: tid=0 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 client.6552 shutdown 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 msgr2=0x562913f290a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbf2005ad00 tx=0x7fbf20014040 comp rx=0 tx=0).stop 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 shutdown_connections 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x56291394eb30 0x562913ee4900 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562913d52bd0 0x562913ee3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562913d422d0 0x562913f290a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 --2- 172.21.15.116:0/87842892 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56291393c580 0x562913f28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf25600640 0 client.6552 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 >> 172.21.15.116:0/87842892 conn(0x56291397b760 msgr2=0x562913ec3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 shutdown_connections 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 1 -- 172.21.15.116:0/87842892 wait complete. 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf46a58000 20 client.6552 trim_cache size 0 max 16384 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:21:07.001 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "dentry", 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 3722144321, 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627776, 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "frag": "*", 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "dname": "file_to_be_damaged", 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_id": "head", 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:07.002 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 msgr2=0x7fbf14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 0x7fbf14071910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbf280b0000 tx=0x7fbf280a8070 comp rx=0 tx=0).stop 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 msgr2=0x7fbf38167410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c3f0 tx=0x7fbf2000c8b0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 shutdown_connections 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbf1406f400 0x7fbf14071910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbf380b0770 0x7fbf38167950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf38072a30 0x7fbf38167410 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 --2- 172.21.15.116:0/1745968023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3806f730 0x7fbf38152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 >> 172.21.15.116:0/1745968023 conn(0x7fbf380b4620 msgr2=0x7fbf380b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 shutdown_connections 2024-05-14T11:21:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:06.998+0000 7fbf3f400640 1 -- 172.21.15.116:0/1745968023 wait complete. 2024-05-14T11:21:07.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:21:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.162+0000 7f421c200640 1 Processor -- start 2024-05-14T11:21:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.162+0000 7f421c200640 1 -- start start 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421c200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f421415fe00 0x7f42140d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421c200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42140cfeb0 con 0x7f42140b70a0 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421c200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42140cf110 con 0x7f42140b9160 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421c200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f42140b6d80 con 0x7f421415fe00 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421a200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f421415fe00 0x7f42140d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f4219800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f42140b70a0 0x7f42140b8a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37198/0 (socket says 172.21.15.116:37198) 2024-05-14T11:21:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f4219800640 1 -- 172.21.15.116:0/215957328 learned_addr learned my addr 172.21.15.116:0/215957328 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421a200640 1 -- 172.21.15.116:0/215957328 >> v1:172.21.15.149:6789/0 conn(0x7f42140b9160 legacy=0x7f42140d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421a200640 1 -- 172.21.15.116:0/215957328 >> v1:172.21.15.116:6789/0 conn(0x7f42140b70a0 legacy=0x7f42140b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f421a200640 1 -- 172.21.15.116:0/215957328 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42140cf3d0 con 0x7f421415fe00 2024-05-14T11:21:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.166+0000 7f4213e00640 1 -- 172.21.15.116:0/215957328 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215285550 0 0) 0x7f42140cfeb0 con 0x7f42140b70a0 2024-05-14T11:21:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421a200640 1 --2- 172.21.15.116:0/215957328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421415fe00 0x7f42140d1a20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f420c009d90 tx=0x7f420c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270cd47eda73152 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f4213e00640 1 -- 172.21.15.116:0/215957328 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0b2070 con 0x7f421415fe00 2024-05-14T11:21:07.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f4213e00640 1 -- 172.21.15.116:0/215957328 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0a5830 con 0x7f421415fe00 2024-05-14T11:21:07.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f4213e00640 1 -- 172.21.15.116:0/215957328 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0a5c00 con 0x7f421415fe00 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 -- 172.21.15.116:0/215957328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421415fe00 msgr2=0x7f42140d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 --2- 172.21.15.116:0/215957328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421415fe00 0x7f42140d1a20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f420c009d90 tx=0x7f420c01c970 comp rx=0 tx=0).stop 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 -- 172.21.15.116:0/215957328 shutdown_connections 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 --2- 172.21.15.116:0/215957328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421415fe00 0x7f42140d1a20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 -- 172.21.15.116:0/215957328 >> 172.21.15.116:0/215957328 conn(0x7f42140d3740 msgr2=0x7f42140d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 -- 172.21.15.116:0/215957328 shutdown_connections 2024-05-14T11:21:07.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.170+0000 7f421c200640 1 -- 172.21.15.116:0/215957328 wait complete. 2024-05-14T11:21:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 Processor -- start 2024-05-14T11:21:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 -- start start 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42140b70a0 0x7f42140c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f421415fe00 0x7f421414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42140b6b40 con 0x7f42140b9160 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42140b69c0 con 0x7f421415fe00 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421c200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f421416e640 con 0x7f42140b70a0 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f4218e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f421a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f421415fe00 0x7f421414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f4218e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42002/0 (socket says 172.21.15.116:42002) 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f4218e00640 1 -- 172.21.15.116:0/4016195010 learned_addr learned my addr 172.21.15.116:0/4016195010 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.174+0000 7f4219800640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42140b70a0 0x7f42140c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 -- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42140b70a0 msgr2=0x7f42140c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42140b70a0 0x7f42140c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 -- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f421415fe00 msgr2=0x7f421414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f421415fe00 0x7f421414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 -- 172.21.15.116:0/4016195010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421414ed80 con 0x7f42140b9160 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4218e00640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f420000c480 tx=0x7f420000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000052d0 con 0x7f42140b9160 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200005470 con 0x7f42140b9160 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200012850 con 0x7f42140b9160 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4214155570 con 0x7f42140b9160 2024-05-14T11:21:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4214155b30 con 0x7f42140b9160 2024-05-14T11:21:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.178+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421406a9c0 con 0x7f42140b9160 2024-05-14T11:21:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.182+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f42000129f0 con 0x7f42140b9160 2024-05-14T11:21:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.182+0000 7f4211600640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 0x7f41f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.182+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f420008ef40 con 0x7f42140b9160 2024-05-14T11:21:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.182+0000 7f4219800640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 0x7f41f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.182+0000 7f4219800640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 0x7f41f4071910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4208000f70 tx=0x7f4208007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.186+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f420005b680 con 0x7f42140b9160 2024-05-14T11:21:07.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.378+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4214075f50 con 0x7f42140b9160 2024-05-14T11:21:07.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.378+0000 7f4211600640 1 -- 172.21.15.116:0/4016195010 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f420005f980 con 0x7f42140b9160 2024-05-14T11:21:07.379 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:07.379 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 msgr2=0x7f41f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 0x7f41f4071910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4208000f70 tx=0x7f4208007240 comp rx=0 tx=0).stop 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 msgr2=0x7f421414e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f420000c480 tx=0x7f420000c940 comp rx=0 tx=0).stop 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 shutdown_connections 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f41f406f400 0x7f41f4071910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f421415fe00 0x7f421414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42140b9160 0x7f421414e840 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 --2- 172.21.15.116:0/4016195010 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42140b70a0 0x7f42140c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 >> 172.21.15.116:0/4016195010 conn(0x7f42140d3740 msgr2=0x7f42140cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 shutdown_connections 2024-05-14T11:21:07.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.382+0000 7f421c200640 1 -- 172.21.15.116:0/4016195010 wait complete. 2024-05-14T11:21:07.401 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-14T11:21:07.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:07.401 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-14T11:21:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.550+0000 7f0928200640 1 Processor -- start 2024-05-14T11:21:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.550+0000 7f0928200640 1 -- start start 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f09200b5f30 0x7f09200b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f092006cf90 0x7f092006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200cf280 0x7f09200cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f092015d0f0 con 0x7f092006cf90 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f09200b6880 con 0x7f09200b5f30 2024-05-14T11:21:07.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0928200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f092006d8e0 con 0x7f09200cf280 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f092006cf90 0x7f092006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f092006cf90 0x7f092006d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36914/0 (socket says 172.21.15.116:36914) 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 -- 172.21.15.116:0/1067765257 learned_addr learned my addr 172.21.15.116:0/1067765257 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0925800640 1 --2- 172.21.15.116:0/1067765257 >> v2:172.21.15.149:3300/0 conn(0x7f09200b5f30 0x7f09200b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0926200640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200cf280 0x7f09200cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 -- 172.21.15.116:0/1067765257 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200cf280 msgr2=0x7f09200cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200cf280 0x7f09200cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 -- 172.21.15.116:0/1067765257 >> v2:172.21.15.149:3300/0 conn(0x7f09200b5f30 msgr2=0x7f09200b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 --2- 172.21.15.116:0/1067765257 >> v2:172.21.15.149:3300/0 conn(0x7f09200b5f30 0x7f09200b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.554+0000 7f0924e00640 1 -- 172.21.15.116:0/1067765257 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092015c800 con 0x7f092006cf90 2024-05-14T11:21:07.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0924e00640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f092006d3a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f09200ceb40 tx=0x7f090c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc02ca1286395303 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f091fe00640 1 -- 172.21.15.116:0/1067765257 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c0b0070 con 0x7f092006cf90 2024-05-14T11:21:07.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f091fe00640 1 -- 172.21.15.116:0/1067765257 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c004230 con 0x7f092006cf90 2024-05-14T11:21:07.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f091fe00640 1 -- 172.21.15.116:0/1067765257 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c003620 con 0x7f092006cf90 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 -- 172.21.15.116:0/1067765257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 msgr2=0x7f092006d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f092006d3a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f09200ceb40 tx=0x7f090c01c520 comp rx=0 tx=0).stop 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 -- 172.21.15.116:0/1067765257 shutdown_connections 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f092006d3a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 --2- 172.21.15.116:0/1067765257 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200cf280 0x7f09200cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 --2- 172.21.15.116:0/1067765257 >> v2:172.21.15.149:3300/0 conn(0x7f09200b5f30 0x7f09200b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 -- 172.21.15.116:0/1067765257 >> 172.21.15.116:0/1067765257 conn(0x7f09200cde80 msgr2=0x7f09200ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 -- 172.21.15.116:0/1067765257 shutdown_connections 2024-05-14T11:21:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.558+0000 7f0928200640 1 -- 172.21.15.116:0/1067765257 wait complete. 2024-05-14T11:21:07.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 Processor -- start 2024-05-14T11:21:07.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- start start 2024-05-14T11:21:07.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200b5f30 0x7f09201fd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09200cf280 0x7f09201fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f092015d580 con 0x7f092006cf90 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09200cffe0 con 0x7f09200cf280 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09201fe3d0 con 0x7f09200b5f30 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36920/0 (socket says 172.21.15.116:36920) 2024-05-14T11:21:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 -- 172.21.15.116:0/3672342271 learned_addr learned my addr 172.21.15.116:0/3672342271 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0924e00640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200b5f30 0x7f09201fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0926200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09200cf280 0x7f09201fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 -- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200b5f30 msgr2=0x7f09201fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200b5f30 0x7f09201fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 -- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09200cf280 msgr2=0x7f09201fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09200cf280 0x7f09201fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 -- 172.21.15.116:0/3672342271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092015fe60 con 0x7f092006cf90 2024-05-14T11:21:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0925800640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f091400e8a0 tx=0x7f091400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f091d600640 1 -- 172.21.15.116:0/3672342271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914017070 con 0x7f092006cf90 2024-05-14T11:21:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f091d600640 1 -- 172.21.15.116:0/3672342271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091400c940 con 0x7f092006cf90 2024-05-14T11:21:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0920160140 con 0x7f092006cf90 2024-05-14T11:21:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f091d600640 1 -- 172.21.15.116:0/3672342271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091400ccf0 con 0x7f092006cf90 2024-05-14T11:21:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.562+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09201fe110 con 0x7f092006cf90 2024-05-14T11:21:07.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.566+0000 7f091d600640 1 -- 172.21.15.116:0/3672342271 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f091401a070 con 0x7f092006cf90 2024-05-14T11:21:07.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.566+0000 7f091d600640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 0x7f0900071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f091d600640 1 -- 172.21.15.116:0/3672342271 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f091408f5b0 con 0x7f092006cf90 2024-05-14T11:21:07.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 0x7f0900071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 Processor -- start 2024-05-14T11:21:07.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- start start 2024-05-14T11:21:07.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 0x7f0900071910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f090c0ad000 tx=0x7f090c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303792680 0x55b303d0bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b303ba9390 con 0x55b303ba4880 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b303b10f20 con 0x55b303bba490 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b303aca000 con 0x55b303792680 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36932/0 (socket says 172.21.15.116:36932) 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 -- 172.21.15.116:0/1564769478 learned_addr learned my addr 172.21.15.116:0/1564769478 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0926200640 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303792680 0x55b303d0bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0925800640 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 -- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303792680 msgr2=0x55b303d0bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303792680 0x55b303d0bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 -- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 msgr2=0x55b303d0b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d0b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 -- 172.21.15.116:0/1564769478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b303bbbc00 con 0x55b303ba4880 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0924e00640 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f090c01c560 tx=0x7f090c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69792f238050806 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0912000640 1 -- 172.21.15.116:0/1564769478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c0b0040 con 0x55b303ba4880 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0912000640 1 -- 172.21.15.116:0/1564769478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c003620 con 0x55b303ba4880 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f0912000640 1 -- 172.21.15.116:0/1564769478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c0039d0 con 0x55b303ba4880 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- 172.21.15.116:0/1564769478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 msgr2=0x55b303d0ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f090c01c560 tx=0x7f090c004740 comp rx=0 tx=0).stop 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- 172.21.15.116:0/1564769478 shutdown_connections 2024-05-14T11:21:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303792680 0x55b303d0bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303ba4880 0x55b303d0ba20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 --2- 172.21.15.116:0/1564769478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d0b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.570+0000 7f092f89b000 1 -- 172.21.15.116:0/1564769478 >> 172.21.15.116:0/1564769478 conn(0x55b3037d0b30 msgr2=0x55b303cee6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- 172.21.15.116:0/1564769478 shutdown_connections 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- 172.21.15.116:0/1564769478 wait complete. 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- start start 2024-05-14T11:21:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 Processor -- start 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 0x55b303d12310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d12850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b303a94bc0 con 0x55b303bba490 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b303aa68d0 con 0x55b303792680 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b303b1e320 con 0x55b303ba4880 2024-05-14T11:21:07.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f092f89b000 10 client.0 _ll_register_callbacks cb 0x55b303cace10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36936/0 (socket says 172.21.15.116:36936) 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 -- 172.21.15.116:0/2831087812 learned_addr learned my addr 172.21.15.116:0/2831087812 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0925800640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 0x55b303d12310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0924e00640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d12850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 -- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 msgr2=0x55b303d12850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d12850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 -- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 msgr2=0x55b303d12310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 0x55b303d12310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 -- 172.21.15.116:0/2831087812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b303bb7bb0 con 0x55b303bba490 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0925800640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 0x55b303d12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0926200640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0918014d90 tx=0x7f0918012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8f6c2ba5d027ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0906a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.574+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918003ba0 con 0x55b303bba490 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f090c003620 con 0x55b303bba490 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f0918003d40 con 0x55b303bba490 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0918027050 con 0x55b303bba490 2024-05-14T11:21:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 10 client.6573 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55b303b56030 con 0x55b303bba490 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c003620 con 0x55b303bba490 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f091802a020 con 0x55b303bba490 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 10 client.6573 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 10 client.6573 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 20 client.6573 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037b6bd0 0x55b303cce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 4 client.6573 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b303aa9ce0 con 0x55b3037b6bd0 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0925800640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037b6bd0 0x55b303cce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0925800640 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037b6bd0 0x55b303cce020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b7e667101200d2 server_cookie=ef2dccf99324b418 in_seq=0 out_seq=0 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 10 client.6573 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 1 -- 172.21.15.116:0/2831087812 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b303aa9ce0 con 0x55b3037b6bd0 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 10 client.6573 handle_command_reply: tid=0 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 client.6573 shutdown 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 msgr2=0x55b303ccd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0918014d90 tx=0x7f0918012670 comp rx=0 tx=0).stop 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 shutdown_connections 2024-05-14T11:21:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037b6bd0 0x55b303cce020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303bba490 0x55b303ccd2e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d12850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 --2- 172.21.15.116:0/2831087812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303792680 0x55b303d12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f0906a00640 0 client.6573 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 >> 172.21.15.116:0/2831087812 conn(0x55b3037d0b30 msgr2=0x55b303cc76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 shutdown_connections 2024-05-14T11:21:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 1 -- 172.21.15.116:0/2831087812 wait complete. 2024-05-14T11:21:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.578+0000 7f092f89b000 20 client.6573 trim_cache size 0 max 16384 2024-05-14T11:21:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 Processor -- start 2024-05-14T11:21:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- start start 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d22c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303cbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f092015eaa0 con 0x55b303792680 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f092015de00 con 0x55b303bba490 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f092015e450 con 0x55b303ba4880 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36948/0 (socket says 172.21.15.116:36948) 2024-05-14T11:21:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 -- 172.21.15.116:0/79528005 learned_addr learned my addr 172.21.15.116:0/79528005 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 -- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 msgr2=0x55b303d22c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d22c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 -- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 msgr2=0x55b303cbf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303cbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 -- 172.21.15.116:0/79528005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b303aaf980 con 0x55b303792680 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0925800640 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0914058550 tx=0x7f09140583e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e795f74baee8de02 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0911600640 1 -- 172.21.15.116:0/79528005 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914021040 con 0x55b303792680 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0911600640 1 -- 172.21.15.116:0/79528005 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091408b070 con 0x55b303792680 2024-05-14T11:21:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f0911600640 1 -- 172.21.15.116:0/79528005 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091405b8a0 con 0x55b303792680 2024-05-14T11:21:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- 172.21.15.116:0/79528005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 msgr2=0x55b303d226c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0914058550 tx=0x7f09140583e0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- 172.21.15.116:0/79528005 shutdown_connections 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303cbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d22c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 --2- 172.21.15.116:0/79528005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303d226c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- 172.21.15.116:0/79528005 >> 172.21.15.116:0/79528005 conn(0x55b3037d0b30 msgr2=0x55b303d18b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- 172.21.15.116:0/79528005 shutdown_connections 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.598+0000 7f092f89b000 1 -- 172.21.15.116:0/79528005 wait complete. 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- start start 2024-05-14T11:21:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 Processor -- start 2024-05-14T11:21:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d3b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d3b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b303abb4d0 con 0x55b303792680 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b303bc8a30 con 0x55b303bba490 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b303ad2330 con 0x55b303ba4880 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 10 client.0 _ll_register_callbacks cb 0x7f0918009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36960/0 (socket says 172.21.15.116:36960) 2024-05-14T11:21:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 -- 172.21.15.116:0/977570446 learned_addr learned my addr 172.21.15.116:0/977570446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 -- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 msgr2=0x55b303d3b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0924e00640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d3b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0926200640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d3b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d3b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 -- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 msgr2=0x55b303d3b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d3b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 -- 172.21.15.116:0/977570446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b303ac37f0 con 0x55b303792680 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0924e00640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d3b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0926200640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d3b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0925800640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f091400c520 tx=0x7f0914000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5f663944e5c789 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0906a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914081060 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f0914086d30 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091408b070 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f091407b070 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 10 client.6579 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55b303b8cf20 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.602+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091408b070 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f091405aff0 con 0x55b303792680 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 10 client.6579 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 10 client.6579 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 20 client.6579 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037a4c30 0x55b303d7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 4 client.6579 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55b303b1eb30 con 0x55b3037a4c30 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0924e00640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037a4c30 0x55b303d7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0906a00640 10 client.6579 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0924e00640 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037a4c30 0x55b303d7ec90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ddb5d75c25e8fa server_cookie=2b089dd3d941cab9 in_seq=0 out_seq=0 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0906a00640 1 -- 172.21.15.116:0/977570446 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b303b1eb30 con 0x55b3037a4c30 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0906a00640 10 client.6579 handle_command_reply: tid=0 2024-05-14T11:21:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 client.6579 shutdown 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 msgr2=0x55b303cc2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f091400c520 tx=0x7f0914000bd0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 shutdown_connections 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55b3037a4c30 0x55b303d7ec90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b303bba490 0x55b303d3b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b303ba4880 0x55b303d3b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 --2- 172.21.15.116:0/977570446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b303792680 0x55b303cc2c50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0906a00640 0 client.6579 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 >> 172.21.15.116:0/977570446 conn(0x55b3037d0b30 msgr2=0x55b303d216b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 shutdown_connections 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 1 -- 172.21.15.116:0/977570446 wait complete. 2024-05-14T11:21:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f092f89b000 20 client.6579 trim_cache size 0 max 16384 2024-05-14T11:21:07.613 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:07.613 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:21:07.613 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "c07453b5-3b2e-40c3-8ee0-2f5087b2df4c", 2024-05-14T11:21:07.613 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:21:07.613 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:07.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 msgr2=0x7f0900071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 0x7f0900071910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f090c0ad000 tx=0x7f090c0a6000 comp rx=0 tx=0).stop 2024-05-14T11:21:07.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 msgr2=0x7f0920154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f091400e8a0 tx=0x7f091400eca0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 shutdown_connections 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f090006f400 0x7f0900071910 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09200cf280 0x7f09201fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09200b5f30 0x7f09201fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 --2- 172.21.15.116:0/3672342271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092006cf90 0x7f0920154730 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 >> 172.21.15.116:0/3672342271 conn(0x7f09200cde80 msgr2=0x7f0920165c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 shutdown_connections 2024-05-14T11:21:07.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.606+0000 7f0928200640 1 -- 172.21.15.116:0/3672342271 wait complete. 2024-05-14T11:21:07.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:07.626 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-14T11:21:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.774+0000 7fd52c800640 1 Processor -- start 2024-05-14T11:21:07.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.774+0000 7fd52c800640 1 -- start start 2024-05-14T11:21:07.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 0x7fd5240d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52c800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd5240af380 0x7fd5240af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52c800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5240d1fc0 con 0x7fd524156fa0 2024-05-14T11:21:07.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52c800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd5240706f0 con 0x7fd5240af380 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5241573b0 con 0x7fd5240d1670 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd524156fa0 0x7fd52406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45550/0 (socket says 172.21.15.116:45550) 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529e00640 1 -- 172.21.15.116:0/372625777 learned_addr learned my addr 172.21.15.116:0/372625777 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529400640 1 --2- 172.21.15.116:0/372625777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 0x7fd5240d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd52a800640 1 --2- 172.21.15.116:0/372625777 >> v2:172.21.15.149:3300/0 conn(0x7fd5240af380 0x7fd5240af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd528a00640 1 -- 172.21.15.116:0/372625777 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832090822 0 0) 0x7fd518001120 con 0x7fd524156fa0 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd528a00640 1 -- 172.21.15.116:0/372625777 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd50c003530 con 0x7fd524156fa0 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529400640 1 -- 172.21.15.116:0/372625777 >> v2:172.21.15.149:3300/0 conn(0x7fd5240af380 msgr2=0x7fd5240af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529400640 1 --2- 172.21.15.116:0/372625777 >> v2:172.21.15.149:3300/0 conn(0x7fd5240af380 0x7fd5240af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529400640 1 -- 172.21.15.116:0/372625777 >> v1:172.21.15.116:6789/0 conn(0x7fd524156fa0 legacy=0x7fd52406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.778+0000 7fd529400640 1 -- 172.21.15.116:0/372625777 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd524160510 con 0x7fd5240d1670 2024-05-14T11:21:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd529400640 1 --2- 172.21.15.116:0/372625777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 0x7fd5240d1a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5140097c0 tx=0x7fd51401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a70846c384b303b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd528a00640 1 -- 172.21.15.116:0/372625777 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140b2070 con 0x7fd5240d1670 2024-05-14T11:21:07.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd528a00640 1 -- 172.21.15.116:0/372625777 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5140ad640 con 0x7fd5240d1670 2024-05-14T11:21:07.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd528a00640 1 -- 172.21.15.116:0/372625777 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140ada10 con 0x7fd5240d1670 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 -- 172.21.15.116:0/372625777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 msgr2=0x7fd5240d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 --2- 172.21.15.116:0/372625777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 0x7fd5240d1a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5140097c0 tx=0x7fd51401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 -- 172.21.15.116:0/372625777 shutdown_connections 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 --2- 172.21.15.116:0/372625777 >> v2:172.21.15.149:3300/0 conn(0x7fd5240af380 0x7fd5240af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 --2- 172.21.15.116:0/372625777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240d1670 0x7fd5240d1a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 -- 172.21.15.116:0/372625777 >> 172.21.15.116:0/372625777 conn(0x7fd5240b4620 msgr2=0x7fd5240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 -- 172.21.15.116:0/372625777 shutdown_connections 2024-05-14T11:21:07.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 -- 172.21.15.116:0/372625777 wait complete. 2024-05-14T11:21:07.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.782+0000 7fd52c800640 1 Processor -- start 2024-05-14T11:21:07.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- start start 2024-05-14T11:21:07.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240af380 0x7fd524151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5240d1670 0x7fd52415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd5240b0660 con 0x7fd524156fa0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5240b04e0 con 0x7fd5240d1670 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd524169810 con 0x7fd5240af380 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36964/0 (socket says 172.21.15.116:36964) 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 -- 172.21.15.116:0/1143866357 learned_addr learned my addr 172.21.15.116:0/1143866357 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd529e00640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240af380 0x7fd524151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd529400640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5240d1670 0x7fd52415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 -- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240af380 msgr2=0x7fd524151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240af380 0x7fd524151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 -- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5240d1670 msgr2=0x7fd52415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5240d1670 0x7fd52415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 -- 172.21.15.116:0/1143866357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd524152000 con 0x7fd524156fa0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52a800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd520009cb0 tx=0x7fd52000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd51e000640 1 -- 172.21.15.116:0/1143866357 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520018020 con 0x7fd524156fa0 2024-05-14T11:21:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd51e000640 1 -- 172.21.15.116:0/1143866357 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd520014480 con 0x7fd524156fa0 2024-05-14T11:21:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd524152d10 con 0x7fd524156fa0 2024-05-14T11:21:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd51e000640 1 -- 172.21.15.116:0/1143866357 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520014870 con 0x7fd524156fa0 2024-05-14T11:21:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.786+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd524152750 con 0x7fd524156fa0 2024-05-14T11:21:07.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd51e000640 1 -- 172.21.15.116:0/1143866357 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd52001b020 con 0x7fd524156fa0 2024-05-14T11:21:07.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd51e000640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 0x7fd508071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd51e000640 1 -- 172.21.15.116:0/1143866357 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fd5200901e0 con 0x7fd524156fa0 2024-05-14T11:21:07.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd529e00640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 0x7fd508071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd533f01000 1 Processor -- start 2024-05-14T11:21:07.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd533f01000 1 -- start start 2024-05-14T11:21:07.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.790+0000 7fd529e00640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 0x7fd508071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd518005890 tx=0x7fd518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080af8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080af8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5600808fd100 con 0x56008097ef00 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600808b5ca0 con 0x56008057e3e0 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5600808f90c0 con 0x560080991120 2024-05-14T11:21:07.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36972/0 (socket says 172.21.15.116:36972) 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 -- 172.21.15.116:0/2717007400 learned_addr learned my addr 172.21.15.116:0/2717007400 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080af8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd52a800640 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080af8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 -- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 msgr2=0x560080af8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080af8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 -- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 msgr2=0x560080af8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080af8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 -- 172.21.15.116:0/2717007400 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600808ba800 con 0x56008097ef00 2024-05-14T11:21:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd529400640 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd51401a0d0 tx=0x7fd5140a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deef4c9ee564f820 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd512600640 1 -- 172.21.15.116:0/2717007400 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140b2040 con 0x56008097ef00 2024-05-14T11:21:07.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd512600640 1 -- 172.21.15.116:0/2717007400 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5140ad440 con 0x56008097ef00 2024-05-14T11:21:07.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd512600640 1 -- 172.21.15.116:0/2717007400 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140ad7f0 con 0x56008097ef00 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- 172.21.15.116:0/2717007400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 msgr2=0x560080af8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd51401a0d0 tx=0x7fd5140a7070 comp rx=0 tx=0).stop 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- 172.21.15.116:0/2717007400 shutdown_connections 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080af8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080af8890 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 --2- 172.21.15.116:0/2717007400 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080af8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- 172.21.15.116:0/2717007400 >> 172.21.15.116:0/2717007400 conn(0x5600805be400 msgr2=0x560080adb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- 172.21.15.116:0/2717007400 shutdown_connections 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- 172.21.15.116:0/2717007400 wait complete. 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.794+0000 7fd533f01000 1 -- start start 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 Processor -- start 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080afedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080ab1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5600809c82a0 con 0x56008097ef00 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600809b8f40 con 0x56008057e3e0 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5600808b9080 con 0x560080991120 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 10 client.0 _ll_register_callbacks cb 0x560080a99c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36978/0 (socket says 172.21.15.116:36978) 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 -- 172.21.15.116:0/654585254 learned_addr learned my addr 172.21.15.116:0/654585254 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd52a800640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080ab1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529e00640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080afedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 -- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 msgr2=0x560080ab1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080ab1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 -- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 msgr2=0x560080afedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080afedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 -- 172.21.15.116:0/654585254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600808a7860 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529400640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd51401a0d0 tx=0x7fd51401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a6a6de9390d32c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140c6060 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5140ad440 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fd5140069d0 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd5140cc050 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 10 client.6591 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5600808b5ef0 con 0x56008097ef00 2024-05-14T11:21:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5140ad440 con 0x56008097ef00 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fd5140cf020 con 0x56008097ef00 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 10 client.6591 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 10 client.6591 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 20 client.6591 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x5600805a2930 0x560080ab2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 4 client.6591 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560080895df0 con 0x5600805a2930 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.798+0000 7fd529e00640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x5600805a2930 0x560080ab2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd529e00640 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x5600805a2930 0x560080ab2cd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def4460f7279a42a server_cookie=a3e5b3fceb82f1cb in_seq=0 out_seq=0 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd506a00640 10 client.6591 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd506a00640 1 -- 172.21.15.116:0/654585254 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560080895df0 con 0x5600805a2930 2024-05-14T11:21:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd506a00640 10 client.6591 handle_command_reply: tid=0 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 client.6591 shutdown 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 msgr2=0x560080aff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd51401a0d0 tx=0x7fd51401ace0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 shutdown_connections 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x5600805a2930 0x560080ab2cd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080ab1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008097ef00 0x560080aff310 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd506a00640 0 client.6591 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 --2- 172.21.15.116:0/654585254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008057e3e0 0x560080afedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 >> 172.21.15.116:0/654585254 conn(0x5600805be400 msgr2=0x560080adb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 shutdown_connections 2024-05-14T11:21:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 1 -- 172.21.15.116:0/654585254 wait complete. 2024-05-14T11:21:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.802+0000 7fd533f01000 20 client.6591 trim_cache size 0 max 16384 2024-05-14T11:21:07.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.818+0000 7fd533f01000 1 Processor -- start 2024-05-14T11:21:07.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.818+0000 7fd533f01000 1 -- start start 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b67180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd524205220 con 0x56008057e3e0 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd52414f690 con 0x56008097ef00 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd524204bd0 con 0x560080991120 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36988/0 (socket says 172.21.15.116:36988) 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 -- 172.21.15.116:0/3191099927 learned_addr learned my addr 172.21.15.116:0/3191099927 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529400640 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd52a800640 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 -- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 msgr2=0x560080b67180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b67180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 -- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 msgr2=0x560080b66c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b66c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 -- 172.21.15.116:0/3191099927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560080ab0070 con 0x56008057e3e0 2024-05-14T11:21:07.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd529e00640 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd51800fc80 tx=0x7fd51800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3785d2598753569f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd511c00640 1 -- 172.21.15.116:0/3191099927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518030070 con 0x56008057e3e0 2024-05-14T11:21:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd511c00640 1 -- 172.21.15.116:0/3191099927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51801b040 con 0x56008057e3e0 2024-05-14T11:21:07.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd511c00640 1 -- 172.21.15.116:0/3191099927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51802a2f0 con 0x56008057e3e0 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- 172.21.15.116:0/3191099927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 msgr2=0x560080b66700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd51800fc80 tx=0x7fd51800a000 comp rx=0 tx=0).stop 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- 172.21.15.116:0/3191099927 shutdown_connections 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b67180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 --2- 172.21.15.116:0/3191099927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b66700 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- 172.21.15.116:0/3191099927 >> 172.21.15.116:0/3191099927 conn(0x5600805be400 msgr2=0x560080ada3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- 172.21.15.116:0/3191099927 shutdown_connections 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- 172.21.15.116:0/3191099927 wait complete. 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 -- start start 2024-05-14T11:21:07.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.822+0000 7fd533f01000 1 Processor -- start 2024-05-14T11:21:07.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b6b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560080901730 con 0x56008057e3e0 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600808fc410 con 0x56008097ef00 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56008090a500 con 0x560080991120 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 10 client.0 _ll_register_callbacks cb 0x7fd520009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37000/0 (socket says 172.21.15.116:37000) 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 -- 172.21.15.116:0/2332795370 learned_addr learned my addr 172.21.15.116:0/2332795370 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529400640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b6b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd52a800640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 -- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 msgr2=0x560080b02c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b02c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 -- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 msgr2=0x560080b6b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b6b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 -- 172.21.15.116:0/2332795370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008097e340 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd52a800640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b02c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529e00640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5600808f5560 tx=0x7fd518007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f4a6e2ca1f2143 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518030030 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fd51802a2f0 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51801b040 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd51803c050 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 10 client.6597 fetch_fsmap learned FSMap version 52 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5600808998e0 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51801b040 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fd51803f020 con 0x56008057e3e0 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 10 client.6597 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T11:21:07.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 10 client.6597 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 20 client.6597 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560080590990 0x560080b03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 4 client.6597 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56008090ad10 con 0x560080590990 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529400640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560080590990 0x560080b03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd506a00640 10 client.6597 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.826+0000 7fd529400640 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560080590990 0x560080b03580 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468701a8f8e144cb server_cookie=dad8e120bde86783 in_seq=0 out_seq=0 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd506a00640 1 -- 172.21.15.116:0/2332795370 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56008090ad10 con 0x560080590990 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd506a00640 10 client.6597 handle_command_reply: tid=0 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 client.6597 shutdown 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 msgr2=0x560080b6ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5600808f5560 tx=0x7fd518007b30 comp rx=0 tx=0).stop 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 shutdown_connections 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x560080590990 0x560080b03580 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56008097ef00 0x560080b6b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560080991120 0x560080b02c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd506a00640 0 client.6597 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 --2- 172.21.15.116:0/2332795370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56008057e3e0 0x560080b6ad40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 >> 172.21.15.116:0/2332795370 conn(0x5600805be400 msgr2=0x560080ada100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 shutdown_connections 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 1 -- 172.21.15.116:0/2332795370 wait complete. 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd533f01000 20 client.6597 trim_cache size 0 max 16384 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:21:07.832 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: "c07453b5-3b2e-40c3-8ee0-2f5087b2df4c": { 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/subdir", 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "c07453b5-3b2e-40c3-8ee0-2f5087b2df4c", 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,repair" 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:07.833 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 msgr2=0x7fd508071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 0x7fd508071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd518005890 tx=0x7fd518009040 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 msgr2=0x7fd52415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd520009cb0 tx=0x7fd52000c490 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 shutdown_connections 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd50806f400 0x7fd508071910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5240d1670 0x7fd52415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd524156fa0 0x7fd52415c030 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 --2- 172.21.15.116:0/1143866357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5240af380 0x7fd524151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 >> 172.21.15.116:0/1143866357 conn(0x7fd5240b4620 msgr2=0x7fd524150360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 shutdown_connections 2024-05-14T11:21:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:07.830+0000 7fd52c800640 1 -- 172.21.15.116:0/1143866357 wait complete. 2024-05-14T11:21:07.847 INFO:tasks.cephfs.filesystem:scrub status for tag:c07453b5-3b2e-40c3-8ee0-2f5087b2df4c - {'path': '/subdir', 'tag': 'c07453b5-3b2e-40c3-8ee0-2f5087b2df4c', 'options': 'recursive,repair'} 2024-05-14T11:21:12.140 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:21:12.143 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:21:37.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:37.849 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-14T11:21:38.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 Processor -- start 2024-05-14T11:21:38.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 -- start start 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c800b0390 0x7f8c80167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c80160510 con 0x7f8c80156fa0 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c801607f0 con 0x7f8c800d1670 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c88200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8c800706f0 con 0x7f8c800b0390 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c85800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8c80156fa0 0x7f8c8006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47714/0 (socket says 172.21.15.116:47714) 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c85800640 1 -- 172.21.15.116:0/3057391410 learned_addr learned my addr 172.21.15.116:0/3057391410 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:38.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.010+0000 7f8c86200640 1 --2- 172.21.15.116:0/3057391410 >> v2:172.21.15.116:3300/0 conn(0x7f8c800b0390 0x7f8c80167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c7fe00640 1 -- 172.21.15.116:0/3057391410 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191163714 0 0) 0x7f8c80160510 con 0x7f8c80156fa0 2024-05-14T11:21:38.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c7fe00640 1 -- 172.21.15.116:0/3057391410 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c68005740 con 0x7f8c80156fa0 2024-05-14T11:21:38.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c86200640 1 -- 172.21.15.116:0/3057391410 >> v1:172.21.15.149:6789/0 conn(0x7f8c800d1670 legacy=0x7f8c800d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c86200640 1 -- 172.21.15.116:0/3057391410 >> v1:172.21.15.116:6789/0 conn(0x7f8c80156fa0 legacy=0x7f8c8006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c86200640 1 -- 172.21.15.116:0/3057391410 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c80160ad0 con 0x7f8c800b0390 2024-05-14T11:21:38.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c86200640 1 --2- 172.21.15.116:0/3057391410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80167800 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d90 tx=0x7f8c7801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93c5761ec2b3386 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c7fe00640 1 -- 172.21.15.116:0/3057391410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780b2070 con 0x7f8c800b0390 2024-05-14T11:21:38.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c7fe00640 1 -- 172.21.15.116:0/3057391410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c780a5730 con 0x7f8c800b0390 2024-05-14T11:21:38.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.014+0000 7f8c7fe00640 1 -- 172.21.15.116:0/3057391410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780a5ae0 con 0x7f8c800b0390 2024-05-14T11:21:38.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- 172.21.15.116:0/3057391410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 msgr2=0x7f8c80167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 --2- 172.21.15.116:0/3057391410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80167800 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d90 tx=0x7f8c7801c970 comp rx=0 tx=0).stop 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- 172.21.15.116:0/3057391410 shutdown_connections 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 --2- 172.21.15.116:0/3057391410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80167800 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- 172.21.15.116:0/3057391410 >> 172.21.15.116:0/3057391410 conn(0x7f8c800b4620 msgr2=0x7f8c800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- 172.21.15.116:0/3057391410 shutdown_connections 2024-05-14T11:21:38.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- 172.21.15.116:0/3057391410 wait complete. 2024-05-14T11:21:38.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 Processor -- start 2024-05-14T11:21:38.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.018+0000 7f8c88200640 1 -- start start 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c800d1670 0x7f8c8014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 0x7f8c801536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006d2a0 con 0x7f8c800b0390 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006d120 con 0x7f8c800d1670 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8006d420 con 0x7f8c80156fa0 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35340/0 (socket says 172.21.15.116:35340) 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 -- 172.21.15.116:0/300947677 learned_addr learned my addr 172.21.15.116:0/300947677 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c84e00640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c800d1670 0x7f8c8014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c86200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 0x7f8c801536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 -- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 msgr2=0x7f8c801536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 0x7f8c801536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 -- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c800d1670 msgr2=0x7f8c8014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c800d1670 0x7f8c8014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 -- 172.21.15.116:0/300947677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800d1fd0 con 0x7f8c800b0390 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c86200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 0x7f8c801536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c85800640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c430 tx=0x7f8c7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c7d600640 1 -- 172.21.15.116:0/300947677 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74017070 con 0x7f8c800b0390 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c7d600640 1 -- 172.21.15.116:0/300947677 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74005cf0 con 0x7f8c800b0390 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c801523c0 con 0x7f8c800b0390 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c7d600640 1 -- 172.21.15.116:0/300947677 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740072f0 con 0x7f8c800b0390 2024-05-14T11:21:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c80152bf0 con 0x7f8c800b0390 2024-05-14T11:21:38.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.022+0000 7f8c7d600640 1 -- 172.21.15.116:0/300947677 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c7401a070 con 0x7f8c800b0390 2024-05-14T11:21:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c7d600640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 0x7f8c60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c7d600640 1 -- 172.21.15.116:0/300947677 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8c7408e750 con 0x7f8c800b0390 2024-05-14T11:21:38.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c84e00640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 0x7f8c60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 Processor -- start 2024-05-14T11:21:38.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 -- start start 2024-05-14T11:21:38.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c84e00640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 0x7f8c60071940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c801566c0 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d43c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d43cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d241100 con 0x55ce2d2c2f00 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d1f9ca0 con 0x55ce2cec23e0 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce2d23d0c0 con 0x55ce2d2d5120 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35352/0 (socket says 172.21.15.116:35352) 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.026+0000 7f8c84e00640 1 -- 172.21.15.116:0/3651203175 learned_addr learned my addr 172.21.15.116:0/3651203175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c85800640 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d43c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c86200640 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d43cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 -- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 msgr2=0x55ce2d43c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d43c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 -- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 msgr2=0x55ce2d43cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d43cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 -- 172.21.15.116:0/3651203175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce2d1fe800 con 0x55ce2d2c2f00 2024-05-14T11:21:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00f960 tx=0x7f8c6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc60b0128e465e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c72000640 1 -- 172.21.15.116:0/3651203175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c025070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c72000640 1 -- 172.21.15.116:0/3651203175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01b050 con 0x55ce2d2c2f00 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c72000640 1 -- 172.21.15.116:0/3651203175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c014e20 con 0x55ce2d2c2f00 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3651203175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 msgr2=0x55ce2d43c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00f960 tx=0x7f8c6c012000 comp rx=0 tx=0).stop 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3651203175 shutdown_connections 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d43cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d43c890 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3651203175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d43c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3651203175 >> 172.21.15.116:0/3651203175 conn(0x55ce2cf02400 msgr2=0x55ce2d41f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3651203175 shutdown_connections 2024-05-14T11:21:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3651203175 wait complete. 2024-05-14T11:21:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 -- start start 2024-05-14T11:21:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.030+0000 7f8c8f8e0000 1 Processor -- start 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d442dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d3f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d30c2a0 con 0x55ce2d2c2f00 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d2fcf40 con 0x55ce2cec23e0 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce2d1fd080 con 0x55ce2d2d5120 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 10 client.0 _ll_register_callbacks cb 0x55ce2d3ddc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35366/0 (socket says 172.21.15.116:35366) 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 -- 172.21.15.116:0/4105497737 learned_addr learned my addr 172.21.15.116:0/4105497737 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c85800640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d442dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c86200640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d3f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 -- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 msgr2=0x55ce2d3f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d3f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 -- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 msgr2=0x55ce2d442dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d442dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 -- 172.21.15.116:0/4105497737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce2d1eb860 con 0x55ce2d2c2f00 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c86200640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d3f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c84e00640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c023040 tx=0x7f8c6c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492c5e72db31296 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c030060 con 0x55ce2d2c2f00 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6c01b050 con 0x55ce2d2c2f00 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8c6c020a20 con 0x55ce2d2c2f00 2024-05-14T11:21:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c036050 con 0x55ce2d2c2f00 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 10 client.6612 fetch_fsmap learned FSMap version 53 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55ce2d1f9ef0 con 0x55ce2d2c2f00 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01b050 con 0x55ce2d2c2f00 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8c6c020c20 con 0x55ce2d2c2f00 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 10 client.6612 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 10 client.6612 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 20 client.6612 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2cee6930 0x55ce2d3f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 4 client.6612 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce2d1d9df0 con 0x55ce2cee6930 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c85800640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2cee6930 0x55ce2d3f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c85800640 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2cee6930 0x55ce2d3f6d90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40437bc49bd89cd1 server_cookie=90a313a74234c4d3 in_seq=0 out_seq=0 2024-05-14T11:21:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 10 client.6612 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 1 -- 172.21.15.116:0/4105497737 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce2d1d9df0 con 0x55ce2cee6930 2024-05-14T11:21:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c66a00640 10 client.6612 handle_command_reply: tid=0 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.034+0000 7f8c8f8e0000 1 client.6612 shutdown 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 msgr2=0x55ce2d443310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c023040 tx=0x7f8c6c000ee0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 shutdown_connections 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2cee6930 0x55ce2d3f6d90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d3f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d443310 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/4105497737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d442dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c66a00640 0 client.6612 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 >> 172.21.15.116:0/4105497737 conn(0x55ce2cf02400 msgr2=0x55ce2d41f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 shutdown_connections 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/4105497737 wait complete. 2024-05-14T11:21:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.038+0000 7f8c8f8e0000 20 client.6612 trim_cache size 0 max 16384 2024-05-14T11:21:38.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 Processor -- start 2024-05-14T11:21:38.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 -- start start 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d4aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d4ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80204f90 con 0x55ce2d2c2f00 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801507f0 con 0x55ce2cec23e0 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.054+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8c80204a10 con 0x55ce2d2d5120 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35372/0 (socket says 172.21.15.116:35372) 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 -- 172.21.15.116:0/3652734518 learned_addr learned my addr 172.21.15.116:0/3652734518 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c85800640 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d4aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c86200640 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d4ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 -- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 msgr2=0x55ce2d4ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d4ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 -- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 msgr2=0x55ce2d4aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d4aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 -- 172.21.15.116:0/3652734518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce2d3f4070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c84e00640 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02b450 tx=0x7f8c6c029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa43dec3f2b4f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c71600640 1 -- 172.21.15.116:0/3652734518 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c03f070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c71600640 1 -- 172.21.15.116:0/3652734518 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01b070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c71600640 1 -- 172.21.15.116:0/3652734518 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c025040 con 0x55ce2d2c2f00 2024-05-14T11:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3652734518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 msgr2=0x55ce2d4ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02b450 tx=0x7f8c6c029000 comp rx=0 tx=0).stop 2024-05-14T11:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3652734518 shutdown_connections 2024-05-14T11:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2d2d5120 0x55ce2d4ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d4ab270 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/3652734518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2cec23e0 0x55ce2d4aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3652734518 >> 172.21.15.116:0/3652734518 conn(0x55ce2cf02400 msgr2=0x55ce2d3f4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3652734518 shutdown_connections 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/3652734518 wait complete. 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 -- start start 2024-05-14T11:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.058+0000 7f8c8f8e0000 1 Processor -- start 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 0x55ce2d447900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 0x55ce2d46be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d245730 con 0x55ce2d2c2f00 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce2d240410 con 0x55ce2d2d5120 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce2d24e500 con 0x55ce2cec23e0 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 10 client.0 _ll_register_callbacks cb 0x7f8c6c023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35384/0 (socket says 172.21.15.116:35384) 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 -- 172.21.15.116:0/2264845475 learned_addr learned my addr 172.21.15.116:0/2264845475 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c85800640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 0x55ce2d447900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c86200640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 0x55ce2d46be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 -- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 msgr2=0x55ce2d447900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 0x55ce2d447900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 -- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 msgr2=0x55ce2d46be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 0x55ce2d46be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 -- 172.21.15.116:0/2264845475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce2d2c2340 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c86200640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 0x55ce2d46be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c85800640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 0x55ce2d447900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c84e00640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00f5a0 tx=0x7f8c6c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7d437e21c04f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c030070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8c6c02b840 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6c01b070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c049050 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 10 client.6618 fetch_fsmap learned FSMap version 53 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55ce2d1dd8e0 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c01b070 con 0x55ce2d2c2f00 2024-05-14T11:21:38.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8c6c04c020 con 0x55ce2d2c2f00 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 10 client.6618 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 10 client.6618 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 20 client.6618 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2ced4990 0x55ce2d46c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 4 client.6618 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce2d24ed10 con 0x55ce2ced4990 2024-05-14T11:21:38.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c85800640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2ced4990 0x55ce2d46c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 10 client.6618 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c85800640 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2ced4990 0x55ce2d46c350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d8f0221d9a3cb2 server_cookie=47f96b27efefc1fd in_seq=0 out_seq=0 2024-05-14T11:21:38.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 1 -- 172.21.15.116:0/2264845475 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce2d24ed10 con 0x55ce2ced4990 2024-05-14T11:21:38.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 10 client.6618 handle_command_reply: tid=0 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 client.6618 shutdown 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 msgr2=0x55ce2d46b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00f5a0 tx=0x7f8c6c00a000 comp rx=0 tx=0).stop 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 shutdown_connections 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55ce2ced4990 0x55ce2d46c350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce2d2d5120 0x55ce2d46be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce2d2c2f00 0x55ce2d46b8d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c66a00640 0 client.6618 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 --2- 172.21.15.116:0/2264845475 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce2cec23e0 0x55ce2d447900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 >> 172.21.15.116:0/2264845475 conn(0x55ce2cf02400 msgr2=0x55ce2d452990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 shutdown_connections 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.062+0000 7f8c8f8e0000 1 -- 172.21.15.116:0/2264845475 wait complete. 2024-05-14T11:21:38.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c8f8e0000 20 client.6618 trim_cache size 0 max 16384 2024-05-14T11:21:38.067 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:38.067 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:21:38.067 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:21:38.067 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 msgr2=0x7f8c60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 0x7f8c60071940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c801566c0 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 msgr2=0x7f8c80151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c430 tx=0x7f8c7400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 shutdown_connections 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6006f430 0x7f8c60071940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c800d1670 0x7f8c8014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c80156fa0 0x7f8c801536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 --2- 172.21.15.116:0/300947677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c800b0390 0x7f8c80151a90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 >> 172.21.15.116:0/300947677 conn(0x7f8c800b4620 msgr2=0x7f8c800b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 shutdown_connections 2024-05-14T11:21:38.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.066+0000 7f8c88200640 1 -- 172.21.15.116:0/300947677 wait complete. 2024-05-14T11:21:38.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:21:38.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:38.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 Processor -- start 2024-05-14T11:21:38.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 -- start start 2024-05-14T11:21:38.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb7d415eb60 0x7fb7d40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7d41432c0 0x7fb7d41436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d40cdac0 0x7fb7d40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d4140f90 con 0x7fb7d41432c0 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d40cf5f0 con 0x7fb7d415eb60 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7dae00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d406cd60 con 0x7fb7d40cdac0 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7d41432c0 0x7fb7d41436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7d41432c0 0x7fb7d41436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35386/0 (socket says 172.21.15.116:35386) 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 -- 172.21.15.116:0/312621855 learned_addr learned my addr 172.21.15.116:0/312621855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7d3e00640 1 --2- 172.21.15.116:0/312621855 >> v2:172.21.15.149:3300/0 conn(0x7fb7d415eb60 0x7fb7d40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7d8e00640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d40cdac0 0x7fb7d40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 -- 172.21.15.116:0/312621855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d40cdac0 msgr2=0x7fb7d40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d40cdac0 0x7fb7d40cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 -- 172.21.15.116:0/312621855 >> v2:172.21.15.149:3300/0 conn(0x7fb7d415eb60 msgr2=0x7fb7d40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 --2- 172.21.15.116:0/312621855 >> v2:172.21.15.149:3300/0 conn(0x7fb7d415eb60 0x7fb7d40b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.230+0000 7fb7cb400640 1 -- 172.21.15.116:0/312621855 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d4143c10 con 0x7fb7d41432c0 2024-05-14T11:21:38.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7cb400640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d41436d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4072a00 tx=0x7fb7bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9f0d951ff03b5c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7d3400640 1 -- 172.21.15.116:0/312621855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b0070 con 0x7fb7d41432c0 2024-05-14T11:21:38.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7d3400640 1 -- 172.21.15.116:0/312621855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0ab600 con 0x7fb7d41432c0 2024-05-14T11:21:38.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7d3400640 1 -- 172.21.15.116:0/312621855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0ab9d0 con 0x7fb7d41432c0 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 -- 172.21.15.116:0/312621855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 msgr2=0x7fb7d41436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d41436d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4072a00 tx=0x7fb7bc01c490 comp rx=0 tx=0).stop 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 -- 172.21.15.116:0/312621855 shutdown_connections 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d41436d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 --2- 172.21.15.116:0/312621855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d40cdac0 0x7fb7d40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 --2- 172.21.15.116:0/312621855 >> v2:172.21.15.149:3300/0 conn(0x7fb7d415eb60 0x7fb7d40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 -- 172.21.15.116:0/312621855 >> 172.21.15.116:0/312621855 conn(0x7fb7d40ce6b0 msgr2=0x7fb7d40ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 -- 172.21.15.116:0/312621855 shutdown_connections 2024-05-14T11:21:38.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.234+0000 7fb7dae00640 1 -- 172.21.15.116:0/312621855 wait complete. 2024-05-14T11:21:38.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 Processor -- start 2024-05-14T11:21:38.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 -- start start 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d40cdac0 0x7fb7d4139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d415eb60 0x7fb7d40d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d415ef70 con 0x7fb7d41432c0 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d40ce410 con 0x7fb7d40cdac0 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7dae00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d40b0bd0 con 0x7fb7d415eb60 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35388/0 (socket says 172.21.15.116:35388) 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7cb400640 1 -- 172.21.15.116:0/1539485157 learned_addr learned my addr 172.21.15.116:0/1539485157 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7d3e00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d40cdac0 0x7fb7d4139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7d8e00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d415eb60 0x7fb7d40d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7cb400640 1 -- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d415eb60 msgr2=0x7fb7d40d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.238+0000 7fb7cb400640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d415eb60 0x7fb7d40d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7cb400640 1 -- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d40cdac0 msgr2=0x7fb7d4139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7cb400640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d40cdac0 0x7fb7d4139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7cb400640 1 -- 172.21.15.116:0/1539485157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d413a400 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7cb400640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc01cd30 tx=0x7fb7bc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b0040 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0b4620 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b4a10 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d4141380 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d4141940 con 0x7fb7d41432c0 2024-05-14T11:21:38.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d406a9c0 con 0x7fb7d41432c0 2024-05-14T11:21:38.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7bc0ab4c0 con 0x7fb7d41432c0 2024-05-14T11:21:38.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 0x7fb7b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.242+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fb7bc128fe0 con 0x7fb7d41432c0 2024-05-14T11:21:38.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.246+0000 7fb7d3e00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 0x7fb7b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.246+0000 7fb7d3e00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 0x7fb7b0071910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7d40b1720 tx=0x7fb7c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.250+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb7d413a400 con 0x7fb7d41432c0 2024-05-14T11:21:38.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d4074310 con 0x7fb7d41432c0 2024-05-14T11:21:38.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7d0c00640 1 -- 172.21.15.116:0/1539485157 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7fb7d4143c10 con 0x7fb7d41432c0 2024-05-14T11:21:38.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:38.441 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:21:11.017738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":16,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:38.441 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 53 2024-05-14T11:21:38.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 msgr2=0x7fb7b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 0x7fb7b0071910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7d40b1720 tx=0x7fb7c4008040 comp rx=0 tx=0).stop 2024-05-14T11:21:38.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 msgr2=0x7fb7d40d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc01cd30 tx=0x7fb7bc019230 comp rx=0 tx=0).stop 2024-05-14T11:21:38.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 shutdown_connections 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb7b006f400 0x7fb7b0071910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb7d415eb60 0x7fb7d40d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7d41432c0 0x7fb7d40d0b80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 --2- 172.21.15.116:0/1539485157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d40cdac0 0x7fb7d4139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 >> 172.21.15.116:0/1539485157 conn(0x7fb7d40ce6b0 msgr2=0x7fb7d415bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 shutdown_connections 2024-05-14T11:21:38.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.441+0000 7fb7dae00640 1 -- 172.21.15.116:0/1539485157 wait complete. 2024-05-14T11:21:38.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-14T11:21:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 Processor -- start 2024-05-14T11:21:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 -- start start 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88880d1670 0x7f88880d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f888806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f888806ff20 con 0x7f888815db30 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f88880ae100 con 0x7f88880d1670 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f888de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f88880ae870 con 0x7f888806c700 2024-05-14T11:21:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88880d1670 0x7f88880d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f8886a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f888815db30 0x7f88880aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47720/0 (socket says 172.21.15.116:47720) 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88880d1670 0x7f88880d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35402/0 (socket says 172.21.15.116:35402) 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f8886a00640 1 -- 172.21.15.116:0/3723290685 learned_addr learned my addr 172.21.15.116:0/3723290685 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f8887400640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f888806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 -- 172.21.15.116:0/3723290685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 msgr2=0x7f888806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f888806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 -- 172.21.15.116:0/3723290685 >> v1:172.21.15.116:6789/0 conn(0x7f888815db30 legacy=0x7f88880aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f887fe00640 1 -- 172.21.15.116:0/3723290685 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880d1fc0 con 0x7f88880d1670 2024-05-14T11:21:38.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f8886000640 1 -- 172.21.15.116:0/3723290685 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706857591 0 0) 0x7f888806ff20 con 0x7f888815db30 2024-05-14T11:21:38.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.613+0000 7f8887400640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f888806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f887fe00640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88880d1a80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f88700097c0 tx=0x7f887001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ab04605eaf68d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f8886000640 1 -- 172.21.15.116:0/3723290685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700b0070 con 0x7f88880d1670 2024-05-14T11:21:38.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f8886000640 1 -- 172.21.15.116:0/3723290685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88700ab600 con 0x7f88880d1670 2024-05-14T11:21:38.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f8886000640 1 -- 172.21.15.116:0/3723290685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700ab9d0 con 0x7f88880d1670 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 -- 172.21.15.116:0/3723290685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 msgr2=0x7f88880d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88880d1a80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f88700097c0 tx=0x7f887001c340 comp rx=0 tx=0).stop 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 -- 172.21.15.116:0/3723290685 shutdown_connections 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f888806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 --2- 172.21.15.116:0/3723290685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88880d1a80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 -- 172.21.15.116:0/3723290685 >> 172.21.15.116:0/3723290685 conn(0x7f88880b4620 msgr2=0x7f88880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 -- 172.21.15.116:0/3723290685 shutdown_connections 2024-05-14T11:21:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.617+0000 7f888de00640 1 -- 172.21.15.116:0/3723290685 wait complete. 2024-05-14T11:21:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 Processor -- start 2024-05-14T11:21:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 -- start start 2024-05-14T11:21:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f8888160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 0x7f8888150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8888072a30 con 0x7f88880d1670 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f888815df40 con 0x7f888815db30 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8888072bb0 con 0x7f888806c700 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35418/0 (socket says 172.21.15.116:35418) 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 -- 172.21.15.116:0/3803336350 learned_addr learned my addr 172.21.15.116:0/3803336350 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f8887400640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 0x7f8888150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f8886a00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f8888160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 -- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 msgr2=0x7f8888160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f8888160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 -- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 msgr2=0x7f8888150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 0x7f8888150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 -- 172.21.15.116:0/3803336350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880cd880 con 0x7f88880d1670 2024-05-14T11:21:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f8887400640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 0x7f8888150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887fe00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88700ae040 tx=0x7f887001ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887f400640 1 -- 172.21.15.116:0/3803336350 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700b0050 con 0x7f88880d1670 2024-05-14T11:21:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887f400640 1 -- 172.21.15.116:0/3803336350 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88700b1040 con 0x7f88880d1670 2024-05-14T11:21:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88880cdb60 con 0x7f88880d1670 2024-05-14T11:21:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.621+0000 7f887f400640 1 -- 172.21.15.116:0/3803336350 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870004220 con 0x7f88880d1670 2024-05-14T11:21:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.625+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88880ce120 con 0x7f88880d1670 2024-05-14T11:21:38.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.625+0000 7f887f400640 1 -- 172.21.15.116:0/3803336350 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f88700be060 con 0x7f88880d1670 2024-05-14T11:21:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887f400640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 0x7f8864071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 Processor -- start 2024-05-14T11:21:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887f400640 1 -- 172.21.15.116:0/3803336350 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8870128eb0 con 0x7f88880d1670 2024-05-14T11:21:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8886a00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 0x7f8864071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- start start 2024-05-14T11:21:38.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8886a00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 0x7f8864071940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8878000f70 tx=0x7f8878007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e369e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e36a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50e16f1a0 con 0x55a50e1f62d0 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a50e127b80 con 0x55a50ddf0580 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a50e16b160 con 0x55a50e206bd0 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35428/0 (socket says 172.21.15.116:35428) 2024-05-14T11:21:38.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 -- 172.21.15.116:0/339223360 learned_addr learned my addr 172.21.15.116:0/339223360 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8886a00640 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e369e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8887400640 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e36a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 -- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 msgr2=0x55a50e369e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e369e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 -- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 msgr2=0x55a50e36a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e36a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 -- 172.21.15.116:0/339223360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50e17b630 con 0x55a50e1f62d0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f887fe00640 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88700f54e0 tx=0x7f88700190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296c1c8ccbdcd3e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8877400640 1 -- 172.21.15.116:0/339223360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887010e070 con 0x55a50e1f62d0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8877400640 1 -- 172.21.15.116:0/339223360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88700f5550 con 0x55a50e1f62d0 2024-05-14T11:21:38.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f8877400640 1 -- 172.21.15.116:0/339223360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88700f5320 con 0x55a50e1f62d0 2024-05-14T11:21:38.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- 172.21.15.116:0/339223360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 msgr2=0x55a50e36a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88700f54e0 tx=0x7f88700190e0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- 172.21.15.116:0/339223360 shutdown_connections 2024-05-14T11:21:38.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e36a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e36a3b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 --2- 172.21.15.116:0/339223360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e369e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- 172.21.15.116:0/339223360 >> 172.21.15.116:0/339223360 conn(0x55a50de2f760 msgr2=0x55a50e34ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- 172.21.15.116:0/339223360 shutdown_connections 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.629+0000 7f88954cc000 1 -- 172.21.15.116:0/339223360 wait complete. 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- start start 2024-05-14T11:21:38.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 Processor -- start 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e3714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e3232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50e10eab0 con 0x55a50e1f62d0 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a50e123f20 con 0x55a50ddf0580 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a50e1ceda0 con 0x55a50e206bd0 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 10 client.0 _ll_register_callbacks cb 0x55a50e30af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35432/0 (socket says 172.21.15.116:35432) 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 -- 172.21.15.116:0/97069217 learned_addr learned my addr 172.21.15.116:0/97069217 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f8887400640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e3232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f8886a00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e3714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 -- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 msgr2=0x55a50e3232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e3232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 -- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 msgr2=0x55a50e3714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e3714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 -- 172.21.15.116:0/97069217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50e12dce0 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f8886a00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e3714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f887fe00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f887001c820 tx=0x7f88701270d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f78fba881c5398 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f886be00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887010e040 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f88700f47b0 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88700f5550 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f88700fe320 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 10 client.6639 fetch_fsmap learned FSMap version 53 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55a50e119740 con 0x55a50e1f62d0 2024-05-14T11:21:38.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.633+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88700f5550 con 0x55a50e1f62d0 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f88700f0ec0 con 0x55a50e1f62d0 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 10 client.6639 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 10 client.6639 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 20 client.6639 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de14ad0 0x55a50e324050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 4 client.6639 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50e107e60 con 0x55a50de14ad0 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f8886a00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de14ad0 0x55a50e324050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f8886a00640 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de14ad0 0x55a50e324050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f013156ca97e7f41 server_cookie=27d466c133fb22f1 in_seq=0 out_seq=0 2024-05-14T11:21:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f886be00640 10 client.6639 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f886be00640 1 -- 172.21.15.116:0/97069217 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a50e107e60 con 0x55a50de14ad0 2024-05-14T11:21:38.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f886be00640 10 client.6639 handle_command_reply: tid=0 2024-05-14T11:21:38.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 client.6639 shutdown 2024-05-14T11:21:38.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 msgr2=0x55a50e371a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f887001c820 tx=0x7f88701270d0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 shutdown_connections 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de14ad0 0x55a50e324050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e3232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e1f62d0 0x55a50e371a20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 --2- 172.21.15.116:0/97069217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50ddf0580 0x55a50e3714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f886be00640 0 client.6639 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 >> 172.21.15.116:0/97069217 conn(0x55a50de2f760 msgr2=0x55a50e34b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 shutdown_connections 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 1 -- 172.21.15.116:0/97069217 wait complete. 2024-05-14T11:21:38.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.637+0000 7f88954cc000 20 client.6639 trim_cache size 0 max 16384 2024-05-14T11:21:38.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.649+0000 7f88954cc000 1 Processor -- start 2024-05-14T11:21:38.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.649+0000 7f88954cc000 1 -- start start 2024-05-14T11:21:38.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50ddf0580 0x55a50e381680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e381bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88882051b0 con 0x55a50e206bd0 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f888814e4e0 con 0x55a50e1f62d0 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f888814eb30 con 0x55a50ddf0580 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35436/0 (socket says 172.21.15.116:35436) 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 -- 172.21.15.116:0/2968357023 learned_addr learned my addr 172.21.15.116:0/2968357023 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8886a00640 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50ddf0580 0x55a50e381680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f887fe00640 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e381bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 -- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50ddf0580 msgr2=0x55a50e381680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50ddf0580 0x55a50e381680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 -- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 msgr2=0x55a50e381bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e381bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 -- 172.21.15.116:0/2968357023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50e127dd0 con 0x55a50e206bd0 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8887400640 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8880009d00 tx=0x7f8880007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53cb839d5d563cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8876a00640 1 -- 172.21.15.116:0/2968357023 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888001a020 con 0x55a50e206bd0 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8876a00640 1 -- 172.21.15.116:0/2968357023 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880009d30 con 0x55a50e206bd0 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f8876a00640 1 -- 172.21.15.116:0/2968357023 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880016640 con 0x55a50e206bd0 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- 172.21.15.116:0/2968357023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 msgr2=0x55a50e373310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8880009d00 tx=0x7f8880007700 comp rx=0 tx=0).stop 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- 172.21.15.116:0/2968357023 shutdown_connections 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e381bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50e206bd0 0x55a50e373310 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 --2- 172.21.15.116:0/2968357023 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50ddf0580 0x55a50e381680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- 172.21.15.116:0/2968357023 >> 172.21.15.116:0/2968357023 conn(0x55a50de2f760 msgr2=0x55a50e322c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- 172.21.15.116:0/2968357023 shutdown_connections 2024-05-14T11:21:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.653+0000 7f88954cc000 1 -- 172.21.15.116:0/2968357023 wait complete. 2024-05-14T11:21:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- start start 2024-05-14T11:21:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 Processor -- start 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e397fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e398510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50e1185f0 con 0x55a50ddf0580 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a50e169e80 con 0x55a50e1f62d0 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a50e12af60 con 0x55a50e206bd0 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 10 client.0 _ll_register_callbacks cb 0x7f88700f4d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35446/0 (socket says 172.21.15.116:35446) 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 -- 172.21.15.116:0/3096985207 learned_addr learned my addr 172.21.15.116:0/3096985207 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8887400640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e398510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f887fe00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e397fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 -- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 msgr2=0x55a50e398510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e398510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 -- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 msgr2=0x55a50e397fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e397fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 -- 172.21.15.116:0/3096985207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50e1efc40 con 0x55a50ddf0580 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8887400640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e398510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f887fe00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e397fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f8886a00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8878006010 tx=0x7f8878000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc68dcf61f65d7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887801e040 con 0x55a50ddf0580 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f8878010040 con 0x55a50ddf0580 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8880009d30 con 0x55a50ddf0580 2024-05-14T11:21:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8878013b00 con 0x55a50ddf0580 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 10 client.6645 fetch_fsmap learned FSMap version 53 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55a50de283c0 con 0x55a50ddf0580 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8880009d30 con 0x55a50ddf0580 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f8878013ca0 con 0x55a50ddf0580 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 10 client.6645 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 10 client.6645 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 20 client.6645 populate_metadata read hostname 'smithi116' 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de02b30 0x55a50e3dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 4 client.6645 mds_command: new command op to 6396 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 --> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55a50e17cdb0 con 0x55a50de02b30 2024-05-14T11:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.657+0000 7f887fe00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de02b30 0x55a50e3dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f887fe00640 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de02b30 0x55a50e3dc480 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90fedcaad499aab4 server_cookie=cadbe6f12ac1753c in_seq=0 out_seq=0 2024-05-14T11:21:38.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f886be00640 10 client.6645 ms_handle_connect on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f886be00640 1 -- 172.21.15.116:0/3096985207 <== mds.0 v2:172.21.15.116:6834/2154548589 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a50e17cdb0 con 0x55a50de02b30 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f886be00640 10 client.6645 handle_command_reply: tid=0 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 client.6645 shutdown 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 msgr2=0x55a50e376700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8878006010 tx=0x7f8878000ee0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 shutdown_connections 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:6834/2154548589,v1:172.21.15.116:6836/2154548589] conn(0x55a50de02b30 0x55a50e3dc480 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50e1f62d0 0x55a50e397fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a50e206bd0 0x55a50e398510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f886be00640 0 client.6645 ms_handle_reset on v2:172.21.15.116:6834/2154548589 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 --2- 172.21.15.116:0/3096985207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50ddf0580 0x55a50e376700 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 >> 172.21.15.116:0/3096985207 conn(0x55a50de2f760 msgr2=0x55a50e3707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 shutdown_connections 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 1 -- 172.21.15.116:0/3096985207 wait complete. 2024-05-14T11:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f88954cc000 20 client.6645 trim_cache size 0 max 16384 2024-05-14T11:21:38.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:38.664 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-05-14T11:21:38.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 msgr2=0x7f8864071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 0x7f8864071940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8878000f70 tx=0x7f8878007c10 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 msgr2=0x7f88881503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88700ae040 tx=0x7f887001ab80 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 shutdown_connections 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406f430 0x7f8864071940 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f888815db30 0x7f8888150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88880d1670 0x7f88881503c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 --2- 172.21.15.116:0/3803336350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f888806c700 0x7f8888160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 >> 172.21.15.116:0/3803336350 conn(0x7f88880b4620 msgr2=0x7f888806f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 shutdown_connections 2024-05-14T11:21:38.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.661+0000 7f888de00640 1 -- 172.21.15.116:0/3803336350 wait complete. 2024-05-14T11:21:38.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:21:38.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 Processor -- start 2024-05-14T11:21:38.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 -- start start 2024-05-14T11:21:38.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f693c161e30 0x7f693c15f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c162450 con 0x7f693c0706c0 2024-05-14T11:21:38.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c15e320 con 0x7f693c0b1b60 2024-05-14T11:21:38.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6944600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f693c0b1f70 con 0x7f693c161e30 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6941200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f693c0706c0 0x7f693c161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47732/0 (socket says 172.21.15.116:47732) 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6941200640 1 -- 172.21.15.116:0/2349035522 learned_addr learned my addr 172.21.15.116:0/2349035522 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6942600640 1 --2- 172.21.15.116:0/2349035522 >> v2:172.21.15.149:3300/0 conn(0x7f693c161e30 0x7f693c15f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f693be00640 1 -- 172.21.15.116:0/2349035522 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493057930 0 0) 0x7f693c162450 con 0x7f693c0706c0 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f693be00640 1 -- 172.21.15.116:0/2349035522 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6924003530 con 0x7f693c0706c0 2024-05-14T11:21:38.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6942600640 1 -- 172.21.15.116:0/2349035522 >> v1:172.21.15.149:6789/0 conn(0x7f693c0b1b60 legacy=0x7f693c06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.825+0000 7f6942600640 1 -- 172.21.15.116:0/2349035522 >> v1:172.21.15.116:6789/0 conn(0x7f693c0706c0 legacy=0x7f693c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f6942600640 1 -- 172.21.15.116:0/2349035522 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c15e600 con 0x7f693c161e30 2024-05-14T11:21:38.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f6942600640 1 --2- 172.21.15.116:0/2349035522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c15f2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d90 tx=0x7f693401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f186e33bd872e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f693be00640 1 -- 172.21.15.116:0/2349035522 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340b2070 con 0x7f693c161e30 2024-05-14T11:21:38.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f693be00640 1 -- 172.21.15.116:0/2349035522 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69340a5a50 con 0x7f693c161e30 2024-05-14T11:21:38.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f693be00640 1 -- 172.21.15.116:0/2349035522 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340a5e00 con 0x7f693c161e30 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f6944600640 1 -- 172.21.15.116:0/2349035522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 msgr2=0x7f693c15f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.829+0000 7f6944600640 1 --2- 172.21.15.116:0/2349035522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c15f2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d90 tx=0x7f693401c970 comp rx=0 tx=0).stop 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- 172.21.15.116:0/2349035522 shutdown_connections 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 --2- 172.21.15.116:0/2349035522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c15f2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- 172.21.15.116:0/2349035522 >> 172.21.15.116:0/2349035522 conn(0x7f693c071370 msgr2=0x7f693c071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- 172.21.15.116:0/2349035522 shutdown_connections 2024-05-14T11:21:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- 172.21.15.116:0/2349035522 wait complete. 2024-05-14T11:21:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 Processor -- start 2024-05-14T11:21:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- start start 2024-05-14T11:21:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f693c0706c0 0x7f693c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f693c16c670 con 0x7f693c0b1b60 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f693c16c4f0 con 0x7f693c161e30 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6944600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f693c16c7f0 con 0x7f693c0706c0 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.833+0000 7f6941200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35448/0 (socket says 172.21.15.116:35448) 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 -- 172.21.15.116:0/595653483 learned_addr learned my addr 172.21.15.116:0/595653483 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941c00640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f693c0706c0 0x7f693c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6942600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 -- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f693c0706c0 msgr2=0x7f693c13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f693c0706c0 0x7f693c13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 -- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 msgr2=0x7f693c145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 -- 172.21.15.116:0/595653483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c144130 con 0x7f693c0b1b60 2024-05-14T11:21:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6941200640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3e0 tx=0x7f692800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928017070 con 0x7f693c0b1b60 2024-05-14T11:21:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6928005d10 con 0x7f693c0b1b60 2024-05-14T11:21:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c13e340 con 0x7f693c0b1b60 2024-05-14T11:21:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928007310 con 0x7f693c0b1b60 2024-05-14T11:21:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1446d0 con 0x7f693c0b1b60 2024-05-14T11:21:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c13e0b0 con 0x7f693c0b1b60 2024-05-14T11:21:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.837+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69280074b0 con 0x7f693c0b1b60 2024-05-14T11:21:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.841+0000 7f6939600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 0x7f691c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.841+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f692808e8f0 con 0x7f693c0b1b60 2024-05-14T11:21:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.841+0000 7f6941c00640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 0x7f691c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.841+0000 7f6941c00640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 0x7f691c071940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6930000c10 tx=0x7f6930009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:38.845+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f692805b4e0 con 0x7f693c0b1b60 2024-05-14T11:21:39.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.033+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f693c06a9c0 con 0x7f693c0b1b60 2024-05-14T11:21:39.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.033+0000 7f6939600640 1 -- 172.21.15.116:0/595653483 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f692805ade0 con 0x7f693c0b1b60 2024-05-14T11:21:39.035 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:39.035 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 msgr2=0x7f691c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 0x7f691c071940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6930000c10 tx=0x7f6930009040 comp rx=0 tx=0).stop 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 msgr2=0x7f693c144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3e0 tx=0x7f692800c8a0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 shutdown_connections 2024-05-14T11:21:39.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f691c06f430 0x7f691c071940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f693c161e30 0x7f693c145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693c0b1b60 0x7f693c144bf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 --2- 172.21.15.116:0/595653483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f693c0706c0 0x7f693c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 >> 172.21.15.116:0/595653483 conn(0x7f693c071370 msgr2=0x7f693c0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 shutdown_connections 2024-05-14T11:21:39.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.037+0000 7f6944600640 1 -- 172.21.15.116:0/595653483 wait complete. 2024-05-14T11:21:39.056 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-14T11:21:39.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:21:39.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 Processor -- start 2024-05-14T11:21:39.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 -- start start 2024-05-14T11:21:39.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe92c1604f0 0x7fe92c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:39.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c160f90 con 0x7fe92c073d40 2024-05-14T11:21:39.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c161270 con 0x7fe92c0b7230 2024-05-14T11:21:39.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92be00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c0b6a30 con 0x7fe92c1604f0 2024-05-14T11:21:39.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92a000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe92c073d40 0x7fe92c074150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47746/0 (socket says 172.21.15.116:47746) 2024-05-14T11:21:39.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92a000640 1 -- 172.21.15.116:0/2933409826 learned_addr learned my addr 172.21.15.116:0/2933409826 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:39.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92aa00640 1 --2- 172.21.15.116:0/2933409826 >> v2:172.21.15.149:3300/0 conn(0x7fe92c1604f0 0x7fe92c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:39.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe929600640 1 -- 172.21.15.116:0/2933409826 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932535847 0 0) 0x7fe92c160f90 con 0x7fe92c073d40 2024-05-14T11:21:39.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe929600640 1 -- 172.21.15.116:0/2933409826 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe910003530 con 0x7fe92c073d40 2024-05-14T11:21:39.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92aa00640 1 -- 172.21.15.116:0/2933409826 >> v1:172.21.15.149:6789/0 conn(0x7fe92c0b7230 legacy=0x7fe92c0b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92aa00640 1 -- 172.21.15.116:0/2933409826 >> v1:172.21.15.116:6789/0 conn(0x7fe92c073d40 legacy=0x7fe92c074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.205+0000 7fe92aa00640 1 -- 172.21.15.116:0/2933409826 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0b6d10 con 0x7fe92c1604f0 2024-05-14T11:21:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.209+0000 7fe92aa00640 1 --2- 172.21.15.116:0/2933409826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c1604f0 0x7fe92c160900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097a0 tx=0x7fe91c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f94871ba212c424 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.209+0000 7fe929600640 1 -- 172.21.15.116:0/2933409826 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0b0070 con 0x7fe92c1604f0 2024-05-14T11:21:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.209+0000 7fe929600640 1 -- 172.21.15.116:0/2933409826 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c0ab600 con 0x7fe92c1604f0 2024-05-14T11:21:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.209+0000 7fe929600640 1 -- 172.21.15.116:0/2933409826 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0ab9b0 con 0x7fe92c1604f0 2024-05-14T11:21:39.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- 172.21.15.116:0/2933409826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c1604f0 msgr2=0x7fe92c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 --2- 172.21.15.116:0/2933409826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c1604f0 0x7fe92c160900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097a0 tx=0x7fe91c01c3f0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- 172.21.15.116:0/2933409826 shutdown_connections 2024-05-14T11:21:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 --2- 172.21.15.116:0/2933409826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c1604f0 0x7fe92c160900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- 172.21.15.116:0/2933409826 >> 172.21.15.116:0/2933409826 conn(0x7fe92c0d26e0 msgr2=0x7fe92c0d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- 172.21.15.116:0/2933409826 shutdown_connections 2024-05-14T11:21:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- 172.21.15.116:0/2933409826 wait complete. 2024-05-14T11:21:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 Processor -- start 2024-05-14T11:21:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- start start 2024-05-14T11:21:39.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c073d40 0x7fe92c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c0b7230 0x7fe92c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c16aa00 con 0x7fe92c1604f0 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c16a880 con 0x7fe92c073d40 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.213+0000 7fe92be00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c16ab80 con 0x7fe92c0b7230 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35464/0 (socket says 172.21.15.116:35464) 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 -- 172.21.15.116:0/796129158 learned_addr learned my addr 172.21.15.116:0/796129158 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92a000640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c0b7230 0x7fe92c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92aa00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c073d40 0x7fe92c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 -- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c0b7230 msgr2=0x7fe92c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c0b7230 0x7fe92c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 -- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c073d40 msgr2=0x7fe92c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c073d40 0x7fe92c15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 -- 172.21.15.116:0/796129158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c06e500 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92b400640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c740 tx=0x7fe92400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924003b80 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92400ce50 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c06e7e0 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92400a850 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c06eda0 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c050880 con 0x7fe92c1604f0 2024-05-14T11:21:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.217+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe92400a9f0 con 0x7fe92c1604f0 2024-05-14T11:21:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.221+0000 7fe922a00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.221+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fe92408f210 con 0x7fe92c1604f0 2024-05-14T11:21:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.221+0000 7fe92aa00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.221+0000 7fe92aa00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ad000 tx=0x7fe91c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:39.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.225+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe92405b8c0 con 0x7fe92c1604f0 2024-05-14T11:21:39.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.417+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe92c15a650 con 0x7fe92c1604f0 2024-05-14T11:21:39.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.417+0000 7fe922a00640 1 -- 172.21.15.116:0/796129158 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7fe92405fbc0 con 0x7fe92c1604f0 2024-05-14T11:21:39.419 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:21:39.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 msgr2=0x7fe90c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ad000 tx=0x7fe91c0a6000 comp rx=0 tx=0).stop 2024-05-14T11:21:39.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 msgr2=0x7fe92c06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:39.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c740 tx=0x7fe92400cc00 comp rx=0 tx=0).stop 2024-05-14T11:21:39.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 shutdown_connections 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c1604f0 0x7fe92c06f2c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c0b7230 0x7fe92c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 --2- 172.21.15.116:0/796129158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c073d40 0x7fe92c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 >> 172.21.15.116:0/796129158 conn(0x7fe92c0d26e0 msgr2=0x7fe92c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:39.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 shutdown_connections 2024-05-14T11:21:39.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39.421+0000 7fe92be00640 1 -- 172.21.15.116:0/796129158 wait complete. 2024-05-14T11:21:39.440 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:21:39.440 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:21:39.440 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:21:39.440 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:21:39.440 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:39.441 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:21:39.441 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:21:39.441 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:21:39.441 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:21:39.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.441 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:21:39.450 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:21:39.450 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:21:39.451 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42103sec preferred_lft 42103sec 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:21:39.452 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:21:39.453 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:21:39.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.454 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:21:39.512 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:21:39.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.514 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:21:39.566 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:21:39.567 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:21:39.567 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:21:39.567 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:21:39.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:21:39.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:21:39.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.699 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:21:39.707 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:21:39.708 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:21:39.708 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:21:39.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.709 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:21:39.770 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:21:39.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:21:39.770 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:39.818 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:21:39.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:21:39.819 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:39.871 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:21:39.935 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:21:39.992 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:21:39.992 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:21:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:21:39.994 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:21:39.995 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:21:39.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:39.996 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:21:40.046 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:21:40.047 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:21:40.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:21:40.109 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:21:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:21:40.162 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:21:40.163 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:21:40.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:40.164 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:21:40.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:21:40.193+0000 7f08f40e2580 -1 init, newargv = 0x55d4798010f0 newargc=15 2024-05-14T11:21:40.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42914]: starting ceph client 2024-05-14T11:21:40.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42914]: starting fuse 2024-05-14T11:21:40.203 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:21:40.203 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:21:40.203 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:21:40.204 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:21:40.205 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:21:40.327 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.42914.asok 2024-05-14T11:21:40.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:21:40.338 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42914.asok 2024-05-14T11:21:40.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:40.338 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42914.asok status 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:21:40.521 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "42914", 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6663, 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 6663 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:21:40.522 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 323080059 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 323080059 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.6663 192.168.0.1:0/323080059", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/323080059", 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 53, 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 73, 2024-05-14T11:21:40.523 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:21:40.524 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:21:40.524 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:21:40.524 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:40.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:40.537 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:21:40.537 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:40.546 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:21:40.546 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:40.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:40.546 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:40.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:40.606 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-14T11:21:40.658 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:21:40.659 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-14T11:21:40.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 Processor -- start 2024-05-14T11:21:40.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 -- start start 2024-05-14T11:21:40.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fcc6f100 0x5567fcd42d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcd26560 0x5567fccb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5567fccb5f60 con 0x5567fccb5570 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5567fcc12aa0 con 0x5567fcc6f100 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79bbc87080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5567fccb6200 con 0x5567fcd26560 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35480/0 (socket says 172.21.15.116:35480) 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79afe00640 1 -- 172.21.15.116:0/2823754004 learned_addr learned my addr 172.21.15.116:0/2823754004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79b4e00640 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fcc6f100 0x5567fcd42d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.757+0000 7f79b5800640 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcd26560 0x5567fccb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 -- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcd26560 msgr2=0x5567fccb24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcd26560 0x5567fccb24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 -- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fcc6f100 msgr2=0x5567fcd42d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:40.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fcc6f100 0x5567fcd42d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 -- 172.21.15.116:0/2823754004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fcc97990 con 0x5567fccb5570 2024-05-14T11:21:40.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79afe00640 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b220 tx=0x7f79a400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364e653fd079bf81 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:40.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79aea00640 1 -- 172.21.15.116:0/2823754004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a400a040 con 0x5567fccb5570 2024-05-14T11:21:40.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79aea00640 1 -- 172.21.15.116:0/2823754004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a400f620 con 0x5567fccb5570 2024-05-14T11:21:40.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79aea00640 1 -- 172.21.15.116:0/2823754004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a400fa10 con 0x5567fccb5570 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- 172.21.15.116:0/2823754004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 msgr2=0x5567fccb1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b220 tx=0x7f79a400b6e0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- 172.21.15.116:0/2823754004 shutdown_connections 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcd26560 0x5567fccb24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fccb5570 0x5567fccb1f70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 --2- 172.21.15.116:0/2823754004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fcc6f100 0x5567fcd42d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- 172.21.15.116:0/2823754004 >> 172.21.15.116:0/2823754004 conn(0x5567fcc57e10 msgr2=0x5567fcd43640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- 172.21.15.116:0/2823754004 shutdown_connections 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- 172.21.15.116:0/2823754004 wait complete. 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 Processor -- start 2024-05-14T11:21:40.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 -- start start 2024-05-14T11:21:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.761+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcc6f100 0x5567fcd23b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fccb5570 0x5567fcd24510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 0x5567fcd43640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5567fcc9b8f0 con 0x5567fcd26560 2024-05-14T11:21:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5567fcc9b690 con 0x5567fccb5570 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5567fcc97c30 con 0x5567fcc6f100 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 0x5567fcd43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 0x5567fcd43640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35496/0 (socket says 172.21.15.116:35496) 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 -- 172.21.15.116:0/3608366537 learned_addr learned my addr 172.21.15.116:0/3608366537 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b4e00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcc6f100 0x5567fcd23b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79afe00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fccb5570 0x5567fcd24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 -- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcc6f100 msgr2=0x5567fcd23b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5567fcc6f100 0x5567fcd23b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 -- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fccb5570 msgr2=0x5567fcd24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fccb5570 0x5567fcd24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 -- 172.21.15.116:0/3608366537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fcd24f70 con 0x5567fcd26560 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79afe00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567fccb5570 0x5567fcd24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79b5800640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 0x5567fcd43640 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f79b0008720 tx=0x7f79b0008be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79ad600640 1 -- 172.21.15.116:0/3608366537 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0006d00 con 0x5567fcd26560 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79ad600640 1 -- 172.21.15.116:0/3608366537 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b001a070 con 0x5567fcd26560 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79ad600640 1 -- 172.21.15.116:0/3608366537 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b00156a0 con 0x5567fcd26560 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567fcd25210 con 0x5567fcd26560 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fcd25750 con 0x5567fcd26560 2024-05-14T11:21:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.765+0000 7f79ad600640 1 -- 172.21.15.116:0/3608366537 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79b0013030 con 0x5567fcd26560 2024-05-14T11:21:40.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79ad600640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f799806dde0 0x7f79980702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79ad600640 1 -- 172.21.15.116:0/3608366537 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f79b008e9e0 con 0x5567fcd26560 2024-05-14T11:21:40.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79b4e00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f799806dde0 0x7f79980702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5567fcd240d0 0x5567fcd3c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73) -- 0x5567fcd24a50 con 0x5567fcd240d0 2024-05-14T11:21:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79afe00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5567fcd240d0 0x5567fcd3c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79b4e00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f799806dde0 0x7f79980702f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f79a800a560 tx=0x7f79a80b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79afe00640 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5567fcd240d0 0x5567fcd3c4e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79afe00640 1 -- 172.21.15.116:0/3608366537 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v73'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f79a400d040 con 0x5567fcd240d0 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5567fcd240d0 msgr2=0x5567fcd3c4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5567fcd240d0 0x5567fcd3c4e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f799806dde0 msgr2=0x7f79980702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f799806dde0 0x7f79980702f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f79a800a560 tx=0x7f79a80b2040 comp rx=0 tx=0).stop 2024-05-14T11:21:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 msgr2=0x5567fcd43640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 --2- 172.21.15.116:0/3608366537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567fcd26560 0x5567fcd43640 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f79b0008720 tx=0x7f79b0008be0 comp rx=0 tx=0).stop 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79b5800640 1 -- 172.21.15.116:0/3608366537 reap_dead start 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 shutdown_connections 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 >> 172.21.15.116:0/3608366537 conn(0x5567fcc57e10 msgr2=0x5567fcd42d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.769+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 shutdown_connections 2024-05-14T11:21:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:40.773+0000 7f79bbc87080 1 -- 172.21.15.116:0/3608366537 wait complete. 2024-05-14T11:21:40.776 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:21:40.776 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:21:40.874 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:21:40.924 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:21:40.924 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:21:40.924 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:21:40.924 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:40.932 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:21:40.933 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:40.983 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:21:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 Processor -- start 2024-05-14T11:21:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 -- start start 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c0d1670 0x7fd40c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd40c06c700 0x7fd40c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40c06ff20 con 0x7fd40c15db30 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c0ae100 con 0x7fd40c06c700 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd413400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c0ae870 con 0x7fd40c0d1670 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd410a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd40c15db30 0x7fd40c0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47760/0 (socket says 172.21.15.116:47760) 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd410a00640 1 -- 172.21.15.116:0/1707613939 learned_addr learned my addr 172.21.15.116:0/1707613939 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd411400640 1 --2- 172.21.15.116:0/1707613939 >> v2:172.21.15.116:3300/0 conn(0x7fd40c06c700 0x7fd40c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd40be00640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c0d1670 0x7fd40c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd40b400640 1 -- 172.21.15.116:0/1707613939 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524436529 0 0) 0x7fd40c06ff20 con 0x7fd40c15db30 2024-05-14T11:21:41.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd40b400640 1 -- 172.21.15.116:0/1707613939 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f0003530 con 0x7fd40c15db30 2024-05-14T11:21:41.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd411400640 1 -- 172.21.15.116:0/1707613939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c0d1670 msgr2=0x7fd40c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:41.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd411400640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c0d1670 0x7fd40c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd411400640 1 -- 172.21.15.116:0/1707613939 >> v1:172.21.15.116:6789/0 conn(0x7fd40c15db30 legacy=0x7fd40c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.177+0000 7fd411400640 1 -- 172.21.15.116:0/1707613939 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0d1fc0 con 0x7fd40c06c700 2024-05-14T11:21:41.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd411400640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c06c700 0x7fd40c06cb10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd4040097c0 tx=0x7fd40401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6a15f095e5bcf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd40b400640 1 -- 172.21.15.116:0/1707613939 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040b0070 con 0x7fd40c06c700 2024-05-14T11:21:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd40b400640 1 -- 172.21.15.116:0/1707613939 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4040ab600 con 0x7fd40c06c700 2024-05-14T11:21:41.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd40b400640 1 -- 172.21.15.116:0/1707613939 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040ab9b0 con 0x7fd40c06c700 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 -- 172.21.15.116:0/1707613939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c06c700 msgr2=0x7fd40c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c06c700 0x7fd40c06cb10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd4040097c0 tx=0x7fd40401c340 comp rx=0 tx=0).stop 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 -- 172.21.15.116:0/1707613939 shutdown_connections 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c06c700 0x7fd40c06cb10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 --2- 172.21.15.116:0/1707613939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c0d1670 0x7fd40c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 -- 172.21.15.116:0/1707613939 >> 172.21.15.116:0/1707613939 conn(0x7fd40c0b4620 msgr2=0x7fd40c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 -- 172.21.15.116:0/1707613939 shutdown_connections 2024-05-14T11:21:41.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.181+0000 7fd413400640 1 -- 172.21.15.116:0/1707613939 wait complete. 2024-05-14T11:21:41.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 Processor -- start 2024-05-14T11:21:41.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 -- start start 2024-05-14T11:21:41.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 0x7fd40c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 0x7fd40c0b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c16a770 con 0x7fd40c0d1670 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c15df40 con 0x7fd40c15db30 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd413400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c16a8f0 con 0x7fd40c06c700 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd40be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd40be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35516/0 (socket says 172.21.15.116:35516) 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd40be00640 1 -- 172.21.15.116:0/2210174888 learned_addr learned my addr 172.21.15.116:0/2210174888 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd410a00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 0x7fd40c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd411400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 0x7fd40c0b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd40be00640 1 -- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 msgr2=0x7fd40c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.185+0000 7fd40be00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 0x7fd40c14d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd40be00640 1 -- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 msgr2=0x7fd40c0b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd40be00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 0x7fd40c0b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd40be00640 1 -- 172.21.15.116:0/2210174888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0cedc0 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd410a00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 0x7fd40c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd411400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 0x7fd40c0b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd40be00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800d620 tx=0x7fd3f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800b620 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8019070 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c0cf030 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800b9e0 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c0cf5f0 con 0x7fd40c0d1670 2024-05-14T11:21:41.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c06a9c0 con 0x7fd40c0d1670 2024-05-14T11:21:41.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3f800bb80 con 0x7fd40c0d1670 2024-05-14T11:21:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 0x7fd3ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fd3f808f9f0 con 0x7fd40c0d1670 2024-05-14T11:21:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.189+0000 7fd410a00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 0x7fd3ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.193+0000 7fd410a00640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 0x7fd3ec071910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc005c10 tx=0x7fd3fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.197+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd3f805c660 con 0x7fd40c0d1670 2024-05-14T11:21:41.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.361+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd40c068dd0 con 0x7fd40c0d1670 2024-05-14T11:21:41.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd408c00640 1 -- 172.21.15.116:0/2210174888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) ==== 80+0+124 (secure 0 0 0) 0x7fd3f805bf60 con 0x7fd40c0d1670 2024-05-14T11:21:41.363 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:21:41.364 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6834/2430466110 2024-05-15T11:20:19.998743+0000 2024-05-14T11:21:41.364 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:21:41.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 msgr2=0x7fd3ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 0x7fd3ec071910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc005c10 tx=0x7fd3fc009040 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 msgr2=0x7fd40c1fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800d620 tx=0x7fd3f800dae0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 shutdown_connections 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd3ec06f400 0x7fd3ec071910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40c15db30 0x7fd40c0b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd40c0d1670 0x7fd40c1fcd60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 --2- 172.21.15.116:0/2210174888 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd40c06c700 0x7fd40c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 >> 172.21.15.116:0/2210174888 conn(0x7fd40c0b4620 msgr2=0x7fd40c166b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 shutdown_connections 2024-05-14T11:21:41.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.365+0000 7fd413400640 1 -- 172.21.15.116:0/2210174888 wait complete. 2024-05-14T11:21:41.385 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:21:41.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:41.385 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:21:41.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[42914]: fuse finished with error 0 and tester_r 0 2024-05-14T11:21:41.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:21:41.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:21:41.405 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:21:41.564 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:21:41.564 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:21:41.565 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:41.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:21:41.575 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:21:41.575 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:21:41.575 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:41.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:21:41.619 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:21:41.619 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:21:41.619 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:41.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:21:41.628 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 Processor -- start 2024-05-14T11:21:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 -- start start 2024-05-14T11:21:41.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f17e415db30 0x7f17e40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f17e40d1670 0x7f17e40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 0x7f17e4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f17e40ae100 con 0x7f17e415db30 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f17e40ae870 con 0x7f17e40d1670 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17ea200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f17e406ff20 con 0x7f17e4161600 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f17e415db30 0x7f17e40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f17e415db30 0x7f17e40aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35526/0 (socket says 172.21.15.116:35526) 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 -- 172.21.15.116:0/3414751773 learned_addr learned my addr 172.21.15.116:0/3414751773 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3e00640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 0x7f17e4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e2a00640 1 --2- 172.21.15.116:0/3414751773 >> v2:172.21.15.149:3300/0 conn(0x7f17e40d1670 0x7f17e40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 -- 172.21.15.116:0/3414751773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 msgr2=0x7f17e4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 0x7f17e4161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 -- 172.21.15.116:0/3414751773 >> v2:172.21.15.149:3300/0 conn(0x7f17e40d1670 msgr2=0x7f17e40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 --2- 172.21.15.116:0/3414751773 >> v2:172.21.15.149:3300/0 conn(0x7f17e40d1670 0x7f17e40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3400640 1 -- 172.21.15.116:0/3414751773 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e40d1fc0 con 0x7f17e415db30 2024-05-14T11:21:41.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.813+0000 7f17e3e00640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 0x7f17e4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:41.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17e3400640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e40aeaa0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f17e40d2370 tx=0x7f17d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f588d529fec1ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17e2000640 1 -- 172.21.15.116:0/3414751773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40b0070 con 0x7f17e415db30 2024-05-14T11:21:41.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17e2000640 1 -- 172.21.15.116:0/3414751773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d40ab600 con 0x7f17e415db30 2024-05-14T11:21:41.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17e2000640 1 -- 172.21.15.116:0/3414751773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40ab9d0 con 0x7f17e415db30 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 -- 172.21.15.116:0/3414751773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 msgr2=0x7f17e40aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e40aeaa0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f17e40d2370 tx=0x7f17d401c490 comp rx=0 tx=0).stop 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 -- 172.21.15.116:0/3414751773 shutdown_connections 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e4161600 0x7f17e4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 --2- 172.21.15.116:0/3414751773 >> v2:172.21.15.149:3300/0 conn(0x7f17e40d1670 0x7f17e40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 --2- 172.21.15.116:0/3414751773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e40aeaa0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 -- 172.21.15.116:0/3414751773 >> 172.21.15.116:0/3414751773 conn(0x7f17e40b4620 msgr2=0x7f17e40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 -- 172.21.15.116:0/3414751773 shutdown_connections 2024-05-14T11:21:41.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.817+0000 7f17ea200640 1 -- 172.21.15.116:0/3414751773 wait complete. 2024-05-14T11:21:41.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 Processor -- start 2024-05-14T11:21:41.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 -- start start 2024-05-14T11:21:41.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e40d1670 0x7f17e414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 0x7f17e4154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4162360 con 0x7f17e415db30 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17e40aefe0 con 0x7f17e4161600 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17ea200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f17e415df40 con 0x7f17e40d1670 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17e2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17e2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35536/0 (socket says 172.21.15.116:35536) 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17e2a00640 1 -- 172.21.15.116:0/2930217204 learned_addr learned my addr 172.21.15.116:0/2930217204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.821+0000 7f17e3400640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e40d1670 0x7f17e414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e3e00640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 0x7f17e4154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 -- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e40d1670 msgr2=0x7f17e414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e40d1670 0x7f17e414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 -- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 msgr2=0x7f17e4154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 0x7f17e4154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:41.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 -- 172.21.15.116:0/2930217204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e41544a0 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e3e00640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 0x7f17e4154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17e2a00640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f17d000d620 tx=0x7f17d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000b620 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0019070 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000b9e0 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e4158860 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e4153f80 con 0x7f17e415db30 2024-05-14T11:21:41.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e406a9c0 con 0x7f17e415db30 2024-05-14T11:21:41.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17d000bb80 con 0x7f17e415db30 2024-05-14T11:21:41.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 0x7f17c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:41.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.825+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f17d008f890 con 0x7f17e415db30 2024-05-14T11:21:41.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.829+0000 7f17e3400640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 0x7f17c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:41.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.829+0000 7f17e3400640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 0x7f17c4071940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17e4071f20 tx=0x7f17d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:41.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:41.833+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f17d005c590 con 0x7f17e415db30 2024-05-14T11:21:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.021+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17e40b3050 con 0x7f17e415db30 2024-05-14T11:21:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.021+0000 7f17db400640 1 -- 172.21.15.116:0/2930217204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f17d005be90 con 0x7f17e415db30 2024-05-14T11:21:42.022 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:42.022 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:21:11.017738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":16,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 53 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 msgr2=0x7f17c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 0x7f17c4071940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17e4071f20 tx=0x7f17d4008040 comp rx=0 tx=0).stop 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 msgr2=0x7f17e41548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f17d000d620 tx=0x7f17d000dae0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 shutdown_connections 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17c406f430 0x7f17c4071940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e4161600 0x7f17e4154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17e415db30 0x7f17e41548d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 --2- 172.21.15.116:0/2930217204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17e40d1670 0x7f17e414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 >> 172.21.15.116:0/2930217204 conn(0x7f17e40b4620 msgr2=0x7f17e415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 shutdown_connections 2024-05-14T11:21:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.025+0000 7f17ea200640 1 -- 172.21.15.116:0/2930217204 wait complete. 2024-05-14T11:21:42.044 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:42.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.189+0000 7fe7f4200640 1 Processor -- start 2024-05-14T11:21:42.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.189+0000 7fe7f4200640 1 -- start start 2024-05-14T11:21:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f4200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe7ec0d1670 0x7fe7ec0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f4200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ec0d1fc0 con 0x7fe7ec0af380 2024-05-14T11:21:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f4200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ec160510 con 0x7fe7ec156fa0 2024-05-14T11:21:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f4200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ec0706f0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f0e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe7ec0d1670 0x7fe7ec0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f2200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe7ec0af380 0x7fe7ec0af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47776/0 (socket says 172.21.15.116:47776) 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f2200640 1 -- 172.21.15.116:0/2182585365 learned_addr learned my addr 172.21.15.116:0/2182585365 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f0e00640 1 -- 172.21.15.116:0/2182585365 >> v1:172.21.15.149:6789/0 conn(0x7fe7ec156fa0 legacy=0x7fe7ec06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f0e00640 1 -- 172.21.15.116:0/2182585365 >> v1:172.21.15.116:6789/0 conn(0x7fe7ec0af380 legacy=0x7fe7ec0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.193+0000 7fe7f0e00640 1 -- 172.21.15.116:0/2182585365 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1607f0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f0e00640 1 --2- 172.21.15.116:0/2182585365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0d1a80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097c0 tx=0x7fe7d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2efc15a4963c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7ebe00640 1 -- 172.21.15.116:0/2182585365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80b0070 con 0x7fe7ec0d1670 2024-05-14T11:21:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7ebe00640 1 -- 172.21.15.116:0/2182585365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d80ab600 con 0x7fe7ec0d1670 2024-05-14T11:21:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7ebe00640 1 -- 172.21.15.116:0/2182585365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ab9d0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 -- 172.21.15.116:0/2182585365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 msgr2=0x7fe7ec0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 --2- 172.21.15.116:0/2182585365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0d1a80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097c0 tx=0x7fe7d801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 -- 172.21.15.116:0/2182585365 shutdown_connections 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 --2- 172.21.15.116:0/2182585365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0d1a80 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 -- 172.21.15.116:0/2182585365 >> 172.21.15.116:0/2182585365 conn(0x7fe7ec0b4620 msgr2=0x7fe7ec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 -- 172.21.15.116:0/2182585365 shutdown_connections 2024-05-14T11:21:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 -- 172.21.15.116:0/2182585365 wait complete. 2024-05-14T11:21:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.197+0000 7fe7f4200640 1 Processor -- start 2024-05-14T11:21:42.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- start start 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ec0af380 0x7fe7ec09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7ec156fa0 0x7fe7ec1f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec16abb0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec16aa30 con 0x7fe7ec0af380 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ec16ad30 con 0x7fe7ec156fa0 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35560/0 (socket says 172.21.15.116:35560) 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 -- 172.21.15.116:0/1299032978 learned_addr learned my addr 172.21.15.116:0/1299032978 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f1800640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ec0af380 0x7fe7ec09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f2200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7ec156fa0 0x7fe7ec1f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 -- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7ec156fa0 msgr2=0x7fe7ec1f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7ec156fa0 0x7fe7ec1f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 -- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ec0af380 msgr2=0x7fe7ec09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ec0af380 0x7fe7ec09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 -- 172.21.15.116:0/1299032978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec0d0270 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f0e00640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80190b0 tx=0x7fe7d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80b8020 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d80b4610 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec0d0550 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80b0040 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec153020 con 0x7fe7ec0d1670 2024-05-14T11:21:42.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.201+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec06a9c0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.205+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7d80bb020 con 0x7fe7ec0d1670 2024-05-14T11:21:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.205+0000 7fe7e9600640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 0x7fe7cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.205+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7fe7d8128f20 con 0x7fe7ec0d1670 2024-05-14T11:21:42.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.205+0000 7fe7f1800640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 0x7fe7cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.205+0000 7fe7f1800640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 0x7fe7cc071910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000f70 tx=0x7fe7e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.209+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe7d80c2070 con 0x7fe7ec0d1670 2024-05-14T11:21:42.303 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:21:42.306 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:21:42.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7ec06fdc0 con 0x7fe7ec0d1670 2024-05-14T11:21:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7e9600640 1 -- 172.21.15.116:0/1299032978 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7fe7d8016090 con 0x7fe7ec0d1670 2024-05-14T11:21:42.405 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:42.405 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:21:11.017738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":16,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:42.405 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 53 2024-05-14T11:21:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 msgr2=0x7fe7cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 0x7fe7cc071910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000f70 tx=0x7fe7e0009040 comp rx=0 tx=0).stop 2024-05-14T11:21:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 msgr2=0x7fe7ec0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80190b0 tx=0x7fe7d8019240 comp rx=0 tx=0).stop 2024-05-14T11:21:42.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 shutdown_connections 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe7cc06f400 0x7fe7cc071910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7ec156fa0 0x7fe7ec1f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7ec0d1670 0x7fe7ec0cf790 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 --2- 172.21.15.116:0/1299032978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ec0af380 0x7fe7ec09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.405+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 >> 172.21.15.116:0/1299032978 conn(0x7fe7ec0b4620 msgr2=0x7fe7ec0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.409+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 shutdown_connections 2024-05-14T11:21:42.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.409+0000 7fe7f4200640 1 -- 172.21.15.116:0/1299032978 wait complete. 2024-05-14T11:21:42.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:42.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 Processor -- start 2024-05-14T11:21:42.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 -- start start 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f866c0b0770 0x7f866c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f866c072a30 0x7f866c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f866c06cc40 con 0x7f866c06f730 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f866c06db20 con 0x7f866c0b0770 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8672e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f866c06cf10 con 0x7f866c072a30 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f866be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f866c0b0770 0x7f866c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f866be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f866c0b0770 0x7f866c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35576/0 (socket says 172.21.15.116:35576) 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f866be00640 1 -- 172.21.15.116:0/3437658224 learned_addr learned my addr 172.21.15.116:0/3437658224 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.573+0000 7f8670e00640 1 --2- 172.21.15.116:0/3437658224 >> v2:172.21.15.149:3300/0 conn(0x7f866c072a30 0x7f866c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866be00640 1 -- 172.21.15.116:0/3437658224 >> v2:172.21.15.149:3300/0 conn(0x7f866c072a30 msgr2=0x7f866c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:42.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866be00640 1 --2- 172.21.15.116:0/3437658224 >> v2:172.21.15.149:3300/0 conn(0x7f866c072a30 0x7f866c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866be00640 1 -- 172.21.15.116:0/3437658224 >> v1:172.21.15.149:6789/0 conn(0x7f866c06f730 legacy=0x7f866c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866be00640 1 -- 172.21.15.116:0/3437658224 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c06d1f0 con 0x7f866c0b0770 2024-05-14T11:21:42.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866be00640 1 --2- 172.21.15.116:0/3437658224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0b0770 0x7f866c06c700 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f866c0729c0 tx=0x7f865c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46af9797c955a765 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866b400640 1 -- 172.21.15.116:0/3437658224 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b1070 con 0x7f866c0b0770 2024-05-14T11:21:42.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866b400640 1 -- 172.21.15.116:0/3437658224 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c0a8050 con 0x7f866c0b0770 2024-05-14T11:21:42.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f866b400640 1 -- 172.21.15.116:0/3437658224 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0ac860 con 0x7f866c0b0770 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f8672e00640 1 -- 172.21.15.116:0/3437658224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0b0770 msgr2=0x7f866c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.577+0000 7f8672e00640 1 --2- 172.21.15.116:0/3437658224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0b0770 0x7f866c06c700 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f866c0729c0 tx=0x7f865c01c460 comp rx=0 tx=0).stop 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- 172.21.15.116:0/3437658224 shutdown_connections 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 --2- 172.21.15.116:0/3437658224 >> v2:172.21.15.149:3300/0 conn(0x7f866c072a30 0x7f866c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 --2- 172.21.15.116:0/3437658224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0b0770 0x7f866c06c700 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- 172.21.15.116:0/3437658224 >> 172.21.15.116:0/3437658224 conn(0x7f866c0b4620 msgr2=0x7f866c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- 172.21.15.116:0/3437658224 shutdown_connections 2024-05-14T11:21:42.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- 172.21.15.116:0/3437658224 wait complete. 2024-05-14T11:21:42.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 Processor -- start 2024-05-14T11:21:42.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- start start 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 0x7f866c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f866c0b0770 0x7f866c146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f866c073500 con 0x7f866c06f730 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f866c073380 con 0x7f866c072a30 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f8672e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f866c073680 con 0x7f866c0b0770 2024-05-14T11:21:42.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f866be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.581+0000 7f866be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35584/0 (socket says 172.21.15.116:35584) 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 -- 172.21.15.116:0/3977656295 learned_addr learned my addr 172.21.15.116:0/3977656295 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8663400640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 0x7f866c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 -- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f866c0b0770 msgr2=0x7f866c146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f866c0b0770 0x7f866c146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 -- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 msgr2=0x7f866c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 0x7f866c15d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 -- 172.21.15.116:0/3977656295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c166460 con 0x7f866c06f730 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8663400640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 0x7f866c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f866be00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f865c01a090 tx=0x7f865c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b1040 con 0x7f866c06f730 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c0bc070 con 0x7f866c06f730 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c15e270 con 0x7f866c06f730 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b7650 con 0x7f866c06f730 2024-05-14T11:21:42.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c165720 con 0x7f866c06f730 2024-05-14T11:21:42.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866c06a9c0 con 0x7f866c06f730 2024-05-14T11:21:42.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.585+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f865c0b77f0 con 0x7f866c06f730 2024-05-14T11:21:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.589+0000 7f8668c00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 0x7f8648071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.589+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f865c129620 con 0x7f866c06f730 2024-05-14T11:21:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.589+0000 7f8663400640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 0x7f8648071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.589+0000 7f8663400640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 0x7f8648071910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f866c15a3e0 tx=0x7f8654009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.593+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f865c016110 con 0x7f866c06f730 2024-05-14T11:21:42.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.749+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f866c06f510 con 0x7f866c06f730 2024-05-14T11:21:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.749+0000 7f8668c00640 1 -- 172.21.15.116:0/3977656295 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17124 (secure 0 0 0) 0x7f865c0f63c0 con 0x7f866c06f730 2024-05-14T11:21:42.751 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:42.751 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":73,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:20:19.998776+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:20:01.282761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-14T11:20:02.552345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6836/2430466110":"2024-05-15T11:20:19.998743+0000","172.21.15.116:6834/2430466110":"2024-05-15T11:20:19.998743+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 msgr2=0x7f8648071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 0x7f8648071910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f866c15a3e0 tx=0x7f8654009040 comp rx=0 tx=0).stop 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 msgr2=0x7f866c165f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f865c01a090 tx=0x7f865c01aca0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 shutdown_connections 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f864806f400 0x7f8648071910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f866c072a30 0x7f866c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f866c0b0770 0x7f866c146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 --2- 172.21.15.116:0/3977656295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c06f730 0x7f866c165f20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 >> 172.21.15.116:0/3977656295 conn(0x7f866c0b4620 msgr2=0x7f866c0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 shutdown_connections 2024-05-14T11:21:42.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.753+0000 7f8672e00640 1 -- 172.21.15.116:0/3977656295 wait complete. 2024-05-14T11:21:42.772 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:21:42.773 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:42.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.917+0000 7f0e75000640 1 Processor -- start 2024-05-14T11:21:42.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.917+0000 7f0e75000640 1 -- start start 2024-05-14T11:21:42.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e75000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e700d15e0 0x7f0e700d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e75000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 0x7f0e700cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e75000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700d1f30 con 0x7f0e7006ead0 2024-05-14T11:21:42.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e75000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7006f5f0 con 0x7f0e700d15e0 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e75000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7015ce20 con 0x7f0e700cbaf0 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e700d15e0 0x7f0e700d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e700d15e0 0x7f0e700d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35590/0 (socket says 172.21.15.116:35590) 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 -- 172.21.15.116:0/1903719041 learned_addr learned my addr 172.21.15.116:0/1903719041 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6fe00640 1 --2- 172.21.15.116:0/1903719041 >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 0x7f0e700cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 -- 172.21.15.116:0/1903719041 >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 msgr2=0x7f0e700cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 --2- 172.21.15.116:0/1903719041 >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 0x7f0e700cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 -- 172.21.15.116:0/1903719041 >> v1:172.21.15.149:6789/0 conn(0x7f0e7006ead0 legacy=0x7f0e7006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6f400640 1 -- 172.21.15.116:0/1903719041 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7006f850 con 0x7f0e700d15e0 2024-05-14T11:21:42.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.921+0000 7f0e6fe00640 1 --2- 172.21.15.116:0/1903719041 >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 0x7f0e700cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:42.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e6f400640 1 --2- 172.21.15.116:0/1903719041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700d15e0 0x7f0e700d19f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0e700cb7c0 tx=0x7f0e5801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a301dc479244f790 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e6ea00640 1 -- 172.21.15.116:0/1903719041 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580b4070 con 0x7f0e700d15e0 2024-05-14T11:21:42.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e6ea00640 1 -- 172.21.15.116:0/1903719041 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e580a7cc0 con 0x7f0e700d15e0 2024-05-14T11:21:42.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e6ea00640 1 -- 172.21.15.116:0/1903719041 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580af660 con 0x7f0e700d15e0 2024-05-14T11:21:42.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- 172.21.15.116:0/1903719041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700d15e0 msgr2=0x7f0e700d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 --2- 172.21.15.116:0/1903719041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700d15e0 0x7f0e700d19f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0e700cb7c0 tx=0x7f0e5801ebf0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- 172.21.15.116:0/1903719041 shutdown_connections 2024-05-14T11:21:42.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 --2- 172.21.15.116:0/1903719041 >> v2:172.21.15.149:3300/0 conn(0x7f0e700cbaf0 0x7f0e700cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 --2- 172.21.15.116:0/1903719041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700d15e0 0x7f0e700d19f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- 172.21.15.116:0/1903719041 >> 172.21.15.116:0/1903719041 conn(0x7f0e700b4530 msgr2=0x7f0e700b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:42.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- 172.21.15.116:0/1903719041 shutdown_connections 2024-05-14T11:21:42.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- 172.21.15.116:0/1903719041 wait complete. 2024-05-14T11:21:42.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 Processor -- start 2024-05-14T11:21:42.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.925+0000 7f0e75000640 1 -- start start 2024-05-14T11:21:42.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e7006ead0 0x7f0e70074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e700d15e0 0x7f0e70072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700cdc40 con 0x7f0e700cbaf0 2024-05-14T11:21:42.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700703d0 con 0x7f0e700d15e0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700cddc0 con 0x7f0e7006ead0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35602/0 (socket says 172.21.15.116:35602) 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 -- 172.21.15.116:0/2331975455 learned_addr learned my addr 172.21.15.116:0/2331975455 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e6f400640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e7006ead0 0x7f0e70074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e6fe00640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e700d15e0 0x7f0e70072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 -- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e7006ead0 msgr2=0x7f0e70074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e7006ead0 0x7f0e70074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 -- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e700d15e0 msgr2=0x7f0e70072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e700d15e0 0x7f0e70072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 -- 172.21.15.116:0/2331975455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70073860 con 0x7f0e700cbaf0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e66a00640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c3c0 tx=0x7f0e6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60017070 con 0x7f0e700cbaf0 2024-05-14T11:21:42.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600052d0 con 0x7f0e700cbaf0 2024-05-14T11:21:42.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60003820 con 0x7f0e700cbaf0 2024-05-14T11:21:42.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e70073b40 con 0x7f0e700cbaf0 2024-05-14T11:21:42.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e70074100 con 0x7f0e700cbaf0 2024-05-14T11:21:42.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.929+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e700735b0 con 0x7f0e700cbaf0 2024-05-14T11:21:42.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.933+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e6001a070 con 0x7f0e700cbaf0 2024-05-14T11:21:42.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.933+0000 7f0e67e00640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 0x7f0e4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:42.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.933+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f0e6008e7d0 con 0x7f0e700cbaf0 2024-05-14T11:21:42.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.933+0000 7f0e6f400640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 0x7f0e4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:42.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.933+0000 7f0e6f400640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 0x7f0e4c071910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e580b1000 tx=0x7f0e580a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:42.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:42.937+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0e700d1f30 con 0x7f0e700cbaf0 2024-05-14T11:21:43.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e70050880 con 0x7f0e700cbaf0 2024-05-14T11:21:43.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e67e00640 1 -- 172.21.15.116:0/2331975455 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f0e60005480 con 0x7f0e700cbaf0 2024-05-14T11:21:43.128 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:43.128 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:21:11.017738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":16,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:43.128 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 53 2024-05-14T11:21:43.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 msgr2=0x7f0e4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 0x7f0e4c071910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e580b1000 tx=0x7f0e580a9070 comp rx=0 tx=0).stop 2024-05-14T11:21:43.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 msgr2=0x7f0e70158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c3c0 tx=0x7f0e6000c880 comp rx=0 tx=0).stop 2024-05-14T11:21:43.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 shutdown_connections 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0e4c06f400 0x7f0e4c071910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e700d15e0 0x7f0e70072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e700cbaf0 0x7f0e70158040 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 --2- 172.21.15.116:0/2331975455 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0e7006ead0 0x7f0e70074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 >> 172.21.15.116:0/2331975455 conn(0x7f0e700b4530 msgr2=0x7f0e7006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 shutdown_connections 2024-05-14T11:21:43.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.129+0000 7f0e75000640 1 -- 172.21.15.116:0/2331975455 wait complete. 2024-05-14T11:21:43.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:43.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 Processor -- start 2024-05-14T11:21:43.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 -- start start 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c4006ead0 0x7f1c4006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400d1f30 con 0x7f1c4006c700 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1c4015ce20 con 0x7f1c4006ead0 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4006f850 con 0x7f1c400d15e0 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c4006ead0 0x7f1c4006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1c4006c700 0x7f1c4006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47788/0 (socket says 172.21.15.116:47788) 2024-05-14T11:21:43.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c45400640 1 -- 172.21.15.116:0/1490706419 learned_addr learned my addr 172.21.15.116:0/1490706419 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c44a00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3fe00640 1 -- 172.21.15.116:0/1490706419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 msgr2=0x7f1c400d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3fe00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3fe00640 1 -- 172.21.15.116:0/1490706419 >> v1:172.21.15.116:6789/0 conn(0x7f1c4006c700 legacy=0x7f1c4006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3fe00640 1 -- 172.21.15.116:0/1490706419 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4006fb30 con 0x7f1c4006ead0 2024-05-14T11:21:43.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.297+0000 7f1c3f400640 1 -- 172.21.15.116:0/1490706419 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982860650 0 0) 0x7f1c38001120 con 0x7f1c4006c700 2024-05-14T11:21:43.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c3fe00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006ead0 0x7f1c4006eee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0097c0 tx=0x7f1c2c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f1c022a536fbc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c3f400640 1 -- 172.21.15.116:0/1490706419 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0aa600 con 0x7f1c4006ead0 2024-05-14T11:21:43.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c3f400640 1 -- 172.21.15.116:0/1490706419 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c0af070 con 0x7f1c4006ead0 2024-05-14T11:21:43.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c3f400640 1 -- 172.21.15.116:0/1490706419 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0aa9c0 con 0x7f1c4006ead0 2024-05-14T11:21:43.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 -- 172.21.15.116:0/1490706419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006ead0 msgr2=0x7f1c4006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006ead0 0x7f1c4006eee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0097c0 tx=0x7f1c2c01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 -- 172.21.15.116:0/1490706419 shutdown_connections 2024-05-14T11:21:43.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006ead0 0x7f1c4006eee0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 --2- 172.21.15.116:0/1490706419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 -- 172.21.15.116:0/1490706419 >> 172.21.15.116:0/1490706419 conn(0x7f1c400b4530 msgr2=0x7f1c400b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:43.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 -- 172.21.15.116:0/1490706419 shutdown_connections 2024-05-14T11:21:43.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.301+0000 7f1c45e00640 1 -- 172.21.15.116:0/1490706419 wait complete. 2024-05-14T11:21:43.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 Processor -- start 2024-05-14T11:21:43.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 -- start start 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c4006ead0 0x7f1c400c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400bde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4006f5a0 con 0x7f1c4006c700 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4006f420 con 0x7f1c4006ead0 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1c40169d80 con 0x7f1c400d15e0 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35630/0 (socket says 172.21.15.116:35630) 2024-05-14T11:21:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 -- 172.21.15.116:0/3206337546 learned_addr learned my addr 172.21.15.116:0/3206337546 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c45400640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c3fe00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c4006ead0 0x7f1c400c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 -- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 msgr2=0x7f1c400bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 -- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c4006ead0 msgr2=0x7f1c400c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c4006ead0 0x7f1c400c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.305+0000 7f1c44a00640 1 -- 172.21.15.116:0/3206337546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400ca2c0 con 0x7f1c4006c700 2024-05-14T11:21:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c44a00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1c30007e40 tx=0x7f1c3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30014620 con 0x7f1c4006c700 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c30019070 con 0x7f1c4006c700 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c400ca5a0 con 0x7f1c4006c700 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30014a00 con 0x7f1c4006c700 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c400cab60 con 0x7f1c4006c700 2024-05-14T11:21:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c400ca050 con 0x7f1c4006c700 2024-05-14T11:21:43.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c30014ba0 con 0x7f1c4006c700 2024-05-14T11:21:43.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 0x7f1c20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.309+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f1c3008fc70 con 0x7f1c4006c700 2024-05-14T11:21:43.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.313+0000 7f1c3fe00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 0x7f1c20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.313+0000 7f1c3fe00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 0x7f1c20071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ae000 tx=0x7f1c2c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.317+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1c3005c290 con 0x7f1c4006c700 2024-05-14T11:21:43.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.509+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c40050880 con 0x7f1c4006c700 2024-05-14T11:21:43.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.509+0000 7f1c3cc00640 1 -- 172.21.15.116:0/3206337546 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f1c30060590 con 0x7f1c4006c700 2024-05-14T11:21:43.510 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:43.510 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:20:03.585967+0000","modified":"2024-05-14T11:21:11.017738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6396":{"gid":6396,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":16,"addr":"172.21.15.116:6836/2154548589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2154548589},{"type":"v1","addr":"172.21.15.116:6836","nonce":2154548589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6396,"qdb_cluster":[6396]},"id":5}]} 2024-05-14T11:21:43.511 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 53 2024-05-14T11:21:43.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 msgr2=0x7f1c20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 0x7f1c20071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ae000 tx=0x7f1c2c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 msgr2=0x7f1c400c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1c30007e40 tx=0x7f1c3000e4b0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 shutdown_connections 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1c2006f430 0x7f1c20071940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c4006ead0 0x7f1c400c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1c400d15e0 0x7f1c400bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 --2- 172.21.15.116:0/3206337546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4006c700 0x7f1c400c8290 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 >> 172.21.15.116:0/3206337546 conn(0x7f1c400b4530 msgr2=0x7f1c400b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 shutdown_connections 2024-05-14T11:21:43.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.513+0000 7f1c45e00640 1 -- 172.21.15.116:0/3206337546 wait complete. 2024-05-14T11:21:43.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:43.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 Processor -- start 2024-05-14T11:21:43.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 -- start start 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c3c164220 0x7f4c3c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 0x7f4c3c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 0x7f4c3c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3c06d590 con 0x7f4c3c164220 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3c0d1fc0 con 0x7f4c3c06cc40 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c40a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c1600a0 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.677+0000 7f4c39e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c3c164220 0x7f4c3c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c3c164220 0x7f4c3c0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35638/0 (socket says 172.21.15.116:35638) 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 -- 172.21.15.116:0/3645123270 learned_addr learned my addr 172.21.15.116:0/3645123270 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c31400640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 0x7f4c3c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 -- 172.21.15.116:0/3645123270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 msgr2=0x7f4c3c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c3a800640 1 --2- 172.21.15.116:0/3645123270 >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 0x7f4c3c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 0x7f4c3c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 -- 172.21.15.116:0/3645123270 >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 msgr2=0x7f4c3c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 --2- 172.21.15.116:0/3645123270 >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 0x7f4c3c06d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 -- 172.21.15.116:0/3645123270 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c06f5d0 con 0x7f4c3c164220 2024-05-14T11:21:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c31400640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 0x7f4c3c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c3a800640 1 --2- 172.21.15.116:0/3645123270 >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 0x7f4c3c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:43.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39e00640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c164220 0x7f4c3c0d1670 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c072800 tx=0x7f4c2c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd46b727a62ddd07 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39400640 1 -- 172.21.15.116:0/3645123270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5840 con 0x7f4c3c164220 2024-05-14T11:21:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39400640 1 -- 172.21.15.116:0/3645123270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0a59e0 con 0x7f4c3c164220 2024-05-14T11:21:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.681+0000 7f4c39400640 1 -- 172.21.15.116:0/3645123270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5db0 con 0x7f4c3c164220 2024-05-14T11:21:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- 172.21.15.116:0/3645123270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c164220 msgr2=0x7f4c3c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c164220 0x7f4c3c0d1670 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c072800 tx=0x7f4c2c01c9c0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- 172.21.15.116:0/3645123270 shutdown_connections 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 --2- 172.21.15.116:0/3645123270 >> v2:172.21.15.149:3300/0 conn(0x7f4c3c06cc40 0x7f4c3c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c0d1bb0 0x7f4c3c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 --2- 172.21.15.116:0/3645123270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c164220 0x7f4c3c0d1670 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- 172.21.15.116:0/3645123270 >> 172.21.15.116:0/3645123270 conn(0x7f4c3c0b4620 msgr2=0x7f4c3c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- 172.21.15.116:0/3645123270 shutdown_connections 2024-05-14T11:21:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- 172.21.15.116:0/3645123270 wait complete. 2024-05-14T11:21:43.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 Processor -- start 2024-05-14T11:21:43.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.685+0000 7f4c40a00640 1 -- start start 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c06cc40 0x7f4c3c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 0x7f4c3c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c073650 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c070150 con 0x7f4c3c164220 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c164630 con 0x7f4c3c06cc40 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35648/0 (socket says 172.21.15.116:35648) 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 -- 172.21.15.116:0/1148653068 learned_addr learned my addr 172.21.15.116:0/1148653068 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c3a800640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 0x7f4c3c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c39e00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c06cc40 0x7f4c3c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 -- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c06cc40 msgr2=0x7f4c3c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c06cc40 0x7f4c3c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 -- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 msgr2=0x7f4c3c150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 0x7f4c3c150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 -- 172.21.15.116:0/1148653068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c0b2390 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c3a800640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 0x7f4c3c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:43.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c31400640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c820 tx=0x7f4c2400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24015070 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24007a30 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24007e40 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c159c90 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c15a250 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c06a9c0 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.689+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c24018070 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.693+0000 7f4c32a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 0x7f4c18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.693+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7f4c2408f300 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.693+0000 7f4c39e00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 0x7f4c18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.693+0000 7f4c39e00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 0x7f4c18071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ae000 tx=0x7f4c2c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.697+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4c2405b9b0 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c3c0b2120 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c32a00640 1 -- 172.21.15.116:0/1148653068 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17124 (secure 0 0 0) 0x7f4c2405fcb0 con 0x7f4c3c0d1bb0 2024-05-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:43.857 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":73,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:20:19.998776+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:20:01.282761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-14T11:20:02.552345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6836/2430466110":"2024-05-15T11:20:19.998743+0000","172.21.15.116:6834/2430466110":"2024-05-15T11:20:19.998743+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 msgr2=0x7f4c18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 0x7f4c18071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ae000 tx=0x7f4c2c0a7070 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 msgr2=0x7f4c3c0af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c820 tx=0x7f4c2400cce0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 shutdown_connections 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4c1806f400 0x7f4c18071910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c3c164220 0x7f4c3c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c3c0d1bb0 0x7f4c3c0af770 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 --2- 172.21.15.116:0/1148653068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4c3c06cc40 0x7f4c3c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 >> 172.21.15.116:0/1148653068 conn(0x7f4c3c0b4620 msgr2=0x7f4c3c15d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:43.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 shutdown_connections 2024-05-14T11:21:43.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:43.857+0000 7f4c40a00640 1 -- 172.21.15.116:0/1148653068 wait complete. 2024-05-14T11:21:43.879 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:21:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.025+0000 7ff2ff800640 1 Processor -- start 2024-05-14T11:21:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.025+0000 7ff2ff800640 1 -- start start 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2ff800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2f80b0390 0x7ff2f8167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2ff800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f8160510 con 0x7ff2f8156fa0 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2ff800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f81607f0 con 0x7ff2f80d1670 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2ff800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff2f80706f0 con 0x7ff2f80b0390 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fce00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff2f8156fa0 0x7ff2f806ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47798/0 (socket says 172.21.15.116:47798) 2024-05-14T11:21:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fce00640 1 -- 172.21.15.116:0/3952967128 learned_addr learned my addr 172.21.15.116:0/3952967128 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fd800640 1 --2- 172.21.15.116:0/3952967128 >> v2:172.21.15.116:3300/0 conn(0x7ff2f80b0390 0x7ff2f8167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2f7400640 1 -- 172.21.15.116:0/3952967128 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653149990 0 0) 0x7ff2f8160510 con 0x7ff2f8156fa0 2024-05-14T11:21:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2f7400640 1 -- 172.21.15.116:0/3952967128 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2dc003530 con 0x7ff2f8156fa0 2024-05-14T11:21:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fd800640 1 -- 172.21.15.116:0/3952967128 >> v1:172.21.15.149:6789/0 conn(0x7ff2f80d1670 legacy=0x7ff2f80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fd800640 1 -- 172.21.15.116:0/3952967128 >> v1:172.21.15.116:6789/0 conn(0x7ff2f8156fa0 legacy=0x7ff2f806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.029+0000 7ff2fd800640 1 -- 172.21.15.116:0/3952967128 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f8160ad0 con 0x7ff2f80b0390 2024-05-14T11:21:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2fd800640 1 --2- 172.21.15.116:0/3952967128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80b0390 0x7ff2f8167800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009d90 tx=0x7ff2f001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b1cc2c7ee328b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2f7400640 1 -- 172.21.15.116:0/3952967128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00b2070 con 0x7ff2f80b0390 2024-05-14T11:21:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2f7400640 1 -- 172.21.15.116:0/3952967128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f00a5830 con 0x7ff2f80b0390 2024-05-14T11:21:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2f7400640 1 -- 172.21.15.116:0/3952967128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00a5c00 con 0x7ff2f80b0390 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 -- 172.21.15.116:0/3952967128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80b0390 msgr2=0x7ff2f8167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 --2- 172.21.15.116:0/3952967128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80b0390 0x7ff2f8167800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009d90 tx=0x7ff2f001c970 comp rx=0 tx=0).stop 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 -- 172.21.15.116:0/3952967128 shutdown_connections 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 --2- 172.21.15.116:0/3952967128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80b0390 0x7ff2f8167800 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 -- 172.21.15.116:0/3952967128 >> 172.21.15.116:0/3952967128 conn(0x7ff2f80b4620 msgr2=0x7ff2f80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 -- 172.21.15.116:0/3952967128 shutdown_connections 2024-05-14T11:21:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.033+0000 7ff2ff800640 1 -- 172.21.15.116:0/3952967128 wait complete. 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 Processor -- start 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- start start 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f80b0390 0x7ff2f814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 0x7ff2f815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f816ad90 con 0x7ff2f80d1670 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f80b07a0 con 0x7ff2f80b0390 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f816af10 con 0x7ff2f8156fa0 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35660/0 (socket says 172.21.15.116:35660) 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 -- 172.21.15.116:0/2339850726 learned_addr learned my addr 172.21.15.116:0/2339850726 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2fce00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f80b0390 0x7ff2f814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2fd800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 0x7ff2f815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 -- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 msgr2=0x7ff2f815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 0x7ff2f815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 -- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f80b0390 msgr2=0x7ff2f814eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f80b0390 0x7ff2f814eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 -- 172.21.15.116:0/2339850726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f8155bb0 con 0x7ff2f80d1670 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2fd800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 0x7ff2f815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f7e00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c3c0 tx=0x7ff2e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4017070 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e40052d0 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4003820 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f8155e90 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.037+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f8156450 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f8155900 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2e401a070 con 0x7ff2f80d1670 2024-05-14T11:21:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2f4c00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 0x7ff2d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6187+0+0 (secure 0 0 0) 0x7ff2e408e780 con 0x7ff2f80d1670 2024-05-14T11:21:44.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2fce00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 0x7ff2d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.041+0000 7ff2fce00640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 0x7ff2d8071940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8007a70 tx=0x7ff2e80079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.045+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2e4005480 con 0x7ff2f80d1670 2024-05-14T11:21:44.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.233+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff2f8156690 con 0x7ff2f80d1670 2024-05-14T11:21:44.235 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:21:44.233+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:21:44.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.237+0000 7ff2f4c00640 1 -- 172.21.15.116:0/2339850726 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7ff2e405ada0 con 0x7ff2f80d1670 2024-05-14T11:21:44.237 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 msgr2=0x7ff2d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 0x7ff2d8071940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8007a70 tx=0x7ff2e80079c0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 msgr2=0x7ff2f8155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c3c0 tx=0x7ff2e400c880 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 shutdown_connections 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2d806f430 0x7ff2d8071940 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f8156fa0 0x7ff2f815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2f80d1670 0x7ff2f8155010 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 --2- 172.21.15.116:0/2339850726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f80b0390 0x7ff2f814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 >> 172.21.15.116:0/2339850726 conn(0x7ff2f80b4620 msgr2=0x7ff2f8153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:44.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 shutdown_connections 2024-05-14T11:21:44.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.241+0000 7ff2ff800640 1 -- 172.21.15.116:0/2339850726 wait complete. 2024-05-14T11:21:44.250 INFO:tasks.ceph.mds.c.smithi116.stderr: -4382> 2024-05-14T11:20:26.010+0000 7fe335000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:21:44.262 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:21:44.377 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:21:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.413+0000 7fc766600640 1 Processor -- start 2024-05-14T11:21:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.413+0000 7fc766600640 1 -- start start 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc766600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc76006f730 0x7fc76006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc766600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc760070080 con 0x7fc760072a30 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc766600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76006cf10 con 0x7fc7600b0770 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc766600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc76006db20 con 0x7fc76006f730 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc75fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc760072a30 0x7fc760072e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47800/0 (socket says 172.21.15.116:47800) 2024-05-14T11:21:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc75fe00640 1 -- 172.21.15.116:0/2943901308 learned_addr learned my addr 172.21.15.116:0/2943901308 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc756a00640 1 --2- 172.21.15.116:0/2943901308 >> v2:172.21.15.149:3300/0 conn(0x7fc76006f730 0x7fc76006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc75ea00640 1 -- 172.21.15.116:0/2943901308 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239306623 0 0) 0x7fc7580014a0 con 0x7fc760072a30 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc75ea00640 1 -- 172.21.15.116:0/2943901308 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc744003450 con 0x7fc760072a30 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc756a00640 1 -- 172.21.15.116:0/2943901308 >> v1:172.21.15.149:6789/0 conn(0x7fc7600b0770 legacy=0x7fc76006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc756a00640 1 -- 172.21.15.116:0/2943901308 >> v1:172.21.15.116:6789/0 conn(0x7fc760072a30 legacy=0x7fc760072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.417+0000 7fc756a00640 1 -- 172.21.15.116:0/2943901308 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76006d1f0 con 0x7fc76006f730 2024-05-14T11:21:44.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc756a00640 1 --2- 172.21.15.116:0/2943901308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76006f730 0x7fc76006fb40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc7480097c0 tx=0x7fc74801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a45a4495b34cf00 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc75ea00640 1 -- 172.21.15.116:0/2943901308 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480b2070 con 0x7fc76006f730 2024-05-14T11:21:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc75ea00640 1 -- 172.21.15.116:0/2943901308 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7480a9070 con 0x7fc76006f730 2024-05-14T11:21:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc75ea00640 1 -- 172.21.15.116:0/2943901308 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480ad860 con 0x7fc76006f730 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 -- 172.21.15.116:0/2943901308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76006f730 msgr2=0x7fc76006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 --2- 172.21.15.116:0/2943901308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76006f730 0x7fc76006fb40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc7480097c0 tx=0x7fc74801c3e0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 -- 172.21.15.116:0/2943901308 shutdown_connections 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 --2- 172.21.15.116:0/2943901308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc76006f730 0x7fc76006fb40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 -- 172.21.15.116:0/2943901308 >> 172.21.15.116:0/2943901308 conn(0x7fc7600b4620 msgr2=0x7fc7600b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 -- 172.21.15.116:0/2943901308 shutdown_connections 2024-05-14T11:21:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.421+0000 7fc766600640 1 -- 172.21.15.116:0/2943901308 wait complete. 2024-05-14T11:21:44.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 Processor -- start 2024-05-14T11:21:44.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- start start 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 0x7fc7600d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc760072a30 0x7fc760158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc760073ae0 con 0x7fc7600b0770 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc760073960 con 0x7fc760072a30 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc760169fb0 con 0x7fc76006f730 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35680/0 (socket says 172.21.15.116:35680) 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 -- 172.21.15.116:0/16406604 learned_addr learned my addr 172.21.15.116:0/16406604 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc756a00640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc760072a30 0x7fc760158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75f400640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 0x7fc7600d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 -- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 msgr2=0x7fc7600d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 0x7fc7600d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 -- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc760072a30 msgr2=0x7fc760158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc760072a30 0x7fc760158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 -- 172.21.15.116:0/16406604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760156e70 con 0x7fc7600b0770 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75f400640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 0x7fc7600d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc75fe00640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc75800ca70 tx=0x7fc75800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75800abd0 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75800ad70 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7601fca60 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758012600 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.425+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7601fd240 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7601fccd0 con 0x7fc7600b0770 2024-05-14T11:21:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc7580127a0 con 0x7fc7600b0770 2024-05-14T11:21:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc757e00640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 0x7fc73c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6273+0+0 (secure 0 0 0) 0x7fc75808fa70 con 0x7fc7600b0770 2024-05-14T11:21:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc75f400640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 0x7fc73c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.429+0000 7fc75f400640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 0x7fc73c071940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc750000f70 tx=0x7fc750009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.433+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc760070080 con 0x7fc7600b0770 2024-05-14T11:21:44.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:44.625+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc76006a9c0 con 0x7fc7600b0770 2024-05-14T11:21:45.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc757e00640 1 -- 172.21.15.116:0/16406604 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7fc75805c680 con 0x7fc7600b0770 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 msgr2=0x7fc73c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 0x7fc73c071940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc750000f70 tx=0x7fc750009040 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 msgr2=0x7fc760158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc75800ca70 tx=0x7fc75800cf30 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 shutdown_connections 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc73c06f430 0x7fc73c071940 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc760072a30 0x7fc760158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7600b0770 0x7fc760158c60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 --2- 172.21.15.116:0/16406604 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc76006f730 0x7fc7600d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 >> 172.21.15.116:0/16406604 conn(0x7fc7600b4620 msgr2=0x7fc7600b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:45.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 shutdown_connections 2024-05-14T11:21:45.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.281+0000 7fc766600640 1 -- 172.21.15.116:0/16406604 wait complete. 2024-05-14T11:21:45.305 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:21:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.457+0000 7f7474400640 1 Processor -- start 2024-05-14T11:21:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.457+0000 7f7474400640 1 -- start start 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.457+0000 7f7474400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f746c164260 0x7f746c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.457+0000 7f7474400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0b5750 0x7f746c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7474400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c0b60a0 con 0x7f746c0af380 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7474400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f746c06c700 con 0x7f746c164260 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7474400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f746c06c9a0 con 0x7f746c0b5750 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f746c164260 0x7f746c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f746c164260 0x7f746c06da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35696/0 (socket says 172.21.15.116:35696) 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 -- 172.21.15.116:0/1433979785 learned_addr learned my addr 172.21.15.116:0/1433979785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471000640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0b5750 0x7f746c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 -- 172.21.15.116:0/1433979785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0b5750 msgr2=0x7f746c0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0b5750 0x7f746c0b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 -- 172.21.15.116:0/1433979785 >> v1:172.21.15.149:6789/0 conn(0x7f746c0af380 legacy=0x7f746c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 -- 172.21.15.116:0/1433979785 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c06cc80 con 0x7f746c164260 2024-05-14T11:21:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f7471a00640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c06da40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7460009e50 tx=0x7f746001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3211da879c33543 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f746be00640 1 -- 172.21.15.116:0/1433979785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74600a58d0 con 0x7f746c164260 2024-05-14T11:21:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f746be00640 1 -- 172.21.15.116:0/1433979785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74600a5a70 con 0x7f746c164260 2024-05-14T11:21:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.461+0000 7f746be00640 1 -- 172.21.15.116:0/1433979785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74600a5e40 con 0x7f746c164260 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- 172.21.15.116:0/1433979785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 msgr2=0x7f746c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c06da40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7460009e50 tx=0x7f746001ca00 comp rx=0 tx=0).stop 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- 172.21.15.116:0/1433979785 shutdown_connections 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0b5750 0x7f746c0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 --2- 172.21.15.116:0/1433979785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c06da40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- 172.21.15.116:0/1433979785 >> 172.21.15.116:0/1433979785 conn(0x7f746c0cde80 msgr2=0x7f746c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- 172.21.15.116:0/1433979785 shutdown_connections 2024-05-14T11:21:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- 172.21.15.116:0/1433979785 wait complete. 2024-05-14T11:21:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 Processor -- start 2024-05-14T11:21:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.465+0000 7f7474400640 1 -- start start 2024-05-14T11:21:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0af380 0x7f746c140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f746c0b5750 0x7f746c15b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f746c164670 con 0x7f746c164260 2024-05-14T11:21:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f746c06df80 con 0x7f746c0b5750 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f746c162d40 con 0x7f746c0af380 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35708/0 (socket says 172.21.15.116:35708) 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 -- 172.21.15.116:0/4053622279 learned_addr learned my addr 172.21.15.116:0/4053622279 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7471a00640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0af380 0x7f746c140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7471000640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f746c0b5750 0x7f746c15b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 -- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0af380 msgr2=0x7f746c140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0af380 0x7f746c140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 -- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f746c0b5750 msgr2=0x7f746c15b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f746c0b5750 0x7f746c15b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 -- 172.21.15.116:0/4053622279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c162f90 con 0x7f746c164260 2024-05-14T11:21:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7472400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f746400c3c0 tx=0x7f746400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464017070 con 0x7f746c164260 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74640052d0 con 0x7f746c164260 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c153bd0 con 0x7f746c164260 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74640054c0 con 0x7f746c164260 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c154190 con 0x7f746c164260 2024-05-14T11:21:45.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.469+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c06aa30 con 0x7f746c164260 2024-05-14T11:21:45.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.473+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f746401a070 con 0x7f746c164260 2024-05-14T11:21:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.473+0000 7f7469600640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 0x7f744c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.473+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6273+0+0 (secure 0 0 0) 0x7f746408e800 con 0x7f746c164260 2024-05-14T11:21:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.473+0000 7f7471a00640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 0x7f744c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:45.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.473+0000 7f7471a00640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 0x7f744c071940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74600ae000 tx=0x7f74600a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.477+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f746c0b60a0 con 0x7f746c164260 2024-05-14T11:21:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:45.637+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f746c0b1260 con 0x7f746c164260 2024-05-14T11:21:46.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.281+0000 7f7469600640 1 -- 172.21.15.116:0/4053622279 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7f746405ae50 con 0x7f746c164260 2024-05-14T11:21:46.283 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:21:46.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 msgr2=0x7f744c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 0x7f744c071940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74600ae000 tx=0x7f74600a7070 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 msgr2=0x7f746c1635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f746400c3c0 tx=0x7f746400c880 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 shutdown_connections 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f744c06f430 0x7f744c071940 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f746c0b5750 0x7f746c15b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c164260 0x7f746c1635c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 --2- 172.21.15.116:0/4053622279 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f746c0af380 0x7f746c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 >> 172.21.15.116:0/4053622279 conn(0x7f746c0cde80 msgr2=0x7f746c15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 shutdown_connections 2024-05-14T11:21:46.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.285+0000 7f7474400640 1 -- 172.21.15.116:0/4053622279 wait complete. 2024-05-14T11:21:46.310 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:21:46.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 Processor -- start 2024-05-14T11:21:46.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 -- start start 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb8b8070640 0x7fb8b806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b815d360 con 0x7fb8b81421f0 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b815dc90 con 0x7fb8b8070640 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8c0800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b815d640 con 0x7fb8b8070a50 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8be800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb8b81421f0 0x7fb8b8142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47806/0 (socket says 172.21.15.116:47806) 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8be800640 1 -- 172.21.15.116:0/1399809473 learned_addr learned my addr 172.21.15.116:0/1399809473 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8bd400640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8bde00640 1 --2- 172.21.15.116:0/1399809473 >> v2:172.21.15.149:3300/0 conn(0x7fb8b8070640 0x7fb8b806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195652495 0 0) 0x7fb8b815d360 con 0x7fb8b81421f0 2024-05-14T11:21:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8a0003530 con 0x7fb8b81421f0 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.469+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1624121692 0 0) 0x7fb8a0003530 con 0x7fb8b81421f0 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 msgr2=0x7fb8b806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8b806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 >> v2:172.21.15.149:3300/0 conn(0x7fb8b8070640 msgr2=0x7fb8b806e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 --2- 172.21.15.116:0/1399809473 >> v2:172.21.15.149:3300/0 conn(0x7fb8b8070640 0x7fb8b806e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b815d920 con 0x7fb8b81421f0 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fb8b4001a80 con 0x7fb8b81421f0 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 >> v1:172.21.15.116:6789/0 conn(0x7fb8b81421f0 legacy=0x7fb8b8142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a0003a10 0x7fb8a0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a0004810 0x7fb8a0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8a0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b815dc90 con 0x7fb8a0003a10 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b815d640 con 0x7fb8a0004810 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a000dce0 con 0x7fb8b8070a50 2024-05-14T11:21:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a0003a10 0x7fb8a0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bd400640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a0004810 0x7fb8a0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8be800640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8a0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 -- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 msgr2=0x7fb8a0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8a0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 -- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a0004810 msgr2=0x7fb8a0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a0004810 0x7fb8a0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 -- 172.21.15.116:0/1399809473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a0003530 con 0x7fb8a0003a10 2024-05-14T11:21:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8be800640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8b8070a50 0x7fb8a0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:46.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bde00640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a0003a10 0x7fb8a0003e80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8b81467b0 tx=0x7fb8ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71c7da9e28dceb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:46.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0096b0 con 0x7fb8a0003a10 2024-05-14T11:21:46.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ac0a64f0 con 0x7fb8a0003a10 2024-05-14T11:21:46.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.473+0000 7fb8bca00640 1 -- 172.21.15.116:0/1399809473 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0a68a0 con 0x7fb8a0003a10 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- 172.21.15.116:0/1399809473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a0003a10 msgr2=0x7fb8a0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 --2- 172.21.15.116:0/1399809473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a0003a10 0x7fb8a0003e80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8b81467b0 tx=0x7fb8ac01c9f0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8be800640 1 -- 172.21.15.116:0/1399809473 reap_dead start 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- 172.21.15.116:0/1399809473 shutdown_connections 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- 172.21.15.116:0/1399809473 >> 172.21.15.116:0/1399809473 conn(0x7fb8b80cde80 msgr2=0x7fb8b80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- 172.21.15.116:0/1399809473 shutdown_connections 2024-05-14T11:21:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- 172.21.15.116:0/1399809473 wait complete. 2024-05-14T11:21:46.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 Processor -- start 2024-05-14T11:21:46.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.477+0000 7fb8c0800640 1 -- start start 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8a0003a10 0x7fb8b8163240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8b81576d0 0x7fb8b813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b806edb0 con 0x7fb8b81452f0 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b80cda10 con 0x7fb8b81576d0 2024-05-14T11:21:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b806ef30 con 0x7fb8a0003a10 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35722/0 (socket says 172.21.15.116:35722) 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 -- 172.21.15.116:0/2891831599 learned_addr learned my addr 172.21.15.116:0/2891831599 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8bde00640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8a0003a10 0x7fb8b8163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8bd400640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8b81576d0 0x7fb8b813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 -- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8a0003a10 msgr2=0x7fb8b8163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8a0003a10 0x7fb8b8163240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 -- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8b81576d0 msgr2=0x7fb8b813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8b81576d0 0x7fb8b813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 -- 172.21.15.116:0/2891831599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b8146230 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8be800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb8b806cd00 tx=0x7fb8b400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400eb30 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b400ecd0 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b80b17e0 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4013420 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b8162050 con 0x7fb8b81452f0 2024-05-14T11:21:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.481+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b806aa30 con 0x7fb8b81452f0 2024-05-14T11:21:46.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.485+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb8b401a020 con 0x7fb8b81452f0 2024-05-14T11:21:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.485+0000 7fb8b2000640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 0x7fb89c0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.485+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5835+0+0 (secure 0 0 0) 0x7fb8b408ebc0 con 0x7fb8b81452f0 2024-05-14T11:21:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.485+0000 7fb8bde00640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 0x7fb89c0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:46.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.485+0000 7fb8bde00640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 0x7fb89c0718b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0af000 tx=0x7fb8ac0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.489+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb8b4021020 con 0x7fb8b81452f0 2024-05-14T11:21:46.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:46.645+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8b806cd30 con 0x7fb8b81452f0 2024-05-14T11:21:47.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.301+0000 7fb8b2000640 1 -- 172.21.15.116:0/2891831599 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7fb8b405bab0 con 0x7fb8b81452f0 2024-05-14T11:21:47.302 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:21:47.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 msgr2=0x7fb89c0718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 0x7fb89c0718b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0af000 tx=0x7fb8ac0046d0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 msgr2=0x7fb8b8145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb8b806cd00 tx=0x7fb8b400e530 comp rx=0 tx=0).stop 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 shutdown_connections 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb89c06f3a0 0x7fb89c0718b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8b81576d0 0x7fb8b813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b81452f0 0x7fb8b8145700 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 --2- 172.21.15.116:0/2891831599 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8a0003a10 0x7fb8b8163240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 >> 172.21.15.116:0/2891831599 conn(0x7fb8b80cde80 msgr2=0x7fb8b80d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:47.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 shutdown_connections 2024-05-14T11:21:47.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.305+0000 7fb8c0800640 1 -- 172.21.15.116:0/2891831599 wait complete. 2024-05-14T11:21:47.330 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:21:47.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 Processor -- start 2024-05-14T11:21:47.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 -- start start 2024-05-14T11:21:47.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 0x7f01e806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f01e80af380 0x7f01e80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e8160510 con 0x7f01e80d1670 2024-05-14T11:21:47.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f01e81573b0 con 0x7f01e80af380 2024-05-14T11:21:47.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01f0200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01e8161120 con 0x7f01e8156fa0 2024-05-14T11:21:47.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ece00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f01e80d1670 0x7f01e80d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:47808/0 (socket says 172.21.15.116:47808) 2024-05-14T11:21:47.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ece00640 1 -- 172.21.15.116:0/1536162403 learned_addr learned my addr 172.21.15.116:0/1536162403 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:47.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ed800640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 0x7f01e806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ee200640 1 --2- 172.21.15.116:0/1536162403 >> v2:172.21.15.149:3300/0 conn(0x7f01e80af380 0x7f01e80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01e7e00640 1 -- 172.21.15.116:0/1536162403 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099217806 0 0) 0x7f01e8160510 con 0x7f01e80d1670 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01e7e00640 1 -- 172.21.15.116:0/1536162403 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01d0003530 con 0x7f01e80d1670 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ee200640 1 -- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 msgr2=0x7f01e806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ee200640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 0x7f01e806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ee200640 1 -- 172.21.15.116:0/1536162403 >> v1:172.21.15.116:6789/0 conn(0x7f01e80d1670 legacy=0x7f01e80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ee200640 1 -- 172.21.15.116:0/1536162403 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e81607f0 con 0x7f01e80af380 2024-05-14T11:21:47.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.481+0000 7f01ed800640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 0x7f01e806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:21:47.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01ee200640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e80af380 0x7f01e80af790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009d90 tx=0x7f01e001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d15d62214e5fa26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01e7e00640 1 -- 172.21.15.116:0/1536162403 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00b2070 con 0x7f01e80af380 2024-05-14T11:21:47.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01e7e00640 1 -- 172.21.15.116:0/1536162403 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e00a5a00 con 0x7f01e80af380 2024-05-14T11:21:47.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01e7e00640 1 -- 172.21.15.116:0/1536162403 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00a5dd0 con 0x7f01e80af380 2024-05-14T11:21:47.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 -- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e80af380 msgr2=0x7f01e80af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e80af380 0x7f01e80af790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009d90 tx=0x7f01e001c940 comp rx=0 tx=0).stop 2024-05-14T11:21:47.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 -- 172.21.15.116:0/1536162403 shutdown_connections 2024-05-14T11:21:47.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e80af380 0x7f01e80af790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 --2- 172.21.15.116:0/1536162403 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e8156fa0 0x7f01e806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 -- 172.21.15.116:0/1536162403 >> 172.21.15.116:0/1536162403 conn(0x7f01e80b4620 msgr2=0x7f01e80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:47.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 -- 172.21.15.116:0/1536162403 shutdown_connections 2024-05-14T11:21:47.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.485+0000 7f01f0200640 1 -- 172.21.15.116:0/1536162403 wait complete. 2024-05-14T11:21:47.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 Processor -- start 2024-05-14T11:21:47.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 -- start start 2024-05-14T11:21:47.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 0x7f01e8073cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 0x7f01e815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80b0660 con 0x7f01e80af380 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01e80b04e0 con 0x7f01e8156fa0 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01f0200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01e806c700 con 0x7f01e80d1670 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35724/0 (socket says 172.21.15.116:35724) 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 -- 172.21.15.116:0/1898884431 learned_addr learned my addr 172.21.15.116:0/1898884431 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 -- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 msgr2=0x7f01e8073cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ee200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 0x7f01e815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ece00640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 0x7f01e8073cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 0x7f01e8073cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 -- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 msgr2=0x7f01e815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 0x7f01e815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ed800640 1 -- 172.21.15.116:0/1898884431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e81f9190 con 0x7f01e80af380 2024-05-14T11:21:47.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ee200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 0x7f01e815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.489+0000 7f01ece00640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 0x7f01e8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01ed800640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00e960 tx=0x7f01dc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc017070 con 0x7f01e80af380 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc00ca00 con 0x7f01e80af380 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e814f610 con 0x7f01e80af380 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00cdd0 con 0x7f01e80af380 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e814fbd0 con 0x7f01e80af380 2024-05-14T11:21:47.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e806a9c0 con 0x7f01e80af380 2024-05-14T11:21:47.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01dc01a070 con 0x7f01e80af380 2024-05-14T11:21:47.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 0x7f01c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7f01dc08f400 con 0x7f01e80af380 2024-05-14T11:21:47.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.493+0000 7f01ece00640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 0x7f01c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.497+0000 7f01ece00640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 0x7f01c8071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f01e80708a0 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.501+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f01dc05c400 con 0x7f01e80af380 2024-05-14T11:21:47.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.657+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f01e80d1cd0 con 0x7f01e80af380 2024-05-14T11:21:47.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.661+0000 7f01e5600640 1 -- 172.21.15.116:0/1898884431 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f01dc05bd00 con 0x7f01e80af380 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 msgr2=0x7f01c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 0x7f01c8071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f01e80708a0 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 msgr2=0x7f01e81f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00e960 tx=0x7f01dc00ee20 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 shutdown_connections 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01c806f430 0x7f01c8071940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e8156fa0 0x7f01e815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01e80d1670 0x7f01e8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 --2- 172.21.15.116:0/1898884431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01e80af380 0x7f01e81f8c50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 >> 172.21.15.116:0/1898884431 conn(0x7f01e80b4620 msgr2=0x7f01e80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 shutdown_connections 2024-05-14T11:21:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.665+0000 7f01f0200640 1 -- 172.21.15.116:0/1898884431 wait complete. 2024-05-14T11:21:47.683 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-14T11:21:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 Processor -- start 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 -- start start 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd16015db30 0x7fd1600aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd1600d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd16006ff20 con 0x7fd160161600 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd16006d710 con 0x7fd16015db30 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd167a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd1600ae100 con 0x7fd1600d1670 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd16015db30 0x7fd1600aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd16015db30 0x7fd1600aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52612/0 (socket says 172.21.15.116:52612) 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 -- 172.21.15.116:0/2937575109 learned_addr learned my addr 172.21.15.116:0/2937575109 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd157e00640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd1600d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 -- 172.21.15.116:0/2937575109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 msgr2=0x7fd1600d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd1600d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 -- 172.21.15.116:0/2937575109 >> v1:172.21.15.149:6789/0 conn(0x7fd160161600 legacy=0x7fd160161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.833+0000 7fd165000640 1 -- 172.21.15.116:0/2937575109 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600d1fc0 con 0x7fd16015db30 2024-05-14T11:21:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd165000640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd16015db30 0x7fd1600aeaa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd150009870 tx=0x7fd15001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7ed47bed685388 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd15fe00640 1 -- 172.21.15.116:0/2937575109 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500b0070 con 0x7fd16015db30 2024-05-14T11:21:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd15fe00640 1 -- 172.21.15.116:0/2937575109 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1500ab600 con 0x7fd16015db30 2024-05-14T11:21:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd15fe00640 1 -- 172.21.15.116:0/2937575109 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500ab9d0 con 0x7fd16015db30 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 -- 172.21.15.116:0/2937575109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd16015db30 msgr2=0x7fd1600aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd16015db30 0x7fd1600aeaa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd150009870 tx=0x7fd15001c4d0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 -- 172.21.15.116:0/2937575109 shutdown_connections 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd1600d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 --2- 172.21.15.116:0/2937575109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd16015db30 0x7fd1600aeaa0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 -- 172.21.15.116:0/2937575109 >> 172.21.15.116:0/2937575109 conn(0x7fd1600b4620 msgr2=0x7fd1600b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 -- 172.21.15.116:0/2937575109 shutdown_connections 2024-05-14T11:21:47.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.837+0000 7fd167a00640 1 -- 172.21.15.116:0/2937575109 wait complete. 2024-05-14T11:21:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 Processor -- start 2024-05-14T11:21:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 -- start start 2024-05-14T11:21:47.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd16015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 0x7fd160160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd16015df40 con 0x7fd160161600 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd1600aefe0 con 0x7fd16015db30 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd167a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd16016af80 con 0x7fd1600d1670 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd157e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 0x7fd160160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd16015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52614/0 (socket says 172.21.15.116:52614) 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd157e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 0x7fd160160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:52076/0 (socket says 172.21.15.116:52076) 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 -- 172.21.15.116:0/3475422461 learned_addr learned my addr 172.21.15.116:0/3475422461 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 -- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 msgr2=0x7fd16015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd16015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 -- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 msgr2=0x7fd160160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 0x7fd160160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 -- 172.21.15.116:0/3475422461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160161050 con 0x7fd160161600 2024-05-14T11:21:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd165a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c6e0 tx=0x7fd15800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158003b80 con 0x7fd160161600 2024-05-14T11:21:47.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.841+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd158003d20 con 0x7fd160161600 2024-05-14T11:21:47.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd160157f40 con 0x7fd160161600 2024-05-14T11:21:47.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1580052f0 con 0x7fd160161600 2024-05-14T11:21:47.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd160158500 con 0x7fd160161600 2024-05-14T11:21:47.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16006a9c0 con 0x7fd160161600 2024-05-14T11:21:47.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd158005490 con 0x7fd160161600 2024-05-14T11:21:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd15d600640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 0x7fd13c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7fd15808eef0 con 0x7fd160161600 2024-05-14T11:21:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.845+0000 7fd165000640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 0x7fd13c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.849+0000 7fd165000640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 0x7fd13c071910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd1500ad000 tx=0x7fd1500a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:47.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:47.853+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd15805b8a0 con 0x7fd160161600 2024-05-14T11:21:48.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.065+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fd160161e00 con 0x7fd160161600 2024-05-14T11:21:48.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd15d600640 1 -- 172.21.15.116:0/3475422461 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v284) ==== 166+0+0 (secure 0 0 0) 0x7fd1580620c0 con 0x7fd160161600 2024-05-14T11:21:48.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 msgr2=0x7fd13c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 0x7fd13c071910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd1500ad000 tx=0x7fd1500a6000 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 msgr2=0x7fd160155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c6e0 tx=0x7fd15800cba0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 shutdown_connections 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd13c06f400 0x7fd13c071910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd16015db30 0x7fd160160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd160161600 0x7fd160155440 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 --2- 172.21.15.116:0/3475422461 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd1600d1670 0x7fd16015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 >> 172.21.15.116:0/3475422461 conn(0x7fd1600b4620 msgr2=0x7fd1601572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 shutdown_connections 2024-05-14T11:21:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.305+0000 7fd167a00640 1 -- 172.21.15.116:0/3475422461 wait complete. 2024-05-14T11:21:48.326 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-14T11:21:48.326 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-14T11:21:48.326 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:21:48.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 Processor -- start 2024-05-14T11:21:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 -- start start 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9bd815c080 0x7f9bd80af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9bd806c700 0x7f9bd806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd815c490 con 0x7f9bd80afb40 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd80af170 con 0x7f9bd815c080 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd7e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd80ae560 con 0x7f9bd806c700 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9bd815c080 0x7f9bd80af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9bd80afb40 0x7f9bd806e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49854/0 (socket says 172.21.15.116:49854) 2024-05-14T11:21:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6000640 1 -- 172.21.15.116:0/665178345 learned_addr learned my addr 172.21.15.116:0/665178345 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6a00640 1 -- 172.21.15.116:0/665178345 >> v2:172.21.15.149:3300/0 conn(0x7f9bd806c700 msgr2=0x7f9bd806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6a00640 1 --2- 172.21.15.116:0/665178345 >> v2:172.21.15.149:3300/0 conn(0x7f9bd806c700 0x7f9bd806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6a00640 1 -- 172.21.15.116:0/665178345 >> v1:172.21.15.116:6789/0 conn(0x7f9bd80afb40 legacy=0x7f9bd806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.477+0000 7f9bd6a00640 1 -- 172.21.15.116:0/665178345 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd80ae840 con 0x7f9bd815c080 2024-05-14T11:21:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd6a00640 1 --2- 172.21.15.116:0/665178345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd815c080 0x7f9bd80af600 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80728e0 tx=0x7f9bc801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfd1036e0754179 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd5600640 1 -- 172.21.15.116:0/665178345 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80ab650 con 0x7f9bd815c080 2024-05-14T11:21:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd5600640 1 -- 172.21.15.116:0/665178345 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc80b0070 con 0x7f9bd815c080 2024-05-14T11:21:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd5600640 1 -- 172.21.15.116:0/665178345 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80aba10 con 0x7f9bd815c080 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 -- 172.21.15.116:0/665178345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd815c080 msgr2=0x7f9bd80af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/665178345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd815c080 0x7f9bd80af600 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80728e0 tx=0x7f9bc801c450 comp rx=0 tx=0).stop 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 -- 172.21.15.116:0/665178345 shutdown_connections 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/665178345 >> v2:172.21.15.149:3300/0 conn(0x7f9bd806c700 0x7f9bd806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/665178345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd815c080 0x7f9bd80af600 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 -- 172.21.15.116:0/665178345 >> 172.21.15.116:0/665178345 conn(0x7f9bd80b4530 msgr2=0x7f9bd80b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 -- 172.21.15.116:0/665178345 shutdown_connections 2024-05-14T11:21:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.481+0000 7f9bd7e00640 1 -- 172.21.15.116:0/665178345 wait complete. 2024-05-14T11:21:48.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 Processor -- start 2024-05-14T11:21:48.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- start start 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bd806c700 0x7f9bd809dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9bd815c080 0x7f9bd8159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd80b0880 con 0x7f9bd80afb40 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd80b0700 con 0x7f9bd806c700 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd806f0c0 con 0x7f9bd815c080 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52638/0 (socket says 172.21.15.116:52638) 2024-05-14T11:21:48.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 -- 172.21.15.116:0/766231054 learned_addr learned my addr 172.21.15.116:0/766231054 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6a00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bd806c700 0x7f9bd809dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7400640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9bd815c080 0x7f9bd8159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 -- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9bd815c080 msgr2=0x7f9bd8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9bd815c080 0x7f9bd8159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 -- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bd806c700 msgr2=0x7f9bd809dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bd806c700 0x7f9bd809dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 -- 172.21.15.116:0/766231054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd80b0aa0 con 0x7f9bd80afb40 2024-05-14T11:21:48.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd6000640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c460 tx=0x7f9bc400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4017070 con 0x7f9bd80afb40 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc4005d10 con 0x7f9bd80afb40 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40072b0 con 0x7f9bd80afb40 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd809eed0 con 0x7f9bd80afb40 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.485+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd80b22b0 con 0x7f9bd80afb40 2024-05-14T11:21:48.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd80cabe0 con 0x7f9bd80afb40 2024-05-14T11:21:48.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9bc401a070 con 0x7f9bd80afb40 2024-05-14T11:21:48.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bcea00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 0x7f9bb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7f9bc408e400 con 0x7f9bd80afb40 2024-05-14T11:21:48.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bd6a00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 0x7f9bb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.489+0000 7f9bd6a00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 0x7f9bb8071940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80af000 tx=0x7f9bc80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.493+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9bc405adb0 con 0x7f9bd80afb40 2024-05-14T11:21:48.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bd8050880 con 0x7f9bd80afb40 2024-05-14T11:21:48.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bcea00640 1 -- 172.21.15.116:0/766231054 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13476 (secure 0 0 0) 0x7f9bc405f0b0 con 0x7f9bd80afb40 2024-05-14T11:21:48.653 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:48.653 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":76,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:21:47.297463+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6836/2154548589":"2024-05-15T11:21:44.238943+0000","172.21.15.116:6834/2154548589":"2024-05-15T11:21:44.238943+0000","172.21.15.116:6836/2430466110":"2024-05-15T11:20:19.998743+0000","172.21.15.116:6834/2430466110":"2024-05-15T11:20:19.998743+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 msgr2=0x7f9bb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 0x7f9bb8071940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80af000 tx=0x7f9bc80a7000 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 msgr2=0x7f9bd8159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c460 tx=0x7f9bc400c920 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 shutdown_connections 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9bb806f430 0x7f9bb8071940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9bd815c080 0x7f9bd8159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9bd80afb40 0x7f9bd8159200 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 --2- 172.21.15.116:0/766231054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bd806c700 0x7f9bd809dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 >> 172.21.15.116:0/766231054 conn(0x7f9bd80b4530 msgr2=0x7f9bd80d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:48.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 shutdown_connections 2024-05-14T11:21:48.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.653+0000 7f9bd7e00640 1 -- 172.21.15.116:0/766231054 wait complete. 2024-05-14T11:21:48.674 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:21:48.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:21:48.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.821+0000 7f40dae00640 1 Processor -- start 2024-05-14T11:21:48.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.821+0000 7f40dae00640 1 -- start start 2024-05-14T11:21:48.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40dae00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f40d40cd5d0 0x7f40d406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40dae00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d406f0a0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40dae00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d40cbb60 con 0x7f40d40d1e40 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40dae00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f40d406f420 con 0x7f40d40cd5d0 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f40d40cd1c0 0x7f40d40d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49860/0 (socket says 172.21.15.116:49860) 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d3e00640 1 -- 172.21.15.116:0/1097940208 learned_addr learned my addr 172.21.15.116:0/1097940208 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:48.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d8e00640 1 --2- 172.21.15.116:0/1097940208 >> v2:172.21.15.116:3300/0 conn(0x7f40d40cd5d0 0x7f40d406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d3400640 1 -- 172.21.15.116:0/1097940208 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334427542 0 0) 0x7f40d406f0a0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d3400640 1 -- 172.21.15.116:0/1097940208 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40b8003530 con 0x7f40d40cd1c0 2024-05-14T11:21:48.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d8e00640 1 -- 172.21.15.116:0/1097940208 >> v1:172.21.15.149:6789/0 conn(0x7f40d40d1e40 legacy=0x7f40d406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d8e00640 1 -- 172.21.15.116:0/1097940208 >> v1:172.21.15.116:6789/0 conn(0x7f40d40cd1c0 legacy=0x7f40d40d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.825+0000 7f40d8e00640 1 -- 172.21.15.116:0/1097940208 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d406f700 con 0x7f40d40cd5d0 2024-05-14T11:21:48.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40d8e00640 1 --2- 172.21.15.116:0/1097940208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd5d0 0x7f40d406ce10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009d90 tx=0x7f40cc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7096f4004a808561 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40d3400640 1 -- 172.21.15.116:0/1097940208 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b2070 con 0x7f40d40cd5d0 2024-05-14T11:21:48.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40d3400640 1 -- 172.21.15.116:0/1097940208 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40cc0a5a00 con 0x7f40d40cd5d0 2024-05-14T11:21:48.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40d3400640 1 -- 172.21.15.116:0/1097940208 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0a5dd0 con 0x7f40d40cd5d0 2024-05-14T11:21:48.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 -- 172.21.15.116:0/1097940208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd5d0 msgr2=0x7f40d406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 --2- 172.21.15.116:0/1097940208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd5d0 0x7f40d406ce10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009d90 tx=0x7f40cc01c940 comp rx=0 tx=0).stop 2024-05-14T11:21:48.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 -- 172.21.15.116:0/1097940208 shutdown_connections 2024-05-14T11:21:48.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 --2- 172.21.15.116:0/1097940208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd5d0 0x7f40d406ce10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 -- 172.21.15.116:0/1097940208 >> 172.21.15.116:0/1097940208 conn(0x7f40d40b4720 msgr2=0x7f40d40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:48.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 -- 172.21.15.116:0/1097940208 shutdown_connections 2024-05-14T11:21:48.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.829+0000 7f40dae00640 1 -- 172.21.15.116:0/1097940208 wait complete. 2024-05-14T11:21:48.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 Processor -- start 2024-05-14T11:21:48.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- start start 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 0x7f40d4074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40d409a520 0x7f40d4155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40d415cf10 con 0x7f40d40cd1c0 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f40d40cd9e0 con 0x7f40d40cd5d0 2024-05-14T11:21:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f40d406d760 con 0x7f40d409a520 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52666/0 (socket says 172.21.15.116:52666) 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 -- 172.21.15.116:0/2898900705 learned_addr learned my addr 172.21.15.116:0/2898900705 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d8e00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40d409a520 0x7f40d4155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40cbe00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 0x7f40d4074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 -- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40d409a520 msgr2=0x7f40d4155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40d409a520 0x7f40d4155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 -- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 msgr2=0x7f40d4074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 0x7f40d4074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 -- 172.21.15.116:0/2898900705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d40caa10 con 0x7f40d40cd1c0 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40cbe00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 0x7f40d4074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d3e00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c430 tx=0x7f40c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4017070 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c4005cf0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d40cacf0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40072d0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.833+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d40b19a0 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d40b1c00 con 0x7f40d40cd1c0 2024-05-14T11:21:48.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40c401a070 con 0x7f40d40cd1c0 2024-05-14T11:21:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40d0c00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 0x7f40b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7f40c408e470 con 0x7f40d40cd1c0 2024-05-14T11:21:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40cbe00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 0x7f40b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:48.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.837+0000 7f40cbe00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 0x7f40b0071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40d40709d0 tx=0x7f40bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:48.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:48.841+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f40c405aec0 con 0x7f40d40cd1c0 2024-05-14T11:21:49.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.001+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40d406a9c0 con 0x7f40d40cd1c0 2024-05-14T11:21:49.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.001+0000 7f40d0c00640 1 -- 172.21.15.116:0/2898900705 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f40c405f1c0 con 0x7f40d40cd1c0 2024-05-14T11:21:49.002 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:21:49.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 msgr2=0x7f40b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 0x7f40b0071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40d40709d0 tx=0x7f40bc009040 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 msgr2=0x7f40d4074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c430 tx=0x7f40c400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 shutdown_connections 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f40b006f400 0x7f40b0071910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40d40cd5d0 0x7f40d4074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40d409a520 0x7f40d4155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 --2- 172.21.15.116:0/2898900705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d40cd1c0 0x7f40d4074020 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 >> 172.21.15.116:0/2898900705 conn(0x7f40d40b4720 msgr2=0x7f40d4153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 shutdown_connections 2024-05-14T11:21:49.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.005+0000 7f40dae00640 1 -- 172.21.15.116:0/2898900705 wait complete. 2024-05-14T11:21:49.024 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-14T11:21:49.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.169+0000 7fe08d400640 1 Processor -- start 2024-05-14T11:21:49.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.169+0000 7fe08d400640 1 -- start start 2024-05-14T11:21:49.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe08d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe088164220 0x7fe0880d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe08d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe08806ce10 0x7fe08816aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe08d400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0880d1fc0 con 0x7fe0880d1bb0 2024-05-14T11:21:49.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe08d400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe0881600a0 con 0x7fe088164220 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe08d400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe08806f5d0 con 0x7fe08806ce10 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe086800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe088164220 0x7fe0880d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe085e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe0880d1bb0 0x7fe08806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49866/0 (socket says 172.21.15.116:49866) 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe085e00640 1 -- 172.21.15.116:0/4143914718 learned_addr learned my addr 172.21.15.116:0/4143914718 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe087200640 1 --2- 172.21.15.116:0/4143914718 >> v2:172.21.15.149:3300/0 conn(0x7fe08806ce10 0x7fe08816aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe086800640 1 -- 172.21.15.116:0/4143914718 >> v2:172.21.15.149:3300/0 conn(0x7fe08806ce10 msgr2=0x7fe08816aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:49.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe086800640 1 --2- 172.21.15.116:0/4143914718 >> v2:172.21.15.149:3300/0 conn(0x7fe08806ce10 0x7fe08816aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe086800640 1 -- 172.21.15.116:0/4143914718 >> v1:172.21.15.116:6789/0 conn(0x7fe0880d1bb0 legacy=0x7fe08806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.173+0000 7fe086800640 1 -- 172.21.15.116:0/4143914718 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08806f8b0 con 0x7fe088164220 2024-05-14T11:21:49.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe086800640 1 --2- 172.21.15.116:0/4143914718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe088164220 0x7fe0880d1670 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe088070c50 tx=0x7fe07801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2d290c5f131aff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:49.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe085400640 1 -- 172.21.15.116:0/4143914718 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780b1070 con 0x7fe088164220 2024-05-14T11:21:49.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe085400640 1 -- 172.21.15.116:0/4143914718 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0780ac620 con 0x7fe088164220 2024-05-14T11:21:49.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe085400640 1 -- 172.21.15.116:0/4143914718 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780ac9f0 con 0x7fe088164220 2024-05-14T11:21:49.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 -- 172.21.15.116:0/4143914718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe088164220 msgr2=0x7fe0880d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 --2- 172.21.15.116:0/4143914718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe088164220 0x7fe0880d1670 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe088070c50 tx=0x7fe07801c3c0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 -- 172.21.15.116:0/4143914718 shutdown_connections 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 --2- 172.21.15.116:0/4143914718 >> v2:172.21.15.149:3300/0 conn(0x7fe08806ce10 0x7fe08816aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 --2- 172.21.15.116:0/4143914718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe088164220 0x7fe0880d1670 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 -- 172.21.15.116:0/4143914718 >> 172.21.15.116:0/4143914718 conn(0x7fe0880b4620 msgr2=0x7fe0880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 -- 172.21.15.116:0/4143914718 shutdown_connections 2024-05-14T11:21:49.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.177+0000 7fe08d400640 1 -- 172.21.15.116:0/4143914718 wait complete. 2024-05-14T11:21:49.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 Processor -- start 2024-05-14T11:21:49.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- start start 2024-05-14T11:21:49.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0880d1bb0 0x7fe088138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088164220 0x7fe0881391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0880af2f0 con 0x7fe08806ce10 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe088073650 con 0x7fe088164220 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe088164630 con 0x7fe0880d1bb0 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52672/0 (socket says 172.21.15.116:52672) 2024-05-14T11:21:49.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 -- 172.21.15.116:0/2955612445 learned_addr learned my addr 172.21.15.116:0/2955612445 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:49.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe085e00640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0880d1bb0 0x7fe088138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe087200640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088164220 0x7fe0881391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 -- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0880d1bb0 msgr2=0x7fe088138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0880d1bb0 0x7fe088138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 -- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088164220 msgr2=0x7fe0881391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088164220 0x7fe0881391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 -- 172.21.15.116:0/2955612445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe088162530 con 0x7fe08806ce10 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe086800640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe0780af040 tx=0x7fe07801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780b1050 con 0x7fe08806ce10 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0780bd070 con 0x7fe08806ce10 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe088162810 con 0x7fe08806ce10 2024-05-14T11:21:49.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780b86b0 con 0x7fe08806ce10 2024-05-14T11:21:49.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0880b21d0 con 0x7fe08806ce10 2024-05-14T11:21:49.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.181+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08806a9c0 con 0x7fe08806ce10 2024-05-14T11:21:49.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.185+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe0780b8850 con 0x7fe08806ce10 2024-05-14T11:21:49.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.185+0000 7fe07ea00640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 0x7fe068071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:49.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.185+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7fe0781297d0 con 0x7fe08806ce10 2024-05-14T11:21:49.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.185+0000 7fe085e00640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 0x7fe068071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:49.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.185+0000 7fe085e00640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 0x7fe068071940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe0880d10d0 tx=0x7fe074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:49.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.189+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe0780c7020 con 0x7fe08806ce10 2024-05-14T11:21:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:49.413+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fe08806ecf0 con 0x7fe08806ce10 2024-05-14T11:21:50.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe07ea00640 1 -- 172.21.15.116:0/2955612445 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v286) ==== 170+0+0 (secure 0 0 0) 0x7fe078016070 con 0x7fe08806ce10 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 msgr2=0x7fe068071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 0x7fe068071940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe0880d10d0 tx=0x7fe074009040 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 msgr2=0x7fe0880b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe0780af040 tx=0x7fe07801ac00 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 shutdown_connections 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe06806f430 0x7fe068071940 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088164220 0x7fe0881391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0880d1bb0 0x7fe088138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 --2- 172.21.15.116:0/2955612445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe08806ce10 0x7fe0880b1c90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 >> 172.21.15.116:0/2955612445 conn(0x7fe0880b4620 msgr2=0x7fe0881634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 shutdown_connections 2024-05-14T11:21:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.313+0000 7fe08d400640 1 -- 172.21.15.116:0/2955612445 wait complete. 2024-05-14T11:21:50.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:21:50.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 Processor -- start 2024-05-14T11:21:50.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 -- start start 2024-05-14T11:21:50.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 0x7f757c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f757c06c700 0x7f757c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c06f0a0 con 0x7f757c0cd1c0 2024-05-14T11:21:50.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f757c0cbb60 con 0x7f757c06c700 2024-05-14T11:21:50.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7581200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f757c06f420 con 0x7f757c0d1e40 2024-05-14T11:21:50.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7579e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f757c0cd1c0 0x7f757c0d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49868/0 (socket says 172.21.15.116:49868) 2024-05-14T11:21:50.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7579e00640 1 -- 172.21.15.116:0/2828258033 learned_addr learned my addr 172.21.15.116:0/2828258033 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:50.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f757a800640 1 --2- 172.21.15.116:0/2828258033 >> v2:172.21.15.116:3300/0 conn(0x7f757c06c700 0x7f757c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7579400640 1 -- 172.21.15.116:0/2828258033 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581339998 0 0) 0x7f757c06f0a0 con 0x7f757c0cd1c0 2024-05-14T11:21:50.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7579400640 1 -- 172.21.15.116:0/2828258033 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7560003530 con 0x7f757c0cd1c0 2024-05-14T11:21:50.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7571400640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 0x7f757c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f757a800640 1 -- 172.21.15.116:0/2828258033 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 msgr2=0x7f757c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:50.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f757a800640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 0x7f757c0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f757a800640 1 -- 172.21.15.116:0/2828258033 >> v1:172.21.15.116:6789/0 conn(0x7f757c0cd1c0 legacy=0x7f757c0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f757a800640 1 -- 172.21.15.116:0/2828258033 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c06f700 con 0x7f757c06c700 2024-05-14T11:21:50.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.493+0000 7f7571400640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 0x7f757c0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f757a800640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c06cb10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7574009d90 tx=0x7f757401c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfb8d3c43fee486 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7579400640 1 -- 172.21.15.116:0/2828258033 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b2070 con 0x7f757c06c700 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7579400640 1 -- 172.21.15.116:0/2828258033 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740a5850 con 0x7f757c06c700 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7579400640 1 -- 172.21.15.116:0/2828258033 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740a5c00 con 0x7f757c06c700 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 -- 172.21.15.116:0/2828258033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 msgr2=0x7f757c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c06cb10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7574009d90 tx=0x7f757401c790 comp rx=0 tx=0).stop 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 -- 172.21.15.116:0/2828258033 shutdown_connections 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c06cb10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 --2- 172.21.15.116:0/2828258033 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0d1e40 0x7f757c0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 -- 172.21.15.116:0/2828258033 >> 172.21.15.116:0/2828258033 conn(0x7f757c0b4720 msgr2=0x7f757c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:50.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 -- 172.21.15.116:0/2828258033 shutdown_connections 2024-05-14T11:21:50.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.497+0000 7f7581200640 1 -- 172.21.15.116:0/2828258033 wait complete. 2024-05-14T11:21:50.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 Processor -- start 2024-05-14T11:21:50.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 -- start start 2024-05-14T11:21:50.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0cd1c0 0x7f757c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757c0c6f20 0x7f757c0c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f757c1699b0 con 0x7f757c06c700 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f757c15cf10 con 0x7f757c0c6f20 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f757c169b30 con 0x7f757c0cd1c0 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52692/0 (socket says 172.21.15.116:52692) 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 -- 172.21.15.116:0/1497185684 learned_addr learned my addr 172.21.15.116:0/1497185684 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7571400640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0cd1c0 0x7f757c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f757a800640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757c0c6f20 0x7f757c0c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 -- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0cd1c0 msgr2=0x7f757c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0cd1c0 0x7f757c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 -- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757c0c6f20 msgr2=0x7f757c0c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757c0c6f20 0x7f757c0c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 -- 172.21.15.116:0/1497185684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c165ea0 con 0x7f757c06c700 2024-05-14T11:21:50.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7579e00640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f756400c460 tx=0x7f756400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564017070 con 0x7f757c06c700 2024-05-14T11:21:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7564005cf0 con 0x7f757c06c700 2024-05-14T11:21:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564007270 con 0x7f757c06c700 2024-05-14T11:21:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.501+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c166180 con 0x7f757c06c700 2024-05-14T11:21:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c166740 con 0x7f757c06c700 2024-05-14T11:21:50.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c06a9c0 con 0x7f757c06c700 2024-05-14T11:21:50.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f756401a070 con 0x7f757c06c700 2024-05-14T11:21:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7572a00640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 0x7f7558071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7f756408e4a0 con 0x7f757c06c700 2024-05-14T11:21:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.505+0000 7f7571400640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 0x7f7558071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:50.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.509+0000 7f7571400640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 0x7f7558071910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b1a90 tx=0x7f756c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.509+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f756405adc0 con 0x7f757c06c700 2024-05-14T11:21:50.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.669+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f757c069de0 con 0x7f757c06c700 2024-05-14T11:21:50.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7572a00640 1 -- 172.21.15.116:0/1497185684 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f756405f0c0 con 0x7f757c06c700 2024-05-14T11:21:50.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 msgr2=0x7f7558071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 0x7f7558071910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b1a90 tx=0x7f756c008040 comp rx=0 tx=0).stop 2024-05-14T11:21:50.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 msgr2=0x7f757c0c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:50.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f756400c460 tx=0x7f756400c920 comp rx=0 tx=0).stop 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 shutdown_connections 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f755806f400 0x7f7558071910 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757c0c6f20 0x7f757c0c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f757c0cd1c0 0x7f757c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 --2- 172.21.15.116:0/1497185684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f757c06c700 0x7f757c0c69e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 >> 172.21.15.116:0/1497185684 conn(0x7f757c0b4720 msgr2=0x7f757c0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 shutdown_connections 2024-05-14T11:21:50.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:50.673+0000 7f7581200640 1 -- 172.21.15.116:0/1497185684 wait complete. 2024-05-14T11:21:50.694 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:21:50.694 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:21:50.797 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:21:50.895 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:21:50.896 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:21:50.896 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:50.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:21:50.905 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:21:50.905 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:50.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:21:50.915 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:21:50.915 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:50.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:21:50.955 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:21:50.956 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:21:50.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:21:50.964 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:21:50.964 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:51.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.145+0000 7f37d6400640 1 Processor -- start 2024-05-14T11:21:51.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 -- start start 2024-05-14T11:21:51.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 0x7f37d006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37d00d1670 0x7f37d00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d00d1fc0 con 0x7f37d006e9e0 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f37d01573b0 con 0x7f37d00d1670 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37d6400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f37d0161120 con 0x7f37d0156fa0 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37c6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37d00d1670 0x7f37d00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37cfe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f37d006e9e0 0x7f37d006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49882/0 (socket says 172.21.15.116:49882) 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37cfe00640 1 -- 172.21.15.116:0/3611154595 learned_addr learned my addr 172.21.15.116:0/3611154595 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37c6a00640 1 -- 172.21.15.116:0/3611154595 >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 msgr2=0x7f37d006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37cf400640 1 --2- 172.21.15.116:0/3611154595 >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 0x7f37d006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37cea00640 1 -- 172.21.15.116:0/3611154595 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414378543 0 0) 0x7f37c8001120 con 0x7f37d006e9e0 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37c6a00640 1 --2- 172.21.15.116:0/3611154595 >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 0x7f37d006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37c6a00640 1 -- 172.21.15.116:0/3611154595 >> v1:172.21.15.116:6789/0 conn(0x7f37d006e9e0 legacy=0x7f37d006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37c6a00640 1 -- 172.21.15.116:0/3611154595 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d0160510 con 0x7f37d00d1670 2024-05-14T11:21:51.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.149+0000 7f37cf400640 1 --2- 172.21.15.116:0/3611154595 >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 0x7f37d006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:51.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37c6a00640 1 --2- 172.21.15.116:0/3611154595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d00d1a80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f37d0071f20 tx=0x7f37b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3b2a64cfcf522d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37cea00640 1 -- 172.21.15.116:0/3611154595 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80ab650 con 0x7f37d00d1670 2024-05-14T11:21:51.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37cea00640 1 -- 172.21.15.116:0/3611154595 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b80b0070 con 0x7f37d00d1670 2024-05-14T11:21:51.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37cea00640 1 -- 172.21.15.116:0/3611154595 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80aba10 con 0x7f37d00d1670 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 -- 172.21.15.116:0/3611154595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 msgr2=0x7f37d00d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 --2- 172.21.15.116:0/3611154595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d00d1a80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f37d0071f20 tx=0x7f37b801c450 comp rx=0 tx=0).stop 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 -- 172.21.15.116:0/3611154595 shutdown_connections 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 --2- 172.21.15.116:0/3611154595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d00d1a80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 --2- 172.21.15.116:0/3611154595 >> v2:172.21.15.149:3300/0 conn(0x7f37d0156fa0 0x7f37d006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 -- 172.21.15.116:0/3611154595 >> 172.21.15.116:0/3611154595 conn(0x7f37d00b4620 msgr2=0x7f37d00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 -- 172.21.15.116:0/3611154595 shutdown_connections 2024-05-14T11:21:51.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.153+0000 7f37d6400640 1 -- 172.21.15.116:0/3611154595 wait complete. 2024-05-14T11:21:51.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 Processor -- start 2024-05-14T11:21:51.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- start start 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37d006e9e0 0x7f37d014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 0x7f37d00af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0070520 con 0x7f37d00d1670 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00b07a0 con 0x7f37d006e9e0 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f37d00706a0 con 0x7f37d0156fa0 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52710/0 (socket says 172.21.15.116:52710) 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 -- 172.21.15.116:0/4116661509 learned_addr learned my addr 172.21.15.116:0/4116661509 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37cfe00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 0x7f37d00af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37cf400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37d006e9e0 0x7f37d014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 -- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 msgr2=0x7f37d00af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 0x7f37d00af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 -- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37d006e9e0 msgr2=0x7f37d014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37d006e9e0 0x7f37d014e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 -- 172.21.15.116:0/4116661509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d014eea0 con 0x7f37d00d1670 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37cfe00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 0x7f37d00af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c6a00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f37b801a080 tx=0x7f37b801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80bf610 con 0x7f37d00d1670 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b80bf7d0 con 0x7f37d00d1670 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d01562b0 con 0x7f37d00d1670 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80bfbc0 con 0x7f37d00d1670 2024-05-14T11:21:51.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.157+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01f9c60 con 0x7f37d00d1670 2024-05-14T11:21:51.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d01f9770 con 0x7f37d00d1670 2024-05-14T11:21:51.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f37b80bd020 con 0x7f37d00d1670 2024-05-14T11:21:51.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37c7e00640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 0x7f37ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7f37b8129d60 con 0x7f37d00d1670 2024-05-14T11:21:51.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37cf400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 0x7f37ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.161+0000 7f37cf400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 0x7f37ac071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f37c00070b0 tx=0x7f37c0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.165+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f37b8016070 con 0x7f37d00d1670 2024-05-14T11:21:51.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37d006a9c0 con 0x7f37d00d1670 2024-05-14T11:21:51.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37c7e00640 1 -- 172.21.15.116:0/4116661509 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3878 (secure 0 0 0) 0x7f37b80f6d60 con 0x7f37d00d1670 2024-05-14T11:21:51.357 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:51.357 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/1660784628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":1660784628},{"type":"v1","addr":"172.21.15.116:6837","nonce":1660784628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/1296198183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1296198183},{"type":"v1","addr":"172.21.15.149:6834","nonce":1296198183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5957,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/2595029675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2595029675},{"type":"v1","addr":"172.21.15.116:6836","nonce":2595029675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/839554016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":839554016},{"type":"v1","addr":"172.21.15.149:6835","nonce":839554016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[]} 2024-05-14T11:21:51.358 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 55 2024-05-14T11:21:51.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 msgr2=0x7f37ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 0x7f37ac071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f37c00070b0 tx=0x7f37c0007040 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 msgr2=0x7f37d0155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.357+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f37b801a080 tx=0x7f37b801ac90 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 shutdown_connections 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f37ac06f400 0x7f37ac071910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37d0156fa0 0x7f37d00af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37d00d1670 0x7f37d0155680 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 --2- 172.21.15.116:0/4116661509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37d006e9e0 0x7f37d014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 >> 172.21.15.116:0/4116661509 conn(0x7f37d00b4620 msgr2=0x7f37d00b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 shutdown_connections 2024-05-14T11:21:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.361+0000 7f37d6400640 1 -- 172.21.15.116:0/4116661509 wait complete. 2024-05-14T11:21:51.379 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:21:51.379 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:21:51.379 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:21:51.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:21:51.380 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:21:51.380 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:21:51.380 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:21:51.380 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:21:51.380 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:21:51.380 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:21:51.380 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:21:51.381 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:21:51.381 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:21:51.381 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:21:51.381 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:21:51.381 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:21:51.392 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:21:51.392 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:21:51.394 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:21:51.394 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:21:51.395 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:21:51.395 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:21:51.399 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:21:51.400 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:21:51.439 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:21:51.439 DEBUG:teuthology.parallel:result is None 2024-05-14T11:21:51.440 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:21:51.440 DEBUG:teuthology.parallel:result is None 2024-05-14T11:21:51.445 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:21:51.445 DEBUG:teuthology.parallel:result is None 2024-05-14T11:21:51.447 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:21:51.448 DEBUG:teuthology.parallel:result is None 2024-05-14T11:21:51.448 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:21:51.539 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:21:51.539 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:21:51.545 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:21:51.546 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:21:51.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.601+0000 7fc4ad200640 1 Processor -- start 2024-05-14T11:21:51.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 -- start start 2024-05-14T11:21:51.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc4a806f250 0x7fc4a80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4a8071a00 0x7fc4a815a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a8071e10 con 0x7fc4a806f660 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a80d1f90 con 0x7fc4a8071a00 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4ad200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a8160fc0 con 0x7fc4a806f250 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a6800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4a8071a00 0x7fc4a815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc49fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc4a806f660 0x7fc4a81598f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49896/0 (socket says 172.21.15.116:49896) 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc49fe00640 1 -- 172.21.15.116:0/1959369921 learned_addr learned my addr 172.21.15.116:0/1959369921 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1959369921 >> v2:172.21.15.149:3300/0 conn(0x7fc4a806f250 0x7fc4a80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5e00640 1 -- 172.21.15.116:0/1959369921 >> v2:172.21.15.116:3300/0 conn(0x7fc4a8071a00 msgr2=0x7fc4a815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1959369921 >> v2:172.21.15.116:3300/0 conn(0x7fc4a8071a00 0x7fc4a815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5e00640 1 -- 172.21.15.116:0/1959369921 >> v1:172.21.15.116:6789/0 conn(0x7fc4a806f660 legacy=0x7fc4a81598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5400640 1 -- 172.21.15.116:0/1959369921 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825135189 0 0) 0x7fc490001120 con 0x7fc4a806f660 2024-05-14T11:21:51.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.605+0000 7fc4a5e00640 1 -- 172.21.15.116:0/1959369921 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81603b0 con 0x7fc4a806f250 2024-05-14T11:21:51.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1959369921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a806f250 0x7fc4a80714c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc4980097c0 tx=0x7fc49801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359923615b5fd8f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4a5400640 1 -- 172.21.15.116:0/1959369921 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980b1070 con 0x7fc4a806f250 2024-05-14T11:21:51.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4a5400640 1 -- 172.21.15.116:0/1959369921 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4980ac600 con 0x7fc4a806f250 2024-05-14T11:21:51.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4a5400640 1 -- 172.21.15.116:0/1959369921 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980ac9d0 con 0x7fc4a806f250 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 -- 172.21.15.116:0/1959369921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a806f250 msgr2=0x7fc4a80714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1959369921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a806f250 0x7fc4a80714c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc4980097c0 tx=0x7fc49801c3a0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 -- 172.21.15.116:0/1959369921 shutdown_connections 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1959369921 >> v2:172.21.15.116:3300/0 conn(0x7fc4a8071a00 0x7fc4a815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1959369921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a806f250 0x7fc4a80714c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 -- 172.21.15.116:0/1959369921 >> 172.21.15.116:0/1959369921 conn(0x7fc4a8073ca0 msgr2=0x7fc4a80740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 -- 172.21.15.116:0/1959369921 shutdown_connections 2024-05-14T11:21:51.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.609+0000 7fc4ad200640 1 -- 172.21.15.116:0/1959369921 wait complete. 2024-05-14T11:21:51.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 Processor -- start 2024-05-14T11:21:51.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- start start 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4a806f250 0x7fc4a8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8071a00 0x7fc4a8147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a815a6c0 con 0x7fc4a806f660 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a815a540 con 0x7fc4a8071a00 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a8164540 con 0x7fc4a806f250 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52750/0 (socket says 172.21.15.116:52750) 2024-05-14T11:21:51.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4a6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8071a00 0x7fc4a8147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 -- 172.21.15.116:0/1933612706 learned_addr learned my addr 172.21.15.116:0/1933612706 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:51.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4a806f250 0x7fc4a8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 -- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4a806f250 msgr2=0x7fc4a8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4a806f250 0x7fc4a8156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 -- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8071a00 msgr2=0x7fc4a8147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8071a00 0x7fc4a8147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 -- 172.21.15.116:0/1933612706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a815a0e0 con 0x7fc4a806f660 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49fe00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c430 tx=0x7fc49000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900074e0 con 0x7fc4a806f660 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4900047f0 con 0x7fc4a806f660 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81574c0 con 0x7fc4a806f660 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490007680 con 0x7fc4a806f660 2024-05-14T11:21:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.613+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a8153df0 con 0x7fc4a806f660 2024-05-14T11:21:51.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a8159e50 con 0x7fc4a806f660 2024-05-14T11:21:51.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc490011070 con 0x7fc4a806f660 2024-05-14T11:21:51.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc49e000640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 0x7fc484071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:51.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5421+0+0 (secure 0 0 0) 0x7fc49008ead0 con 0x7fc4a806f660 2024-05-14T11:21:51.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 0x7fc484071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:51.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.617+0000 7fc4a5e00640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 0x7fc484071940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc4980ae000 tx=0x7fc4980a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:51.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.621+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc4a8071e10 con 0x7fc4a806f660 2024-05-14T11:21:51.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:51.789+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc4a806a9c0 con 0x7fc4a806f660 2024-05-14T11:21:52.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.329+0000 7fc49e000640 1 -- 172.21.15.116:0/1933612706 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) ==== 97+0+0 (secure 0 0 0) 0x7fc49005b520 con 0x7fc4a806f660 2024-05-14T11:21:52.330 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:21:52.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 msgr2=0x7fc484071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 0x7fc484071940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc4980ae000 tx=0x7fc4980a7000 comp rx=0 tx=0).stop 2024-05-14T11:21:52.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 msgr2=0x7fc4a81473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c430 tx=0x7fc49000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 shutdown_connections 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc48406f430 0x7fc484071940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4a8071a00 0x7fc4a8147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4a806f660 0x7fc4a81473c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 --2- 172.21.15.116:0/1933612706 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc4a806f250 0x7fc4a8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 >> 172.21.15.116:0/1933612706 conn(0x7fc4a8073ca0 msgr2=0x7fc4a8162540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 shutdown_connections 2024-05-14T11:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.333+0000 7fc4ad200640 1 -- 172.21.15.116:0/1933612706 wait complete. 2024-05-14T11:21:52.358 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:21:52.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 Processor -- start 2024-05-14T11:21:52.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 -- start start 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 0x7fa5400d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa54006c700 0x7fa54006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa54006ff20 con 0x7fa54015db30 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa5400ae100 con 0x7fa54006c700 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa548200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400ae870 con 0x7fa5400d1670 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa545800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa54015db30 0x7fa5400aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49906/0 (socket says 172.21.15.116:49906) 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa545800640 1 -- 172.21.15.116:0/2918107238 learned_addr learned my addr 172.21.15.116:0/2918107238 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:52.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa546200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa54006c700 0x7fa54006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa546200640 1 -- 172.21.15.116:0/2918107238 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 msgr2=0x7fa5400d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa544e00640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 0x7fa5400d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa53fe00640 1 -- 172.21.15.116:0/2918107238 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206109281 0 0) 0x7fa54006ff20 con 0x7fa54015db30 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa546200640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 0x7fa5400d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa546200640 1 -- 172.21.15.116:0/2918107238 >> v1:172.21.15.116:6789/0 conn(0x7fa54015db30 legacy=0x7fa5400aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa546200640 1 -- 172.21.15.116:0/2918107238 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400d1fc0 con 0x7fa54006c700 2024-05-14T11:21:52.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.509+0000 7fa544e00640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 0x7fa5400d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:52.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa546200640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa54006c700 0x7fa54006cb10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa538009d90 tx=0x7fa53801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27965d00e30f1132 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa53fe00640 1 -- 172.21.15.116:0/2918107238 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380b2070 con 0x7fa54006c700 2024-05-14T11:21:52.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa53fe00640 1 -- 172.21.15.116:0/2918107238 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5380a57c0 con 0x7fa54006c700 2024-05-14T11:21:52.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa53fe00640 1 -- 172.21.15.116:0/2918107238 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380a5b90 con 0x7fa54006c700 2024-05-14T11:21:52.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 -- 172.21.15.116:0/2918107238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa54006c700 msgr2=0x7fa54006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa54006c700 0x7fa54006cb10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa538009d90 tx=0x7fa53801c810 comp rx=0 tx=0).stop 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 -- 172.21.15.116:0/2918107238 shutdown_connections 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa54006c700 0x7fa54006cb10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 --2- 172.21.15.116:0/2918107238 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa5400d1670 0x7fa5400d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 -- 172.21.15.116:0/2918107238 >> 172.21.15.116:0/2918107238 conn(0x7fa5400b4620 msgr2=0x7fa5400b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 -- 172.21.15.116:0/2918107238 shutdown_connections 2024-05-14T11:21:52.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.513+0000 7fa548200640 1 -- 172.21.15.116:0/2918107238 wait complete. 2024-05-14T11:21:52.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 Processor -- start 2024-05-14T11:21:52.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 -- start start 2024-05-14T11:21:52.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa54006c700 0x7fa5401f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa54015db30 0x7fa5400cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa54016a770 con 0x7fa5400d1670 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa54015df40 con 0x7fa54006c700 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa548200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa54016a8f0 con 0x7fa54015db30 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa544e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa544e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52780/0 (socket says 172.21.15.116:52780) 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa544e00640 1 -- 172.21.15.116:0/1602510764 learned_addr learned my addr 172.21.15.116:0/1602510764 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa545800640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa54006c700 0x7fa5401f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.517+0000 7fa546200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa54015db30 0x7fa5400cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 -- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa54015db30 msgr2=0x7fa5400cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa54015db30 0x7fa5400cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 -- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa54006c700 msgr2=0x7fa5401f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa54006c700 0x7fa5401f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 -- 172.21.15.116:0/1602510764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400b0d70 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa544e00640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c7f0 tx=0x7fa52c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c015070 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c0077c0 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401f9980 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c007bb0 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401f9f40 con 0x7fa5400d1670 2024-05-14T11:21:52.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5401f96d0 con 0x7fa5400d1670 2024-05-14T11:21:52.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa52c018070 con 0x7fa5400d1670 2024-05-14T11:21:52.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.521+0000 7fa53d600640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 0x7fa520071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.525+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5249+0+0 (secure 0 0 0) 0x7fa52c08eee0 con 0x7fa5400d1670 2024-05-14T11:21:52.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.525+0000 7fa545800640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 0x7fa520071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.525+0000 7fa545800640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 0x7fa520071940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f70 tx=0x7fa534007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.529+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa52c05b9b0 con 0x7fa5400d1670 2024-05-14T11:21:52.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa54006a9c0 con 0x7fa5400d1670 2024-05-14T11:21:52.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa53d600640 1 -- 172.21.15.116:0/1602510764 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fa52c05fcb0 con 0x7fa5400d1670 2024-05-14T11:21:52.740 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:52.740 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:21:52.740 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.741 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.742 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:21:52.743 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.744 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:21:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:21:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:52.753 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:21:52.753 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 msgr2=0x7fa520071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 0x7fa520071940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa534000f70 tx=0x7fa534007df0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 msgr2=0x7fa540138ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c7f0 tx=0x7fa52c00ccb0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 shutdown_connections 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa52006f430 0x7fa520071940 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa54015db30 0x7fa5400cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5400d1670 0x7fa540138ca0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 --2- 172.21.15.116:0/1602510764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa54006c700 0x7fa5401f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 >> 172.21.15.116:0/1602510764 conn(0x7fa5400b4620 msgr2=0x7fa540073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:52.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 shutdown_connections 2024-05-14T11:21:52.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.741+0000 7fa548200640 1 -- 172.21.15.116:0/1602510764 wait complete. 2024-05-14T11:21:52.762 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:21:52.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 Processor -- start 2024-05-14T11:21:52.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 -- start start 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe41605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7effe4070610 0x7effe4160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe406f5c0 con 0x7effe40d0730 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7effe4161450 con 0x7effe4070610 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effea400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7effe40d1fb0 con 0x7effe4070130 2024-05-14T11:21:52.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effdaa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7effe4070610 0x7effe4160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effe3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7effe40d0730 0x7effe40d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49908/0 (socket says 172.21.15.116:49908) 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effe3e00640 1 -- 172.21.15.116:0/1900880438 learned_addr learned my addr 172.21.15.116:0/1900880438 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.909+0000 7effe3400640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe41605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effdaa00640 1 -- 172.21.15.116:0/1900880438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 msgr2=0x7effe41605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effdaa00640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe41605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effdaa00640 1 -- 172.21.15.116:0/1900880438 >> v1:172.21.15.116:6789/0 conn(0x7effe40d0730 legacy=0x7effe40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effdaa00640 1 -- 172.21.15.116:0/1900880438 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe40d2290 con 0x7effe4070610 2024-05-14T11:21:52.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effdaa00640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe4070610 0x7effe4160af0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c81d5a08e0403b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effe2a00640 1 -- 172.21.15.116:0/1900880438 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b0070 con 0x7effe4070610 2024-05-14T11:21:52.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effe2a00640 1 -- 172.21.15.116:0/1900880438 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0ab600 con 0x7effe4070610 2024-05-14T11:21:52.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effe2a00640 1 -- 172.21.15.116:0/1900880438 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0ab9d0 con 0x7effe4070610 2024-05-14T11:21:52.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effea400640 1 -- 172.21.15.116:0/1900880438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe4070610 msgr2=0x7effe4160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.913+0000 7effea400640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe4070610 0x7effe4160af0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- 172.21.15.116:0/1900880438 shutdown_connections 2024-05-14T11:21:52.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe4070610 0x7effe4160af0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 --2- 172.21.15.116:0/1900880438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe41605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- 172.21.15.116:0/1900880438 >> 172.21.15.116:0/1900880438 conn(0x7effe4071370 msgr2=0x7effe4071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:52.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- 172.21.15.116:0/1900880438 shutdown_connections 2024-05-14T11:21:52.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- 172.21.15.116:0/1900880438 wait complete. 2024-05-14T11:21:52.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 Processor -- start 2024-05-14T11:21:52.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- start start 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 0x7effe415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7effe41611b0 con 0x7effe40d0730 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7effe4161030 con 0x7effe4070610 2024-05-14T11:21:52.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.917+0000 7effea400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7effe4169f50 con 0x7effe4070130 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52794/0 (socket says 172.21.15.116:52794) 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 -- 172.21.15.116:0/2691167802 learned_addr learned my addr 172.21.15.116:0/2691167802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdaa00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 0x7effe415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 -- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 msgr2=0x7effe415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe415eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 -- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 msgr2=0x7effe415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 0x7effe415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 -- 172.21.15.116:0/2691167802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe415eff0 con 0x7effe40d0730 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdaa00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 0x7effe415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effe3e00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7effdc00caa0 tx=0x7effdc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc017070 con 0x7effe40d0730 2024-05-14T11:21:52.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effdc00ac00 con 0x7effe40d0730 2024-05-14T11:21:52.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc012400 con 0x7effe40d0730 2024-05-14T11:21:52.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe4165e30 con 0x7effe40d0730 2024-05-14T11:21:52.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41663f0 con 0x7effe40d0730 2024-05-14T11:21:52.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe406a9c0 con 0x7effe40d0730 2024-05-14T11:21:52.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.921+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7effdc01a070 con 0x7effe40d0730 2024-05-14T11:21:52.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.925+0000 7effdbe00640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 0x7effc0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:52.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.925+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5249+0+0 (secure 0 0 0) 0x7effdc08ed40 con 0x7effe40d0730 2024-05-14T11:21:52.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.925+0000 7effe3400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 0x7effc0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:52.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.925+0000 7effe3400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 0x7effc0071940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7effe406ecf0 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:52.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:52.929+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7effdc05be80 con 0x7effe40d0730 2024-05-14T11:21:53.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.085+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7effe40cf3b0 con 0x7effe40d0730 2024-05-14T11:21:53.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.085+0000 7effdbe00640 1 -- 172.21.15.116:0/2691167802 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7effdc062e20 con 0x7effe40d0730 2024-05-14T11:21:53.086 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:53.086 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:21:53.087 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:21:53.087 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:21:53.087 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:21:53.087 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:21:53.087 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:21:53.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 msgr2=0x7effc0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 0x7effc0071940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7effe406ecf0 tx=0x7effd4008040 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 msgr2=0x7effe415bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7effdc00caa0 tx=0x7effdc00cf60 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 shutdown_connections 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7effc006f430 0x7effc0071940 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effe4070610 0x7effe415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effe40d0730 0x7effe415bbd0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 --2- 172.21.15.116:0/2691167802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7effe4070130 0x7effe415eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 >> 172.21.15.116:0/2691167802 conn(0x7effe4071370 msgr2=0x7effe4160540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 shutdown_connections 2024-05-14T11:21:53.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.089+0000 7effea400640 1 -- 172.21.15.116:0/2691167802 wait complete. 2024-05-14T11:21:53.108 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:21:53.108 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:21:53.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 Processor -- start 2024-05-14T11:21:53.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 -- start start 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 0x7f567406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 0x7f567406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56740d04c0 con 0x7f56740cda30 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f56740d0270 con 0x7f567406d710 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f567aa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56740b0c30 con 0x7f56740cd620 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f56740cda30 0x7f5674161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49924/0 (socket says 172.21.15.116:49924) 2024-05-14T11:21:53.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673400640 1 -- 172.21.15.116:0/2413084327 learned_addr learned my addr 172.21.15.116:0/2413084327 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:53.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673e00640 1 --2- 172.21.15.116:0/2413084327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 0x7f567406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5678a00640 1 --2- 172.21.15.116:0/2413084327 >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 0x7f567406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379070754 0 0) 0x7f56740d04c0 con 0x7f56740cda30 2024-05-14T11:21:53.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5658003530 con 0x7f56740cda30 2024-05-14T11:21:53.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673e00640 1 -- 172.21.15.116:0/2413084327 >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 msgr2=0x7f567406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:53.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673e00640 1 --2- 172.21.15.116:0/2413084327 >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 0x7f567406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673e00640 1 -- 172.21.15.116:0/2413084327 >> v1:172.21.15.116:6789/0 conn(0x7f56740cda30 legacy=0x7f5674161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1821806858 0 0) 0x7f5658003530 con 0x7f56740cda30 2024-05-14T11:21:53.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5673e00640 1 -- 172.21.15.116:0/2413084327 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56740d07a0 con 0x7f56740cd620 2024-05-14T11:21:53.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.261+0000 7f5678a00640 1 --2- 172.21.15.116:0/2413084327 >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 0x7f567406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:21:53.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f5673e00640 1 --2- 172.21.15.116:0/2413084327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 0x7f567406f6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56740d03f0 tx=0x7f566401c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd2de097c94d387 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:53.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660001700 con 0x7f56740cd620 2024-05-14T11:21:53.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640b0400 con 0x7f56740cd620 2024-05-14T11:21:53.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f5672a00640 1 -- 172.21.15.116:0/2413084327 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640b07d0 con 0x7f56740cd620 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 -- 172.21.15.116:0/2413084327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 msgr2=0x7f567406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 --2- 172.21.15.116:0/2413084327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 0x7f567406f6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56740d03f0 tx=0x7f566401c870 comp rx=0 tx=0).stop 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 -- 172.21.15.116:0/2413084327 shutdown_connections 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 --2- 172.21.15.116:0/2413084327 >> v2:172.21.15.149:3300/0 conn(0x7f567406d710 0x7f567406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 --2- 172.21.15.116:0/2413084327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56740cd620 0x7f567406f6b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 -- 172.21.15.116:0/2413084327 >> 172.21.15.116:0/2413084327 conn(0x7f56740cde80 msgr2=0x7f56740ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 -- 172.21.15.116:0/2413084327 shutdown_connections 2024-05-14T11:21:53.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.265+0000 7f567aa00640 1 -- 172.21.15.116:0/2413084327 wait complete. 2024-05-14T11:21:53.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 Processor -- start 2024-05-14T11:21:53.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 -- start start 2024-05-14T11:21:53.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f567406d710 0x7f567414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5674146db0 0x7f5674157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f567416ae40 con 0x7f56740cd620 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f567416acc0 con 0x7f5674146db0 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f567aa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f567416afc0 con 0x7f567406d710 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52808/0 (socket says 172.21.15.116:52808) 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 -- 172.21.15.116:0/257277031 learned_addr learned my addr 172.21.15.116:0/257277031 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673e00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f567406d710 0x7f567414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5678a00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5674146db0 0x7f5674157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 -- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f567406d710 msgr2=0x7f567414f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f567406d710 0x7f567414f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 -- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5674146db0 msgr2=0x7f5674157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5674146db0 0x7f5674157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 -- 172.21.15.116:0/257277031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567414fd10 con 0x7f56740cd620 2024-05-14T11:21:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.269+0000 7f5673400640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f566000c500 tx=0x7f566000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660017070 con 0x7f56740cd620 2024-05-14T11:21:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5660005cf0 con 0x7f56740cd620 2024-05-14T11:21:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600072f0 con 0x7f56740cd620 2024-05-14T11:21:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741555d0 con 0x7f56740cd620 2024-05-14T11:21:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567414efd0 con 0x7f56740cd620 2024-05-14T11:21:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567406aa30 con 0x7f56740cd620 2024-05-14T11:21:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f566001a070 con 0x7f56740cd620 2024-05-14T11:21:53.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.273+0000 7f566be00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 0x7f5654071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:53.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.277+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5249+0+0 (secure 0 0 0) 0x7f566008e400 con 0x7f56740cd620 2024-05-14T11:21:53.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.277+0000 7f5673e00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 0x7f5654071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:53.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.277+0000 7f5673e00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 0x7f5654071940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56640af000 tx=0x7f56640a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.281+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f566005aee0 con 0x7f56740cd620 2024-05-14T11:21:53.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:53.449+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56741552f0 con 0x7f56740cd620 2024-05-14T11:21:54.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.345+0000 7f566be00640 1 -- 172.21.15.116:0/257277031 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7f566005f1e0 con 0x7f56740cd620 2024-05-14T11:21:54.345 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:21:54.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 msgr2=0x7f5654071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 0x7f5654071940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f56640af000 tx=0x7f56640a7000 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 msgr2=0x7f567415eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f566000c500 tx=0x7f566000c900 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 shutdown_connections 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f565406f430 0x7f5654071940 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5674146db0 0x7f5674157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56740cd620 0x7f567415eec0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 --2- 172.21.15.116:0/257277031 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f567406d710 0x7f567414f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 >> 172.21.15.116:0/257277031 conn(0x7f56740cde80 msgr2=0x7f56740ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 shutdown_connections 2024-05-14T11:21:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.349+0000 7f567aa00640 1 -- 172.21.15.116:0/257277031 wait complete. 2024-05-14T11:21:54.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:21:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.529+0000 7fe15a400640 1 Processor -- start 2024-05-14T11:21:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.529+0000 7fe15a400640 1 -- start start 2024-05-14T11:21:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe15a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe15415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe15a400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe154161770 con 0x7fe15406ff70 2024-05-14T11:21:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe15a400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe154161a50 con 0x7fe1540cdd20 2024-05-14T11:21:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe15a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe154162530 con 0x7fe154074530 2024-05-14T11:21:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe14be00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe15406ff70 0x7fe154070380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49936/0 (socket says 172.21.15.116:49936) 2024-05-14T11:21:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe14be00640 1 -- 172.21.15.116:0/802466723 learned_addr learned my addr 172.21.15.116:0/802466723 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe15415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4115467512 0 0) 0x7fe154161770 con 0x7fe15406ff70 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe138003530 con 0x7fe15406ff70 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861567281 0 0) 0x7fe154161a50 con 0x7fe1540cdd20 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe154161770 con 0x7fe1540cdd20 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1138501229 0 0) 0x7fe138003530 con 0x7fe15406ff70 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 msgr2=0x7fe15415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe15415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 >> v1:172.21.15.149:6789/0 conn(0x7fe1540cdd20 legacy=0x7fe1540ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe154161d30 con 0x7fe15406ff70 2024-05-14T11:21:54.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fe13c001700 con 0x7fe15406ff70 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 >> v1:172.21.15.116:6789/0 conn(0x7fe15406ff70 legacy=0x7fe154070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138003c20 0x7fe138004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe138004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe138004da0 0x7fe138005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe154162530 con 0x7fe138004da0 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe13800bbe0 con 0x7fe138003c20 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe13800bde0 con 0x7fe154074530 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138003c20 0x7fe138004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159a00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe138004da0 0x7fe138005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe14be00640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe138004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 -- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 msgr2=0x7fe138004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe154074530 0x7fe138004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 -- 172.21.15.116:0/802466723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe138004da0 msgr2=0x7fe138005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe138004da0 0x7fe138005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.533+0000 7fe159000640 1 -- 172.21.15.116:0/802466723 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe138003530 con 0x7fe138003c20 2024-05-14T11:21:54.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe159000640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138003c20 0x7fe138004070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fc0 tx=0x7fe14401cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d297aa687c0f6c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:54.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440b2070 con 0x7fe138003c20 2024-05-14T11:21:54.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1440a5cb0 con 0x7fe138003c20 2024-05-14T11:21:54.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe153e00640 1 -- 172.21.15.116:0/802466723 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144009c10 con 0x7fe138003c20 2024-05-14T11:21:54.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 -- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138003c20 msgr2=0x7fe138004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 --2- 172.21.15.116:0/802466723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138003c20 0x7fe138004070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fc0 tx=0x7fe14401cca0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe159a00640 1 -- 172.21.15.116:0/802466723 reap_dead start 2024-05-14T11:21:54.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 -- 172.21.15.116:0/802466723 shutdown_connections 2024-05-14T11:21:54.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 -- 172.21.15.116:0/802466723 >> 172.21.15.116:0/802466723 conn(0x7fe1540b4820 msgr2=0x7fe1540b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:54.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 -- 172.21.15.116:0/802466723 shutdown_connections 2024-05-14T11:21:54.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.537+0000 7fe15a400640 1 -- 172.21.15.116:0/802466723 wait complete. 2024-05-14T11:21:54.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 Processor -- start 2024-05-14T11:21:54.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- start start 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe138003c20 0x7fe154159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154157c50 0x7fe1540af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540ce840 con 0x7fe1540cb040 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540b0a70 con 0x7fe154157c50 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe154074c40 con 0x7fe138003c20 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52826/0 (socket says 172.21.15.116:52826) 2024-05-14T11:21:54.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 -- 172.21.15.116:0/2683443056 learned_addr learned my addr 172.21.15.116:0/2683443056 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe159000640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe138003c20 0x7fe154159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe159a00640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154157c50 0x7fe1540af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 -- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe138003c20 msgr2=0x7fe154159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe138003c20 0x7fe154159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 -- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154157c50 msgr2=0x7fe1540af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154157c50 0x7fe1540af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 -- 172.21.15.116:0/2683443056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1540b0050 con 0x7fe1540cb040 2024-05-14T11:21:54.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe14be00640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe13c004f80 tx=0x7fe13c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00f750 con 0x7fe1540cb040 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c018070 con 0x7fe1540cb040 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe154156c30 con 0x7fe1540cb040 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00fb10 con 0x7fe1540cb040 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.541+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541571f0 con 0x7fe1540cb040 2024-05-14T11:21:54.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.545+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1540508f0 con 0x7fe1540cb040 2024-05-14T11:21:54.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.545+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe13c01d460 con 0x7fe1540cb040 2024-05-14T11:21:54.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.545+0000 7fe151600640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 0x7fe130071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:54.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.545+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5619+0+0 (secure 0 0 0) 0x7fe13c08fe30 con 0x7fe1540cb040 2024-05-14T11:21:54.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.545+0000 7fe159000640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 0x7fe130071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:54.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.549+0000 7fe159000640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 0x7fe130071910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1440af000 tx=0x7fe144004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:54.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.549+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe13c05cde0 con 0x7fe1540cb040 2024-05-14T11:21:54.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:54.709+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe1540b3a40 con 0x7fe1540cb040 2024-05-14T11:21:55.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.353+0000 7fe151600640 1 -- 172.21.15.116:0/2683443056 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7fe13c05c6e0 con 0x7fe1540cb040 2024-05-14T11:21:55.353 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 msgr2=0x7fe130071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 0x7fe130071910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1440af000 tx=0x7fe144004ac0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 msgr2=0x7fe154157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe13c004f80 tx=0x7fe13c0074f0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 shutdown_connections 2024-05-14T11:21:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe13006f400 0x7fe130071910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154157c50 0x7fe1540af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1540cb040 0x7fe154157710 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 --2- 172.21.15.116:0/2683443056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe138003c20 0x7fe154159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 >> 172.21.15.116:0/2683443056 conn(0x7fe1540b4820 msgr2=0x7fe1540b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 shutdown_connections 2024-05-14T11:21:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.357+0000 7fe15a400640 1 -- 172.21.15.116:0/2683443056 wait complete. 2024-05-14T11:21:55.391 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:21:55.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.541+0000 7f6135c00640 1 Processor -- start 2024-05-14T11:21:55.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.541+0000 7f6135c00640 1 -- start start 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6135c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f613006f450 0x7f613006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6135c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f61300d1730 0x7f61300d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6135c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613006f860 con 0x7f61300cce60 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6135c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f613006db80 con 0x7f613006f450 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6135c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f61300d2080 con 0x7f61300d1730 2024-05-14T11:21:55.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f613006f450 0x7f613006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f613006f450 0x7f613006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52834/0 (socket says 172.21.15.116:52834) 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 -- 172.21.15.116:0/4279295304 learned_addr learned my addr 172.21.15.116:0/4279295304 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f6126000640 1 --2- 172.21.15.116:0/4279295304 >> v2:172.21.15.149:3300/0 conn(0x7f61300d1730 0x7f61300d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 -- 172.21.15.116:0/4279295304 >> v2:172.21.15.149:3300/0 conn(0x7f61300d1730 msgr2=0x7f61300d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 --2- 172.21.15.116:0/4279295304 >> v2:172.21.15.149:3300/0 conn(0x7f61300d1730 0x7f61300d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 -- 172.21.15.116:0/4279295304 >> v1:172.21.15.149:6789/0 conn(0x7f61300cce60 legacy=0x7f61300cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.545+0000 7f612ea00640 1 -- 172.21.15.116:0/4279295304 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613006fae0 con 0x7f613006f450 2024-05-14T11:21:55.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f612ea00640 1 --2- 172.21.15.116:0/4279295304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613006d640 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f61300d2430 tx=0x7f612001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c836a74bd776631 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:55.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f612e000640 1 -- 172.21.15.116:0/4279295304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300ce190 con 0x7f613006f450 2024-05-14T11:21:55.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f612e000640 1 -- 172.21.15.116:0/4279295304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61200b0070 con 0x7f613006f450 2024-05-14T11:21:55.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f612e000640 1 -- 172.21.15.116:0/4279295304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200ab830 con 0x7f613006f450 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 -- 172.21.15.116:0/4279295304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 msgr2=0x7f613006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 --2- 172.21.15.116:0/4279295304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613006d640 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f61300d2430 tx=0x7f612001c520 comp rx=0 tx=0).stop 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 -- 172.21.15.116:0/4279295304 shutdown_connections 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 --2- 172.21.15.116:0/4279295304 >> v2:172.21.15.149:3300/0 conn(0x7f61300d1730 0x7f61300d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 --2- 172.21.15.116:0/4279295304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613006d640 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 -- 172.21.15.116:0/4279295304 >> 172.21.15.116:0/4279295304 conn(0x7f61300b4720 msgr2=0x7f61300b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 -- 172.21.15.116:0/4279295304 shutdown_connections 2024-05-14T11:21:55.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.549+0000 7f6135c00640 1 -- 172.21.15.116:0/4279295304 wait complete. 2024-05-14T11:21:55.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 Processor -- start 2024-05-14T11:21:55.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- start start 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61300cce60 0x7f6130073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61300d1730 0x7f6130073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f613016a610 con 0x7f613006f450 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6130070660 con 0x7f61300cce60 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f613016a790 con 0x7f61300d1730 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52846/0 (socket says 172.21.15.116:52846) 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 -- 172.21.15.116:0/2558598568 learned_addr learned my addr 172.21.15.116:0/2558598568 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:55.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612f400640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61300d1730 0x7f6130073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6126000640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61300cce60 0x7f6130073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 -- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61300d1730 msgr2=0x7f6130073b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61300d1730 0x7f6130073b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 -- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61300cce60 msgr2=0x7f6130073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61300cce60 0x7f6130073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 -- 172.21.15.116:0/2558598568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613015a5d0 con 0x7f613006f450 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f612ea00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f612001a150 tx=0x7f612001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200be5f0 con 0x7f613006f450 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61200be7b0 con 0x7f613006f450 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6130072af0 con 0x7f613006f450 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200beba0 con 0x7f613006f450 2024-05-14T11:21:55.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61300b1b50 con 0x7f613006f450 2024-05-14T11:21:55.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.553+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613006a9c0 con 0x7f613006f450 2024-05-14T11:21:55.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.557+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61200b43f0 con 0x7f613006f450 2024-05-14T11:21:55.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.557+0000 7f6127400640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:55.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.557+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6001+0+0 (secure 0 0 0) 0x7f6120129e90 con 0x7f613006f450 2024-05-14T11:21:55.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.557+0000 7f6126000640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.561+0000 7f6126000640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 0x7f610c071910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61300cff70 tx=0x7f6118008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:55.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.561+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f61200c4060 con 0x7f613006f450 2024-05-14T11:21:55.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:55.753+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6130069de0 con 0x7f613006f450 2024-05-14T11:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.357+0000 7f6127400640 1 -- 172.21.15.116:0/2558598568 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v58) ==== 174+0+0 (secure 0 0 0) 0x7f61200a8050 con 0x7f613006f450 2024-05-14T11:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 13 and data pool 14 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.357+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 msgr2=0x7f610c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.357+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 0x7f610c071910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61300cff70 tx=0x7f6118008040 comp rx=0 tx=0).stop 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.357+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 msgr2=0x7f613009a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.357+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f612001a150 tx=0x7f612001ad60 comp rx=0 tx=0).stop 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 shutdown_connections 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61300cce60 0x7f6130073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61300d1730 0x7f6130073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 --2- 172.21.15.116:0/2558598568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f613006f450 0x7f613009a9d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 >> 172.21.15.116:0/2558598568 conn(0x7f61300b4720 msgr2=0x7f61300ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 shutdown_connections 2024-05-14T11:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.361+0000 7f6135c00640 1 -- 172.21.15.116:0/2558598568 wait complete. 2024-05-14T11:21:56.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:56.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 Processor -- start 2024-05-14T11:21:56.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 -- start start 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f74440af420 0x7f7444074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744415d340 con 0x7f74440d1d80 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744415d620 con 0x7f7444074b70 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.553+0000 7f744a600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f744415d900 con 0x7f74440af420 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74440d1d80 0x7f744406edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49948/0 (socket says 172.21.15.116:49948) 2024-05-14T11:21:56.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 -- 172.21.15.116:0/3524489638 learned_addr learned my addr 172.21.15.116:0/3524489638 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:56.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443400640 1 --2- 172.21.15.116:0/3524489638 >> v2:172.21.15.149:3300/0 conn(0x7f74440af420 0x7f7444074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310187368 0 0) 0x7f744415d340 con 0x7f74440d1d80 2024-05-14T11:21:56.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7428003530 con 0x7f74440d1d80 2024-05-14T11:21:56.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3618520627 0 0) 0x7f7428003530 con 0x7f74440d1d80 2024-05-14T11:21:56.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 >> v2:172.21.15.149:3300/0 conn(0x7f74440af420 msgr2=0x7f7444074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 --2- 172.21.15.116:0/3524489638 >> v2:172.21.15.149:3300/0 conn(0x7f74440af420 0x7f7444074630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 >> v1:172.21.15.149:6789/0 conn(0x7f7444074b70 legacy=0x7f74440d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744415dbe0 con 0x7f74440d1d80 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f743c001700 con 0x7f74440d1d80 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443400640 1 --2- 172.21.15.116:0/3524489638 >> v2:172.21.15.149:3300/0 conn(0x7f74440af420 0x7f7444074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 >> v1:172.21.15.116:6789/0 conn(0x7f74440d1d80 legacy=0x7f744406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428003a60 0x7f7428003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7428004810 0x7f7428004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 0x7f742800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f744415d900 con 0x7f7428005260 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f742800c1f0 con 0x7f7428003a60 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f742800e600 con 0x7f7428004810 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 0x7f742800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443400640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428003a60 0x7f7428003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7442a00640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7428004810 0x7f7428004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 -- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7428004810 msgr2=0x7f7428004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7428004810 0x7f7428004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 -- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428003a60 msgr2=0x7f7428003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428003a60 0x7f7428003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443e00640 1 -- 172.21.15.116:0/3524489638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7428003530 con 0x7f7428005260 2024-05-14T11:21:56.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.557+0000 7f7443400640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428003a60 0x7f7428003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:21:56.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f7443e00640 1 -- 172.21.15.116:0/3524489638 reap_dead start 2024-05-14T11:21:56.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f7443e00640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 0x7f742800b9d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c210 tx=0x7f743c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f071f53c5172a1c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434009de0 con 0x7f7428005260 2024-05-14T11:21:56.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c0a9070 con 0x7f7428005260 2024-05-14T11:21:56.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f7442000640 1 -- 172.21.15.116:0/3524489638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c007d80 con 0x7f7428005260 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 -- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 msgr2=0x7f742800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 0x7f742800b9d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c210 tx=0x7f743c01cba0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 -- 172.21.15.116:0/3524489638 shutdown_connections 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 --2- 172.21.15.116:0/3524489638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7428005260 0x7f742800b9d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 -- 172.21.15.116:0/3524489638 >> 172.21.15.116:0/3524489638 conn(0x7f74440b4620 msgr2=0x7f74440b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 -- 172.21.15.116:0/3524489638 shutdown_connections 2024-05-14T11:21:56.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.561+0000 7f744a600640 1 -- 172.21.15.116:0/3524489638 wait complete. 2024-05-14T11:21:56.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 Processor -- start 2024-05-14T11:21:56.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- start start 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428005260 0x7f744414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f74440d08c0 0x7f744415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f744415cec0 con 0x7f74441f5b90 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74440afa40 con 0x7f7428005260 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f744415d040 con 0x7f74440d08c0 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52860/0 (socket says 172.21.15.116:52860) 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 -- 172.21.15.116:0/3160290068 learned_addr learned my addr 172.21.15.116:0/3160290068 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:56.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7443400640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428005260 0x7f744414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7443e00640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f74440d08c0 0x7f744415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 -- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f74440d08c0 msgr2=0x7f744415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f74440d08c0 0x7f744415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 -- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428005260 msgr2=0x7f744414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428005260 0x7f744414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 -- 172.21.15.116:0/3160290068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744414bd70 con 0x7f74441f5b90 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f7442a00640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f743000f5c0 tx=0x7f743000f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743001a070 con 0x7f74441f5b90 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7430005320 con 0x7f74441f5b90 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300069e0 con 0x7f74441f5b90 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74440cde50 con 0x7f74441f5b90 2024-05-14T11:21:56.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.565+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74440ce410 con 0x7f74441f5b90 2024-05-14T11:21:56.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744406a9c0 con 0x7f74441f5b90 2024-05-14T11:21:56.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f743001d070 con 0x7f74441f5b90 2024-05-14T11:21:56.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f743b400640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 0x7f7424071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f743008ea50 con 0x7f74441f5b90 2024-05-14T11:21:56.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f7443400640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 0x7f7424071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.569+0000 7f7443400640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 0x7f7424071940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7444070ac0 tx=0x7f743400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.573+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f743005b150 con 0x7f74441f5b90 2024-05-14T11:21:56.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f744406fe00 con 0x7f74441f5b90 2024-05-14T11:21:56.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f743b400640 1 -- 172.21.15.116:0/3160290068 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16946 (secure 0 0 0) 0x7f743005f450 con 0x7f74441f5b90 2024-05-14T11:21:56.733 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:56.733 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:21:55.758154+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:21:53.462471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-14T11:21:54.724572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 msgr2=0x7f7424071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 0x7f7424071940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7444070ac0 tx=0x7f743400b040 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 msgr2=0x7f74440d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f743000f5c0 tx=0x7f743000f9c0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 shutdown_connections 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f742406f430 0x7f7424071940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f74440d08c0 0x7f744415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74441f5b90 0x7f74440d0380 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 --2- 172.21.15.116:0/3160290068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428005260 0x7f744414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 >> 172.21.15.116:0/3160290068 conn(0x7f74440b4620 msgr2=0x7f74440d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 shutdown_connections 2024-05-14T11:21:56.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.733+0000 7f744a600640 1 -- 172.21.15.116:0/3160290068 wait complete. 2024-05-14T11:21:56.755 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:56.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 Processor -- start 2024-05-14T11:21:56.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 -- start start 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f568015f9b0 0x7f568006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f568015fdc0 0x7f568006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f568006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5680163b10 con 0x7f568015f9b0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5680163df0 con 0x7f568015fdc0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f5685200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56801640d0 con 0x7f568006f1c0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f568015f9b0 0x7f568006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f568015f9b0 0x7f568006dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52870/0 (socket says 172.21.15.116:52870) 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 -- 172.21.15.116:0/2785271314 learned_addr learned my addr 172.21.15.116:0/2785271314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567fe00640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f568006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567ea00640 1 --2- 172.21.15.116:0/2785271314 >> v2:172.21.15.149:3300/0 conn(0x7f568015fdc0 0x7f568006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 -- 172.21.15.116:0/2785271314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 msgr2=0x7f568006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:56.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f568006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 -- 172.21.15.116:0/2785271314 >> v2:172.21.15.149:3300/0 conn(0x7f568015fdc0 msgr2=0x7f568006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:56.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 --2- 172.21.15.116:0/2785271314 >> v2:172.21.15.149:3300/0 conn(0x7f568015fdc0 0x7f568006e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567f400640 1 -- 172.21.15.116:0/2785271314 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56801643b0 con 0x7f568015f9b0 2024-05-14T11:21:56.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.901+0000 7f567fe00640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f568006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:56.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f567f400640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f568006dfa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5680072db0 tx=0x7f567001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca11d9848b7b0d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f567e000640 1 -- 172.21.15.116:0/2785271314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700a7070 con 0x7f568015f9b0 2024-05-14T11:21:56.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f567e000640 1 -- 172.21.15.116:0/2785271314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56700b0070 con 0x7f568015f9b0 2024-05-14T11:21:56.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f567e000640 1 -- 172.21.15.116:0/2785271314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab890 con 0x7f568015f9b0 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 -- 172.21.15.116:0/2785271314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 msgr2=0x7f568006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f568006dfa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5680072db0 tx=0x7f567001c490 comp rx=0 tx=0).stop 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 -- 172.21.15.116:0/2785271314 shutdown_connections 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f568006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 --2- 172.21.15.116:0/2785271314 >> v2:172.21.15.149:3300/0 conn(0x7f568015fdc0 0x7f568006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 --2- 172.21.15.116:0/2785271314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f568006dfa0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 -- 172.21.15.116:0/2785271314 >> 172.21.15.116:0/2785271314 conn(0x7f56800731a0 msgr2=0x7f56800735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 -- 172.21.15.116:0/2785271314 shutdown_connections 2024-05-14T11:21:56.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.905+0000 7f5685200640 1 -- 172.21.15.116:0/2785271314 wait complete. 2024-05-14T11:21:56.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 Processor -- start 2024-05-14T11:21:56.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 -- start start 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f56801425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f568015fdc0 0x7f56801619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56801638b0 con 0x7f568015f9b0 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5680163730 con 0x7f568015fdc0 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f5685200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56800b5560 con 0x7f568006f1c0 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f567ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f567ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52884/0 (socket says 172.21.15.116:52884) 2024-05-14T11:21:56.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f567ea00640 1 -- 172.21.15.116:0/2227110303 learned_addr learned my addr 172.21.15.116:0/2227110303 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f567f400640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f56801425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.909+0000 7f567fe00640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f568015fdc0 0x7f56801619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 -- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 msgr2=0x7f56801425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f56801425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 -- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f568015fdc0 msgr2=0x7f56801619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f568015fdc0 0x7f56801619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 -- 172.21.15.116:0/2227110303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568015f290 con 0x7f568015f9b0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f567ea00640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d620 tx=0x7f566c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00b620 con 0x7f568015f9b0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566c019070 con 0x7f568015f9b0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5680154750 con 0x7f568015f9b0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00b9e0 con 0x7f568015f9b0 2024-05-14T11:21:56.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5680154d10 con 0x7f568015f9b0 2024-05-14T11:21:56.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56800704f0 con 0x7f568015f9b0 2024-05-14T11:21:56.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.913+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f566c00bb80 con 0x7f568015f9b0 2024-05-14T11:21:56.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.917+0000 7f5677400640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 0x7f5660071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:56.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.917+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f566c08f720 con 0x7f568015f9b0 2024-05-14T11:21:56.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.917+0000 7f567f400640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 0x7f5660071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:56.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.917+0000 7f567f400640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 0x7f5660071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f56700af000 tx=0x7f56700a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:56.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:56.921+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f566c05be20 con 0x7f568015f9b0 2024-05-14T11:21:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.077+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f568015f020 con 0x7f568015f9b0 2024-05-14T11:21:57.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.077+0000 7f5677400640 1 -- 172.21.15.116:0/2227110303 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16946 (secure 0 0 0) 0x7f566c060120 con 0x7f568015f9b0 2024-05-14T11:21:57.079 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:57.079 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:21:55.758154+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:21:53.462471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-14T11:21:54.724572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 msgr2=0x7f5660071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 0x7f5660071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f56700af000 tx=0x7f56700a7000 comp rx=0 tx=0).stop 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 msgr2=0x7f5680159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d620 tx=0x7f566c00dae0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 shutdown_connections 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f566006f400 0x7f5660071910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f568015fdc0 0x7f56801619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f568015f9b0 0x7f5680159ab0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 --2- 172.21.15.116:0/2227110303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f568006f1c0 0x7f56801425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 >> 172.21.15.116:0/2227110303 conn(0x7f56800731a0 msgr2=0x7f56801601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 shutdown_connections 2024-05-14T11:21:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.081+0000 7f5685200640 1 -- 172.21.15.116:0/2227110303 wait complete. 2024-05-14T11:21:57.100 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:21:57.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.249+0000 7f50d3400640 1 Processor -- start 2024-05-14T11:21:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.249+0000 7f50d3400640 1 -- start start 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d3400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50cc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d3400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f50cc0d1730 0x7f50cc0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d3400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50cc06f0a0 con 0x7f50cc160670 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d3400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc0cbb60 con 0x7f50cc0d1730 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d3400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc0d2080 con 0x7f50cc0cd1c0 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d1400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f50cc160670 0x7f50cc160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:49954/0 (socket says 172.21.15.116:49954) 2024-05-14T11:21:57.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d1400640 1 -- 172.21.15.116:0/4180306876 learned_addr learned my addr 172.21.15.116:0/4180306876 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:57.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 --2- 172.21.15.116:0/4180306876 >> v2:172.21.15.149:3300/0 conn(0x7f50cc0d1730 0x7f50cc0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1700750657 0 0) 0x7f50cc06f0a0 con 0x7f50cc160670 2024-05-14T11:21:57.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d0a00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50cc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50b0003530 con 0x7f50cc160670 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2727146702 0 0) 0x7f50b0003530 con 0x7f50cc160670 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 msgr2=0x7f50cc0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50cc0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 >> v2:172.21.15.149:3300/0 conn(0x7f50cc0d1730 msgr2=0x7f50cc0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 --2- 172.21.15.116:0/4180306876 >> v2:172.21.15.149:3300/0 conn(0x7f50cc0d1730 0x7f50cc0d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc06f420 con 0x7f50cc160670 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f50c4001a80 con 0x7f50cc160670 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d0a00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50cc0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 >> v1:172.21.15.116:6789/0 conn(0x7f50cc160670 legacy=0x7f50cc160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50b0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50b0004040 0x7f50b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004d50 0x7f50b0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0cbb60 con 0x7f50b0004040 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0d2080 con 0x7f50b0004d50 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50b000ddf0 con 0x7f50cc0cd1c0 2024-05-14T11:21:57.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50b0004040 0x7f50b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d1400640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004d50 0x7f50b0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50d0a00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50b0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 -- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 msgr2=0x7f50b0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc0cd1c0 0x7f50b0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 -- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004d50 msgr2=0x7f50b0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004d50 0x7f50b0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.253+0000 7f50c3e00640 1 -- 172.21.15.116:0/4180306876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b0003530 con 0x7f50b0004040 2024-05-14T11:21:57.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50c3e00640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50b0004040 0x7f50b0004810 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f50cc070cd0 tx=0x7f50b401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5281691692303f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:57.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b40b2070 con 0x7f50b0004040 2024-05-14T11:21:57.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b40a5530 con 0x7f50b0004040 2024-05-14T11:21:57.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50cbe00640 1 -- 172.21.15.116:0/4180306876 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b40096b0 con 0x7f50b0004040 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 -- 172.21.15.116:0/4180306876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50b0004040 msgr2=0x7f50b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 --2- 172.21.15.116:0/4180306876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50b0004040 0x7f50b0004810 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f50cc070cd0 tx=0x7f50b401ca50 comp rx=0 tx=0).stop 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d1400640 1 -- 172.21.15.116:0/4180306876 reap_dead start 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 -- 172.21.15.116:0/4180306876 shutdown_connections 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 -- 172.21.15.116:0/4180306876 >> 172.21.15.116:0/4180306876 conn(0x7f50cc0b4720 msgr2=0x7f50cc0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 -- 172.21.15.116:0/4180306876 shutdown_connections 2024-05-14T11:21:57.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.257+0000 7f50d3400640 1 -- 172.21.15.116:0/4180306876 wait complete. 2024-05-14T11:21:57.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 Processor -- start 2024-05-14T11:21:57.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 -- start start 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 0x7f50cc0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 0x7f50cc0686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc06fcc0 con 0x7f50cc154150 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc06f9e0 con 0x7f50b0004040 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d3400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc06ffa0 con 0x7f50cc151a40 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52914/0 (socket says 172.21.15.116:52914) 2024-05-14T11:21:57.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 -- 172.21.15.116:0/1866700672 learned_addr learned my addr 172.21.15.116:0/1866700672 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:57.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d0a00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 0x7f50cc0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d1400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 0x7f50cc0686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 -- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 msgr2=0x7f50cc0686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 0x7f50cc0686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 -- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 msgr2=0x7f50cc0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 0x7f50cc0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 -- 172.21.15.116:0/1866700672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc15f340 con 0x7f50cc154150 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d1400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 0x7f50cc0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50d0a00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 0x7f50cc0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:21:57.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.261+0000 7f50c3e00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f50b40b0040 tx=0x7f50b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b40b2040 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b40a5ce0 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc0c8450 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4004260 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc0c8a10 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc15f0b0 con 0x7f50cc154150 2024-05-14T11:21:57.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50b40c1030 con 0x7f50cc154150 2024-05-14T11:21:57.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 0x7f50a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:57.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6101+0+0 (secure 0 0 0) 0x7f50b41290a0 con 0x7f50cc154150 2024-05-14T11:21:57.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.265+0000 7f50d0a00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 0x7f50a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:57.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.269+0000 7f50d0a00640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 0x7f50a8071940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009910 tx=0x7f50bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.269+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f50b4016760 con 0x7f50cc154150 2024-05-14T11:21:57.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:57.469+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f50cc06a9c0 con 0x7f50cc154150 2024-05-14T11:21:58.335 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:21:58.335+0000 7fc49e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:21:58.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.377+0000 7f50c9600640 1 -- 172.21.15.116:0/1866700672 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f50b40f5840 con 0x7f50cc154150 2024-05-14T11:21:58.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 msgr2=0x7f50a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 0x7f50a8071940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009910 tx=0x7f50bc008040 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 msgr2=0x7f50cc151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f50b40b0040 tx=0x7f50b0005840 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 shutdown_connections 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f50a806f430 0x7f50a8071940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f50cc151a40 0x7f50cc0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50cc154150 0x7f50cc151500 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 --2- 172.21.15.116:0/1866700672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b0004040 0x7f50cc0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 >> 172.21.15.116:0/1866700672 conn(0x7f50cc0b4720 msgr2=0x7f50cc1581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 shutdown_connections 2024-05-14T11:21:58.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.381+0000 7f50d3400640 1 -- 172.21.15.116:0/1866700672 wait complete. 2024-05-14T11:21:58.403 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:21:58.403 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:21:58.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 Processor -- start 2024-05-14T11:21:58.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 -- start start 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde280cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fde280d1730 0x7fde280d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fde28160670 0x7fde28160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fde2806f0a0 con 0x7fde280d1730 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fde280cbb60 con 0x7fde28160670 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde30600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fde280d2080 con 0x7fde280cd1c0 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fde280d1730 0x7fde280d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fde280d1730 0x7fde280d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42410/0 (socket says 172.21.15.116:42410) 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 -- 172.21.15.116:0/1225388994 learned_addr learned my addr 172.21.15.116:0/1225388994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2dc00640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde280cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2e600640 1 --2- 172.21.15.116:0/1225388994 >> v2:172.21.15.149:3300/0 conn(0x7fde28160670 0x7fde28160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 -- 172.21.15.116:0/1225388994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 msgr2=0x7fde280cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:58.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde280cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 -- 172.21.15.116:0/1225388994 >> v2:172.21.15.149:3300/0 conn(0x7fde28160670 msgr2=0x7fde28160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:58.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 --2- 172.21.15.116:0/1225388994 >> v2:172.21.15.149:3300/0 conn(0x7fde28160670 0x7fde28160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.557+0000 7fde2d200640 1 -- 172.21.15.116:0/1225388994 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2806f420 con 0x7fde280d1730 2024-05-14T11:21:58.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde2d200640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde280d1b40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fde14009870 tx=0x7fde1401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde44d22d77a98cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:58.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde27e00640 1 -- 172.21.15.116:0/1225388994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140a6040 con 0x7fde280d1730 2024-05-14T11:21:58.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde27e00640 1 -- 172.21.15.116:0/1225388994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde140af070 con 0x7fde280d1730 2024-05-14T11:21:58.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde27e00640 1 -- 172.21.15.116:0/1225388994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140aa860 con 0x7fde280d1730 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 -- 172.21.15.116:0/1225388994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 msgr2=0x7fde280d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde280d1b40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fde14009870 tx=0x7fde1401c520 comp rx=0 tx=0).stop 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 -- 172.21.15.116:0/1225388994 shutdown_connections 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde280d1b40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 --2- 172.21.15.116:0/1225388994 >> v2:172.21.15.149:3300/0 conn(0x7fde28160670 0x7fde28160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 --2- 172.21.15.116:0/1225388994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde280cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 -- 172.21.15.116:0/1225388994 >> 172.21.15.116:0/1225388994 conn(0x7fde280b4720 msgr2=0x7fde280b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 -- 172.21.15.116:0/1225388994 shutdown_connections 2024-05-14T11:21:58.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.561+0000 7fde30600640 1 -- 172.21.15.116:0/1225388994 wait complete. 2024-05-14T11:21:58.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 Processor -- start 2024-05-14T11:21:58.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 -- start start 2024-05-14T11:21:58.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde281f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde28160670 0x7fde28074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde2815cf10 con 0x7fde280d1730 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde2806ffa0 con 0x7fde28160670 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde30600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fde281613d0 con 0x7fde280cd1c0 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde2d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde2d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42418/0 (socket says 172.21.15.116:42418) 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.565+0000 7fde2d200640 1 -- 172.21.15.116:0/2296872107 learned_addr learned my addr 172.21.15.116:0/2296872107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2e600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde28160670 0x7fde28074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2dc00640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde281f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 -- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 msgr2=0x7fde281f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde281f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 -- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde28160670 msgr2=0x7fde28074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde28160670 0x7fde28074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:58.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 -- 172.21.15.116:0/2296872107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28074c60 con 0x7fde280d1730 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde2d200640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fde1401cdc0 tx=0x7fde1401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140aadb0 con 0x7fde280d1730 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde140bd430 con 0x7fde280d1730 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140bd820 con 0x7fde280d1730 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281fdab0 con 0x7fde280d1730 2024-05-14T11:21:58.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281fe070 con 0x7fde280d1730 2024-05-14T11:21:58.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2806a9c0 con 0x7fde280d1730 2024-05-14T11:21:58.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.569+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde140b33f0 con 0x7fde280d1730 2024-05-14T11:21:58.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.573+0000 7fde25600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 0x7fde08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:58.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.573+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fde14128ee0 con 0x7fde280d1730 2024-05-14T11:21:58.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.573+0000 7fde2dc00640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 0x7fde08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:58.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.573+0000 7fde2dc00640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 0x7fde08071910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde1c011200 tx=0x7fde1c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:58.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.577+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fde14016110 con 0x7fde280d1730 2024-05-14T11:21:58.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:58.765+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fde280749f0 con 0x7fde280d1730 2024-05-14T11:21:59.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.381+0000 7fde25600640 1 -- 172.21.15.116:0/2296872107 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7fde140f5670 con 0x7fde280d1730 2024-05-14T11:21:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 msgr2=0x7fde08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 0x7fde08071910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde1c011200 tx=0x7fde1c00d040 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 msgr2=0x7fde28073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fde1401cdc0 tx=0x7fde1401ad60 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 shutdown_connections 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fde0806f400 0x7fde08071910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde28160670 0x7fde28074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde280d1730 0x7fde28073f50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 --2- 172.21.15.116:0/2296872107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde280cd1c0 0x7fde281f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 >> 172.21.15.116:0/2296872107 conn(0x7fde280b4720 msgr2=0x7fde28165d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 shutdown_connections 2024-05-14T11:21:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.385+0000 7fde30600640 1 -- 172.21.15.116:0/2296872107 wait complete. 2024-05-14T11:21:59.406 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:21:59.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:21:59.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.553+0000 7f0c07c00640 1 Processor -- start 2024-05-14T11:21:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.553+0000 7f0c07c00640 1 -- start start 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.553+0000 7f0c07c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0015db30 0x7f0c000aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.553+0000 7f0c07c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c0006c700 0x7f0c0006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c07c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0006ff20 con 0x7f0c000d1670 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c07c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0c000ae100 con 0x7f0c0006c700 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c07c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000ae870 con 0x7f0c0015db30 2024-05-14T11:21:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c0006c700 0x7f0c0006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bf7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0c000d1670 0x7f0c000d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37118/0 (socket says 172.21.15.116:37118) 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bf7e00640 1 -- 172.21.15.116:0/3778831240 learned_addr learned my addr 172.21.15.116:0/3778831240 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05200640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0015db30 0x7f0c000aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 -- 172.21.15.116:0/3778831240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0015db30 msgr2=0x7f0c000aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bffe00640 1 -- 172.21.15.116:0/3778831240 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784916401 0 0) 0x7f0c0006ff20 con 0x7f0c000d1670 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0015db30 0x7f0c000aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 -- 172.21.15.116:0/3778831240 >> v1:172.21.15.116:6789/0 conn(0x7f0c000d1670 legacy=0x7f0c000d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 -- 172.21.15.116:0/3778831240 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c00070470 con 0x7f0c0006c700 2024-05-14T11:21:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0c05c00640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0006c700 0x7f0c0006cb10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009d90 tx=0x7f0bf801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1934dddd56d1dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bffe00640 1 -- 172.21.15.116:0/3778831240 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b2070 con 0x7f0c0006c700 2024-05-14T11:21:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bffe00640 1 -- 172.21.15.116:0/3778831240 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf80a5870 con 0x7f0c0006c700 2024-05-14T11:21:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.557+0000 7f0bffe00640 1 -- 172.21.15.116:0/3778831240 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80a5c40 con 0x7f0c0006c700 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- 172.21.15.116:0/3778831240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0006c700 msgr2=0x7f0c0006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0006c700 0x7f0c0006cb10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009d90 tx=0x7f0bf801c8c0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- 172.21.15.116:0/3778831240 shutdown_connections 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0006c700 0x7f0c0006cb10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 --2- 172.21.15.116:0/3778831240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0015db30 0x7f0c000aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- 172.21.15.116:0/3778831240 >> 172.21.15.116:0/3778831240 conn(0x7f0c000b4620 msgr2=0x7f0c000b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- 172.21.15.116:0/3778831240 shutdown_connections 2024-05-14T11:21:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- 172.21.15.116:0/3778831240 wait complete. 2024-05-14T11:21:59.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 Processor -- start 2024-05-14T11:21:59.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.561+0000 7f0c07c00640 1 -- start start 2024-05-14T11:21:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0006c700 0x7f0c0009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c0015db30 0x7f0c0014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0016a770 con 0x7f0c000d1670 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0015df40 con 0x7f0c0015db30 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0016a8f0 con 0x7f0c0006c700 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42436/0 (socket says 172.21.15.116:42436) 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 -- 172.21.15.116:0/2404030445 learned_addr learned my addr 172.21.15.116:0/2404030445 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c05200640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0006c700 0x7f0c0009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c05c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c0015db30 0x7f0c0014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 -- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0006c700 msgr2=0x7f0c0009a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0006c700 0x7f0c0009a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 -- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c0015db30 msgr2=0x7f0c0014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c0015db30 0x7f0c0014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 -- 172.21.15.116:0/2404030445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0009e8e0 con 0x7f0c000d1670 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bf7e00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c3f0 tx=0x7f0bec001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0074e0 con 0x7f0c000d1670 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec007680 con 0x7f0c000d1670 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec004a10 con 0x7f0c000d1670 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0009ebc0 con 0x7f0c000d1670 2024-05-14T11:21:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0009f180 con 0x7f0c000d1670 2024-05-14T11:21:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.565+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0006a9c0 con 0x7f0c000d1670 2024-05-14T11:21:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.569+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bec011070 con 0x7f0c000d1670 2024-05-14T11:21:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.569+0000 7f0bfd600640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 0x7f0bd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.569+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f0bec08ee80 con 0x7f0c000d1670 2024-05-14T11:21:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.569+0000 7f0c05200640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 0x7f0bd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.569+0000 7f0c05200640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 0x7f0bd4071910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009ac0 tx=0x7f0be4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.573+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0bec05b590 con 0x7f0c000d1670 2024-05-14T11:21:59.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.761+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c0009e670 con 0x7f0c000d1670 2024-05-14T11:21:59.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.761+0000 7f0bfd600640 1 -- 172.21.15.116:0/2404030445 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5189 (secure 0 0 0) 0x7f0bec05f890 con 0x7f0c000d1670 2024-05-14T11:21:59.761 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:21:59.761 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:21:58.770840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6835/536312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":536312712},{"type":"v1","addr":"172.21.15.149:6835","nonce":536312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":6}]} 2024-05-14T11:21:59.762 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 61 2024-05-14T11:21:59.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 msgr2=0x7f0bd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 0x7f0bd4071910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009ac0 tx=0x7f0be4009190 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 msgr2=0x7f0c0014d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c3f0 tx=0x7f0bec001e80 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 shutdown_connections 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0bd406f400 0x7f0bd4071910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c0015db30 0x7f0c0014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c000d1670 0x7f0c0014d8d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 --2- 172.21.15.116:0/2404030445 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c0006c700 0x7f0c0009a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 >> 172.21.15.116:0/2404030445 conn(0x7f0c000b4620 msgr2=0x7f0c0015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 shutdown_connections 2024-05-14T11:21:59.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.765+0000 7f0c07c00640 1 -- 172.21.15.116:0/2404030445 wait complete. 2024-05-14T11:21:59.783 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:21:59.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.929+0000 7f1af6c00640 1 Processor -- start 2024-05-14T11:21:59.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.929+0000 7f1af6c00640 1 -- start start 2024-05-14T11:21:59.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af6c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1af006c700 0x7f1af006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af6c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af006ff20 con 0x7f1af015db30 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af6c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1af00ae100 con 0x7f1af006c700 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00ae870 con 0x7f1af00d1670 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1af006c700 0x7f1af006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1af006c700 0x7f1af006cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42448/0 (socket says 172.21.15.116:42448) 2024-05-14T11:21:59.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 -- 172.21.15.116:0/3155965419 learned_addr learned my addr 172.21.15.116:0/3155965419 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1ae7400640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 -- 172.21.15.116:0/3155965419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 msgr2=0x7f1af00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af00d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 -- 172.21.15.116:0/3155965419 >> v1:172.21.15.149:6789/0 conn(0x7f1af015db30 legacy=0x7f1af00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 -- 172.21.15.116:0/3155965419 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00d1fc0 con 0x7f1af006c700 2024-05-14T11:21:59.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1ae7400640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:59.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.933+0000 7f1af4c00640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af006cb10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009870 tx=0x7f1ae801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5be1e7d529c95a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1aef400640 1 -- 172.21.15.116:0/3155965419 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80b0070 con 0x7f1af006c700 2024-05-14T11:21:59.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1aef400640 1 -- 172.21.15.116:0/3155965419 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae80ab600 con 0x7f1af006c700 2024-05-14T11:21:59.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1aef400640 1 -- 172.21.15.116:0/3155965419 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80ab9d0 con 0x7f1af006c700 2024-05-14T11:21:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- 172.21.15.116:0/3155965419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 msgr2=0x7f1af006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af006cb10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009870 tx=0x7f1ae801c400 comp rx=0 tx=0).stop 2024-05-14T11:21:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- 172.21.15.116:0/3155965419 shutdown_connections 2024-05-14T11:21:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af006cb10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 --2- 172.21.15.116:0/3155965419 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- 172.21.15.116:0/3155965419 >> 172.21.15.116:0/3155965419 conn(0x7f1af00b4620 msgr2=0x7f1af00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:21:59.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- 172.21.15.116:0/3155965419 shutdown_connections 2024-05-14T11:21:59.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- 172.21.15.116:0/3155965419 wait complete. 2024-05-14T11:21:59.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 Processor -- start 2024-05-14T11:21:59.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.937+0000 7f1af6c00640 1 -- start start 2024-05-14T11:21:59.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 0x7f1af015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1af016a770 con 0x7f1af006c700 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1af015df40 con 0x7f1af015db30 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1af016a8f0 con 0x7f1af00d1670 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42462/0 (socket says 172.21.15.116:42462) 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 -- 172.21.15.116:0/2084435211 learned_addr learned my addr 172.21.15.116:0/2084435211 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1ae7400640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 -- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 msgr2=0x7f1af014e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af4c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 0x7f1af015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af014e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 -- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 msgr2=0x7f1af015c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:21:59.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 0x7f1af015c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:21:59.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 -- 172.21.15.116:0/2084435211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af014f190 con 0x7f1af006c700 2024-05-14T11:21:59.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af4c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 0x7f1af015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:59.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1ae7400640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:21:59.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aefe00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000ba50 tx=0x7f1ae000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0019070 con 0x7f1af006c700 2024-05-14T11:21:59.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae000cc20 con 0x7f1af006c700 2024-05-14T11:21:59.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0014680 con 0x7f1af006c700 2024-05-14T11:21:59.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af014f6a0 con 0x7f1af006c700 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af014fc60 con 0x7f1af006c700 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.941+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af006a9c0 con 0x7f1af006c700 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.945+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ae001c070 con 0x7f1af006c700 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.945+0000 7f1aecc00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 0x7f1acc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.945+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f1ae008f100 con 0x7f1af006c700 2024-05-14T11:21:59.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.945+0000 7f1ae7400640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 0x7f1acc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:21:59.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.945+0000 7f1ae7400640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 0x7f1acc071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1af015cdc0 tx=0x7f1ad8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:21:59.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:21:59.949+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1ae005b800 con 0x7f1af006c700 2024-05-14T11:22:00.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.109+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1af0071340 con 0x7f1af006c700 2024-05-14T11:22:00.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.109+0000 7f1aecc00640 1 -- 172.21.15.116:0/2084435211 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16933 (secure 0 0 0) 0x7f1ae005fb00 con 0x7f1af006c700 2024-05-14T11:22:00.156 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:00.156 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":84,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:21:57.370942+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:21:53.462471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-14T11:21:54.724572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:22:00.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 msgr2=0x7f1acc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 0x7f1acc071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1af015cdc0 tx=0x7f1ad8008040 comp rx=0 tx=0).stop 2024-05-14T11:22:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 msgr2=0x7f1af014ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000ba50 tx=0x7f1ae000bf10 comp rx=0 tx=0).stop 2024-05-14T11:22:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 shutdown_connections 2024-05-14T11:22:00.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1acc06f400 0x7f1acc071910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1af015db30 0x7f1af015c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1af00d1670 0x7f1af014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 --2- 172.21.15.116:0/2084435211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af006c700 0x7f1af014ec50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 >> 172.21.15.116:0/2084435211 conn(0x7f1af00b4620 msgr2=0x7f1af0157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 shutdown_connections 2024-05-14T11:22:00.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.113+0000 7f1af6c00640 1 -- 172.21.15.116:0/2084435211 wait complete. 2024-05-14T11:22:00.160 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:22:00.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:22:00.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.305+0000 7ff31fc00640 1 Processor -- start 2024-05-14T11:22:00.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.305+0000 7ff31fc00640 1 -- start start 2024-05-14T11:22:00.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3180d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31fc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3180d1bb0 0x7ff31806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31fc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3180d1fc0 con 0x7ff31806cc40 2024-05-14T11:22:00.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31fc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff3181600a0 con 0x7ff3180d1bb0 2024-05-14T11:22:00.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff31806f5d0 con 0x7ff318164220 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31dc00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff31806cc40 0x7ff3180ae2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37134/0 (socket says 172.21.15.116:37134) 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31dc00640 1 -- 172.21.15.116:0/2005645080 learned_addr learned my addr 172.21.15.116:0/2005645080 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff30fe00640 1 --2- 172.21.15.116:0/2005645080 >> v2:172.21.15.116:3300/0 conn(0x7ff3180d1bb0 0x7ff31806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff31d200640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3180d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff30fe00640 1 -- 172.21.15.116:0/2005645080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 msgr2=0x7ff3180d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff30fe00640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3180d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff30fe00640 1 -- 172.21.15.116:0/2005645080 >> v1:172.21.15.116:6789/0 conn(0x7ff31806cc40 legacy=0x7ff3180ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.309+0000 7ff30fe00640 1 -- 172.21.15.116:0/2005645080 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31806f8b0 con 0x7ff3180d1bb0 2024-05-14T11:22:00.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff30fe00640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31806c700 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff3000097c0 tx=0x7ff30001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbe5ecfa38d4f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff317e00640 1 -- 172.21.15.116:0/2005645080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000b1070 con 0x7ff3180d1bb0 2024-05-14T11:22:00.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff317e00640 1 -- 172.21.15.116:0/2005645080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3000a8050 con 0x7ff3180d1bb0 2024-05-14T11:22:00.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff317e00640 1 -- 172.21.15.116:0/2005645080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000ac860 con 0x7ff3180d1bb0 2024-05-14T11:22:00.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 -- 172.21.15.116:0/2005645080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 msgr2=0x7ff31806c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31806c700 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff3000097c0 tx=0x7ff30001c3e0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 -- 172.21.15.116:0/2005645080 shutdown_connections 2024-05-14T11:22:00.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31806c700 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 --2- 172.21.15.116:0/2005645080 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3180d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 -- 172.21.15.116:0/2005645080 >> 172.21.15.116:0/2005645080 conn(0x7ff3180b4620 msgr2=0x7ff3180b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:00.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 -- 172.21.15.116:0/2005645080 shutdown_connections 2024-05-14T11:22:00.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.313+0000 7ff31fc00640 1 -- 172.21.15.116:0/2005645080 wait complete. 2024-05-14T11:22:00.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 Processor -- start 2024-05-14T11:22:00.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- start start 2024-05-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff31806cc40 0x7ff31814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3181623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff31806d460 con 0x7ff3180d1bb0 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff318073650 con 0x7ff31806cc40 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff318164630 con 0x7ff318164220 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42480/0 (socket says 172.21.15.116:42480) 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 -- 172.21.15.116:0/652424186 learned_addr learned my addr 172.21.15.116:0/652424186 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 -- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 msgr2=0x7ff3181623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3181623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 -- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff31806cc40 msgr2=0x7ff31814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff31806cc40 0x7ff31814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 -- 172.21.15.116:0/652424186 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff318150af0 con 0x7ff3180d1bb0 2024-05-14T11:22:00.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff30fe00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff30001a010 tx=0x7ff30001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000b1040 con 0x7ff3180d1bb0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3000bc070 con 0x7ff3180d1bb0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31814fa50 con 0x7ff3180d1bb0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000b7690 con 0x7ff3180d1bb0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff318150010 con 0x7ff3180d1bb0 2024-05-14T11:22:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.317+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31814f7a0 con 0x7ff3180d1bb0 2024-05-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.321+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3000b7830 con 0x7ff3180d1bb0 2024-05-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.321+0000 7ff315600640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 0x7ff2f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.321+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7ff3001297f0 con 0x7ff3180d1bb0 2024-05-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.321+0000 7ff31d200640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 0x7ff2f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.321+0000 7ff31d200640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 0x7ff2f4071910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff318074920 tx=0x7ff308008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.325+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff300016090 con 0x7ff3180d1bb0 2024-05-14T11:22:00.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.481+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff31806a9c0 con 0x7ff2f406f400 2024-05-14T11:22:00.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.489+0000 7ff315600640 1 -- 172.21.15.116:0/652424186 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251568 (secure 0 0 0) 0x7ff31806a9c0 con 0x7ff2f406f400 2024-05-14T11:22:00.492 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:00.494 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 msgr2=0x7ff2f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 0x7ff2f4071910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff318074920 tx=0x7ff308008040 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 msgr2=0x7ff31814e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff30001a010 tx=0x7ff30001ac20 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 shutdown_connections 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2f406f400 0x7ff2f4071910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff318164220 0x7ff3181623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3180d1bb0 0x7ff31814e520 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 --2- 172.21.15.116:0/652424186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff31806cc40 0x7ff31814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 >> 172.21.15.116:0/652424186 conn(0x7ff3180b4620 msgr2=0x7ff318155170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:00.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 shutdown_connections 2024-05-14T11:22:00.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.493+0000 7ff31fc00640 1 -- 172.21.15.116:0/652424186 wait complete. 2024-05-14T11:22:00.513 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-14T11:22:00.447107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258768,"kb_used_data":9284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715952,"statfs":{"total":773094113280,"available":772829134848,"internally_reserved":0,"allocated":9506816,"data_stored":4956142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403074+0000","last_change":"2024-05-14T11:21:56.378698+0000","last_active":"2024-05-14T11:21:57.403074+0000","last_peered":"2024-05-14T11:21:57.403074+0000","last_clean":"2024-05-14T11:21:57.403074+0000","last_became_active":"2024-05-14T11:21:56.378402+0000","last_became_peered":"2024-05-14T11:21:56.378402+0000","last_unstale":"2024-05-14T11:21:57.403074+0000","last_undegraded":"2024-05-14T11:21:57.403074+0000","last_fullsized":"2024-05-14T11:21:57.403074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369688+0000","last_change":"2024-05-14T11:21:55.369688+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369688+0000","last_undegraded":"2024-05-14T11:21:55.369688+0000","last_fullsized":"2024-05-14T11:21:55.369688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402927+0000","last_change":"2024-05-14T11:21:56.392559+0000","last_active":"2024-05-14T11:21:57.402927+0000","last_peered":"2024-05-14T11:21:57.402927+0000","last_clean":"2024-05-14T11:21:57.402927+0000","last_became_active":"2024-05-14T11:21:56.391449+0000","last_became_peered":"2024-05-14T11:21:56.391449+0000","last_unstale":"2024-05-14T11:21:57.402927+0000","last_undegraded":"2024-05-14T11:21:57.402927+0000","last_fullsized":"2024-05-14T11:21:57.402927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459732+0000","last_change":"2024-05-14T11:21:56.378008+0000","last_active":"2024-05-14T11:21:57.459732+0000","last_peered":"2024-05-14T11:21:57.459732+0000","last_clean":"2024-05-14T11:21:57.459732+0000","last_became_active":"2024-05-14T11:21:56.377707+0000","last_became_peered":"2024-05-14T11:21:56.377707+0000","last_unstale":"2024-05-14T11:21:57.459732+0000","last_undegraded":"2024-05-14T11:21:57.459732+0000","last_fullsized":"2024-05-14T11:21:57.459732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461077+0000","last_change":"2024-05-14T11:21:56.391093+0000","last_active":"2024-05-14T11:21:57.461077+0000","last_peered":"2024-05-14T11:21:57.461077+0000","last_clean":"2024-05-14T11:21:57.461077+0000","last_became_active":"2024-05-14T11:21:56.390712+0000","last_became_peered":"2024-05-14T11:21:56.390712+0000","last_unstale":"2024-05-14T11:21:57.461077+0000","last_undegraded":"2024-05-14T11:21:57.461077+0000","last_fullsized":"2024-05-14T11:21:57.461077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460609+0000","last_change":"2024-05-14T11:21:56.398553+0000","last_active":"2024-05-14T11:21:57.460609+0000","last_peered":"2024-05-14T11:21:57.460609+0000","last_clean":"2024-05-14T11:21:57.460609+0000","last_became_active":"2024-05-14T11:21:56.396664+0000","last_became_peered":"2024-05-14T11:21:56.396664+0000","last_unstale":"2024-05-14T11:21:57.460609+0000","last_undegraded":"2024-05-14T11:21:57.460609+0000","last_fullsized":"2024-05-14T11:21:57.460609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:59:19.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386491+0000","last_change":"2024-05-14T11:21:56.385275+0000","last_active":"2024-05-14T11:21:57.386491+0000","last_peered":"2024-05-14T11:21:57.386491+0000","last_clean":"2024-05-14T11:21:57.386491+0000","last_became_active":"2024-05-14T11:21:56.384904+0000","last_became_peered":"2024-05-14T11:21:56.384904+0000","last_unstale":"2024-05-14T11:21:57.386491+0000","last_undegraded":"2024-05-14T11:21:57.386491+0000","last_fullsized":"2024-05-14T11:21:57.386491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:36.820614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459460+0000","last_change":"2024-05-14T11:21:56.382681+0000","last_active":"2024-05-14T11:21:57.459460+0000","last_peered":"2024-05-14T11:21:57.459460+0000","last_clean":"2024-05-14T11:21:57.459460+0000","last_became_active":"2024-05-14T11:21:56.381079+0000","last_became_peered":"2024-05-14T11:21:56.381079+0000","last_unstale":"2024-05-14T11:21:57.459460+0000","last_undegraded":"2024-05-14T11:21:57.459460+0000","last_fullsized":"2024-05-14T11:21:57.459460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.394964+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.393782+0000","last_became_peered":"2024-05-14T11:21:56.393782+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380908+0000","last_change":"2024-05-14T11:21:56.384798+0000","last_active":"2024-05-14T11:21:57.380908+0000","last_peered":"2024-05-14T11:21:57.380908+0000","last_clean":"2024-05-14T11:21:57.380908+0000","last_became_active":"2024-05-14T11:21:56.384447+0000","last_became_peered":"2024-05-14T11:21:56.384447+0000","last_unstale":"2024-05-14T11:21:57.380908+0000","last_undegraded":"2024-05-14T11:21:57.380908+0000","last_fullsized":"2024-05-14T11:21:57.380908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:12.540726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.391270+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.390667+0000","last_became_peered":"2024-05-14T11:21:56.390667+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380632+0000","last_change":"2024-05-14T11:21:56.379345+0000","last_active":"2024-05-14T11:21:57.380632+0000","last_peered":"2024-05-14T11:21:57.380632+0000","last_clean":"2024-05-14T11:21:57.380632+0000","last_became_active":"2024-05-14T11:21:56.379007+0000","last_became_peered":"2024-05-14T11:21:56.379007+0000","last_unstale":"2024-05-14T11:21:57.380632+0000","last_undegraded":"2024-05-14T11:21:57.380632+0000","last_fullsized":"2024-05-14T11:21:57.380632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:31.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459974+0000","last_change":"2024-05-14T11:21:56.384668+0000","last_active":"2024-05-14T11:21:57.459974+0000","last_peered":"2024-05-14T11:21:57.459974+0000","last_clean":"2024-05-14T11:21:57.459974+0000","last_became_active":"2024-05-14T11:21:56.382366+0000","last_became_peered":"2024-05-14T11:21:56.382366+0000","last_unstale":"2024-05-14T11:21:57.459974+0000","last_undegraded":"2024-05-14T11:21:57.459974+0000","last_fullsized":"2024-05-14T11:21:57.459974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460403+0000","last_change":"2024-05-14T11:21:56.389629+0000","last_active":"2024-05-14T11:21:57.460403+0000","last_peered":"2024-05-14T11:21:57.460403+0000","last_clean":"2024-05-14T11:21:57.460403+0000","last_became_active":"2024-05-14T11:21:56.389299+0000","last_became_peered":"2024-05-14T11:21:56.389299+0000","last_unstale":"2024-05-14T11:21:57.460403+0000","last_undegraded":"2024-05-14T11:21:57.460403+0000","last_fullsized":"2024-05-14T11:21:57.460403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:00:13.582984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459513+0000","last_change":"2024-05-14T11:21:56.387482+0000","last_active":"2024-05-14T11:21:57.459513+0000","last_peered":"2024-05-14T11:21:57.459513+0000","last_clean":"2024-05-14T11:21:57.459513+0000","last_became_active":"2024-05-14T11:21:56.385978+0000","last_became_peered":"2024-05-14T11:21:56.385978+0000","last_unstale":"2024-05-14T11:21:57.459513+0000","last_undegraded":"2024-05-14T11:21:57.459513+0000","last_fullsized":"2024-05-14T11:21:57.459513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460528+0000","last_change":"2024-05-14T11:21:56.386948+0000","last_active":"2024-05-14T11:21:57.460528+0000","last_peered":"2024-05-14T11:21:57.460528+0000","last_clean":"2024-05-14T11:21:57.460528+0000","last_became_active":"2024-05-14T11:21:56.385310+0000","last_became_peered":"2024-05-14T11:21:56.385310+0000","last_unstale":"2024-05-14T11:21:57.460528+0000","last_undegraded":"2024-05-14T11:21:57.460528+0000","last_fullsized":"2024-05-14T11:21:57.460528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:31:18.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461165+0000","last_change":"2024-05-14T11:21:56.400922+0000","last_active":"2024-05-14T11:21:57.461165+0000","last_peered":"2024-05-14T11:21:57.461165+0000","last_clean":"2024-05-14T11:21:57.461165+0000","last_became_active":"2024-05-14T11:21:56.400597+0000","last_became_peered":"2024-05-14T11:21:56.400597+0000","last_unstale":"2024-05-14T11:21:57.461165+0000","last_undegraded":"2024-05-14T11:21:57.461165+0000","last_fullsized":"2024-05-14T11:21:57.461165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:07.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403099+0000","last_change":"2024-05-14T11:21:56.397705+0000","last_active":"2024-05-14T11:21:57.403099+0000","last_peered":"2024-05-14T11:21:57.403099+0000","last_clean":"2024-05-14T11:21:57.403099+0000","last_became_active":"2024-05-14T11:21:56.397144+0000","last_became_peered":"2024-05-14T11:21:56.397144+0000","last_unstale":"2024-05-14T11:21:57.403099+0000","last_undegraded":"2024-05-14T11:21:57.403099+0000","last_fullsized":"2024-05-14T11:21:57.403099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461475+0000","last_change":"2024-05-14T11:21:56.390962+0000","last_active":"2024-05-14T11:21:57.461475+0000","last_peered":"2024-05-14T11:21:57.461475+0000","last_clean":"2024-05-14T11:21:57.461475+0000","last_became_active":"2024-05-14T11:21:56.390072+0000","last_became_peered":"2024-05-14T11:21:56.390072+0000","last_unstale":"2024-05-14T11:21:57.461475+0000","last_undegraded":"2024-05-14T11:21:57.461475+0000","last_fullsized":"2024-05-14T11:21:57.461475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459937+0000","last_change":"2024-05-14T11:21:56.379798+0000","last_active":"2024-05-14T11:21:57.459937+0000","last_peered":"2024-05-14T11:21:57.459937+0000","last_clean":"2024-05-14T11:21:57.459937+0000","last_became_active":"2024-05-14T11:21:56.378722+0000","last_became_peered":"2024-05-14T11:21:56.378722+0000","last_unstale":"2024-05-14T11:21:57.459937+0000","last_undegraded":"2024-05-14T11:21:57.459937+0000","last_fullsized":"2024-05-14T11:21:57.459937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460813+0000","last_change":"2024-05-14T11:21:56.391852+0000","last_active":"2024-05-14T11:21:57.460813+0000","last_peered":"2024-05-14T11:21:57.460813+0000","last_clean":"2024-05-14T11:21:57.460813+0000","last_became_active":"2024-05-14T11:21:56.391495+0000","last_became_peered":"2024-05-14T11:21:56.391495+0000","last_unstale":"2024-05-14T11:21:57.460813+0000","last_undegraded":"2024-05-14T11:21:57.460813+0000","last_fullsized":"2024-05-14T11:21:57.460813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460442+0000","last_change":"2024-05-14T11:21:56.390530+0000","last_active":"2024-05-14T11:21:57.460442+0000","last_peered":"2024-05-14T11:21:57.460442+0000","last_clean":"2024-05-14T11:21:57.460442+0000","last_became_active":"2024-05-14T11:21:56.390215+0000","last_became_peered":"2024-05-14T11:21:56.390215+0000","last_unstale":"2024-05-14T11:21:57.460442+0000","last_undegraded":"2024-05-14T11:21:57.460442+0000","last_fullsized":"2024-05-14T11:21:57.460442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:07.315380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387804+0000","last_change":"2024-05-14T11:21:56.391315+0000","last_active":"2024-05-14T11:21:57.387804+0000","last_peered":"2024-05-14T11:21:57.387804+0000","last_clean":"2024-05-14T11:21:57.387804+0000","last_became_active":"2024-05-14T11:21:56.390649+0000","last_became_peered":"2024-05-14T11:21:56.390649+0000","last_unstale":"2024-05-14T11:21:57.387804+0000","last_undegraded":"2024-05-14T11:21:57.387804+0000","last_fullsized":"2024-05-14T11:21:57.387804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402786+0000","last_change":"2024-05-14T11:21:56.394765+0000","last_active":"2024-05-14T11:21:57.402786+0000","last_peered":"2024-05-14T11:21:57.402786+0000","last_clean":"2024-05-14T11:21:57.402786+0000","last_became_active":"2024-05-14T11:21:56.394490+0000","last_became_peered":"2024-05-14T11:21:56.394490+0000","last_unstale":"2024-05-14T11:21:57.402786+0000","last_undegraded":"2024-05-14T11:21:57.402786+0000","last_fullsized":"2024-05-14T11:21:57.402786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.372405+0000","last_change":"2024-05-14T11:21:55.372405+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.372405+0000","last_undegraded":"2024-05-14T11:21:55.372405+0000","last_fullsized":"2024-05-14T11:21:55.372405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461831+0000","last_change":"2024-05-14T11:21:56.389580+0000","last_active":"2024-05-14T11:21:57.461831+0000","last_peered":"2024-05-14T11:21:57.461831+0000","last_clean":"2024-05-14T11:21:57.461831+0000","last_became_active":"2024-05-14T11:21:56.389252+0000","last_became_peered":"2024-05-14T11:21:56.389252+0000","last_unstale":"2024-05-14T11:21:57.461831+0000","last_undegraded":"2024-05-14T11:21:57.461831+0000","last_fullsized":"2024-05-14T11:21:57.461831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.381007+0000","last_change":"2024-05-14T11:21:56.378063+0000","last_active":"2024-05-14T11:21:57.381007+0000","last_peered":"2024-05-14T11:21:57.381007+0000","last_clean":"2024-05-14T11:21:57.381007+0000","last_became_active":"2024-05-14T11:21:56.377759+0000","last_became_peered":"2024-05-14T11:21:56.377759+0000","last_unstale":"2024-05-14T11:21:57.381007+0000","last_undegraded":"2024-05-14T11:21:57.381007+0000","last_fullsized":"2024-05-14T11:21:57.381007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:43.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460090+0000","last_change":"2024-05-14T11:21:56.398675+0000","last_active":"2024-05-14T11:21:57.460090+0000","last_peered":"2024-05-14T11:21:57.460090+0000","last_clean":"2024-05-14T11:21:57.460090+0000","last_became_active":"2024-05-14T11:21:56.392629+0000","last_became_peered":"2024-05-14T11:21:56.392629+0000","last_unstale":"2024-05-14T11:21:57.460090+0000","last_undegraded":"2024-05-14T11:21:57.460090+0000","last_fullsized":"2024-05-14T11:21:57.460090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386641+0000","last_change":"2024-05-14T11:21:56.371575+0000","last_active":"2024-05-14T11:21:57.386641+0000","last_peered":"2024-05-14T11:21:57.386641+0000","last_clean":"2024-05-14T11:21:57.386641+0000","last_became_active":"2024-05-14T11:21:56.371212+0000","last_became_peered":"2024-05-14T11:21:56.371212+0000","last_unstale":"2024-05-14T11:21:57.386641+0000","last_undegraded":"2024-05-14T11:21:57.386641+0000","last_fullsized":"2024-05-14T11:21:57.386641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:49:03.645090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460626+0000","last_change":"2024-05-14T11:21:56.394494+0000","last_active":"2024-05-14T11:21:57.460626+0000","last_peered":"2024-05-14T11:21:57.460626+0000","last_clean":"2024-05-14T11:21:57.460626+0000","last_became_active":"2024-05-14T11:21:56.394205+0000","last_became_peered":"2024-05-14T11:21:56.394205+0000","last_unstale":"2024-05-14T11:21:57.460626+0000","last_undegraded":"2024-05-14T11:21:57.460626+0000","last_fullsized":"2024-05-14T11:21:57.460626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370175+0000","last_change":"2024-05-14T11:21:55.370175+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370175+0000","last_undegraded":"2024-05-14T11:21:55.370175+0000","last_fullsized":"2024-05-14T11:21:55.370175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460844+0000","last_change":"2024-05-14T11:21:56.382783+0000","last_active":"2024-05-14T11:21:57.460844+0000","last_peered":"2024-05-14T11:21:57.460844+0000","last_clean":"2024-05-14T11:21:57.460844+0000","last_became_active":"2024-05-14T11:21:56.382496+0000","last_became_peered":"2024-05-14T11:21:56.382496+0000","last_unstale":"2024-05-14T11:21:57.460844+0000","last_undegraded":"2024-05-14T11:21:57.460844+0000","last_fullsized":"2024-05-14T11:21:57.460844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:44.636667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461113+0000","last_change":"2024-05-14T11:21:56.399015+0000","last_active":"2024-05-14T11:21:57.461113+0000","last_peered":"2024-05-14T11:21:57.461113+0000","last_clean":"2024-05-14T11:21:57.461113+0000","last_became_active":"2024-05-14T11:21:56.397801+0000","last_became_peered":"2024-05-14T11:21:56.397801+0000","last_unstale":"2024-05-14T11:21:57.461113+0000","last_undegraded":"2024-05-14T11:21:57.461113+0000","last_fullsized":"2024-05-14T11:21:57.461113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:03.545193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387341+0000","last_change":"2024-05-14T11:21:55.377300+0000","last_active":"2024-05-14T11:21:57.387341+0000","last_peered":"2024-05-14T11:21:57.387341+0000","last_clean":"2024-05-14T11:21:57.387341+0000","last_became_active":"2024-05-14T11:21:55.376999+0000","last_became_peered":"2024-05-14T11:21:55.376999+0000","last_unstale":"2024-05-14T11:21:57.387341+0000","last_undegraded":"2024-05-14T11:21:57.387341+0000","last_fullsized":"2024-05-14T11:21:57.387341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:14.148384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460907+0000","last_change":"2024-05-14T11:21:56.381804+0000","last_active":"2024-05-14T11:21:57.460907+0000","last_peered":"2024-05-14T11:21:57.460907+0000","last_clean":"2024-05-14T11:21:57.460907+0000","last_became_active":"2024-05-14T11:21:56.381554+0000","last_became_peered":"2024-05-14T11:21:56.381554+0000","last_unstale":"2024-05-14T11:21:57.460907+0000","last_undegraded":"2024-05-14T11:21:57.460907+0000","last_fullsized":"2024-05-14T11:21:57.460907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:37:23.266696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461538+0000","last_change":"2024-05-14T11:21:55.373916+0000","last_active":"2024-05-14T11:21:57.461538+0000","last_peered":"2024-05-14T11:21:57.461538+0000","last_clean":"2024-05-14T11:21:57.461538+0000","last_became_active":"2024-05-14T11:21:55.373461+0000","last_became_peered":"2024-05-14T11:21:55.373461+0000","last_unstale":"2024-05-14T11:21:57.461538+0000","last_undegraded":"2024-05-14T11:21:57.461538+0000","last_fullsized":"2024-05-14T11:21:57.461538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:01.508933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386940+0000","last_change":"2024-05-14T11:21:56.385784+0000","last_active":"2024-05-14T11:21:57.386940+0000","last_peered":"2024-05-14T11:21:57.386940+0000","last_clean":"2024-05-14T11:21:57.386940+0000","last_became_active":"2024-05-14T11:21:56.385460+0000","last_became_peered":"2024-05-14T11:21:56.385460+0000","last_unstale":"2024-05-14T11:21:57.386940+0000","last_undegraded":"2024-05-14T11:21:57.386940+0000","last_fullsized":"2024-05-14T11:21:57.386940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:02:04.422136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387331+0000","last_change":"2024-05-14T11:21:55.384239+0000","last_active":"2024-05-14T11:21:57.387331+0000","last_peered":"2024-05-14T11:21:57.387331+0000","last_clean":"2024-05-14T11:21:57.387331+0000","last_became_active":"2024-05-14T11:21:55.383906+0000","last_became_peered":"2024-05-14T11:21:55.383906+0000","last_unstale":"2024-05-14T11:21:57.387331+0000","last_undegraded":"2024-05-14T11:21:57.387331+0000","last_fullsized":"2024-05-14T11:21:57.387331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:11.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461213+0000","last_change":"2024-05-14T11:21:56.398729+0000","last_active":"2024-05-14T11:21:57.461213+0000","last_peered":"2024-05-14T11:21:57.461213+0000","last_clean":"2024-05-14T11:21:57.461213+0000","last_became_active":"2024-05-14T11:21:56.397051+0000","last_became_peered":"2024-05-14T11:21:56.397051+0000","last_unstale":"2024-05-14T11:21:57.461213+0000","last_undegraded":"2024-05-14T11:21:57.461213+0000","last_fullsized":"2024-05-14T11:21:57.461213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:22:03.622138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387307+0000","last_change":"2024-05-14T11:21:55.386498+0000","last_active":"2024-05-14T11:21:57.387307+0000","last_peered":"2024-05-14T11:21:57.387307+0000","last_clean":"2024-05-14T11:21:57.387307+0000","last_became_active":"2024-05-14T11:21:55.385760+0000","last_became_peered":"2024-05-14T11:21:55.385760+0000","last_unstale":"2024-05-14T11:21:57.387307+0000","last_undegraded":"2024-05-14T11:21:57.387307+0000","last_fullsized":"2024-05-14T11:21:57.387307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.837126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459652+0000","last_change":"2024-05-14T11:21:56.381076+0000","last_active":"2024-05-14T11:21:57.459652+0000","last_peered":"2024-05-14T11:21:57.459652+0000","last_clean":"2024-05-14T11:21:57.459652+0000","last_became_active":"2024-05-14T11:21:56.380328+0000","last_became_peered":"2024-05-14T11:21:56.380328+0000","last_unstale":"2024-05-14T11:21:57.459652+0000","last_undegraded":"2024-05-14T11:21:57.459652+0000","last_fullsized":"2024-05-14T11:21:57.459652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461212+0000","last_change":"2024-05-14T11:21:55.383653+0000","last_active":"2024-05-14T11:21:57.461212+0000","last_peered":"2024-05-14T11:21:57.461212+0000","last_clean":"2024-05-14T11:21:57.461212+0000","last_became_active":"2024-05-14T11:21:55.383294+0000","last_became_peered":"2024-05-14T11:21:55.383294+0000","last_unstale":"2024-05-14T11:21:57.461212+0000","last_undegraded":"2024-05-14T11:21:57.461212+0000","last_fullsized":"2024-05-14T11:21:57.461212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:14.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386959+0000","last_change":"2024-05-14T11:21:56.378909+0000","last_active":"2024-05-14T11:21:57.386959+0000","last_peered":"2024-05-14T11:21:57.386959+0000","last_clean":"2024-05-14T11:21:57.386959+0000","last_became_active":"2024-05-14T11:21:56.378614+0000","last_became_peered":"2024-05-14T11:21:56.378614+0000","last_unstale":"2024-05-14T11:21:57.386959+0000","last_undegraded":"2024-05-14T11:21:57.386959+0000","last_fullsized":"2024-05-14T11:21:57.386959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460014+0000","last_change":"2024-05-14T11:21:55.387444+0000","last_active":"2024-05-14T11:21:57.460014+0000","last_peered":"2024-05-14T11:21:57.460014+0000","last_clean":"2024-05-14T11:21:57.460014+0000","last_became_active":"2024-05-14T11:21:55.386992+0000","last_became_peered":"2024-05-14T11:21:55.386992+0000","last_unstale":"2024-05-14T11:21:57.460014+0000","last_undegraded":"2024-05-14T11:21:57.460014+0000","last_fullsized":"2024-05-14T11:21:57.460014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:34.670851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460786+0000","last_change":"2024-05-14T11:21:56.389934+0000","last_active":"2024-05-14T11:21:57.460786+0000","last_peered":"2024-05-14T11:21:57.460786+0000","last_clean":"2024-05-14T11:21:57.460786+0000","last_became_active":"2024-05-14T11:21:56.388204+0000","last_became_peered":"2024-05-14T11:21:56.388204+0000","last_unstale":"2024-05-14T11:21:57.460786+0000","last_undegraded":"2024-05-14T11:21:57.460786+0000","last_fullsized":"2024-05-14T11:21:57.460786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:42.638658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402951+0000","last_change":"2024-05-14T11:21:55.379234+0000","last_active":"2024-05-14T11:21:57.402951+0000","last_peered":"2024-05-14T11:21:57.402951+0000","last_clean":"2024-05-14T11:21:57.402951+0000","last_became_active":"2024-05-14T11:21:55.378924+0000","last_became_peered":"2024-05-14T11:21:55.378924+0000","last_unstale":"2024-05-14T11:21:57.402951+0000","last_undegraded":"2024-05-14T11:21:57.402951+0000","last_fullsized":"2024-05-14T11:21:57.402951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460137+0000","last_change":"2024-05-14T11:21:56.383194+0000","last_active":"2024-05-14T11:21:57.460137+0000","last_peered":"2024-05-14T11:21:57.460137+0000","last_clean":"2024-05-14T11:21:57.460137+0000","last_became_active":"2024-05-14T11:21:56.379071+0000","last_became_peered":"2024-05-14T11:21:56.379071+0000","last_unstale":"2024-05-14T11:21:57.460137+0000","last_undegraded":"2024-05-14T11:21:57.460137+0000","last_fullsized":"2024-05-14T11:21:57.460137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386225+0000","last_change":"2024-05-14T11:21:55.384145+0000","last_active":"2024-05-14T11:21:57.386225+0000","last_peered":"2024-05-14T11:21:57.386225+0000","last_clean":"2024-05-14T11:21:57.386225+0000","last_became_active":"2024-05-14T11:21:55.383834+0000","last_became_peered":"2024-05-14T11:21:55.383834+0000","last_unstale":"2024-05-14T11:21:57.386225+0000","last_undegraded":"2024-05-14T11:21:57.386225+0000","last_fullsized":"2024-05-14T11:21:57.386225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:31.347753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461043+0000","last_change":"2024-05-14T11:21:56.396531+0000","last_active":"2024-05-14T11:21:57.461043+0000","last_peered":"2024-05-14T11:21:57.461043+0000","last_clean":"2024-05-14T11:21:57.461043+0000","last_became_active":"2024-05-14T11:21:56.396223+0000","last_became_peered":"2024-05-14T11:21:56.396223+0000","last_unstale":"2024-05-14T11:21:57.461043+0000","last_undegraded":"2024-05-14T11:21:57.461043+0000","last_fullsized":"2024-05-14T11:21:57.461043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.385972+0000","last_change":"2024-05-14T11:21:55.379424+0000","last_active":"2024-05-14T11:21:55.385972+0000","last_peered":"2024-05-14T11:21:55.385972+0000","last_clean":"2024-05-14T11:21:55.385972+0000","last_became_active":"2024-05-14T11:21:55.378552+0000","last_became_peered":"2024-05-14T11:21:55.378552+0000","last_unstale":"2024-05-14T11:21:55.385972+0000","last_undegraded":"2024-05-14T11:21:55.385972+0000","last_fullsized":"2024-05-14T11:21:55.385972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:39.156471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387395+0000","last_change":"2024-05-14T11:21:56.392129+0000","last_active":"2024-05-14T11:21:57.387395+0000","last_peered":"2024-05-14T11:21:57.387395+0000","last_clean":"2024-05-14T11:21:57.387395+0000","last_became_active":"2024-05-14T11:21:56.391814+0000","last_became_peered":"2024-05-14T11:21:56.391814+0000","last_unstale":"2024-05-14T11:21:57.387395+0000","last_undegraded":"2024-05-14T11:21:57.387395+0000","last_fullsized":"2024-05-14T11:21:57.387395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:20.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403200+0000","last_change":"2024-05-14T11:21:55.388521+0000","last_active":"2024-05-14T11:21:57.403200+0000","last_peered":"2024-05-14T11:21:57.403200+0000","last_clean":"2024-05-14T11:21:57.403200+0000","last_became_active":"2024-05-14T11:21:55.387310+0000","last_became_peered":"2024-05-14T11:21:55.387310+0000","last_unstale":"2024-05-14T11:21:57.403200+0000","last_undegraded":"2024-05-14T11:21:57.403200+0000","last_fullsized":"2024-05-14T11:21:57.403200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403125+0000","last_change":"2024-05-14T11:21:56.379479+0000","last_active":"2024-05-14T11:21:57.403125+0000","last_peered":"2024-05-14T11:21:57.403125+0000","last_clean":"2024-05-14T11:21:57.403125+0000","last_became_active":"2024-05-14T11:21:56.379170+0000","last_became_peered":"2024-05-14T11:21:56.379170+0000","last_unstale":"2024-05-14T11:21:57.403125+0000","last_undegraded":"2024-05-14T11:21:57.403125+0000","last_fullsized":"2024-05-14T11:21:57.403125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:45.902619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461559+0000","last_change":"2024-05-14T11:21:55.379532+0000","last_active":"2024-05-14T11:21:57.461559+0000","last_peered":"2024-05-14T11:21:57.461559+0000","last_clean":"2024-05-14T11:21:57.461559+0000","last_became_active":"2024-05-14T11:21:55.379224+0000","last_became_peered":"2024-05-14T11:21:55.379224+0000","last_unstale":"2024-05-14T11:21:57.461559+0000","last_undegraded":"2024-05-14T11:21:57.461559+0000","last_fullsized":"2024-05-14T11:21:57.461559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:23:55.106701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459814+0000","last_change":"2024-05-14T11:21:56.400307+0000","last_active":"2024-05-14T11:21:57.459814+0000","last_peered":"2024-05-14T11:21:57.459814+0000","last_clean":"2024-05-14T11:21:57.459814+0000","last_became_active":"2024-05-14T11:21:56.399940+0000","last_became_peered":"2024-05-14T11:21:56.399940+0000","last_unstale":"2024-05-14T11:21:57.459814+0000","last_undegraded":"2024-05-14T11:21:57.459814+0000","last_fullsized":"2024-05-14T11:21:57.459814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461259+0000","last_change":"2024-05-14T11:21:55.378699+0000","last_active":"2024-05-14T11:21:57.461259+0000","last_peered":"2024-05-14T11:21:57.461259+0000","last_clean":"2024-05-14T11:21:57.461259+0000","last_became_active":"2024-05-14T11:21:55.378372+0000","last_became_peered":"2024-05-14T11:21:55.378372+0000","last_unstale":"2024-05-14T11:21:57.461259+0000","last_undegraded":"2024-05-14T11:21:57.461259+0000","last_fullsized":"2024-05-14T11:21:57.461259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:42.121073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459654+0000","last_change":"2024-05-14T11:21:56.380739+0000","last_active":"2024-05-14T11:21:57.459654+0000","last_peered":"2024-05-14T11:21:57.459654+0000","last_clean":"2024-05-14T11:21:57.459654+0000","last_became_active":"2024-05-14T11:21:56.377804+0000","last_became_peered":"2024-05-14T11:21:56.377804+0000","last_unstale":"2024-05-14T11:21:57.459654+0000","last_undegraded":"2024-05-14T11:21:57.459654+0000","last_fullsized":"2024-05-14T11:21:57.459654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386631+0000","last_change":"2024-05-14T11:21:55.380045+0000","last_active":"2024-05-14T11:21:57.386631+0000","last_peered":"2024-05-14T11:21:57.386631+0000","last_clean":"2024-05-14T11:21:57.386631+0000","last_became_active":"2024-05-14T11:21:55.378906+0000","last_became_peered":"2024-05-14T11:21:55.378906+0000","last_unstale":"2024-05-14T11:21:57.386631+0000","last_undegraded":"2024-05-14T11:21:57.386631+0000","last_fullsized":"2024-05-14T11:21:57.386631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:34.974768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460535+0000","last_change":"2024-05-14T11:21:56.388451+0000","last_active":"2024-05-14T11:21:57.460535+0000","last_peered":"2024-05-14T11:21:57.460535+0000","last_clean":"2024-05-14T11:21:57.460535+0000","last_became_active":"2024-05-14T11:21:56.388148+0000","last_became_peered":"2024-05-14T11:21:56.388148+0000","last_unstale":"2024-05-14T11:21:57.460535+0000","last_undegraded":"2024-05-14T11:21:57.460535+0000","last_fullsized":"2024-05-14T11:21:57.460535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460827+0000","last_change":"2024-05-14T11:21:55.379659+0000","last_active":"2024-05-14T11:21:57.460827+0000","last_peered":"2024-05-14T11:21:57.460827+0000","last_clean":"2024-05-14T11:21:57.460827+0000","last_became_active":"2024-05-14T11:21:55.379357+0000","last_became_peered":"2024-05-14T11:21:55.379357+0000","last_unstale":"2024-05-14T11:21:57.460827+0000","last_undegraded":"2024-05-14T11:21:57.460827+0000","last_fullsized":"2024-05-14T11:21:57.460827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:54.423695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380729+0000","last_change":"2024-05-14T11:21:56.390097+0000","last_active":"2024-05-14T11:21:57.380729+0000","last_peered":"2024-05-14T11:21:57.380729+0000","last_clean":"2024-05-14T11:21:57.380729+0000","last_became_active":"2024-05-14T11:21:56.389775+0000","last_became_peered":"2024-05-14T11:21:56.389775+0000","last_unstale":"2024-05-14T11:21:57.380729+0000","last_undegraded":"2024-05-14T11:21:57.380729+0000","last_fullsized":"2024-05-14T11:21:57.380729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:24.671464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459571+0000","last_change":"2024-05-14T11:21:55.384436+0000","last_active":"2024-05-14T11:21:57.459571+0000","last_peered":"2024-05-14T11:21:57.459571+0000","last_clean":"2024-05-14T11:21:57.459571+0000","last_became_active":"2024-05-14T11:21:55.384104+0000","last_became_peered":"2024-05-14T11:21:55.384104+0000","last_unstale":"2024-05-14T11:21:57.459571+0000","last_undegraded":"2024-05-14T11:21:57.459571+0000","last_fullsized":"2024-05-14T11:21:57.459571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:56.800584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380862+0000","last_change":"2024-05-14T11:21:56.393507+0000","last_active":"2024-05-14T11:21:57.380862+0000","last_peered":"2024-05-14T11:21:57.380862+0000","last_clean":"2024-05-14T11:21:57.380862+0000","last_became_active":"2024-05-14T11:21:56.393227+0000","last_became_peered":"2024-05-14T11:21:56.393227+0000","last_unstale":"2024-05-14T11:21:57.380862+0000","last_undegraded":"2024-05-14T11:21:57.380862+0000","last_fullsized":"2024-05-14T11:21:57.380862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:26.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459792+0000","last_change":"2024-05-14T11:21:55.386065+0000","last_active":"2024-05-14T11:21:57.459792+0000","last_peered":"2024-05-14T11:21:57.459792+0000","last_clean":"2024-05-14T11:21:57.459792+0000","last_became_active":"2024-05-14T11:21:55.385699+0000","last_became_peered":"2024-05-14T11:21:55.385699+0000","last_unstale":"2024-05-14T11:21:57.459792+0000","last_undegraded":"2024-05-14T11:21:57.459792+0000","last_fullsized":"2024-05-14T11:21:57.459792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:34:21.465908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461286+0000","last_change":"2024-05-14T11:21:56.396244+0000","last_active":"2024-05-14T11:21:57.461286+0000","last_peered":"2024-05-14T11:21:57.461286+0000","last_clean":"2024-05-14T11:21:57.461286+0000","last_became_active":"2024-05-14T11:21:56.395934+0000","last_became_peered":"2024-05-14T11:21:56.395934+0000","last_unstale":"2024-05-14T11:21:57.461286+0000","last_undegraded":"2024-05-14T11:21:57.461286+0000","last_fullsized":"2024-05-14T11:21:57.461286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:44.571245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387256+0000","last_change":"2024-05-14T11:21:55.376695+0000","last_active":"2024-05-14T11:21:57.387256+0000","last_peered":"2024-05-14T11:21:57.387256+0000","last_clean":"2024-05-14T11:21:57.387256+0000","last_became_active":"2024-05-14T11:21:55.375674+0000","last_became_peered":"2024-05-14T11:21:55.375674+0000","last_unstale":"2024-05-14T11:21:57.387256+0000","last_undegraded":"2024-05-14T11:21:57.387256+0000","last_fullsized":"2024-05-14T11:21:57.387256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:57.676773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386997+0000","last_change":"2024-05-14T11:21:56.394005+0000","last_active":"2024-05-14T11:21:57.386997+0000","last_peered":"2024-05-14T11:21:57.386997+0000","last_clean":"2024-05-14T11:21:57.386997+0000","last_became_active":"2024-05-14T11:21:56.393708+0000","last_became_peered":"2024-05-14T11:21:56.393708+0000","last_unstale":"2024-05-14T11:21:57.386997+0000","last_undegraded":"2024-05-14T11:21:57.386997+0000","last_fullsized":"2024-05-14T11:21:57.386997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387203+0000","last_change":"2024-05-14T11:21:55.378651+0000","last_active":"2024-05-14T11:21:57.387203+0000","last_peered":"2024-05-14T11:21:57.387203+0000","last_clean":"2024-05-14T11:21:57.387203+0000","last_became_active":"2024-05-14T11:21:55.378348+0000","last_became_peered":"2024-05-14T11:21:55.378348+0000","last_unstale":"2024-05-14T11:21:57.387203+0000","last_undegraded":"2024-05-14T11:21:57.387203+0000","last_fullsized":"2024-05-14T11:21:57.387203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:13.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461456+0000","last_change":"2024-05-14T11:21:56.389463+0000","last_active":"2024-05-14T11:21:57.461456+0000","last_peered":"2024-05-14T11:21:57.461456+0000","last_clean":"2024-05-14T11:21:57.461456+0000","last_became_active":"2024-05-14T11:21:56.389014+0000","last_became_peered":"2024-05-14T11:21:56.389014+0000","last_unstale":"2024-05-14T11:21:57.461456+0000","last_undegraded":"2024-05-14T11:21:57.461456+0000","last_fullsized":"2024-05-14T11:21:57.461456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461621+0000","last_change":"2024-05-14T11:21:55.383442+0000","last_active":"2024-05-14T11:21:57.461621+0000","last_peered":"2024-05-14T11:21:57.461621+0000","last_clean":"2024-05-14T11:21:57.461621+0000","last_became_active":"2024-05-14T11:21:55.383197+0000","last_became_peered":"2024-05-14T11:21:55.383197+0000","last_unstale":"2024-05-14T11:21:57.461621+0000","last_undegraded":"2024-05-14T11:21:57.461621+0000","last_fullsized":"2024-05-14T11:21:57.461621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:52:51.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460829+0000","last_change":"2024-05-14T11:21:56.397428+0000","last_active":"2024-05-14T11:21:57.460829+0000","last_peered":"2024-05-14T11:21:57.460829+0000","last_clean":"2024-05-14T11:21:57.460829+0000","last_became_active":"2024-05-14T11:21:56.397079+0000","last_became_peered":"2024-05-14T11:21:56.397079+0000","last_unstale":"2024-05-14T11:21:57.460829+0000","last_undegraded":"2024-05-14T11:21:57.460829+0000","last_fullsized":"2024-05-14T11:21:57.460829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:05.256399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388807+0000","last_change":"2024-05-14T11:21:55.381263+0000","last_active":"2024-05-14T11:21:55.388807+0000","last_peered":"2024-05-14T11:21:55.388807+0000","last_clean":"2024-05-14T11:21:55.388807+0000","last_became_active":"2024-05-14T11:21:55.380928+0000","last_became_peered":"2024-05-14T11:21:55.380928+0000","last_unstale":"2024-05-14T11:21:55.388807+0000","last_undegraded":"2024-05-14T11:21:55.388807+0000","last_fullsized":"2024-05-14T11:21:55.388807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:43.506348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369616+0000","last_change":"2024-05-14T11:21:55.369616+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369616+0000","last_undegraded":"2024-05-14T11:21:55.369616+0000","last_fullsized":"2024-05-14T11:21:55.369616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380740+0000","last_change":"2024-05-14T11:21:55.386139+0000","last_active":"2024-05-14T11:21:57.380740+0000","last_peered":"2024-05-14T11:21:57.380740+0000","last_clean":"2024-05-14T11:21:57.380740+0000","last_became_active":"2024-05-14T11:21:55.385435+0000","last_became_peered":"2024-05-14T11:21:55.385435+0000","last_unstale":"2024-05-14T11:21:57.380740+0000","last_undegraded":"2024-05-14T11:21:57.380740+0000","last_fullsized":"2024-05-14T11:21:57.380740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:00.748983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370647+0000","last_change":"2024-05-14T11:21:55.370647+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370647+0000","last_undegraded":"2024-05-14T11:21:55.370647+0000","last_fullsized":"2024-05-14T11:21:55.370647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.381246+0000","last_change":"2024-05-14T11:21:55.377847+0000","last_active":"2024-05-14T11:21:55.381246+0000","last_peered":"2024-05-14T11:21:55.381246+0000","last_clean":"2024-05-14T11:21:55.381246+0000","last_became_active":"2024-05-14T11:21:55.377479+0000","last_became_peered":"2024-05-14T11:21:55.377479+0000","last_unstale":"2024-05-14T11:21:55.381246+0000","last_undegraded":"2024-05-14T11:21:55.381246+0000","last_fullsized":"2024-05-14T11:21:55.381246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:40.925873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460650+0000","last_change":"2024-05-14T11:21:56.381486+0000","last_active":"2024-05-14T11:21:57.460650+0000","last_peered":"2024-05-14T11:21:57.460650+0000","last_clean":"2024-05-14T11:21:57.460650+0000","last_became_active":"2024-05-14T11:21:56.381191+0000","last_became_peered":"2024-05-14T11:21:56.381191+0000","last_unstale":"2024-05-14T11:21:57.460650+0000","last_undegraded":"2024-05-14T11:21:57.460650+0000","last_fullsized":"2024-05-14T11:21:57.460650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:11:28.255356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380491+0000","last_change":"2024-05-14T11:21:55.380084+0000","last_active":"2024-05-14T11:21:57.380491+0000","last_peered":"2024-05-14T11:21:57.380491+0000","last_clean":"2024-05-14T11:21:57.380491+0000","last_became_active":"2024-05-14T11:21:55.379631+0000","last_became_peered":"2024-05-14T11:21:55.379631+0000","last_unstale":"2024-05-14T11:21:57.380491+0000","last_undegraded":"2024-05-14T11:21:57.380491+0000","last_fullsized":"2024-05-14T11:21:57.380491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:11.076627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387177+0000","last_change":"2024-05-14T11:21:56.380163+0000","last_active":"2024-05-14T11:21:57.387177+0000","last_peered":"2024-05-14T11:21:57.387177+0000","last_clean":"2024-05-14T11:21:57.387177+0000","last_became_active":"2024-05-14T11:21:56.377500+0000","last_became_peered":"2024-05-14T11:21:56.377500+0000","last_unstale":"2024-05-14T11:21:57.387177+0000","last_undegraded":"2024-05-14T11:21:57.387177+0000","last_fullsized":"2024-05-14T11:21:57.387177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387601+0000","last_change":"2024-05-14T11:21:55.382014+0000","last_active":"2024-05-14T11:21:57.387601+0000","last_peered":"2024-05-14T11:21:57.387601+0000","last_clean":"2024-05-14T11:21:57.387601+0000","last_became_active":"2024-05-14T11:21:55.381670+0000","last_became_peered":"2024-05-14T11:21:55.381670+0000","last_unstale":"2024-05-14T11:21:57.387601+0000","last_undegraded":"2024-05-14T11:21:57.387601+0000","last_fullsized":"2024-05-14T11:21:57.387601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:25.098189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461127+0000","last_change":"2024-05-14T11:21:56.397506+0000","last_active":"2024-05-14T11:21:57.461127+0000","last_peered":"2024-05-14T11:21:57.461127+0000","last_clean":"2024-05-14T11:21:57.461127+0000","last_became_active":"2024-05-14T11:21:56.395519+0000","last_became_peered":"2024-05-14T11:21:56.395519+0000","last_unstale":"2024-05-14T11:21:57.461127+0000","last_undegraded":"2024-05-14T11:21:57.461127+0000","last_fullsized":"2024-05-14T11:21:57.461127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:26:54.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461633+0000","last_change":"2024-05-14T11:21:55.375676+0000","last_active":"2024-05-14T11:21:57.461633+0000","last_peered":"2024-05-14T11:21:57.461633+0000","last_clean":"2024-05-14T11:21:57.461633+0000","last_became_active":"2024-05-14T11:21:55.375356+0000","last_became_peered":"2024-05-14T11:21:55.375356+0000","last_unstale":"2024-05-14T11:21:57.461633+0000","last_undegraded":"2024-05-14T11:21:57.461633+0000","last_fullsized":"2024-05-14T11:21:57.461633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:11.395431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461112+0000","last_change":"2024-05-14T11:21:56.393934+0000","last_active":"2024-05-14T11:21:57.461112+0000","last_peered":"2024-05-14T11:21:57.461112+0000","last_clean":"2024-05-14T11:21:57.461112+0000","last_became_active":"2024-05-14T11:21:56.393609+0000","last_became_peered":"2024-05-14T11:21:56.393609+0000","last_unstale":"2024-05-14T11:21:57.461112+0000","last_undegraded":"2024-05-14T11:21:57.461112+0000","last_fullsized":"2024-05-14T11:21:57.461112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:22.581542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461639+0000","last_change":"2024-05-14T11:21:55.379420+0000","last_active":"2024-05-14T11:21:57.461639+0000","last_peered":"2024-05-14T11:21:57.461639+0000","last_clean":"2024-05-14T11:21:57.461639+0000","last_became_active":"2024-05-14T11:21:55.378985+0000","last_became_peered":"2024-05-14T11:21:55.378985+0000","last_unstale":"2024-05-14T11:21:57.461639+0000","last_undegraded":"2024-05-14T11:21:57.461639+0000","last_fullsized":"2024-05-14T11:21:57.461639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:30.781659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.371911+0000","last_change":"2024-05-14T11:21:55.371911+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.371911+0000","last_undegraded":"2024-05-14T11:21:55.371911+0000","last_fullsized":"2024-05-14T11:21:55.371911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387442+0000","last_change":"2024-05-14T11:21:55.374216+0000","last_active":"2024-05-14T11:21:57.387442+0000","last_peered":"2024-05-14T11:21:57.387442+0000","last_clean":"2024-05-14T11:21:57.387442+0000","last_became_active":"2024-05-14T11:21:55.373713+0000","last_became_peered":"2024-05-14T11:21:55.373713+0000","last_unstale":"2024-05-14T11:21:57.387442+0000","last_undegraded":"2024-05-14T11:21:57.387442+0000","last_fullsized":"2024-05-14T11:21:57.387442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:32:45.264705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380622+0000","last_change":"2024-05-14T11:21:56.383298+0000","last_active":"2024-05-14T11:21:57.380622+0000","last_peered":"2024-05-14T11:21:57.380622+0000","last_clean":"2024-05-14T11:21:57.380622+0000","last_became_active":"2024-05-14T11:21:56.381852+0000","last_became_peered":"2024-05-14T11:21:56.381852+0000","last_unstale":"2024-05-14T11:21:57.380622+0000","last_undegraded":"2024-05-14T11:21:57.380622+0000","last_fullsized":"2024-05-14T11:21:57.380622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:45:25.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460760+0000","last_change":"2024-05-14T11:21:55.379545+0000","last_active":"2024-05-14T11:21:57.460760+0000","last_peered":"2024-05-14T11:21:57.460760+0000","last_clean":"2024-05-14T11:21:57.460760+0000","last_became_active":"2024-05-14T11:21:55.378762+0000","last_became_peered":"2024-05-14T11:21:55.378762+0000","last_unstale":"2024-05-14T11:21:57.460760+0000","last_undegraded":"2024-05-14T11:21:57.460760+0000","last_fullsized":"2024-05-14T11:21:57.460760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:44.899842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402884+0000","last_change":"2024-05-14T11:21:56.384970+0000","last_active":"2024-05-14T11:21:57.402884+0000","last_peered":"2024-05-14T11:21:57.402884+0000","last_clean":"2024-05-14T11:21:57.402884+0000","last_became_active":"2024-05-14T11:21:56.384703+0000","last_became_peered":"2024-05-14T11:21:56.384703+0000","last_unstale":"2024-05-14T11:21:57.402884+0000","last_undegraded":"2024-05-14T11:21:57.402884+0000","last_fullsized":"2024-05-14T11:21:57.402884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388829+0000","last_change":"2024-05-14T11:21:55.381768+0000","last_active":"2024-05-14T11:21:55.388829+0000","last_peered":"2024-05-14T11:21:55.388829+0000","last_clean":"2024-05-14T11:21:55.388829+0000","last_became_active":"2024-05-14T11:21:55.381485+0000","last_became_peered":"2024-05-14T11:21:55.381485+0000","last_unstale":"2024-05-14T11:21:55.388829+0000","last_undegraded":"2024-05-14T11:21:55.388829+0000","last_fullsized":"2024-05-14T11:21:55.388829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:38.207761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386457+0000","last_change":"2024-05-14T11:21:56.392100+0000","last_active":"2024-05-14T11:21:57.386457+0000","last_peered":"2024-05-14T11:21:57.386457+0000","last_clean":"2024-05-14T11:21:57.386457+0000","last_became_active":"2024-05-14T11:21:56.391699+0000","last_became_peered":"2024-05-14T11:21:56.391699+0000","last_unstale":"2024-05-14T11:21:57.386457+0000","last_undegraded":"2024-05-14T11:21:57.386457+0000","last_fullsized":"2024-05-14T11:21:57.386457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387086+0000","last_change":"2024-05-14T11:21:55.373928+0000","last_active":"2024-05-14T11:21:57.387086+0000","last_peered":"2024-05-14T11:21:57.387086+0000","last_clean":"2024-05-14T11:21:57.387086+0000","last_became_active":"2024-05-14T11:21:55.373638+0000","last_became_peered":"2024-05-14T11:21:55.373638+0000","last_unstale":"2024-05-14T11:21:57.387086+0000","last_undegraded":"2024-05-14T11:21:57.387086+0000","last_fullsized":"2024-05-14T11:21:57.387086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:35.515189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460703+0000","last_change":"2024-05-14T11:21:56.395619+0000","last_active":"2024-05-14T11:21:57.460703+0000","last_peered":"2024-05-14T11:21:57.460703+0000","last_clean":"2024-05-14T11:21:57.460703+0000","last_became_active":"2024-05-14T11:21:56.394829+0000","last_became_peered":"2024-05-14T11:21:56.394829+0000","last_unstale":"2024-05-14T11:21:57.460703+0000","last_undegraded":"2024-05-14T11:21:57.460703+0000","last_fullsized":"2024-05-14T11:21:57.460703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:36:28.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386239+0000","last_change":"2024-05-14T11:21:55.387084+0000","last_active":"2024-05-14T11:21:57.386239+0000","last_peered":"2024-05-14T11:21:57.386239+0000","last_clean":"2024-05-14T11:21:57.386239+0000","last_became_active":"2024-05-14T11:21:55.386834+0000","last_became_peered":"2024-05-14T11:21:55.386834+0000","last_unstale":"2024-05-14T11:21:57.386239+0000","last_undegraded":"2024-05-14T11:21:57.386239+0000","last_fullsized":"2024-05-14T11:21:57.386239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:17:10.801403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387496+0000","last_change":"2024-05-14T11:21:56.387037+0000","last_active":"2024-05-14T11:21:57.387496+0000","last_peered":"2024-05-14T11:21:57.387496+0000","last_clean":"2024-05-14T11:21:57.387496+0000","last_became_active":"2024-05-14T11:21:56.386496+0000","last_became_peered":"2024-05-14T11:21:56.386496+0000","last_unstale":"2024-05-14T11:21:57.387496+0000","last_undegraded":"2024-05-14T11:21:57.387496+0000","last_fullsized":"2024-05-14T11:21:57.387496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:21.878567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460199+0000","last_change":"2024-05-14T11:21:55.387153+0000","last_active":"2024-05-14T11:21:57.460199+0000","last_peered":"2024-05-14T11:21:57.460199+0000","last_clean":"2024-05-14T11:21:57.460199+0000","last_became_active":"2024-05-14T11:21:55.386815+0000","last_became_peered":"2024-05-14T11:21:55.386815+0000","last_unstale":"2024-05-14T11:21:57.460199+0000","last_undegraded":"2024-05-14T11:21:57.460199+0000","last_fullsized":"2024-05-14T11:21:57.460199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:44.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386383+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:21:57.386383+0000","last_peered":"2024-05-14T11:21:57.386383+0000","last_clean":"2024-05-14T11:21:57.386383+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:21:57.386383+0000","last_undegraded":"2024-05-14T11:21:57.386383+0000","last_fullsized":"2024-05-14T11:21:57.386383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1064313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":476105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":473959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1066575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:22:00.517 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:22:00.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 Processor -- start 2024-05-14T11:22:00.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 -- start start 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8541604f0 0x7fb854160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 0x7fb8540b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb854160f90 con 0x7fb854073d40 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb854161270 con 0x7fb8541604f0 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb859800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb8540b6a30 con 0x7fb8540b7230 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8541604f0 0x7fb854160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8541604f0 0x7fb854160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42494/0 (socket says 172.21.15.116:42494) 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 -- 172.21.15.116:0/3792416016 learned_addr learned my addr 172.21.15.116:0/3792416016 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb858e00640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 0x7fb8540b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 -- 172.21.15.116:0/3792416016 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 msgr2=0x7fb8540b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 0x7fb8540b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 -- 172.21.15.116:0/3792416016 >> v1:172.21.15.149:6789/0 conn(0x7fb854073d40 legacy=0x7fb854074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb853e00640 1 -- 172.21.15.116:0/3792416016 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8540b6d10 con 0x7fb8541604f0 2024-05-14T11:22:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.705+0000 7fb858e00640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 0x7fb8540b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb853e00640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb854160900 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb844006fd0 tx=0x7fb84401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633990b323df86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb852a00640 1 -- 172.21.15.116:0/3792416016 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440b0070 con 0x7fb8541604f0 2024-05-14T11:22:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb852a00640 1 -- 172.21.15.116:0/3792416016 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8440ab600 con 0x7fb8541604f0 2024-05-14T11:22:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb852a00640 1 -- 172.21.15.116:0/3792416016 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440ab9d0 con 0x7fb8541604f0 2024-05-14T11:22:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 -- 172.21.15.116:0/3792416016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 msgr2=0x7fb854160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb854160900 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb844006fd0 tx=0x7fb84401c4b0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 -- 172.21.15.116:0/3792416016 shutdown_connections 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540b7230 0x7fb8540b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 --2- 172.21.15.116:0/3792416016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb854160900 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 -- 172.21.15.116:0/3792416016 >> 172.21.15.116:0/3792416016 conn(0x7fb8540d26e0 msgr2=0x7fb8540d2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 -- 172.21.15.116:0/3792416016 shutdown_connections 2024-05-14T11:22:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.709+0000 7fb859800640 1 -- 172.21.15.116:0/3792416016 wait complete. 2024-05-14T11:22:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 Processor -- start 2024-05-14T11:22:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- start start 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb854073d40 0x7fb8541549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8540b7230 0x7fb85406f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541699f0 con 0x7fb8541604f0 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb854169870 con 0x7fb8540b7230 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb854169b70 con 0x7fb854073d40 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42500/0 (socket says 172.21.15.116:42500) 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 -- 172.21.15.116:0/763700479 learned_addr learned my addr 172.21.15.116:0/763700479 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb853400640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8540b7230 0x7fb85406f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb853e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb854073d40 0x7fb8541549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 -- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb854073d40 msgr2=0x7fb8541549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb854073d40 0x7fb8541549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 -- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8540b7230 msgr2=0x7fb85406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8540b7230 0x7fb85406f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 -- 172.21.15.116:0/763700479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854154f20 con 0x7fb8541604f0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb858e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00cd50 tx=0x7fb84c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00f680 con 0x7fb8541604f0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c00f840 con 0x7fb8541604f0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85406e590 con 0x7fb8541604f0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00fc30 con 0x7fb8541604f0 2024-05-14T11:22:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.713+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85406eb50 con 0x7fb8541604f0 2024-05-14T11:22:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.717+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85406e300 con 0x7fb8541604f0 2024-05-14T11:22:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.717+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb84c0213f0 con 0x7fb8541604f0 2024-05-14T11:22:00.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.717+0000 7fb84be00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 0x7fb834071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:00.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.717+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fb84c090210 con 0x7fb8541604f0 2024-05-14T11:22:00.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.717+0000 7fb853e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 0x7fb834071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:00.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.721+0000 7fb853e00640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 0x7fb834071910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ad000 tx=0x7fb8440a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.725+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb84c014030 con 0x7fb8541604f0 2024-05-14T11:22:00.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.877+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8540b5310 con 0x7fb83406f400 2024-05-14T11:22:00.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.885+0000 7fb84be00640 1 -- 172.21.15.116:0/763700479 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251568 (secure 0 0 0) 0x7fb8440f86c0 con 0x7fb83406f400 2024-05-14T11:22:00.887 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:00.888 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 msgr2=0x7fb834071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 0x7fb834071910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ad000 tx=0x7fb8440a6000 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 msgr2=0x7fb8541500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00cd50 tx=0x7fb84c00a650 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 shutdown_connections 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f400 0x7fb834071910 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8540b7230 0x7fb85406f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8541604f0 0x7fb8541500d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 --2- 172.21.15.116:0/763700479 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb854073d40 0x7fb8541549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 >> 172.21.15.116:0/763700479 conn(0x7fb8540d26e0 msgr2=0x7fb8540ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:00.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 shutdown_connections 2024-05-14T11:22:00.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:00.889+0000 7fb859800640 1 -- 172.21.15.116:0/763700479 wait complete. 2024-05-14T11:22:00.908 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-14T11:22:00.447107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258768,"kb_used_data":9284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715952,"statfs":{"total":773094113280,"available":772829134848,"internally_reserved":0,"allocated":9506816,"data_stored":4956142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403074+0000","last_change":"2024-05-14T11:21:56.378698+0000","last_active":"2024-05-14T11:21:57.403074+0000","last_peered":"2024-05-14T11:21:57.403074+0000","last_clean":"2024-05-14T11:21:57.403074+0000","last_became_active":"2024-05-14T11:21:56.378402+0000","last_became_peered":"2024-05-14T11:21:56.378402+0000","last_unstale":"2024-05-14T11:21:57.403074+0000","last_undegraded":"2024-05-14T11:21:57.403074+0000","last_fullsized":"2024-05-14T11:21:57.403074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369688+0000","last_change":"2024-05-14T11:21:55.369688+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369688+0000","last_undegraded":"2024-05-14T11:21:55.369688+0000","last_fullsized":"2024-05-14T11:21:55.369688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402927+0000","last_change":"2024-05-14T11:21:56.392559+0000","last_active":"2024-05-14T11:21:57.402927+0000","last_peered":"2024-05-14T11:21:57.402927+0000","last_clean":"2024-05-14T11:21:57.402927+0000","last_became_active":"2024-05-14T11:21:56.391449+0000","last_became_peered":"2024-05-14T11:21:56.391449+0000","last_unstale":"2024-05-14T11:21:57.402927+0000","last_undegraded":"2024-05-14T11:21:57.402927+0000","last_fullsized":"2024-05-14T11:21:57.402927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459732+0000","last_change":"2024-05-14T11:21:56.378008+0000","last_active":"2024-05-14T11:21:57.459732+0000","last_peered":"2024-05-14T11:21:57.459732+0000","last_clean":"2024-05-14T11:21:57.459732+0000","last_became_active":"2024-05-14T11:21:56.377707+0000","last_became_peered":"2024-05-14T11:21:56.377707+0000","last_unstale":"2024-05-14T11:21:57.459732+0000","last_undegraded":"2024-05-14T11:21:57.459732+0000","last_fullsized":"2024-05-14T11:21:57.459732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461077+0000","last_change":"2024-05-14T11:21:56.391093+0000","last_active":"2024-05-14T11:21:57.461077+0000","last_peered":"2024-05-14T11:21:57.461077+0000","last_clean":"2024-05-14T11:21:57.461077+0000","last_became_active":"2024-05-14T11:21:56.390712+0000","last_became_peered":"2024-05-14T11:21:56.390712+0000","last_unstale":"2024-05-14T11:21:57.461077+0000","last_undegraded":"2024-05-14T11:21:57.461077+0000","last_fullsized":"2024-05-14T11:21:57.461077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460609+0000","last_change":"2024-05-14T11:21:56.398553+0000","last_active":"2024-05-14T11:21:57.460609+0000","last_peered":"2024-05-14T11:21:57.460609+0000","last_clean":"2024-05-14T11:21:57.460609+0000","last_became_active":"2024-05-14T11:21:56.396664+0000","last_became_peered":"2024-05-14T11:21:56.396664+0000","last_unstale":"2024-05-14T11:21:57.460609+0000","last_undegraded":"2024-05-14T11:21:57.460609+0000","last_fullsized":"2024-05-14T11:21:57.460609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:59:19.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386491+0000","last_change":"2024-05-14T11:21:56.385275+0000","last_active":"2024-05-14T11:21:57.386491+0000","last_peered":"2024-05-14T11:21:57.386491+0000","last_clean":"2024-05-14T11:21:57.386491+0000","last_became_active":"2024-05-14T11:21:56.384904+0000","last_became_peered":"2024-05-14T11:21:56.384904+0000","last_unstale":"2024-05-14T11:21:57.386491+0000","last_undegraded":"2024-05-14T11:21:57.386491+0000","last_fullsized":"2024-05-14T11:21:57.386491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:36.820614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459460+0000","last_change":"2024-05-14T11:21:56.382681+0000","last_active":"2024-05-14T11:21:57.459460+0000","last_peered":"2024-05-14T11:21:57.459460+0000","last_clean":"2024-05-14T11:21:57.459460+0000","last_became_active":"2024-05-14T11:21:56.381079+0000","last_became_peered":"2024-05-14T11:21:56.381079+0000","last_unstale":"2024-05-14T11:21:57.459460+0000","last_undegraded":"2024-05-14T11:21:57.459460+0000","last_fullsized":"2024-05-14T11:21:57.459460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.394964+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.393782+0000","last_became_peered":"2024-05-14T11:21:56.393782+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380908+0000","last_change":"2024-05-14T11:21:56.384798+0000","last_active":"2024-05-14T11:21:57.380908+0000","last_peered":"2024-05-14T11:21:57.380908+0000","last_clean":"2024-05-14T11:21:57.380908+0000","last_became_active":"2024-05-14T11:21:56.384447+0000","last_became_peered":"2024-05-14T11:21:56.384447+0000","last_unstale":"2024-05-14T11:21:57.380908+0000","last_undegraded":"2024-05-14T11:21:57.380908+0000","last_fullsized":"2024-05-14T11:21:57.380908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:12.540726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.391270+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.390667+0000","last_became_peered":"2024-05-14T11:21:56.390667+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380632+0000","last_change":"2024-05-14T11:21:56.379345+0000","last_active":"2024-05-14T11:21:57.380632+0000","last_peered":"2024-05-14T11:21:57.380632+0000","last_clean":"2024-05-14T11:21:57.380632+0000","last_became_active":"2024-05-14T11:21:56.379007+0000","last_became_peered":"2024-05-14T11:21:56.379007+0000","last_unstale":"2024-05-14T11:21:57.380632+0000","last_undegraded":"2024-05-14T11:21:57.380632+0000","last_fullsized":"2024-05-14T11:21:57.380632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:31.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459974+0000","last_change":"2024-05-14T11:21:56.384668+0000","last_active":"2024-05-14T11:21:57.459974+0000","last_peered":"2024-05-14T11:21:57.459974+0000","last_clean":"2024-05-14T11:21:57.459974+0000","last_became_active":"2024-05-14T11:21:56.382366+0000","last_became_peered":"2024-05-14T11:21:56.382366+0000","last_unstale":"2024-05-14T11:21:57.459974+0000","last_undegraded":"2024-05-14T11:21:57.459974+0000","last_fullsized":"2024-05-14T11:21:57.459974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460403+0000","last_change":"2024-05-14T11:21:56.389629+0000","last_active":"2024-05-14T11:21:57.460403+0000","last_peered":"2024-05-14T11:21:57.460403+0000","last_clean":"2024-05-14T11:21:57.460403+0000","last_became_active":"2024-05-14T11:21:56.389299+0000","last_became_peered":"2024-05-14T11:21:56.389299+0000","last_unstale":"2024-05-14T11:21:57.460403+0000","last_undegraded":"2024-05-14T11:21:57.460403+0000","last_fullsized":"2024-05-14T11:21:57.460403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:00:13.582984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459513+0000","last_change":"2024-05-14T11:21:56.387482+0000","last_active":"2024-05-14T11:21:57.459513+0000","last_peered":"2024-05-14T11:21:57.459513+0000","last_clean":"2024-05-14T11:21:57.459513+0000","last_became_active":"2024-05-14T11:21:56.385978+0000","last_became_peered":"2024-05-14T11:21:56.385978+0000","last_unstale":"2024-05-14T11:21:57.459513+0000","last_undegraded":"2024-05-14T11:21:57.459513+0000","last_fullsized":"2024-05-14T11:21:57.459513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460528+0000","last_change":"2024-05-14T11:21:56.386948+0000","last_active":"2024-05-14T11:21:57.460528+0000","last_peered":"2024-05-14T11:21:57.460528+0000","last_clean":"2024-05-14T11:21:57.460528+0000","last_became_active":"2024-05-14T11:21:56.385310+0000","last_became_peered":"2024-05-14T11:21:56.385310+0000","last_unstale":"2024-05-14T11:21:57.460528+0000","last_undegraded":"2024-05-14T11:21:57.460528+0000","last_fullsized":"2024-05-14T11:21:57.460528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:31:18.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461165+0000","last_change":"2024-05-14T11:21:56.400922+0000","last_active":"2024-05-14T11:21:57.461165+0000","last_peered":"2024-05-14T11:21:57.461165+0000","last_clean":"2024-05-14T11:21:57.461165+0000","last_became_active":"2024-05-14T11:21:56.400597+0000","last_became_peered":"2024-05-14T11:21:56.400597+0000","last_unstale":"2024-05-14T11:21:57.461165+0000","last_undegraded":"2024-05-14T11:21:57.461165+0000","last_fullsized":"2024-05-14T11:21:57.461165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:07.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403099+0000","last_change":"2024-05-14T11:21:56.397705+0000","last_active":"2024-05-14T11:21:57.403099+0000","last_peered":"2024-05-14T11:21:57.403099+0000","last_clean":"2024-05-14T11:21:57.403099+0000","last_became_active":"2024-05-14T11:21:56.397144+0000","last_became_peered":"2024-05-14T11:21:56.397144+0000","last_unstale":"2024-05-14T11:21:57.403099+0000","last_undegraded":"2024-05-14T11:21:57.403099+0000","last_fullsized":"2024-05-14T11:21:57.403099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461475+0000","last_change":"2024-05-14T11:21:56.390962+0000","last_active":"2024-05-14T11:21:57.461475+0000","last_peered":"2024-05-14T11:21:57.461475+0000","last_clean":"2024-05-14T11:21:57.461475+0000","last_became_active":"2024-05-14T11:21:56.390072+0000","last_became_peered":"2024-05-14T11:21:56.390072+0000","last_unstale":"2024-05-14T11:21:57.461475+0000","last_undegraded":"2024-05-14T11:21:57.461475+0000","last_fullsized":"2024-05-14T11:21:57.461475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459937+0000","last_change":"2024-05-14T11:21:56.379798+0000","last_active":"2024-05-14T11:21:57.459937+0000","last_peered":"2024-05-14T11:21:57.459937+0000","last_clean":"2024-05-14T11:21:57.459937+0000","last_became_active":"2024-05-14T11:21:56.378722+0000","last_became_peered":"2024-05-14T11:21:56.378722+0000","last_unstale":"2024-05-14T11:21:57.459937+0000","last_undegraded":"2024-05-14T11:21:57.459937+0000","last_fullsized":"2024-05-14T11:21:57.459937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460813+0000","last_change":"2024-05-14T11:21:56.391852+0000","last_active":"2024-05-14T11:21:57.460813+0000","last_peered":"2024-05-14T11:21:57.460813+0000","last_clean":"2024-05-14T11:21:57.460813+0000","last_became_active":"2024-05-14T11:21:56.391495+0000","last_became_peered":"2024-05-14T11:21:56.391495+0000","last_unstale":"2024-05-14T11:21:57.460813+0000","last_undegraded":"2024-05-14T11:21:57.460813+0000","last_fullsized":"2024-05-14T11:21:57.460813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460442+0000","last_change":"2024-05-14T11:21:56.390530+0000","last_active":"2024-05-14T11:21:57.460442+0000","last_peered":"2024-05-14T11:21:57.460442+0000","last_clean":"2024-05-14T11:21:57.460442+0000","last_became_active":"2024-05-14T11:21:56.390215+0000","last_became_peered":"2024-05-14T11:21:56.390215+0000","last_unstale":"2024-05-14T11:21:57.460442+0000","last_undegraded":"2024-05-14T11:21:57.460442+0000","last_fullsized":"2024-05-14T11:21:57.460442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:07.315380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387804+0000","last_change":"2024-05-14T11:21:56.391315+0000","last_active":"2024-05-14T11:21:57.387804+0000","last_peered":"2024-05-14T11:21:57.387804+0000","last_clean":"2024-05-14T11:21:57.387804+0000","last_became_active":"2024-05-14T11:21:56.390649+0000","last_became_peered":"2024-05-14T11:21:56.390649+0000","last_unstale":"2024-05-14T11:21:57.387804+0000","last_undegraded":"2024-05-14T11:21:57.387804+0000","last_fullsized":"2024-05-14T11:21:57.387804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402786+0000","last_change":"2024-05-14T11:21:56.394765+0000","last_active":"2024-05-14T11:21:57.402786+0000","last_peered":"2024-05-14T11:21:57.402786+0000","last_clean":"2024-05-14T11:21:57.402786+0000","last_became_active":"2024-05-14T11:21:56.394490+0000","last_became_peered":"2024-05-14T11:21:56.394490+0000","last_unstale":"2024-05-14T11:21:57.402786+0000","last_undegraded":"2024-05-14T11:21:57.402786+0000","last_fullsized":"2024-05-14T11:21:57.402786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.372405+0000","last_change":"2024-05-14T11:21:55.372405+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.372405+0000","last_undegraded":"2024-05-14T11:21:55.372405+0000","last_fullsized":"2024-05-14T11:21:55.372405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461831+0000","last_change":"2024-05-14T11:21:56.389580+0000","last_active":"2024-05-14T11:21:57.461831+0000","last_peered":"2024-05-14T11:21:57.461831+0000","last_clean":"2024-05-14T11:21:57.461831+0000","last_became_active":"2024-05-14T11:21:56.389252+0000","last_became_peered":"2024-05-14T11:21:56.389252+0000","last_unstale":"2024-05-14T11:21:57.461831+0000","last_undegraded":"2024-05-14T11:21:57.461831+0000","last_fullsized":"2024-05-14T11:21:57.461831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.381007+0000","last_change":"2024-05-14T11:21:56.378063+0000","last_active":"2024-05-14T11:21:57.381007+0000","last_peered":"2024-05-14T11:21:57.381007+0000","last_clean":"2024-05-14T11:21:57.381007+0000","last_became_active":"2024-05-14T11:21:56.377759+0000","last_became_peered":"2024-05-14T11:21:56.377759+0000","last_unstale":"2024-05-14T11:21:57.381007+0000","last_undegraded":"2024-05-14T11:21:57.381007+0000","last_fullsized":"2024-05-14T11:21:57.381007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:43.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460090+0000","last_change":"2024-05-14T11:21:56.398675+0000","last_active":"2024-05-14T11:21:57.460090+0000","last_peered":"2024-05-14T11:21:57.460090+0000","last_clean":"2024-05-14T11:21:57.460090+0000","last_became_active":"2024-05-14T11:21:56.392629+0000","last_became_peered":"2024-05-14T11:21:56.392629+0000","last_unstale":"2024-05-14T11:21:57.460090+0000","last_undegraded":"2024-05-14T11:21:57.460090+0000","last_fullsized":"2024-05-14T11:21:57.460090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386641+0000","last_change":"2024-05-14T11:21:56.371575+0000","last_active":"2024-05-14T11:21:57.386641+0000","last_peered":"2024-05-14T11:21:57.386641+0000","last_clean":"2024-05-14T11:21:57.386641+0000","last_became_active":"2024-05-14T11:21:56.371212+0000","last_became_peered":"2024-05-14T11:21:56.371212+0000","last_unstale":"2024-05-14T11:21:57.386641+0000","last_undegraded":"2024-05-14T11:21:57.386641+0000","last_fullsized":"2024-05-14T11:21:57.386641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:49:03.645090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460626+0000","last_change":"2024-05-14T11:21:56.394494+0000","last_active":"2024-05-14T11:21:57.460626+0000","last_peered":"2024-05-14T11:21:57.460626+0000","last_clean":"2024-05-14T11:21:57.460626+0000","last_became_active":"2024-05-14T11:21:56.394205+0000","last_became_peered":"2024-05-14T11:21:56.394205+0000","last_unstale":"2024-05-14T11:21:57.460626+0000","last_undegraded":"2024-05-14T11:21:57.460626+0000","last_fullsized":"2024-05-14T11:21:57.460626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370175+0000","last_change":"2024-05-14T11:21:55.370175+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370175+0000","last_undegraded":"2024-05-14T11:21:55.370175+0000","last_fullsized":"2024-05-14T11:21:55.370175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460844+0000","last_change":"2024-05-14T11:21:56.382783+0000","last_active":"2024-05-14T11:21:57.460844+0000","last_peered":"2024-05-14T11:21:57.460844+0000","last_clean":"2024-05-14T11:21:57.460844+0000","last_became_active":"2024-05-14T11:21:56.382496+0000","last_became_peered":"2024-05-14T11:21:56.382496+0000","last_unstale":"2024-05-14T11:21:57.460844+0000","last_undegraded":"2024-05-14T11:21:57.460844+0000","last_fullsized":"2024-05-14T11:21:57.460844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:44.636667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461113+0000","last_change":"2024-05-14T11:21:56.399015+0000","last_active":"2024-05-14T11:21:57.461113+0000","last_peered":"2024-05-14T11:21:57.461113+0000","last_clean":"2024-05-14T11:21:57.461113+0000","last_became_active":"2024-05-14T11:21:56.397801+0000","last_became_peered":"2024-05-14T11:21:56.397801+0000","last_unstale":"2024-05-14T11:21:57.461113+0000","last_undegraded":"2024-05-14T11:21:57.461113+0000","last_fullsized":"2024-05-14T11:21:57.461113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:03.545193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387341+0000","last_change":"2024-05-14T11:21:55.377300+0000","last_active":"2024-05-14T11:21:57.387341+0000","last_peered":"2024-05-14T11:21:57.387341+0000","last_clean":"2024-05-14T11:21:57.387341+0000","last_became_active":"2024-05-14T11:21:55.376999+0000","last_became_peered":"2024-05-14T11:21:55.376999+0000","last_unstale":"2024-05-14T11:21:57.387341+0000","last_undegraded":"2024-05-14T11:21:57.387341+0000","last_fullsized":"2024-05-14T11:21:57.387341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:14.148384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460907+0000","last_change":"2024-05-14T11:21:56.381804+0000","last_active":"2024-05-14T11:21:57.460907+0000","last_peered":"2024-05-14T11:21:57.460907+0000","last_clean":"2024-05-14T11:21:57.460907+0000","last_became_active":"2024-05-14T11:21:56.381554+0000","last_became_peered":"2024-05-14T11:21:56.381554+0000","last_unstale":"2024-05-14T11:21:57.460907+0000","last_undegraded":"2024-05-14T11:21:57.460907+0000","last_fullsized":"2024-05-14T11:21:57.460907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:37:23.266696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461538+0000","last_change":"2024-05-14T11:21:55.373916+0000","last_active":"2024-05-14T11:21:57.461538+0000","last_peered":"2024-05-14T11:21:57.461538+0000","last_clean":"2024-05-14T11:21:57.461538+0000","last_became_active":"2024-05-14T11:21:55.373461+0000","last_became_peered":"2024-05-14T11:21:55.373461+0000","last_unstale":"2024-05-14T11:21:57.461538+0000","last_undegraded":"2024-05-14T11:21:57.461538+0000","last_fullsized":"2024-05-14T11:21:57.461538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:01.508933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386940+0000","last_change":"2024-05-14T11:21:56.385784+0000","last_active":"2024-05-14T11:21:57.386940+0000","last_peered":"2024-05-14T11:21:57.386940+0000","last_clean":"2024-05-14T11:21:57.386940+0000","last_became_active":"2024-05-14T11:21:56.385460+0000","last_became_peered":"2024-05-14T11:21:56.385460+0000","last_unstale":"2024-05-14T11:21:57.386940+0000","last_undegraded":"2024-05-14T11:21:57.386940+0000","last_fullsized":"2024-05-14T11:21:57.386940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:02:04.422136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387331+0000","last_change":"2024-05-14T11:21:55.384239+0000","last_active":"2024-05-14T11:21:57.387331+0000","last_peered":"2024-05-14T11:21:57.387331+0000","last_clean":"2024-05-14T11:21:57.387331+0000","last_became_active":"2024-05-14T11:21:55.383906+0000","last_became_peered":"2024-05-14T11:21:55.383906+0000","last_unstale":"2024-05-14T11:21:57.387331+0000","last_undegraded":"2024-05-14T11:21:57.387331+0000","last_fullsized":"2024-05-14T11:21:57.387331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:11.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461213+0000","last_change":"2024-05-14T11:21:56.398729+0000","last_active":"2024-05-14T11:21:57.461213+0000","last_peered":"2024-05-14T11:21:57.461213+0000","last_clean":"2024-05-14T11:21:57.461213+0000","last_became_active":"2024-05-14T11:21:56.397051+0000","last_became_peered":"2024-05-14T11:21:56.397051+0000","last_unstale":"2024-05-14T11:21:57.461213+0000","last_undegraded":"2024-05-14T11:21:57.461213+0000","last_fullsized":"2024-05-14T11:21:57.461213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:22:03.622138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387307+0000","last_change":"2024-05-14T11:21:55.386498+0000","last_active":"2024-05-14T11:21:57.387307+0000","last_peered":"2024-05-14T11:21:57.387307+0000","last_clean":"2024-05-14T11:21:57.387307+0000","last_became_active":"2024-05-14T11:21:55.385760+0000","last_became_peered":"2024-05-14T11:21:55.385760+0000","last_unstale":"2024-05-14T11:21:57.387307+0000","last_undegraded":"2024-05-14T11:21:57.387307+0000","last_fullsized":"2024-05-14T11:21:57.387307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.837126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459652+0000","last_change":"2024-05-14T11:21:56.381076+0000","last_active":"2024-05-14T11:21:57.459652+0000","last_peered":"2024-05-14T11:21:57.459652+0000","last_clean":"2024-05-14T11:21:57.459652+0000","last_became_active":"2024-05-14T11:21:56.380328+0000","last_became_peered":"2024-05-14T11:21:56.380328+0000","last_unstale":"2024-05-14T11:21:57.459652+0000","last_undegraded":"2024-05-14T11:21:57.459652+0000","last_fullsized":"2024-05-14T11:21:57.459652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461212+0000","last_change":"2024-05-14T11:21:55.383653+0000","last_active":"2024-05-14T11:21:57.461212+0000","last_peered":"2024-05-14T11:21:57.461212+0000","last_clean":"2024-05-14T11:21:57.461212+0000","last_became_active":"2024-05-14T11:21:55.383294+0000","last_became_peered":"2024-05-14T11:21:55.383294+0000","last_unstale":"2024-05-14T11:21:57.461212+0000","last_undegraded":"2024-05-14T11:21:57.461212+0000","last_fullsized":"2024-05-14T11:21:57.461212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:14.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386959+0000","last_change":"2024-05-14T11:21:56.378909+0000","last_active":"2024-05-14T11:21:57.386959+0000","last_peered":"2024-05-14T11:21:57.386959+0000","last_clean":"2024-05-14T11:21:57.386959+0000","last_became_active":"2024-05-14T11:21:56.378614+0000","last_became_peered":"2024-05-14T11:21:56.378614+0000","last_unstale":"2024-05-14T11:21:57.386959+0000","last_undegraded":"2024-05-14T11:21:57.386959+0000","last_fullsized":"2024-05-14T11:21:57.386959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460014+0000","last_change":"2024-05-14T11:21:55.387444+0000","last_active":"2024-05-14T11:21:57.460014+0000","last_peered":"2024-05-14T11:21:57.460014+0000","last_clean":"2024-05-14T11:21:57.460014+0000","last_became_active":"2024-05-14T11:21:55.386992+0000","last_became_peered":"2024-05-14T11:21:55.386992+0000","last_unstale":"2024-05-14T11:21:57.460014+0000","last_undegraded":"2024-05-14T11:21:57.460014+0000","last_fullsized":"2024-05-14T11:21:57.460014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:34.670851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460786+0000","last_change":"2024-05-14T11:21:56.389934+0000","last_active":"2024-05-14T11:21:57.460786+0000","last_peered":"2024-05-14T11:21:57.460786+0000","last_clean":"2024-05-14T11:21:57.460786+0000","last_became_active":"2024-05-14T11:21:56.388204+0000","last_became_peered":"2024-05-14T11:21:56.388204+0000","last_unstale":"2024-05-14T11:21:57.460786+0000","last_undegraded":"2024-05-14T11:21:57.460786+0000","last_fullsized":"2024-05-14T11:21:57.460786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:42.638658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402951+0000","last_change":"2024-05-14T11:21:55.379234+0000","last_active":"2024-05-14T11:21:57.402951+0000","last_peered":"2024-05-14T11:21:57.402951+0000","last_clean":"2024-05-14T11:21:57.402951+0000","last_became_active":"2024-05-14T11:21:55.378924+0000","last_became_peered":"2024-05-14T11:21:55.378924+0000","last_unstale":"2024-05-14T11:21:57.402951+0000","last_undegraded":"2024-05-14T11:21:57.402951+0000","last_fullsized":"2024-05-14T11:21:57.402951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460137+0000","last_change":"2024-05-14T11:21:56.383194+0000","last_active":"2024-05-14T11:21:57.460137+0000","last_peered":"2024-05-14T11:21:57.460137+0000","last_clean":"2024-05-14T11:21:57.460137+0000","last_became_active":"2024-05-14T11:21:56.379071+0000","last_became_peered":"2024-05-14T11:21:56.379071+0000","last_unstale":"2024-05-14T11:21:57.460137+0000","last_undegraded":"2024-05-14T11:21:57.460137+0000","last_fullsized":"2024-05-14T11:21:57.460137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386225+0000","last_change":"2024-05-14T11:21:55.384145+0000","last_active":"2024-05-14T11:21:57.386225+0000","last_peered":"2024-05-14T11:21:57.386225+0000","last_clean":"2024-05-14T11:21:57.386225+0000","last_became_active":"2024-05-14T11:21:55.383834+0000","last_became_peered":"2024-05-14T11:21:55.383834+0000","last_unstale":"2024-05-14T11:21:57.386225+0000","last_undegraded":"2024-05-14T11:21:57.386225+0000","last_fullsized":"2024-05-14T11:21:57.386225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:31.347753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461043+0000","last_change":"2024-05-14T11:21:56.396531+0000","last_active":"2024-05-14T11:21:57.461043+0000","last_peered":"2024-05-14T11:21:57.461043+0000","last_clean":"2024-05-14T11:21:57.461043+0000","last_became_active":"2024-05-14T11:21:56.396223+0000","last_became_peered":"2024-05-14T11:21:56.396223+0000","last_unstale":"2024-05-14T11:21:57.461043+0000","last_undegraded":"2024-05-14T11:21:57.461043+0000","last_fullsized":"2024-05-14T11:21:57.461043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.385972+0000","last_change":"2024-05-14T11:21:55.379424+0000","last_active":"2024-05-14T11:21:55.385972+0000","last_peered":"2024-05-14T11:21:55.385972+0000","last_clean":"2024-05-14T11:21:55.385972+0000","last_became_active":"2024-05-14T11:21:55.378552+0000","last_became_peered":"2024-05-14T11:21:55.378552+0000","last_unstale":"2024-05-14T11:21:55.385972+0000","last_undegraded":"2024-05-14T11:21:55.385972+0000","last_fullsized":"2024-05-14T11:21:55.385972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:39.156471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387395+0000","last_change":"2024-05-14T11:21:56.392129+0000","last_active":"2024-05-14T11:21:57.387395+0000","last_peered":"2024-05-14T11:21:57.387395+0000","last_clean":"2024-05-14T11:21:57.387395+0000","last_became_active":"2024-05-14T11:21:56.391814+0000","last_became_peered":"2024-05-14T11:21:56.391814+0000","last_unstale":"2024-05-14T11:21:57.387395+0000","last_undegraded":"2024-05-14T11:21:57.387395+0000","last_fullsized":"2024-05-14T11:21:57.387395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:20.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403200+0000","last_change":"2024-05-14T11:21:55.388521+0000","last_active":"2024-05-14T11:21:57.403200+0000","last_peered":"2024-05-14T11:21:57.403200+0000","last_clean":"2024-05-14T11:21:57.403200+0000","last_became_active":"2024-05-14T11:21:55.387310+0000","last_became_peered":"2024-05-14T11:21:55.387310+0000","last_unstale":"2024-05-14T11:21:57.403200+0000","last_undegraded":"2024-05-14T11:21:57.403200+0000","last_fullsized":"2024-05-14T11:21:57.403200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403125+0000","last_change":"2024-05-14T11:21:56.379479+0000","last_active":"2024-05-14T11:21:57.403125+0000","last_peered":"2024-05-14T11:21:57.403125+0000","last_clean":"2024-05-14T11:21:57.403125+0000","last_became_active":"2024-05-14T11:21:56.379170+0000","last_became_peered":"2024-05-14T11:21:56.379170+0000","last_unstale":"2024-05-14T11:21:57.403125+0000","last_undegraded":"2024-05-14T11:21:57.403125+0000","last_fullsized":"2024-05-14T11:21:57.403125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:45.902619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461559+0000","last_change":"2024-05-14T11:21:55.379532+0000","last_active":"2024-05-14T11:21:57.461559+0000","last_peered":"2024-05-14T11:21:57.461559+0000","last_clean":"2024-05-14T11:21:57.461559+0000","last_became_active":"2024-05-14T11:21:55.379224+0000","last_became_peered":"2024-05-14T11:21:55.379224+0000","last_unstale":"2024-05-14T11:21:57.461559+0000","last_undegraded":"2024-05-14T11:21:57.461559+0000","last_fullsized":"2024-05-14T11:21:57.461559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:23:55.106701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459814+0000","last_change":"2024-05-14T11:21:56.400307+0000","last_active":"2024-05-14T11:21:57.459814+0000","last_peered":"2024-05-14T11:21:57.459814+0000","last_clean":"2024-05-14T11:21:57.459814+0000","last_became_active":"2024-05-14T11:21:56.399940+0000","last_became_peered":"2024-05-14T11:21:56.399940+0000","last_unstale":"2024-05-14T11:21:57.459814+0000","last_undegraded":"2024-05-14T11:21:57.459814+0000","last_fullsized":"2024-05-14T11:21:57.459814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461259+0000","last_change":"2024-05-14T11:21:55.378699+0000","last_active":"2024-05-14T11:21:57.461259+0000","last_peered":"2024-05-14T11:21:57.461259+0000","last_clean":"2024-05-14T11:21:57.461259+0000","last_became_active":"2024-05-14T11:21:55.378372+0000","last_became_peered":"2024-05-14T11:21:55.378372+0000","last_unstale":"2024-05-14T11:21:57.461259+0000","last_undegraded":"2024-05-14T11:21:57.461259+0000","last_fullsized":"2024-05-14T11:21:57.461259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:42.121073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459654+0000","last_change":"2024-05-14T11:21:56.380739+0000","last_active":"2024-05-14T11:21:57.459654+0000","last_peered":"2024-05-14T11:21:57.459654+0000","last_clean":"2024-05-14T11:21:57.459654+0000","last_became_active":"2024-05-14T11:21:56.377804+0000","last_became_peered":"2024-05-14T11:21:56.377804+0000","last_unstale":"2024-05-14T11:21:57.459654+0000","last_undegraded":"2024-05-14T11:21:57.459654+0000","last_fullsized":"2024-05-14T11:21:57.459654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386631+0000","last_change":"2024-05-14T11:21:55.380045+0000","last_active":"2024-05-14T11:21:57.386631+0000","last_peered":"2024-05-14T11:21:57.386631+0000","last_clean":"2024-05-14T11:21:57.386631+0000","last_became_active":"2024-05-14T11:21:55.378906+0000","last_became_peered":"2024-05-14T11:21:55.378906+0000","last_unstale":"2024-05-14T11:21:57.386631+0000","last_undegraded":"2024-05-14T11:21:57.386631+0000","last_fullsized":"2024-05-14T11:21:57.386631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:34.974768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460535+0000","last_change":"2024-05-14T11:21:56.388451+0000","last_active":"2024-05-14T11:21:57.460535+0000","last_peered":"2024-05-14T11:21:57.460535+0000","last_clean":"2024-05-14T11:21:57.460535+0000","last_became_active":"2024-05-14T11:21:56.388148+0000","last_became_peered":"2024-05-14T11:21:56.388148+0000","last_unstale":"2024-05-14T11:21:57.460535+0000","last_undegraded":"2024-05-14T11:21:57.460535+0000","last_fullsized":"2024-05-14T11:21:57.460535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460827+0000","last_change":"2024-05-14T11:21:55.379659+0000","last_active":"2024-05-14T11:21:57.460827+0000","last_peered":"2024-05-14T11:21:57.460827+0000","last_clean":"2024-05-14T11:21:57.460827+0000","last_became_active":"2024-05-14T11:21:55.379357+0000","last_became_peered":"2024-05-14T11:21:55.379357+0000","last_unstale":"2024-05-14T11:21:57.460827+0000","last_undegraded":"2024-05-14T11:21:57.460827+0000","last_fullsized":"2024-05-14T11:21:57.460827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:54.423695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380729+0000","last_change":"2024-05-14T11:21:56.390097+0000","last_active":"2024-05-14T11:21:57.380729+0000","last_peered":"2024-05-14T11:21:57.380729+0000","last_clean":"2024-05-14T11:21:57.380729+0000","last_became_active":"2024-05-14T11:21:56.389775+0000","last_became_peered":"2024-05-14T11:21:56.389775+0000","last_unstale":"2024-05-14T11:21:57.380729+0000","last_undegraded":"2024-05-14T11:21:57.380729+0000","last_fullsized":"2024-05-14T11:21:57.380729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:24.671464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459571+0000","last_change":"2024-05-14T11:21:55.384436+0000","last_active":"2024-05-14T11:21:57.459571+0000","last_peered":"2024-05-14T11:21:57.459571+0000","last_clean":"2024-05-14T11:21:57.459571+0000","last_became_active":"2024-05-14T11:21:55.384104+0000","last_became_peered":"2024-05-14T11:21:55.384104+0000","last_unstale":"2024-05-14T11:21:57.459571+0000","last_undegraded":"2024-05-14T11:21:57.459571+0000","last_fullsized":"2024-05-14T11:21:57.459571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:56.800584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380862+0000","last_change":"2024-05-14T11:21:56.393507+0000","last_active":"2024-05-14T11:21:57.380862+0000","last_peered":"2024-05-14T11:21:57.380862+0000","last_clean":"2024-05-14T11:21:57.380862+0000","last_became_active":"2024-05-14T11:21:56.393227+0000","last_became_peered":"2024-05-14T11:21:56.393227+0000","last_unstale":"2024-05-14T11:21:57.380862+0000","last_undegraded":"2024-05-14T11:21:57.380862+0000","last_fullsized":"2024-05-14T11:21:57.380862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:26.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459792+0000","last_change":"2024-05-14T11:21:55.386065+0000","last_active":"2024-05-14T11:21:57.459792+0000","last_peered":"2024-05-14T11:21:57.459792+0000","last_clean":"2024-05-14T11:21:57.459792+0000","last_became_active":"2024-05-14T11:21:55.385699+0000","last_became_peered":"2024-05-14T11:21:55.385699+0000","last_unstale":"2024-05-14T11:21:57.459792+0000","last_undegraded":"2024-05-14T11:21:57.459792+0000","last_fullsized":"2024-05-14T11:21:57.459792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:34:21.465908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461286+0000","last_change":"2024-05-14T11:21:56.396244+0000","last_active":"2024-05-14T11:21:57.461286+0000","last_peered":"2024-05-14T11:21:57.461286+0000","last_clean":"2024-05-14T11:21:57.461286+0000","last_became_active":"2024-05-14T11:21:56.395934+0000","last_became_peered":"2024-05-14T11:21:56.395934+0000","last_unstale":"2024-05-14T11:21:57.461286+0000","last_undegraded":"2024-05-14T11:21:57.461286+0000","last_fullsized":"2024-05-14T11:21:57.461286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:44.571245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387256+0000","last_change":"2024-05-14T11:21:55.376695+0000","last_active":"2024-05-14T11:21:57.387256+0000","last_peered":"2024-05-14T11:21:57.387256+0000","last_clean":"2024-05-14T11:21:57.387256+0000","last_became_active":"2024-05-14T11:21:55.375674+0000","last_became_peered":"2024-05-14T11:21:55.375674+0000","last_unstale":"2024-05-14T11:21:57.387256+0000","last_undegraded":"2024-05-14T11:21:57.387256+0000","last_fullsized":"2024-05-14T11:21:57.387256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:57.676773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386997+0000","last_change":"2024-05-14T11:21:56.394005+0000","last_active":"2024-05-14T11:21:57.386997+0000","last_peered":"2024-05-14T11:21:57.386997+0000","last_clean":"2024-05-14T11:21:57.386997+0000","last_became_active":"2024-05-14T11:21:56.393708+0000","last_became_peered":"2024-05-14T11:21:56.393708+0000","last_unstale":"2024-05-14T11:21:57.386997+0000","last_undegraded":"2024-05-14T11:21:57.386997+0000","last_fullsized":"2024-05-14T11:21:57.386997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387203+0000","last_change":"2024-05-14T11:21:55.378651+0000","last_active":"2024-05-14T11:21:57.387203+0000","last_peered":"2024-05-14T11:21:57.387203+0000","last_clean":"2024-05-14T11:21:57.387203+0000","last_became_active":"2024-05-14T11:21:55.378348+0000","last_became_peered":"2024-05-14T11:21:55.378348+0000","last_unstale":"2024-05-14T11:21:57.387203+0000","last_undegraded":"2024-05-14T11:21:57.387203+0000","last_fullsized":"2024-05-14T11:21:57.387203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:13.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461456+0000","last_change":"2024-05-14T11:21:56.389463+0000","last_active":"2024-05-14T11:21:57.461456+0000","last_peered":"2024-05-14T11:21:57.461456+0000","last_clean":"2024-05-14T11:21:57.461456+0000","last_became_active":"2024-05-14T11:21:56.389014+0000","last_became_peered":"2024-05-14T11:21:56.389014+0000","last_unstale":"2024-05-14T11:21:57.461456+0000","last_undegraded":"2024-05-14T11:21:57.461456+0000","last_fullsized":"2024-05-14T11:21:57.461456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461621+0000","last_change":"2024-05-14T11:21:55.383442+0000","last_active":"2024-05-14T11:21:57.461621+0000","last_peered":"2024-05-14T11:21:57.461621+0000","last_clean":"2024-05-14T11:21:57.461621+0000","last_became_active":"2024-05-14T11:21:55.383197+0000","last_became_peered":"2024-05-14T11:21:55.383197+0000","last_unstale":"2024-05-14T11:21:57.461621+0000","last_undegraded":"2024-05-14T11:21:57.461621+0000","last_fullsized":"2024-05-14T11:21:57.461621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:52:51.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460829+0000","last_change":"2024-05-14T11:21:56.397428+0000","last_active":"2024-05-14T11:21:57.460829+0000","last_peered":"2024-05-14T11:21:57.460829+0000","last_clean":"2024-05-14T11:21:57.460829+0000","last_became_active":"2024-05-14T11:21:56.397079+0000","last_became_peered":"2024-05-14T11:21:56.397079+0000","last_unstale":"2024-05-14T11:21:57.460829+0000","last_undegraded":"2024-05-14T11:21:57.460829+0000","last_fullsized":"2024-05-14T11:21:57.460829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:05.256399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388807+0000","last_change":"2024-05-14T11:21:55.381263+0000","last_active":"2024-05-14T11:21:55.388807+0000","last_peered":"2024-05-14T11:21:55.388807+0000","last_clean":"2024-05-14T11:21:55.388807+0000","last_became_active":"2024-05-14T11:21:55.380928+0000","last_became_peered":"2024-05-14T11:21:55.380928+0000","last_unstale":"2024-05-14T11:21:55.388807+0000","last_undegraded":"2024-05-14T11:21:55.388807+0000","last_fullsized":"2024-05-14T11:21:55.388807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:43.506348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369616+0000","last_change":"2024-05-14T11:21:55.369616+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369616+0000","last_undegraded":"2024-05-14T11:21:55.369616+0000","last_fullsized":"2024-05-14T11:21:55.369616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380740+0000","last_change":"2024-05-14T11:21:55.386139+0000","last_active":"2024-05-14T11:21:57.380740+0000","last_peered":"2024-05-14T11:21:57.380740+0000","last_clean":"2024-05-14T11:21:57.380740+0000","last_became_active":"2024-05-14T11:21:55.385435+0000","last_became_peered":"2024-05-14T11:21:55.385435+0000","last_unstale":"2024-05-14T11:21:57.380740+0000","last_undegraded":"2024-05-14T11:21:57.380740+0000","last_fullsized":"2024-05-14T11:21:57.380740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:00.748983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370647+0000","last_change":"2024-05-14T11:21:55.370647+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370647+0000","last_undegraded":"2024-05-14T11:21:55.370647+0000","last_fullsized":"2024-05-14T11:21:55.370647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.381246+0000","last_change":"2024-05-14T11:21:55.377847+0000","last_active":"2024-05-14T11:21:55.381246+0000","last_peered":"2024-05-14T11:21:55.381246+0000","last_clean":"2024-05-14T11:21:55.381246+0000","last_became_active":"2024-05-14T11:21:55.377479+0000","last_became_peered":"2024-05-14T11:21:55.377479+0000","last_unstale":"2024-05-14T11:21:55.381246+0000","last_undegraded":"2024-05-14T11:21:55.381246+0000","last_fullsized":"2024-05-14T11:21:55.381246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:40.925873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460650+0000","last_change":"2024-05-14T11:21:56.381486+0000","last_active":"2024-05-14T11:21:57.460650+0000","last_peered":"2024-05-14T11:21:57.460650+0000","last_clean":"2024-05-14T11:21:57.460650+0000","last_became_active":"2024-05-14T11:21:56.381191+0000","last_became_peered":"2024-05-14T11:21:56.381191+0000","last_unstale":"2024-05-14T11:21:57.460650+0000","last_undegraded":"2024-05-14T11:21:57.460650+0000","last_fullsized":"2024-05-14T11:21:57.460650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:11:28.255356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380491+0000","last_change":"2024-05-14T11:21:55.380084+0000","last_active":"2024-05-14T11:21:57.380491+0000","last_peered":"2024-05-14T11:21:57.380491+0000","last_clean":"2024-05-14T11:21:57.380491+0000","last_became_active":"2024-05-14T11:21:55.379631+0000","last_became_peered":"2024-05-14T11:21:55.379631+0000","last_unstale":"2024-05-14T11:21:57.380491+0000","last_undegraded":"2024-05-14T11:21:57.380491+0000","last_fullsized":"2024-05-14T11:21:57.380491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:11.076627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387177+0000","last_change":"2024-05-14T11:21:56.380163+0000","last_active":"2024-05-14T11:21:57.387177+0000","last_peered":"2024-05-14T11:21:57.387177+0000","last_clean":"2024-05-14T11:21:57.387177+0000","last_became_active":"2024-05-14T11:21:56.377500+0000","last_became_peered":"2024-05-14T11:21:56.377500+0000","last_unstale":"2024-05-14T11:21:57.387177+0000","last_undegraded":"2024-05-14T11:21:57.387177+0000","last_fullsized":"2024-05-14T11:21:57.387177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387601+0000","last_change":"2024-05-14T11:21:55.382014+0000","last_active":"2024-05-14T11:21:57.387601+0000","last_peered":"2024-05-14T11:21:57.387601+0000","last_clean":"2024-05-14T11:21:57.387601+0000","last_became_active":"2024-05-14T11:21:55.381670+0000","last_became_peered":"2024-05-14T11:21:55.381670+0000","last_unstale":"2024-05-14T11:21:57.387601+0000","last_undegraded":"2024-05-14T11:21:57.387601+0000","last_fullsized":"2024-05-14T11:21:57.387601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:25.098189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461127+0000","last_change":"2024-05-14T11:21:56.397506+0000","last_active":"2024-05-14T11:21:57.461127+0000","last_peered":"2024-05-14T11:21:57.461127+0000","last_clean":"2024-05-14T11:21:57.461127+0000","last_became_active":"2024-05-14T11:21:56.395519+0000","last_became_peered":"2024-05-14T11:21:56.395519+0000","last_unstale":"2024-05-14T11:21:57.461127+0000","last_undegraded":"2024-05-14T11:21:57.461127+0000","last_fullsized":"2024-05-14T11:21:57.461127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:26:54.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461633+0000","last_change":"2024-05-14T11:21:55.375676+0000","last_active":"2024-05-14T11:21:57.461633+0000","last_peered":"2024-05-14T11:21:57.461633+0000","last_clean":"2024-05-14T11:21:57.461633+0000","last_became_active":"2024-05-14T11:21:55.375356+0000","last_became_peered":"2024-05-14T11:21:55.375356+0000","last_unstale":"2024-05-14T11:21:57.461633+0000","last_undegraded":"2024-05-14T11:21:57.461633+0000","last_fullsized":"2024-05-14T11:21:57.461633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:11.395431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461112+0000","last_change":"2024-05-14T11:21:56.393934+0000","last_active":"2024-05-14T11:21:57.461112+0000","last_peered":"2024-05-14T11:21:57.461112+0000","last_clean":"2024-05-14T11:21:57.461112+0000","last_became_active":"2024-05-14T11:21:56.393609+0000","last_became_peered":"2024-05-14T11:21:56.393609+0000","last_unstale":"2024-05-14T11:21:57.461112+0000","last_undegraded":"2024-05-14T11:21:57.461112+0000","last_fullsized":"2024-05-14T11:21:57.461112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:22.581542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461639+0000","last_change":"2024-05-14T11:21:55.379420+0000","last_active":"2024-05-14T11:21:57.461639+0000","last_peered":"2024-05-14T11:21:57.461639+0000","last_clean":"2024-05-14T11:21:57.461639+0000","last_became_active":"2024-05-14T11:21:55.378985+0000","last_became_peered":"2024-05-14T11:21:55.378985+0000","last_unstale":"2024-05-14T11:21:57.461639+0000","last_undegraded":"2024-05-14T11:21:57.461639+0000","last_fullsized":"2024-05-14T11:21:57.461639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:30.781659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.371911+0000","last_change":"2024-05-14T11:21:55.371911+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.371911+0000","last_undegraded":"2024-05-14T11:21:55.371911+0000","last_fullsized":"2024-05-14T11:21:55.371911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387442+0000","last_change":"2024-05-14T11:21:55.374216+0000","last_active":"2024-05-14T11:21:57.387442+0000","last_peered":"2024-05-14T11:21:57.387442+0000","last_clean":"2024-05-14T11:21:57.387442+0000","last_became_active":"2024-05-14T11:21:55.373713+0000","last_became_peered":"2024-05-14T11:21:55.373713+0000","last_unstale":"2024-05-14T11:21:57.387442+0000","last_undegraded":"2024-05-14T11:21:57.387442+0000","last_fullsized":"2024-05-14T11:21:57.387442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:32:45.264705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380622+0000","last_change":"2024-05-14T11:21:56.383298+0000","last_active":"2024-05-14T11:21:57.380622+0000","last_peered":"2024-05-14T11:21:57.380622+0000","last_clean":"2024-05-14T11:21:57.380622+0000","last_became_active":"2024-05-14T11:21:56.381852+0000","last_became_peered":"2024-05-14T11:21:56.381852+0000","last_unstale":"2024-05-14T11:21:57.380622+0000","last_undegraded":"2024-05-14T11:21:57.380622+0000","last_fullsized":"2024-05-14T11:21:57.380622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:45:25.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460760+0000","last_change":"2024-05-14T11:21:55.379545+0000","last_active":"2024-05-14T11:21:57.460760+0000","last_peered":"2024-05-14T11:21:57.460760+0000","last_clean":"2024-05-14T11:21:57.460760+0000","last_became_active":"2024-05-14T11:21:55.378762+0000","last_became_peered":"2024-05-14T11:21:55.378762+0000","last_unstale":"2024-05-14T11:21:57.460760+0000","last_undegraded":"2024-05-14T11:21:57.460760+0000","last_fullsized":"2024-05-14T11:21:57.460760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:44.899842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402884+0000","last_change":"2024-05-14T11:21:56.384970+0000","last_active":"2024-05-14T11:21:57.402884+0000","last_peered":"2024-05-14T11:21:57.402884+0000","last_clean":"2024-05-14T11:21:57.402884+0000","last_became_active":"2024-05-14T11:21:56.384703+0000","last_became_peered":"2024-05-14T11:21:56.384703+0000","last_unstale":"2024-05-14T11:21:57.402884+0000","last_undegraded":"2024-05-14T11:21:57.402884+0000","last_fullsized":"2024-05-14T11:21:57.402884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388829+0000","last_change":"2024-05-14T11:21:55.381768+0000","last_active":"2024-05-14T11:21:55.388829+0000","last_peered":"2024-05-14T11:21:55.388829+0000","last_clean":"2024-05-14T11:21:55.388829+0000","last_became_active":"2024-05-14T11:21:55.381485+0000","last_became_peered":"2024-05-14T11:21:55.381485+0000","last_unstale":"2024-05-14T11:21:55.388829+0000","last_undegraded":"2024-05-14T11:21:55.388829+0000","last_fullsized":"2024-05-14T11:21:55.388829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:38.207761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386457+0000","last_change":"2024-05-14T11:21:56.392100+0000","last_active":"2024-05-14T11:21:57.386457+0000","last_peered":"2024-05-14T11:21:57.386457+0000","last_clean":"2024-05-14T11:21:57.386457+0000","last_became_active":"2024-05-14T11:21:56.391699+0000","last_became_peered":"2024-05-14T11:21:56.391699+0000","last_unstale":"2024-05-14T11:21:57.386457+0000","last_undegraded":"2024-05-14T11:21:57.386457+0000","last_fullsized":"2024-05-14T11:21:57.386457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387086+0000","last_change":"2024-05-14T11:21:55.373928+0000","last_active":"2024-05-14T11:21:57.387086+0000","last_peered":"2024-05-14T11:21:57.387086+0000","last_clean":"2024-05-14T11:21:57.387086+0000","last_became_active":"2024-05-14T11:21:55.373638+0000","last_became_peered":"2024-05-14T11:21:55.373638+0000","last_unstale":"2024-05-14T11:21:57.387086+0000","last_undegraded":"2024-05-14T11:21:57.387086+0000","last_fullsized":"2024-05-14T11:21:57.387086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:35.515189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460703+0000","last_change":"2024-05-14T11:21:56.395619+0000","last_active":"2024-05-14T11:21:57.460703+0000","last_peered":"2024-05-14T11:21:57.460703+0000","last_clean":"2024-05-14T11:21:57.460703+0000","last_became_active":"2024-05-14T11:21:56.394829+0000","last_became_peered":"2024-05-14T11:21:56.394829+0000","last_unstale":"2024-05-14T11:21:57.460703+0000","last_undegraded":"2024-05-14T11:21:57.460703+0000","last_fullsized":"2024-05-14T11:21:57.460703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:36:28.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386239+0000","last_change":"2024-05-14T11:21:55.387084+0000","last_active":"2024-05-14T11:21:57.386239+0000","last_peered":"2024-05-14T11:21:57.386239+0000","last_clean":"2024-05-14T11:21:57.386239+0000","last_became_active":"2024-05-14T11:21:55.386834+0000","last_became_peered":"2024-05-14T11:21:55.386834+0000","last_unstale":"2024-05-14T11:21:57.386239+0000","last_undegraded":"2024-05-14T11:21:57.386239+0000","last_fullsized":"2024-05-14T11:21:57.386239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:17:10.801403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387496+0000","last_change":"2024-05-14T11:21:56.387037+0000","last_active":"2024-05-14T11:21:57.387496+0000","last_peered":"2024-05-14T11:21:57.387496+0000","last_clean":"2024-05-14T11:21:57.387496+0000","last_became_active":"2024-05-14T11:21:56.386496+0000","last_became_peered":"2024-05-14T11:21:56.386496+0000","last_unstale":"2024-05-14T11:21:57.387496+0000","last_undegraded":"2024-05-14T11:21:57.387496+0000","last_fullsized":"2024-05-14T11:21:57.387496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:21.878567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460199+0000","last_change":"2024-05-14T11:21:55.387153+0000","last_active":"2024-05-14T11:21:57.460199+0000","last_peered":"2024-05-14T11:21:57.460199+0000","last_clean":"2024-05-14T11:21:57.460199+0000","last_became_active":"2024-05-14T11:21:55.386815+0000","last_became_peered":"2024-05-14T11:21:55.386815+0000","last_unstale":"2024-05-14T11:21:57.460199+0000","last_undegraded":"2024-05-14T11:21:57.460199+0000","last_fullsized":"2024-05-14T11:21:57.460199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:44.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386383+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:21:57.386383+0000","last_peered":"2024-05-14T11:21:57.386383+0000","last_clean":"2024-05-14T11:21:57.386383+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:21:57.386383+0000","last_undegraded":"2024-05-14T11:21:57.386383+0000","last_fullsized":"2024-05-14T11:21:57.386383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1064313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":476105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":473959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1066575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:22:00.912 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:22:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 Processor -- start 2024-05-14T11:22:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 -- start start 2024-05-14T11:22:01.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f431c156fa0 0x7f431c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c0d1670 0x7f431c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c0d1fc0 con 0x7f431c06e9e0 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f431c1573b0 con 0x7f431c156fa0 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4324800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f431c161120 con 0x7f431c0d1670 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f431c156fa0 0x7f431c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f431c156fa0 0x7f431c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42508/0 (socket says 172.21.15.116:42508) 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 -- 172.21.15.116:0/1829662691 learned_addr learned my addr 172.21.15.116:0/1829662691 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321400640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c0d1670 0x7f431c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 -- 172.21.15.116:0/1829662691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c0d1670 msgr2=0x7f431c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c0d1670 0x7f431c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 -- 172.21.15.116:0/1829662691 >> v1:172.21.15.149:6789/0 conn(0x7f431c06e9e0 legacy=0x7f431c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.097+0000 7f4321e00640 1 -- 172.21.15.116:0/1829662691 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c160510 con 0x7f431c156fa0 2024-05-14T11:22:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4321e00640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c156fa0 0x7f431c06ffe0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f431001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3186197d6fcf4ce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4320a00640 1 -- 172.21.15.116:0/1829662691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100a95f0 con 0x7f431c156fa0 2024-05-14T11:22:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4320a00640 1 -- 172.21.15.116:0/1829662691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43100a9790 con 0x7f431c156fa0 2024-05-14T11:22:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4320a00640 1 -- 172.21.15.116:0/1829662691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100a9b60 con 0x7f431c156fa0 2024-05-14T11:22:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 -- 172.21.15.116:0/1829662691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c156fa0 msgr2=0x7f431c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c156fa0 0x7f431c06ffe0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f431001c530 comp rx=0 tx=0).stop 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 -- 172.21.15.116:0/1829662691 shutdown_connections 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c0d1670 0x7f431c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 --2- 172.21.15.116:0/1829662691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c156fa0 0x7f431c06ffe0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 -- 172.21.15.116:0/1829662691 >> 172.21.15.116:0/1829662691 conn(0x7f431c0b4620 msgr2=0x7f431c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 -- 172.21.15.116:0/1829662691 shutdown_connections 2024-05-14T11:22:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.101+0000 7f4324800640 1 -- 172.21.15.116:0/1829662691 wait complete. 2024-05-14T11:22:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 Processor -- start 2024-05-14T11:22:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 -- start start 2024-05-14T11:22:01.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 0x7f431c14d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c156fa0 0x7f431c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f431c070520 con 0x7f431c0d1670 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0b07a0 con 0x7f431c156fa0 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4324800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0706a0 con 0x7f431c06e9e0 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4321400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4321400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42518/0 (socket says 172.21.15.116:42518) 2024-05-14T11:22:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.105+0000 7f4321400640 1 -- 172.21.15.116:0/4025394866 learned_addr learned my addr 172.21.15.116:0/4025394866 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 -- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 msgr2=0x7f431c14d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321e00640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 0x7f431c14d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4322800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c156fa0 0x7f431c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 0x7f431c14d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 -- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c156fa0 msgr2=0x7f431c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c156fa0 0x7f431c152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 -- 172.21.15.116:0/4025394866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c168810 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321e00640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 0x7f431c14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4321400640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d620 tx=0x7f430c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00b620 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c019070 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c168af0 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00b9e0 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1690b0 con 0x7f431c0d1670 2024-05-14T11:22:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c06a9c0 con 0x7f431c0d1670 2024-05-14T11:22:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f430c00bb80 con 0x7f431c0d1670 2024-05-14T11:22:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.109+0000 7f4316000640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 0x7f4300071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.113+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f430c08f950 con 0x7f431c0d1670 2024-05-14T11:22:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.113+0000 7f4321e00640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 0x7f4300071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.113+0000 7f4321e00640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 0x7f4300071940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43100ae000 tx=0x7f43100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.117+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f430c05c610 con 0x7f431c0d1670 2024-05-14T11:22:01.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.269+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f431c06d410 con 0x7f430006f430 2024-05-14T11:22:01.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.273+0000 7f4316000640 1 -- 172.21.15.116:0/4025394866 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251568 (secure 0 0 0) 0x7f431c06d410 con 0x7f430006f430 2024-05-14T11:22:01.277 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:01.278 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:22:01.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 msgr2=0x7f4300071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 0x7f4300071940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43100ae000 tx=0x7f43100a7000 comp rx=0 tx=0).stop 2024-05-14T11:22:01.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 msgr2=0x7f431c073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d620 tx=0x7f430c00dae0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 shutdown_connections 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f430006f430 0x7f4300071940 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c156fa0 0x7f431c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f431c0d1670 0x7f431c073550 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 --2- 172.21.15.116:0/4025394866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f431c06e9e0 0x7f431c14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 >> 172.21.15.116:0/4025394866 conn(0x7f431c0b4620 msgr2=0x7f431c15e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 shutdown_connections 2024-05-14T11:22:01.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.277+0000 7f4324800640 1 -- 172.21.15.116:0/4025394866 wait complete. 2024-05-14T11:22:01.298 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-14T11:22:00.447107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258768,"kb_used_data":9284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715952,"statfs":{"total":773094113280,"available":772829134848,"internally_reserved":0,"allocated":9506816,"data_stored":4956142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403074+0000","last_change":"2024-05-14T11:21:56.378698+0000","last_active":"2024-05-14T11:21:57.403074+0000","last_peered":"2024-05-14T11:21:57.403074+0000","last_clean":"2024-05-14T11:21:57.403074+0000","last_became_active":"2024-05-14T11:21:56.378402+0000","last_became_peered":"2024-05-14T11:21:56.378402+0000","last_unstale":"2024-05-14T11:21:57.403074+0000","last_undegraded":"2024-05-14T11:21:57.403074+0000","last_fullsized":"2024-05-14T11:21:57.403074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369688+0000","last_change":"2024-05-14T11:21:55.369688+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369688+0000","last_undegraded":"2024-05-14T11:21:55.369688+0000","last_fullsized":"2024-05-14T11:21:55.369688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402927+0000","last_change":"2024-05-14T11:21:56.392559+0000","last_active":"2024-05-14T11:21:57.402927+0000","last_peered":"2024-05-14T11:21:57.402927+0000","last_clean":"2024-05-14T11:21:57.402927+0000","last_became_active":"2024-05-14T11:21:56.391449+0000","last_became_peered":"2024-05-14T11:21:56.391449+0000","last_unstale":"2024-05-14T11:21:57.402927+0000","last_undegraded":"2024-05-14T11:21:57.402927+0000","last_fullsized":"2024-05-14T11:21:57.402927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459732+0000","last_change":"2024-05-14T11:21:56.378008+0000","last_active":"2024-05-14T11:21:57.459732+0000","last_peered":"2024-05-14T11:21:57.459732+0000","last_clean":"2024-05-14T11:21:57.459732+0000","last_became_active":"2024-05-14T11:21:56.377707+0000","last_became_peered":"2024-05-14T11:21:56.377707+0000","last_unstale":"2024-05-14T11:21:57.459732+0000","last_undegraded":"2024-05-14T11:21:57.459732+0000","last_fullsized":"2024-05-14T11:21:57.459732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461077+0000","last_change":"2024-05-14T11:21:56.391093+0000","last_active":"2024-05-14T11:21:57.461077+0000","last_peered":"2024-05-14T11:21:57.461077+0000","last_clean":"2024-05-14T11:21:57.461077+0000","last_became_active":"2024-05-14T11:21:56.390712+0000","last_became_peered":"2024-05-14T11:21:56.390712+0000","last_unstale":"2024-05-14T11:21:57.461077+0000","last_undegraded":"2024-05-14T11:21:57.461077+0000","last_fullsized":"2024-05-14T11:21:57.461077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460609+0000","last_change":"2024-05-14T11:21:56.398553+0000","last_active":"2024-05-14T11:21:57.460609+0000","last_peered":"2024-05-14T11:21:57.460609+0000","last_clean":"2024-05-14T11:21:57.460609+0000","last_became_active":"2024-05-14T11:21:56.396664+0000","last_became_peered":"2024-05-14T11:21:56.396664+0000","last_unstale":"2024-05-14T11:21:57.460609+0000","last_undegraded":"2024-05-14T11:21:57.460609+0000","last_fullsized":"2024-05-14T11:21:57.460609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:59:19.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386491+0000","last_change":"2024-05-14T11:21:56.385275+0000","last_active":"2024-05-14T11:21:57.386491+0000","last_peered":"2024-05-14T11:21:57.386491+0000","last_clean":"2024-05-14T11:21:57.386491+0000","last_became_active":"2024-05-14T11:21:56.384904+0000","last_became_peered":"2024-05-14T11:21:56.384904+0000","last_unstale":"2024-05-14T11:21:57.386491+0000","last_undegraded":"2024-05-14T11:21:57.386491+0000","last_fullsized":"2024-05-14T11:21:57.386491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:36.820614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459460+0000","last_change":"2024-05-14T11:21:56.382681+0000","last_active":"2024-05-14T11:21:57.459460+0000","last_peered":"2024-05-14T11:21:57.459460+0000","last_clean":"2024-05-14T11:21:57.459460+0000","last_became_active":"2024-05-14T11:21:56.381079+0000","last_became_peered":"2024-05-14T11:21:56.381079+0000","last_unstale":"2024-05-14T11:21:57.459460+0000","last_undegraded":"2024-05-14T11:21:57.459460+0000","last_fullsized":"2024-05-14T11:21:57.459460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.394964+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.393782+0000","last_became_peered":"2024-05-14T11:21:56.393782+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380908+0000","last_change":"2024-05-14T11:21:56.384798+0000","last_active":"2024-05-14T11:21:57.380908+0000","last_peered":"2024-05-14T11:21:57.380908+0000","last_clean":"2024-05-14T11:21:57.380908+0000","last_became_active":"2024-05-14T11:21:56.384447+0000","last_became_peered":"2024-05-14T11:21:56.384447+0000","last_unstale":"2024-05-14T11:21:57.380908+0000","last_undegraded":"2024-05-14T11:21:57.380908+0000","last_fullsized":"2024-05-14T11:21:57.380908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:12.540726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.391270+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.390667+0000","last_became_peered":"2024-05-14T11:21:56.390667+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380632+0000","last_change":"2024-05-14T11:21:56.379345+0000","last_active":"2024-05-14T11:21:57.380632+0000","last_peered":"2024-05-14T11:21:57.380632+0000","last_clean":"2024-05-14T11:21:57.380632+0000","last_became_active":"2024-05-14T11:21:56.379007+0000","last_became_peered":"2024-05-14T11:21:56.379007+0000","last_unstale":"2024-05-14T11:21:57.380632+0000","last_undegraded":"2024-05-14T11:21:57.380632+0000","last_fullsized":"2024-05-14T11:21:57.380632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:31.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459974+0000","last_change":"2024-05-14T11:21:56.384668+0000","last_active":"2024-05-14T11:21:57.459974+0000","last_peered":"2024-05-14T11:21:57.459974+0000","last_clean":"2024-05-14T11:21:57.459974+0000","last_became_active":"2024-05-14T11:21:56.382366+0000","last_became_peered":"2024-05-14T11:21:56.382366+0000","last_unstale":"2024-05-14T11:21:57.459974+0000","last_undegraded":"2024-05-14T11:21:57.459974+0000","last_fullsized":"2024-05-14T11:21:57.459974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460403+0000","last_change":"2024-05-14T11:21:56.389629+0000","last_active":"2024-05-14T11:21:57.460403+0000","last_peered":"2024-05-14T11:21:57.460403+0000","last_clean":"2024-05-14T11:21:57.460403+0000","last_became_active":"2024-05-14T11:21:56.389299+0000","last_became_peered":"2024-05-14T11:21:56.389299+0000","last_unstale":"2024-05-14T11:21:57.460403+0000","last_undegraded":"2024-05-14T11:21:57.460403+0000","last_fullsized":"2024-05-14T11:21:57.460403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:00:13.582984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459513+0000","last_change":"2024-05-14T11:21:56.387482+0000","last_active":"2024-05-14T11:21:57.459513+0000","last_peered":"2024-05-14T11:21:57.459513+0000","last_clean":"2024-05-14T11:21:57.459513+0000","last_became_active":"2024-05-14T11:21:56.385978+0000","last_became_peered":"2024-05-14T11:21:56.385978+0000","last_unstale":"2024-05-14T11:21:57.459513+0000","last_undegraded":"2024-05-14T11:21:57.459513+0000","last_fullsized":"2024-05-14T11:21:57.459513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460528+0000","last_change":"2024-05-14T11:21:56.386948+0000","last_active":"2024-05-14T11:21:57.460528+0000","last_peered":"2024-05-14T11:21:57.460528+0000","last_clean":"2024-05-14T11:21:57.460528+0000","last_became_active":"2024-05-14T11:21:56.385310+0000","last_became_peered":"2024-05-14T11:21:56.385310+0000","last_unstale":"2024-05-14T11:21:57.460528+0000","last_undegraded":"2024-05-14T11:21:57.460528+0000","last_fullsized":"2024-05-14T11:21:57.460528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:31:18.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461165+0000","last_change":"2024-05-14T11:21:56.400922+0000","last_active":"2024-05-14T11:21:57.461165+0000","last_peered":"2024-05-14T11:21:57.461165+0000","last_clean":"2024-05-14T11:21:57.461165+0000","last_became_active":"2024-05-14T11:21:56.400597+0000","last_became_peered":"2024-05-14T11:21:56.400597+0000","last_unstale":"2024-05-14T11:21:57.461165+0000","last_undegraded":"2024-05-14T11:21:57.461165+0000","last_fullsized":"2024-05-14T11:21:57.461165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:07.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403099+0000","last_change":"2024-05-14T11:21:56.397705+0000","last_active":"2024-05-14T11:21:57.403099+0000","last_peered":"2024-05-14T11:21:57.403099+0000","last_clean":"2024-05-14T11:21:57.403099+0000","last_became_active":"2024-05-14T11:21:56.397144+0000","last_became_peered":"2024-05-14T11:21:56.397144+0000","last_unstale":"2024-05-14T11:21:57.403099+0000","last_undegraded":"2024-05-14T11:21:57.403099+0000","last_fullsized":"2024-05-14T11:21:57.403099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461475+0000","last_change":"2024-05-14T11:21:56.390962+0000","last_active":"2024-05-14T11:21:57.461475+0000","last_peered":"2024-05-14T11:21:57.461475+0000","last_clean":"2024-05-14T11:21:57.461475+0000","last_became_active":"2024-05-14T11:21:56.390072+0000","last_became_peered":"2024-05-14T11:21:56.390072+0000","last_unstale":"2024-05-14T11:21:57.461475+0000","last_undegraded":"2024-05-14T11:21:57.461475+0000","last_fullsized":"2024-05-14T11:21:57.461475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459937+0000","last_change":"2024-05-14T11:21:56.379798+0000","last_active":"2024-05-14T11:21:57.459937+0000","last_peered":"2024-05-14T11:21:57.459937+0000","last_clean":"2024-05-14T11:21:57.459937+0000","last_became_active":"2024-05-14T11:21:56.378722+0000","last_became_peered":"2024-05-14T11:21:56.378722+0000","last_unstale":"2024-05-14T11:21:57.459937+0000","last_undegraded":"2024-05-14T11:21:57.459937+0000","last_fullsized":"2024-05-14T11:21:57.459937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460813+0000","last_change":"2024-05-14T11:21:56.391852+0000","last_active":"2024-05-14T11:21:57.460813+0000","last_peered":"2024-05-14T11:21:57.460813+0000","last_clean":"2024-05-14T11:21:57.460813+0000","last_became_active":"2024-05-14T11:21:56.391495+0000","last_became_peered":"2024-05-14T11:21:56.391495+0000","last_unstale":"2024-05-14T11:21:57.460813+0000","last_undegraded":"2024-05-14T11:21:57.460813+0000","last_fullsized":"2024-05-14T11:21:57.460813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460442+0000","last_change":"2024-05-14T11:21:56.390530+0000","last_active":"2024-05-14T11:21:57.460442+0000","last_peered":"2024-05-14T11:21:57.460442+0000","last_clean":"2024-05-14T11:21:57.460442+0000","last_became_active":"2024-05-14T11:21:56.390215+0000","last_became_peered":"2024-05-14T11:21:56.390215+0000","last_unstale":"2024-05-14T11:21:57.460442+0000","last_undegraded":"2024-05-14T11:21:57.460442+0000","last_fullsized":"2024-05-14T11:21:57.460442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:07.315380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387804+0000","last_change":"2024-05-14T11:21:56.391315+0000","last_active":"2024-05-14T11:21:57.387804+0000","last_peered":"2024-05-14T11:21:57.387804+0000","last_clean":"2024-05-14T11:21:57.387804+0000","last_became_active":"2024-05-14T11:21:56.390649+0000","last_became_peered":"2024-05-14T11:21:56.390649+0000","last_unstale":"2024-05-14T11:21:57.387804+0000","last_undegraded":"2024-05-14T11:21:57.387804+0000","last_fullsized":"2024-05-14T11:21:57.387804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402786+0000","last_change":"2024-05-14T11:21:56.394765+0000","last_active":"2024-05-14T11:21:57.402786+0000","last_peered":"2024-05-14T11:21:57.402786+0000","last_clean":"2024-05-14T11:21:57.402786+0000","last_became_active":"2024-05-14T11:21:56.394490+0000","last_became_peered":"2024-05-14T11:21:56.394490+0000","last_unstale":"2024-05-14T11:21:57.402786+0000","last_undegraded":"2024-05-14T11:21:57.402786+0000","last_fullsized":"2024-05-14T11:21:57.402786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.372405+0000","last_change":"2024-05-14T11:21:55.372405+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.372405+0000","last_undegraded":"2024-05-14T11:21:55.372405+0000","last_fullsized":"2024-05-14T11:21:55.372405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461831+0000","last_change":"2024-05-14T11:21:56.389580+0000","last_active":"2024-05-14T11:21:57.461831+0000","last_peered":"2024-05-14T11:21:57.461831+0000","last_clean":"2024-05-14T11:21:57.461831+0000","last_became_active":"2024-05-14T11:21:56.389252+0000","last_became_peered":"2024-05-14T11:21:56.389252+0000","last_unstale":"2024-05-14T11:21:57.461831+0000","last_undegraded":"2024-05-14T11:21:57.461831+0000","last_fullsized":"2024-05-14T11:21:57.461831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.381007+0000","last_change":"2024-05-14T11:21:56.378063+0000","last_active":"2024-05-14T11:21:57.381007+0000","last_peered":"2024-05-14T11:21:57.381007+0000","last_clean":"2024-05-14T11:21:57.381007+0000","last_became_active":"2024-05-14T11:21:56.377759+0000","last_became_peered":"2024-05-14T11:21:56.377759+0000","last_unstale":"2024-05-14T11:21:57.381007+0000","last_undegraded":"2024-05-14T11:21:57.381007+0000","last_fullsized":"2024-05-14T11:21:57.381007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:43.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460090+0000","last_change":"2024-05-14T11:21:56.398675+0000","last_active":"2024-05-14T11:21:57.460090+0000","last_peered":"2024-05-14T11:21:57.460090+0000","last_clean":"2024-05-14T11:21:57.460090+0000","last_became_active":"2024-05-14T11:21:56.392629+0000","last_became_peered":"2024-05-14T11:21:56.392629+0000","last_unstale":"2024-05-14T11:21:57.460090+0000","last_undegraded":"2024-05-14T11:21:57.460090+0000","last_fullsized":"2024-05-14T11:21:57.460090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386641+0000","last_change":"2024-05-14T11:21:56.371575+0000","last_active":"2024-05-14T11:21:57.386641+0000","last_peered":"2024-05-14T11:21:57.386641+0000","last_clean":"2024-05-14T11:21:57.386641+0000","last_became_active":"2024-05-14T11:21:56.371212+0000","last_became_peered":"2024-05-14T11:21:56.371212+0000","last_unstale":"2024-05-14T11:21:57.386641+0000","last_undegraded":"2024-05-14T11:21:57.386641+0000","last_fullsized":"2024-05-14T11:21:57.386641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:49:03.645090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460626+0000","last_change":"2024-05-14T11:21:56.394494+0000","last_active":"2024-05-14T11:21:57.460626+0000","last_peered":"2024-05-14T11:21:57.460626+0000","last_clean":"2024-05-14T11:21:57.460626+0000","last_became_active":"2024-05-14T11:21:56.394205+0000","last_became_peered":"2024-05-14T11:21:56.394205+0000","last_unstale":"2024-05-14T11:21:57.460626+0000","last_undegraded":"2024-05-14T11:21:57.460626+0000","last_fullsized":"2024-05-14T11:21:57.460626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370175+0000","last_change":"2024-05-14T11:21:55.370175+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370175+0000","last_undegraded":"2024-05-14T11:21:55.370175+0000","last_fullsized":"2024-05-14T11:21:55.370175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460844+0000","last_change":"2024-05-14T11:21:56.382783+0000","last_active":"2024-05-14T11:21:57.460844+0000","last_peered":"2024-05-14T11:21:57.460844+0000","last_clean":"2024-05-14T11:21:57.460844+0000","last_became_active":"2024-05-14T11:21:56.382496+0000","last_became_peered":"2024-05-14T11:21:56.382496+0000","last_unstale":"2024-05-14T11:21:57.460844+0000","last_undegraded":"2024-05-14T11:21:57.460844+0000","last_fullsized":"2024-05-14T11:21:57.460844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:44.636667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461113+0000","last_change":"2024-05-14T11:21:56.399015+0000","last_active":"2024-05-14T11:21:57.461113+0000","last_peered":"2024-05-14T11:21:57.461113+0000","last_clean":"2024-05-14T11:21:57.461113+0000","last_became_active":"2024-05-14T11:21:56.397801+0000","last_became_peered":"2024-05-14T11:21:56.397801+0000","last_unstale":"2024-05-14T11:21:57.461113+0000","last_undegraded":"2024-05-14T11:21:57.461113+0000","last_fullsized":"2024-05-14T11:21:57.461113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:03.545193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387341+0000","last_change":"2024-05-14T11:21:55.377300+0000","last_active":"2024-05-14T11:21:57.387341+0000","last_peered":"2024-05-14T11:21:57.387341+0000","last_clean":"2024-05-14T11:21:57.387341+0000","last_became_active":"2024-05-14T11:21:55.376999+0000","last_became_peered":"2024-05-14T11:21:55.376999+0000","last_unstale":"2024-05-14T11:21:57.387341+0000","last_undegraded":"2024-05-14T11:21:57.387341+0000","last_fullsized":"2024-05-14T11:21:57.387341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:14.148384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460907+0000","last_change":"2024-05-14T11:21:56.381804+0000","last_active":"2024-05-14T11:21:57.460907+0000","last_peered":"2024-05-14T11:21:57.460907+0000","last_clean":"2024-05-14T11:21:57.460907+0000","last_became_active":"2024-05-14T11:21:56.381554+0000","last_became_peered":"2024-05-14T11:21:56.381554+0000","last_unstale":"2024-05-14T11:21:57.460907+0000","last_undegraded":"2024-05-14T11:21:57.460907+0000","last_fullsized":"2024-05-14T11:21:57.460907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:37:23.266696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461538+0000","last_change":"2024-05-14T11:21:55.373916+0000","last_active":"2024-05-14T11:21:57.461538+0000","last_peered":"2024-05-14T11:21:57.461538+0000","last_clean":"2024-05-14T11:21:57.461538+0000","last_became_active":"2024-05-14T11:21:55.373461+0000","last_became_peered":"2024-05-14T11:21:55.373461+0000","last_unstale":"2024-05-14T11:21:57.461538+0000","last_undegraded":"2024-05-14T11:21:57.461538+0000","last_fullsized":"2024-05-14T11:21:57.461538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:01.508933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386940+0000","last_change":"2024-05-14T11:21:56.385784+0000","last_active":"2024-05-14T11:21:57.386940+0000","last_peered":"2024-05-14T11:21:57.386940+0000","last_clean":"2024-05-14T11:21:57.386940+0000","last_became_active":"2024-05-14T11:21:56.385460+0000","last_became_peered":"2024-05-14T11:21:56.385460+0000","last_unstale":"2024-05-14T11:21:57.386940+0000","last_undegraded":"2024-05-14T11:21:57.386940+0000","last_fullsized":"2024-05-14T11:21:57.386940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:02:04.422136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387331+0000","last_change":"2024-05-14T11:21:55.384239+0000","last_active":"2024-05-14T11:21:57.387331+0000","last_peered":"2024-05-14T11:21:57.387331+0000","last_clean":"2024-05-14T11:21:57.387331+0000","last_became_active":"2024-05-14T11:21:55.383906+0000","last_became_peered":"2024-05-14T11:21:55.383906+0000","last_unstale":"2024-05-14T11:21:57.387331+0000","last_undegraded":"2024-05-14T11:21:57.387331+0000","last_fullsized":"2024-05-14T11:21:57.387331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:11.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461213+0000","last_change":"2024-05-14T11:21:56.398729+0000","last_active":"2024-05-14T11:21:57.461213+0000","last_peered":"2024-05-14T11:21:57.461213+0000","last_clean":"2024-05-14T11:21:57.461213+0000","last_became_active":"2024-05-14T11:21:56.397051+0000","last_became_peered":"2024-05-14T11:21:56.397051+0000","last_unstale":"2024-05-14T11:21:57.461213+0000","last_undegraded":"2024-05-14T11:21:57.461213+0000","last_fullsized":"2024-05-14T11:21:57.461213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:22:03.622138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387307+0000","last_change":"2024-05-14T11:21:55.386498+0000","last_active":"2024-05-14T11:21:57.387307+0000","last_peered":"2024-05-14T11:21:57.387307+0000","last_clean":"2024-05-14T11:21:57.387307+0000","last_became_active":"2024-05-14T11:21:55.385760+0000","last_became_peered":"2024-05-14T11:21:55.385760+0000","last_unstale":"2024-05-14T11:21:57.387307+0000","last_undegraded":"2024-05-14T11:21:57.387307+0000","last_fullsized":"2024-05-14T11:21:57.387307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.837126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459652+0000","last_change":"2024-05-14T11:21:56.381076+0000","last_active":"2024-05-14T11:21:57.459652+0000","last_peered":"2024-05-14T11:21:57.459652+0000","last_clean":"2024-05-14T11:21:57.459652+0000","last_became_active":"2024-05-14T11:21:56.380328+0000","last_became_peered":"2024-05-14T11:21:56.380328+0000","last_unstale":"2024-05-14T11:21:57.459652+0000","last_undegraded":"2024-05-14T11:21:57.459652+0000","last_fullsized":"2024-05-14T11:21:57.459652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461212+0000","last_change":"2024-05-14T11:21:55.383653+0000","last_active":"2024-05-14T11:21:57.461212+0000","last_peered":"2024-05-14T11:21:57.461212+0000","last_clean":"2024-05-14T11:21:57.461212+0000","last_became_active":"2024-05-14T11:21:55.383294+0000","last_became_peered":"2024-05-14T11:21:55.383294+0000","last_unstale":"2024-05-14T11:21:57.461212+0000","last_undegraded":"2024-05-14T11:21:57.461212+0000","last_fullsized":"2024-05-14T11:21:57.461212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:14.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386959+0000","last_change":"2024-05-14T11:21:56.378909+0000","last_active":"2024-05-14T11:21:57.386959+0000","last_peered":"2024-05-14T11:21:57.386959+0000","last_clean":"2024-05-14T11:21:57.386959+0000","last_became_active":"2024-05-14T11:21:56.378614+0000","last_became_peered":"2024-05-14T11:21:56.378614+0000","last_unstale":"2024-05-14T11:21:57.386959+0000","last_undegraded":"2024-05-14T11:21:57.386959+0000","last_fullsized":"2024-05-14T11:21:57.386959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460014+0000","last_change":"2024-05-14T11:21:55.387444+0000","last_active":"2024-05-14T11:21:57.460014+0000","last_peered":"2024-05-14T11:21:57.460014+0000","last_clean":"2024-05-14T11:21:57.460014+0000","last_became_active":"2024-05-14T11:21:55.386992+0000","last_became_peered":"2024-05-14T11:21:55.386992+0000","last_unstale":"2024-05-14T11:21:57.460014+0000","last_undegraded":"2024-05-14T11:21:57.460014+0000","last_fullsized":"2024-05-14T11:21:57.460014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:34.670851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460786+0000","last_change":"2024-05-14T11:21:56.389934+0000","last_active":"2024-05-14T11:21:57.460786+0000","last_peered":"2024-05-14T11:21:57.460786+0000","last_clean":"2024-05-14T11:21:57.460786+0000","last_became_active":"2024-05-14T11:21:56.388204+0000","last_became_peered":"2024-05-14T11:21:56.388204+0000","last_unstale":"2024-05-14T11:21:57.460786+0000","last_undegraded":"2024-05-14T11:21:57.460786+0000","last_fullsized":"2024-05-14T11:21:57.460786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:42.638658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402951+0000","last_change":"2024-05-14T11:21:55.379234+0000","last_active":"2024-05-14T11:21:57.402951+0000","last_peered":"2024-05-14T11:21:57.402951+0000","last_clean":"2024-05-14T11:21:57.402951+0000","last_became_active":"2024-05-14T11:21:55.378924+0000","last_became_peered":"2024-05-14T11:21:55.378924+0000","last_unstale":"2024-05-14T11:21:57.402951+0000","last_undegraded":"2024-05-14T11:21:57.402951+0000","last_fullsized":"2024-05-14T11:21:57.402951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460137+0000","last_change":"2024-05-14T11:21:56.383194+0000","last_active":"2024-05-14T11:21:57.460137+0000","last_peered":"2024-05-14T11:21:57.460137+0000","last_clean":"2024-05-14T11:21:57.460137+0000","last_became_active":"2024-05-14T11:21:56.379071+0000","last_became_peered":"2024-05-14T11:21:56.379071+0000","last_unstale":"2024-05-14T11:21:57.460137+0000","last_undegraded":"2024-05-14T11:21:57.460137+0000","last_fullsized":"2024-05-14T11:21:57.460137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386225+0000","last_change":"2024-05-14T11:21:55.384145+0000","last_active":"2024-05-14T11:21:57.386225+0000","last_peered":"2024-05-14T11:21:57.386225+0000","last_clean":"2024-05-14T11:21:57.386225+0000","last_became_active":"2024-05-14T11:21:55.383834+0000","last_became_peered":"2024-05-14T11:21:55.383834+0000","last_unstale":"2024-05-14T11:21:57.386225+0000","last_undegraded":"2024-05-14T11:21:57.386225+0000","last_fullsized":"2024-05-14T11:21:57.386225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:31.347753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461043+0000","last_change":"2024-05-14T11:21:56.396531+0000","last_active":"2024-05-14T11:21:57.461043+0000","last_peered":"2024-05-14T11:21:57.461043+0000","last_clean":"2024-05-14T11:21:57.461043+0000","last_became_active":"2024-05-14T11:21:56.396223+0000","last_became_peered":"2024-05-14T11:21:56.396223+0000","last_unstale":"2024-05-14T11:21:57.461043+0000","last_undegraded":"2024-05-14T11:21:57.461043+0000","last_fullsized":"2024-05-14T11:21:57.461043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.385972+0000","last_change":"2024-05-14T11:21:55.379424+0000","last_active":"2024-05-14T11:21:55.385972+0000","last_peered":"2024-05-14T11:21:55.385972+0000","last_clean":"2024-05-14T11:21:55.385972+0000","last_became_active":"2024-05-14T11:21:55.378552+0000","last_became_peered":"2024-05-14T11:21:55.378552+0000","last_unstale":"2024-05-14T11:21:55.385972+0000","last_undegraded":"2024-05-14T11:21:55.385972+0000","last_fullsized":"2024-05-14T11:21:55.385972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:39.156471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387395+0000","last_change":"2024-05-14T11:21:56.392129+0000","last_active":"2024-05-14T11:21:57.387395+0000","last_peered":"2024-05-14T11:21:57.387395+0000","last_clean":"2024-05-14T11:21:57.387395+0000","last_became_active":"2024-05-14T11:21:56.391814+0000","last_became_peered":"2024-05-14T11:21:56.391814+0000","last_unstale":"2024-05-14T11:21:57.387395+0000","last_undegraded":"2024-05-14T11:21:57.387395+0000","last_fullsized":"2024-05-14T11:21:57.387395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:20.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403200+0000","last_change":"2024-05-14T11:21:55.388521+0000","last_active":"2024-05-14T11:21:57.403200+0000","last_peered":"2024-05-14T11:21:57.403200+0000","last_clean":"2024-05-14T11:21:57.403200+0000","last_became_active":"2024-05-14T11:21:55.387310+0000","last_became_peered":"2024-05-14T11:21:55.387310+0000","last_unstale":"2024-05-14T11:21:57.403200+0000","last_undegraded":"2024-05-14T11:21:57.403200+0000","last_fullsized":"2024-05-14T11:21:57.403200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403125+0000","last_change":"2024-05-14T11:21:56.379479+0000","last_active":"2024-05-14T11:21:57.403125+0000","last_peered":"2024-05-14T11:21:57.403125+0000","last_clean":"2024-05-14T11:21:57.403125+0000","last_became_active":"2024-05-14T11:21:56.379170+0000","last_became_peered":"2024-05-14T11:21:56.379170+0000","last_unstale":"2024-05-14T11:21:57.403125+0000","last_undegraded":"2024-05-14T11:21:57.403125+0000","last_fullsized":"2024-05-14T11:21:57.403125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:45.902619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461559+0000","last_change":"2024-05-14T11:21:55.379532+0000","last_active":"2024-05-14T11:21:57.461559+0000","last_peered":"2024-05-14T11:21:57.461559+0000","last_clean":"2024-05-14T11:21:57.461559+0000","last_became_active":"2024-05-14T11:21:55.379224+0000","last_became_peered":"2024-05-14T11:21:55.379224+0000","last_unstale":"2024-05-14T11:21:57.461559+0000","last_undegraded":"2024-05-14T11:21:57.461559+0000","last_fullsized":"2024-05-14T11:21:57.461559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:23:55.106701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459814+0000","last_change":"2024-05-14T11:21:56.400307+0000","last_active":"2024-05-14T11:21:57.459814+0000","last_peered":"2024-05-14T11:21:57.459814+0000","last_clean":"2024-05-14T11:21:57.459814+0000","last_became_active":"2024-05-14T11:21:56.399940+0000","last_became_peered":"2024-05-14T11:21:56.399940+0000","last_unstale":"2024-05-14T11:21:57.459814+0000","last_undegraded":"2024-05-14T11:21:57.459814+0000","last_fullsized":"2024-05-14T11:21:57.459814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461259+0000","last_change":"2024-05-14T11:21:55.378699+0000","last_active":"2024-05-14T11:21:57.461259+0000","last_peered":"2024-05-14T11:21:57.461259+0000","last_clean":"2024-05-14T11:21:57.461259+0000","last_became_active":"2024-05-14T11:21:55.378372+0000","last_became_peered":"2024-05-14T11:21:55.378372+0000","last_unstale":"2024-05-14T11:21:57.461259+0000","last_undegraded":"2024-05-14T11:21:57.461259+0000","last_fullsized":"2024-05-14T11:21:57.461259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:42.121073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459654+0000","last_change":"2024-05-14T11:21:56.380739+0000","last_active":"2024-05-14T11:21:57.459654+0000","last_peered":"2024-05-14T11:21:57.459654+0000","last_clean":"2024-05-14T11:21:57.459654+0000","last_became_active":"2024-05-14T11:21:56.377804+0000","last_became_peered":"2024-05-14T11:21:56.377804+0000","last_unstale":"2024-05-14T11:21:57.459654+0000","last_undegraded":"2024-05-14T11:21:57.459654+0000","last_fullsized":"2024-05-14T11:21:57.459654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386631+0000","last_change":"2024-05-14T11:21:55.380045+0000","last_active":"2024-05-14T11:21:57.386631+0000","last_peered":"2024-05-14T11:21:57.386631+0000","last_clean":"2024-05-14T11:21:57.386631+0000","last_became_active":"2024-05-14T11:21:55.378906+0000","last_became_peered":"2024-05-14T11:21:55.378906+0000","last_unstale":"2024-05-14T11:21:57.386631+0000","last_undegraded":"2024-05-14T11:21:57.386631+0000","last_fullsized":"2024-05-14T11:21:57.386631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:34.974768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460535+0000","last_change":"2024-05-14T11:21:56.388451+0000","last_active":"2024-05-14T11:21:57.460535+0000","last_peered":"2024-05-14T11:21:57.460535+0000","last_clean":"2024-05-14T11:21:57.460535+0000","last_became_active":"2024-05-14T11:21:56.388148+0000","last_became_peered":"2024-05-14T11:21:56.388148+0000","last_unstale":"2024-05-14T11:21:57.460535+0000","last_undegraded":"2024-05-14T11:21:57.460535+0000","last_fullsized":"2024-05-14T11:21:57.460535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460827+0000","last_change":"2024-05-14T11:21:55.379659+0000","last_active":"2024-05-14T11:21:57.460827+0000","last_peered":"2024-05-14T11:21:57.460827+0000","last_clean":"2024-05-14T11:21:57.460827+0000","last_became_active":"2024-05-14T11:21:55.379357+0000","last_became_peered":"2024-05-14T11:21:55.379357+0000","last_unstale":"2024-05-14T11:21:57.460827+0000","last_undegraded":"2024-05-14T11:21:57.460827+0000","last_fullsized":"2024-05-14T11:21:57.460827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:54.423695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380729+0000","last_change":"2024-05-14T11:21:56.390097+0000","last_active":"2024-05-14T11:21:57.380729+0000","last_peered":"2024-05-14T11:21:57.380729+0000","last_clean":"2024-05-14T11:21:57.380729+0000","last_became_active":"2024-05-14T11:21:56.389775+0000","last_became_peered":"2024-05-14T11:21:56.389775+0000","last_unstale":"2024-05-14T11:21:57.380729+0000","last_undegraded":"2024-05-14T11:21:57.380729+0000","last_fullsized":"2024-05-14T11:21:57.380729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:24.671464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459571+0000","last_change":"2024-05-14T11:21:55.384436+0000","last_active":"2024-05-14T11:21:57.459571+0000","last_peered":"2024-05-14T11:21:57.459571+0000","last_clean":"2024-05-14T11:21:57.459571+0000","last_became_active":"2024-05-14T11:21:55.384104+0000","last_became_peered":"2024-05-14T11:21:55.384104+0000","last_unstale":"2024-05-14T11:21:57.459571+0000","last_undegraded":"2024-05-14T11:21:57.459571+0000","last_fullsized":"2024-05-14T11:21:57.459571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:56.800584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380862+0000","last_change":"2024-05-14T11:21:56.393507+0000","last_active":"2024-05-14T11:21:57.380862+0000","last_peered":"2024-05-14T11:21:57.380862+0000","last_clean":"2024-05-14T11:21:57.380862+0000","last_became_active":"2024-05-14T11:21:56.393227+0000","last_became_peered":"2024-05-14T11:21:56.393227+0000","last_unstale":"2024-05-14T11:21:57.380862+0000","last_undegraded":"2024-05-14T11:21:57.380862+0000","last_fullsized":"2024-05-14T11:21:57.380862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:26.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459792+0000","last_change":"2024-05-14T11:21:55.386065+0000","last_active":"2024-05-14T11:21:57.459792+0000","last_peered":"2024-05-14T11:21:57.459792+0000","last_clean":"2024-05-14T11:21:57.459792+0000","last_became_active":"2024-05-14T11:21:55.385699+0000","last_became_peered":"2024-05-14T11:21:55.385699+0000","last_unstale":"2024-05-14T11:21:57.459792+0000","last_undegraded":"2024-05-14T11:21:57.459792+0000","last_fullsized":"2024-05-14T11:21:57.459792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:34:21.465908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461286+0000","last_change":"2024-05-14T11:21:56.396244+0000","last_active":"2024-05-14T11:21:57.461286+0000","last_peered":"2024-05-14T11:21:57.461286+0000","last_clean":"2024-05-14T11:21:57.461286+0000","last_became_active":"2024-05-14T11:21:56.395934+0000","last_became_peered":"2024-05-14T11:21:56.395934+0000","last_unstale":"2024-05-14T11:21:57.461286+0000","last_undegraded":"2024-05-14T11:21:57.461286+0000","last_fullsized":"2024-05-14T11:21:57.461286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:44.571245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387256+0000","last_change":"2024-05-14T11:21:55.376695+0000","last_active":"2024-05-14T11:21:57.387256+0000","last_peered":"2024-05-14T11:21:57.387256+0000","last_clean":"2024-05-14T11:21:57.387256+0000","last_became_active":"2024-05-14T11:21:55.375674+0000","last_became_peered":"2024-05-14T11:21:55.375674+0000","last_unstale":"2024-05-14T11:21:57.387256+0000","last_undegraded":"2024-05-14T11:21:57.387256+0000","last_fullsized":"2024-05-14T11:21:57.387256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:57.676773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386997+0000","last_change":"2024-05-14T11:21:56.394005+0000","last_active":"2024-05-14T11:21:57.386997+0000","last_peered":"2024-05-14T11:21:57.386997+0000","last_clean":"2024-05-14T11:21:57.386997+0000","last_became_active":"2024-05-14T11:21:56.393708+0000","last_became_peered":"2024-05-14T11:21:56.393708+0000","last_unstale":"2024-05-14T11:21:57.386997+0000","last_undegraded":"2024-05-14T11:21:57.386997+0000","last_fullsized":"2024-05-14T11:21:57.386997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387203+0000","last_change":"2024-05-14T11:21:55.378651+0000","last_active":"2024-05-14T11:21:57.387203+0000","last_peered":"2024-05-14T11:21:57.387203+0000","last_clean":"2024-05-14T11:21:57.387203+0000","last_became_active":"2024-05-14T11:21:55.378348+0000","last_became_peered":"2024-05-14T11:21:55.378348+0000","last_unstale":"2024-05-14T11:21:57.387203+0000","last_undegraded":"2024-05-14T11:21:57.387203+0000","last_fullsized":"2024-05-14T11:21:57.387203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:13.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461456+0000","last_change":"2024-05-14T11:21:56.389463+0000","last_active":"2024-05-14T11:21:57.461456+0000","last_peered":"2024-05-14T11:21:57.461456+0000","last_clean":"2024-05-14T11:21:57.461456+0000","last_became_active":"2024-05-14T11:21:56.389014+0000","last_became_peered":"2024-05-14T11:21:56.389014+0000","last_unstale":"2024-05-14T11:21:57.461456+0000","last_undegraded":"2024-05-14T11:21:57.461456+0000","last_fullsized":"2024-05-14T11:21:57.461456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461621+0000","last_change":"2024-05-14T11:21:55.383442+0000","last_active":"2024-05-14T11:21:57.461621+0000","last_peered":"2024-05-14T11:21:57.461621+0000","last_clean":"2024-05-14T11:21:57.461621+0000","last_became_active":"2024-05-14T11:21:55.383197+0000","last_became_peered":"2024-05-14T11:21:55.383197+0000","last_unstale":"2024-05-14T11:21:57.461621+0000","last_undegraded":"2024-05-14T11:21:57.461621+0000","last_fullsized":"2024-05-14T11:21:57.461621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:52:51.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460829+0000","last_change":"2024-05-14T11:21:56.397428+0000","last_active":"2024-05-14T11:21:57.460829+0000","last_peered":"2024-05-14T11:21:57.460829+0000","last_clean":"2024-05-14T11:21:57.460829+0000","last_became_active":"2024-05-14T11:21:56.397079+0000","last_became_peered":"2024-05-14T11:21:56.397079+0000","last_unstale":"2024-05-14T11:21:57.460829+0000","last_undegraded":"2024-05-14T11:21:57.460829+0000","last_fullsized":"2024-05-14T11:21:57.460829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:05.256399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388807+0000","last_change":"2024-05-14T11:21:55.381263+0000","last_active":"2024-05-14T11:21:55.388807+0000","last_peered":"2024-05-14T11:21:55.388807+0000","last_clean":"2024-05-14T11:21:55.388807+0000","last_became_active":"2024-05-14T11:21:55.380928+0000","last_became_peered":"2024-05-14T11:21:55.380928+0000","last_unstale":"2024-05-14T11:21:55.388807+0000","last_undegraded":"2024-05-14T11:21:55.388807+0000","last_fullsized":"2024-05-14T11:21:55.388807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:43.506348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369616+0000","last_change":"2024-05-14T11:21:55.369616+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369616+0000","last_undegraded":"2024-05-14T11:21:55.369616+0000","last_fullsized":"2024-05-14T11:21:55.369616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380740+0000","last_change":"2024-05-14T11:21:55.386139+0000","last_active":"2024-05-14T11:21:57.380740+0000","last_peered":"2024-05-14T11:21:57.380740+0000","last_clean":"2024-05-14T11:21:57.380740+0000","last_became_active":"2024-05-14T11:21:55.385435+0000","last_became_peered":"2024-05-14T11:21:55.385435+0000","last_unstale":"2024-05-14T11:21:57.380740+0000","last_undegraded":"2024-05-14T11:21:57.380740+0000","last_fullsized":"2024-05-14T11:21:57.380740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:00.748983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370647+0000","last_change":"2024-05-14T11:21:55.370647+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370647+0000","last_undegraded":"2024-05-14T11:21:55.370647+0000","last_fullsized":"2024-05-14T11:21:55.370647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.381246+0000","last_change":"2024-05-14T11:21:55.377847+0000","last_active":"2024-05-14T11:21:55.381246+0000","last_peered":"2024-05-14T11:21:55.381246+0000","last_clean":"2024-05-14T11:21:55.381246+0000","last_became_active":"2024-05-14T11:21:55.377479+0000","last_became_peered":"2024-05-14T11:21:55.377479+0000","last_unstale":"2024-05-14T11:21:55.381246+0000","last_undegraded":"2024-05-14T11:21:55.381246+0000","last_fullsized":"2024-05-14T11:21:55.381246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:40.925873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460650+0000","last_change":"2024-05-14T11:21:56.381486+0000","last_active":"2024-05-14T11:21:57.460650+0000","last_peered":"2024-05-14T11:21:57.460650+0000","last_clean":"2024-05-14T11:21:57.460650+0000","last_became_active":"2024-05-14T11:21:56.381191+0000","last_became_peered":"2024-05-14T11:21:56.381191+0000","last_unstale":"2024-05-14T11:21:57.460650+0000","last_undegraded":"2024-05-14T11:21:57.460650+0000","last_fullsized":"2024-05-14T11:21:57.460650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:11:28.255356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380491+0000","last_change":"2024-05-14T11:21:55.380084+0000","last_active":"2024-05-14T11:21:57.380491+0000","last_peered":"2024-05-14T11:21:57.380491+0000","last_clean":"2024-05-14T11:21:57.380491+0000","last_became_active":"2024-05-14T11:21:55.379631+0000","last_became_peered":"2024-05-14T11:21:55.379631+0000","last_unstale":"2024-05-14T11:21:57.380491+0000","last_undegraded":"2024-05-14T11:21:57.380491+0000","last_fullsized":"2024-05-14T11:21:57.380491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:11.076627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387177+0000","last_change":"2024-05-14T11:21:56.380163+0000","last_active":"2024-05-14T11:21:57.387177+0000","last_peered":"2024-05-14T11:21:57.387177+0000","last_clean":"2024-05-14T11:21:57.387177+0000","last_became_active":"2024-05-14T11:21:56.377500+0000","last_became_peered":"2024-05-14T11:21:56.377500+0000","last_unstale":"2024-05-14T11:21:57.387177+0000","last_undegraded":"2024-05-14T11:21:57.387177+0000","last_fullsized":"2024-05-14T11:21:57.387177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387601+0000","last_change":"2024-05-14T11:21:55.382014+0000","last_active":"2024-05-14T11:21:57.387601+0000","last_peered":"2024-05-14T11:21:57.387601+0000","last_clean":"2024-05-14T11:21:57.387601+0000","last_became_active":"2024-05-14T11:21:55.381670+0000","last_became_peered":"2024-05-14T11:21:55.381670+0000","last_unstale":"2024-05-14T11:21:57.387601+0000","last_undegraded":"2024-05-14T11:21:57.387601+0000","last_fullsized":"2024-05-14T11:21:57.387601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:25.098189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461127+0000","last_change":"2024-05-14T11:21:56.397506+0000","last_active":"2024-05-14T11:21:57.461127+0000","last_peered":"2024-05-14T11:21:57.461127+0000","last_clean":"2024-05-14T11:21:57.461127+0000","last_became_active":"2024-05-14T11:21:56.395519+0000","last_became_peered":"2024-05-14T11:21:56.395519+0000","last_unstale":"2024-05-14T11:21:57.461127+0000","last_undegraded":"2024-05-14T11:21:57.461127+0000","last_fullsized":"2024-05-14T11:21:57.461127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:26:54.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461633+0000","last_change":"2024-05-14T11:21:55.375676+0000","last_active":"2024-05-14T11:21:57.461633+0000","last_peered":"2024-05-14T11:21:57.461633+0000","last_clean":"2024-05-14T11:21:57.461633+0000","last_became_active":"2024-05-14T11:21:55.375356+0000","last_became_peered":"2024-05-14T11:21:55.375356+0000","last_unstale":"2024-05-14T11:21:57.461633+0000","last_undegraded":"2024-05-14T11:21:57.461633+0000","last_fullsized":"2024-05-14T11:21:57.461633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:11.395431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461112+0000","last_change":"2024-05-14T11:21:56.393934+0000","last_active":"2024-05-14T11:21:57.461112+0000","last_peered":"2024-05-14T11:21:57.461112+0000","last_clean":"2024-05-14T11:21:57.461112+0000","last_became_active":"2024-05-14T11:21:56.393609+0000","last_became_peered":"2024-05-14T11:21:56.393609+0000","last_unstale":"2024-05-14T11:21:57.461112+0000","last_undegraded":"2024-05-14T11:21:57.461112+0000","last_fullsized":"2024-05-14T11:21:57.461112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:22.581542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461639+0000","last_change":"2024-05-14T11:21:55.379420+0000","last_active":"2024-05-14T11:21:57.461639+0000","last_peered":"2024-05-14T11:21:57.461639+0000","last_clean":"2024-05-14T11:21:57.461639+0000","last_became_active":"2024-05-14T11:21:55.378985+0000","last_became_peered":"2024-05-14T11:21:55.378985+0000","last_unstale":"2024-05-14T11:21:57.461639+0000","last_undegraded":"2024-05-14T11:21:57.461639+0000","last_fullsized":"2024-05-14T11:21:57.461639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:30.781659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.371911+0000","last_change":"2024-05-14T11:21:55.371911+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.371911+0000","last_undegraded":"2024-05-14T11:21:55.371911+0000","last_fullsized":"2024-05-14T11:21:55.371911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387442+0000","last_change":"2024-05-14T11:21:55.374216+0000","last_active":"2024-05-14T11:21:57.387442+0000","last_peered":"2024-05-14T11:21:57.387442+0000","last_clean":"2024-05-14T11:21:57.387442+0000","last_became_active":"2024-05-14T11:21:55.373713+0000","last_became_peered":"2024-05-14T11:21:55.373713+0000","last_unstale":"2024-05-14T11:21:57.387442+0000","last_undegraded":"2024-05-14T11:21:57.387442+0000","last_fullsized":"2024-05-14T11:21:57.387442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:32:45.264705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380622+0000","last_change":"2024-05-14T11:21:56.383298+0000","last_active":"2024-05-14T11:21:57.380622+0000","last_peered":"2024-05-14T11:21:57.380622+0000","last_clean":"2024-05-14T11:21:57.380622+0000","last_became_active":"2024-05-14T11:21:56.381852+0000","last_became_peered":"2024-05-14T11:21:56.381852+0000","last_unstale":"2024-05-14T11:21:57.380622+0000","last_undegraded":"2024-05-14T11:21:57.380622+0000","last_fullsized":"2024-05-14T11:21:57.380622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:45:25.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460760+0000","last_change":"2024-05-14T11:21:55.379545+0000","last_active":"2024-05-14T11:21:57.460760+0000","last_peered":"2024-05-14T11:21:57.460760+0000","last_clean":"2024-05-14T11:21:57.460760+0000","last_became_active":"2024-05-14T11:21:55.378762+0000","last_became_peered":"2024-05-14T11:21:55.378762+0000","last_unstale":"2024-05-14T11:21:57.460760+0000","last_undegraded":"2024-05-14T11:21:57.460760+0000","last_fullsized":"2024-05-14T11:21:57.460760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:44.899842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402884+0000","last_change":"2024-05-14T11:21:56.384970+0000","last_active":"2024-05-14T11:21:57.402884+0000","last_peered":"2024-05-14T11:21:57.402884+0000","last_clean":"2024-05-14T11:21:57.402884+0000","last_became_active":"2024-05-14T11:21:56.384703+0000","last_became_peered":"2024-05-14T11:21:56.384703+0000","last_unstale":"2024-05-14T11:21:57.402884+0000","last_undegraded":"2024-05-14T11:21:57.402884+0000","last_fullsized":"2024-05-14T11:21:57.402884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388829+0000","last_change":"2024-05-14T11:21:55.381768+0000","last_active":"2024-05-14T11:21:55.388829+0000","last_peered":"2024-05-14T11:21:55.388829+0000","last_clean":"2024-05-14T11:21:55.388829+0000","last_became_active":"2024-05-14T11:21:55.381485+0000","last_became_peered":"2024-05-14T11:21:55.381485+0000","last_unstale":"2024-05-14T11:21:55.388829+0000","last_undegraded":"2024-05-14T11:21:55.388829+0000","last_fullsized":"2024-05-14T11:21:55.388829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:38.207761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386457+0000","last_change":"2024-05-14T11:21:56.392100+0000","last_active":"2024-05-14T11:21:57.386457+0000","last_peered":"2024-05-14T11:21:57.386457+0000","last_clean":"2024-05-14T11:21:57.386457+0000","last_became_active":"2024-05-14T11:21:56.391699+0000","last_became_peered":"2024-05-14T11:21:56.391699+0000","last_unstale":"2024-05-14T11:21:57.386457+0000","last_undegraded":"2024-05-14T11:21:57.386457+0000","last_fullsized":"2024-05-14T11:21:57.386457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387086+0000","last_change":"2024-05-14T11:21:55.373928+0000","last_active":"2024-05-14T11:21:57.387086+0000","last_peered":"2024-05-14T11:21:57.387086+0000","last_clean":"2024-05-14T11:21:57.387086+0000","last_became_active":"2024-05-14T11:21:55.373638+0000","last_became_peered":"2024-05-14T11:21:55.373638+0000","last_unstale":"2024-05-14T11:21:57.387086+0000","last_undegraded":"2024-05-14T11:21:57.387086+0000","last_fullsized":"2024-05-14T11:21:57.387086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:35.515189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460703+0000","last_change":"2024-05-14T11:21:56.395619+0000","last_active":"2024-05-14T11:21:57.460703+0000","last_peered":"2024-05-14T11:21:57.460703+0000","last_clean":"2024-05-14T11:21:57.460703+0000","last_became_active":"2024-05-14T11:21:56.394829+0000","last_became_peered":"2024-05-14T11:21:56.394829+0000","last_unstale":"2024-05-14T11:21:57.460703+0000","last_undegraded":"2024-05-14T11:21:57.460703+0000","last_fullsized":"2024-05-14T11:21:57.460703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:36:28.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386239+0000","last_change":"2024-05-14T11:21:55.387084+0000","last_active":"2024-05-14T11:21:57.386239+0000","last_peered":"2024-05-14T11:21:57.386239+0000","last_clean":"2024-05-14T11:21:57.386239+0000","last_became_active":"2024-05-14T11:21:55.386834+0000","last_became_peered":"2024-05-14T11:21:55.386834+0000","last_unstale":"2024-05-14T11:21:57.386239+0000","last_undegraded":"2024-05-14T11:21:57.386239+0000","last_fullsized":"2024-05-14T11:21:57.386239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:17:10.801403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387496+0000","last_change":"2024-05-14T11:21:56.387037+0000","last_active":"2024-05-14T11:21:57.387496+0000","last_peered":"2024-05-14T11:21:57.387496+0000","last_clean":"2024-05-14T11:21:57.387496+0000","last_became_active":"2024-05-14T11:21:56.386496+0000","last_became_peered":"2024-05-14T11:21:56.386496+0000","last_unstale":"2024-05-14T11:21:57.387496+0000","last_undegraded":"2024-05-14T11:21:57.387496+0000","last_fullsized":"2024-05-14T11:21:57.387496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:21.878567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460199+0000","last_change":"2024-05-14T11:21:55.387153+0000","last_active":"2024-05-14T11:21:57.460199+0000","last_peered":"2024-05-14T11:21:57.460199+0000","last_clean":"2024-05-14T11:21:57.460199+0000","last_became_active":"2024-05-14T11:21:55.386815+0000","last_became_peered":"2024-05-14T11:21:55.386815+0000","last_unstale":"2024-05-14T11:21:57.460199+0000","last_undegraded":"2024-05-14T11:21:57.460199+0000","last_fullsized":"2024-05-14T11:21:57.460199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:44.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386383+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:21:57.386383+0000","last_peered":"2024-05-14T11:21:57.386383+0000","last_clean":"2024-05-14T11:21:57.386383+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:21:57.386383+0000","last_undegraded":"2024-05-14T11:21:57.386383+0000","last_fullsized":"2024-05-14T11:21:57.386383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1064313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":476105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":473959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1066575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3e is not active+clean 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.369688+0000', 'last_change': '2024-05-14T11:21:55.369688+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.369688+0000', 'last_undegraded': '2024-05-14T11:21:55.369688+0000', 'last_fullsized': '2024-05-14T11:21:55.369688+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:PG 14.27 is not active+clean 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.372405+0000', 'last_change': '2024-05-14T11:21:55.372405+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.372405+0000', 'last_undegraded': '2024-05-14T11:21:55.372405+0000', 'last_fullsized': '2024-05-14T11:21:55.372405+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:PG 14.21 is not active+clean 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.370175+0000', 'last_change': '2024-05-14T11:21:55.370175+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.370175+0000', 'last_undegraded': '2024-05-14T11:21:55.370175+0000', 'last_fullsized': '2024-05-14T11:21:55.370175+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.369616+0000', 'last_change': '2024-05-14T11:21:55.369616+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.369616+0000', 'last_undegraded': '2024-05-14T11:21:55.369616+0000', 'last_fullsized': '2024-05-14T11:21:55.369616+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:PG 14.a is not active+clean 2024-05-14T11:22:01.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.370647+0000', 'last_change': '2024-05-14T11:21:55.370647+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.370647+0000', 'last_undegraded': '2024-05-14T11:21:55.370647+0000', 'last_fullsized': '2024-05-14T11:21:55.370647+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.303 INFO:tasks.ceph.ceph_manager.ceph:PG 14.7 is not active+clean 2024-05-14T11:22:01.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2024-05-14T11:21:55.371911+0000', 'last_change': '2024-05-14T11:21:55.371911+0000', 'last_active': '2024-05-14T11:21:55.349508+0000', 'last_peered': '2024-05-14T11:21:55.349508+0000', 'last_clean': '2024-05-14T11:21:55.349508+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-14T11:21:55.371911+0000', 'last_undegraded': '2024-05-14T11:21:55.371911+0000', 'last_fullsized': '2024-05-14T11:21:55.371911+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'last_clean_scrub_stamp': '2024-05-14T11:21:55.349508+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-14T11:22:01.303 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-14T11:22:01.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 Processor -- start 2024-05-14T11:22:01.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 -- start start 2024-05-14T11:22:01.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7eff041621b0 0x7eff040d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7eff040b9020 0x7eff040ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7eff040d1d40 con 0x7eff040b9020 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7eff040cec10 con 0x7eff041621b0 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff09400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7eff040d1f90 con 0x7eff040ce800 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7eff040b9020 0x7eff040ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7eff040b9020 0x7eff040ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42532/0 (socket says 172.21.15.116:42532) 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 -- 172.21.15.116:0/3920521587 learned_addr learned my addr 172.21.15.116:0/3920521587 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff03200640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff02800640 1 --2- 172.21.15.116:0/3920521587 >> v2:172.21.15.149:3300/0 conn(0x7eff041621b0 0x7eff040d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 -- 172.21.15.116:0/3920521587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 msgr2=0x7eff040b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 -- 172.21.15.116:0/3920521587 >> v2:172.21.15.149:3300/0 conn(0x7eff041621b0 msgr2=0x7eff040d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 --2- 172.21.15.116:0/3920521587 >> v2:172.21.15.149:3300/0 conn(0x7eff041621b0 0x7eff040d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.485+0000 7eff01e00640 1 -- 172.21.15.116:0/3920521587 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040d2270 con 0x7eff040b9020 2024-05-14T11:22:01.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff01e00640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff040b9020 0x7eff040ce2c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7eff040b4c00 tx=0x7efef001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3105f826b9b62436 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff01400640 1 -- 172.21.15.116:0/3920521587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00b0070 con 0x7eff040b9020 2024-05-14T11:22:01.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff01400640 1 -- 172.21.15.116:0/3920521587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef00ab600 con 0x7eff040b9020 2024-05-14T11:22:01.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff01400640 1 -- 172.21.15.116:0/3920521587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00ab9d0 con 0x7eff040b9020 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 -- 172.21.15.116:0/3920521587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff040b9020 msgr2=0x7eff040ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff040b9020 0x7eff040ce2c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7eff040b4c00 tx=0x7efef001c520 comp rx=0 tx=0).stop 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 -- 172.21.15.116:0/3920521587 shutdown_connections 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff040b9020 0x7eff040ce2c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 --2- 172.21.15.116:0/3920521587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 --2- 172.21.15.116:0/3920521587 >> v2:172.21.15.149:3300/0 conn(0x7eff041621b0 0x7eff040d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 -- 172.21.15.116:0/3920521587 >> 172.21.15.116:0/3920521587 conn(0x7eff040d0770 msgr2=0x7eff040d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 -- 172.21.15.116:0/3920521587 shutdown_connections 2024-05-14T11:22:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.489+0000 7eff09400640 1 -- 172.21.15.116:0/3920521587 wait complete. 2024-05-14T11:22:01.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 Processor -- start 2024-05-14T11:22:01.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 -- start start 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 0x7eff0415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415d3e0 con 0x7eff041621b0 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415d260 con 0x7eff040b9020 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff09400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7eff0415d560 con 0x7eff040ce800 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff03200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff01e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff02800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 0x7eff0415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff02800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 0x7eff0415b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:44298/0 (socket says 172.21.15.116:44298) 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff01e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:42882/0 (socket says 172.21.15.116:42882) 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff03200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42536/0 (socket says 172.21.15.116:42536) 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.493+0000 7eff02800640 1 -- 172.21.15.116:0/3269461679 learned_addr learned my addr 172.21.15.116:0/3269461679 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 -- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 msgr2=0x7eff040b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 -- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 msgr2=0x7eff0415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 0x7eff0415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 -- 172.21.15.116:0/3269461679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040cc4c0 con 0x7eff041621b0 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff03200640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efefc00d4d0 tx=0x7efefc00d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc018030 con 0x7eff041621b0 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc004020 con 0x7eff041621b0 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff040cc9f0 con 0x7eff041621b0 2024-05-14T11:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc004210 con 0x7eff041621b0 2024-05-14T11:22:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff040ccfb0 con 0x7eff041621b0 2024-05-14T11:22:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0406a9c0 con 0x7eff041621b0 2024-05-14T11:22:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.497+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efefc01b070 con 0x7eff041621b0 2024-05-14T11:22:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.501+0000 7efefaa00640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 0x7efee4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.501+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7efefc08ec40 con 0x7eff041621b0 2024-05-14T11:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.501+0000 7eff02800640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 0x7efee4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.501+0000 7eff02800640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 0x7efee4071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efef4007440 tx=0x7efef40098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.505+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efefc022020 con 0x7eff041621b0 2024-05-14T11:22:01.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.717+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7eff040cc780 con 0x7eff041621b0 2024-05-14T11:22:01.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.717+0000 7efefaa00640 1 -- 172.21.15.116:0/3269461679 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1100 (secure 0 0 0) 0x7efefc05b2b0 con 0x7eff041621b0 2024-05-14T11:22:01.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:01.718 INFO:teuthology.orchestra.run.smithi116.stdout:{"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":374,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":84,"num_osds":8,"num_up_osds":8,"osd_up_since":1715685359,"num_in_osds":8,"osd_in_since":1715685350,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":91},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":264978432,"bytes_avail":772829134848,"bytes_total":773094113280,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":61,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":6825}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-05-14T11:21:44.444256+0000","services":{}},"progress_events":{}} 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 msgr2=0x7efee4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 0x7efee4071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efef4007440 tx=0x7efef40098d0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 msgr2=0x7eff040b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efefc00d4d0 tx=0x7efefc00d990 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 shutdown_connections 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efee406f400 0x7efee4071910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff041621b0 0x7eff040b7dc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7eff040ce800 0x7eff040b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 --2- 172.21.15.116:0/3269461679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff040b9020 0x7eff0415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 >> 172.21.15.116:0/3269461679 conn(0x7eff040d0770 msgr2=0x7eff0415fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 shutdown_connections 2024-05-14T11:22:01.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.721+0000 7eff09400640 1 -- 172.21.15.116:0/3269461679 wait complete. 2024-05-14T11:22:01.739 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-14T11:22:01.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:22:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.885+0000 7faff9800640 1 Processor -- start 2024-05-14T11:22:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.885+0000 7faff9800640 1 -- start start 2024-05-14T11:22:01.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff9800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7faff406f730 0x7faff406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff9800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faff4072a30 0x7faff4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff9800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff4070080 con 0x7faff40b0770 2024-05-14T11:22:01.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff9800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7faff406db20 con 0x7faff4072a30 2024-05-14T11:22:01.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff9800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7faff406cf10 con 0x7faff406f730 2024-05-14T11:22:01.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faff4072a30 0x7faff4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff2a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7faff40b0770 0x7faff406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37138/0 (socket says 172.21.15.116:37138) 2024-05-14T11:22:01.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff2a00640 1 -- 172.21.15.116:0/3225419220 learned_addr learned my addr 172.21.15.116:0/3225419220 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:01.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7fafebe00640 1 --2- 172.21.15.116:0/3225419220 >> v2:172.21.15.149:3300/0 conn(0x7faff406f730 0x7faff406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff2000640 1 -- 172.21.15.116:0/3225419220 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002692714 0 0) 0x7fafe4001120 con 0x7faff40b0770 2024-05-14T11:22:01.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 -- 172.21.15.116:0/3225419220 >> v2:172.21.15.149:3300/0 conn(0x7faff406f730 msgr2=0x7faff406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:01.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 --2- 172.21.15.116:0/3225419220 >> v2:172.21.15.149:3300/0 conn(0x7faff406f730 0x7faff406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 -- 172.21.15.116:0/3225419220 >> v1:172.21.15.116:6789/0 conn(0x7faff40b0770 legacy=0x7faff406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 -- 172.21.15.116:0/3225419220 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff406d1f0 con 0x7faff4072a30 2024-05-14T11:22:01.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff3400640 1 --2- 172.21.15.116:0/3225419220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff4072e40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faff406f4e0 tx=0x7fafec01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841198972e2db3a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.889+0000 7faff2000640 1 -- 172.21.15.116:0/3225419220 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0b2070 con 0x7faff4072a30 2024-05-14T11:22:01.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff2000640 1 -- 172.21.15.116:0/3225419220 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0a58b0 con 0x7faff4072a30 2024-05-14T11:22:01.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff2000640 1 -- 172.21.15.116:0/3225419220 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0a5c80 con 0x7faff4072a30 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- 172.21.15.116:0/3225419220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 msgr2=0x7faff4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 --2- 172.21.15.116:0/3225419220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff4072e40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faff406f4e0 tx=0x7fafec01c870 comp rx=0 tx=0).stop 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- 172.21.15.116:0/3225419220 shutdown_connections 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 --2- 172.21.15.116:0/3225419220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff4072e40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 --2- 172.21.15.116:0/3225419220 >> v2:172.21.15.149:3300/0 conn(0x7faff406f730 0x7faff406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- 172.21.15.116:0/3225419220 >> 172.21.15.116:0/3225419220 conn(0x7faff40b4620 msgr2=0x7faff40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- 172.21.15.116:0/3225419220 shutdown_connections 2024-05-14T11:22:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- 172.21.15.116:0/3225419220 wait complete. 2024-05-14T11:22:01.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 Processor -- start 2024-05-14T11:22:01.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.893+0000 7faff9800640 1 -- start start 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faff406f730 0x7faff40ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faff40b0770 0x7faff414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faff4073500 con 0x7faff4072a30 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faff4073380 con 0x7faff40b0770 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7faff4073680 con 0x7faff406f730 2024-05-14T11:22:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42556/0 (socket says 172.21.15.116:42556) 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 -- 172.21.15.116:0/3501459680 learned_addr learned my addr 172.21.15.116:0/3501459680 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 -- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faff406f730 msgr2=0x7faff40ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faff406f730 0x7faff40ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 -- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faff40b0770 msgr2=0x7faff414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faff40b0770 0x7faff414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 -- 172.21.15.116:0/3501459680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff40cc230 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafebe00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e870 tx=0x7fafdc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc017070 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc00c940 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff40ccfb0 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc00ccf0 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff40cc4d0 con 0x7faff4072a30 2024-05-14T11:22:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.897+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff406a9c0 con 0x7faff4072a30 2024-05-14T11:22:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.901+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fafdc01a070 con 0x7faff4072a30 2024-05-14T11:22:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.901+0000 7fafeaa00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 0x7fafd0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.901+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fafdc08f320 con 0x7faff4072a30 2024-05-14T11:22:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.901+0000 7faff2a00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 0x7fafd0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.901+0000 7faff2a00640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 0x7fafd0071940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafe4005890 tx=0x7fafe4007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:01.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:01.905+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fafdc05c080 con 0x7faff4072a30 2024-05-14T11:22:02.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.061+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faff406f510 con 0x7fafd006f430 2024-05-14T11:22:02.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7fafeaa00640 1 -- 172.21.15.116:0/3501459680 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251568 (secure 0 0 0) 0x7faff406f510 con 0x7fafd006f430 2024-05-14T11:22:02.070 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:02.071 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 msgr2=0x7fafd0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 0x7fafd0071940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafe4005890 tx=0x7fafe4007a10 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 msgr2=0x7faff415f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00e870 tx=0x7fafdc00ec70 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 shutdown_connections 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafd006f430 0x7fafd0071940 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faff40b0770 0x7faff414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faff4072a30 0x7faff415f300 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 --2- 172.21.15.116:0/3501459680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faff406f730 0x7faff40ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.069+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 >> 172.21.15.116:0/3501459680 conn(0x7faff40b4620 msgr2=0x7faff40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.073+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 shutdown_connections 2024-05-14T11:22:02.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:02.073+0000 7faff9800640 1 -- 172.21.15.116:0/3501459680 wait complete. 2024-05-14T11:22:02.091 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-14T11:22:00.447107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258768,"kb_used_data":9284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715952,"statfs":{"total":773094113280,"available":772829134848,"internally_reserved":0,"allocated":9506816,"data_stored":4956142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403074+0000","last_change":"2024-05-14T11:21:56.378698+0000","last_active":"2024-05-14T11:21:57.403074+0000","last_peered":"2024-05-14T11:21:57.403074+0000","last_clean":"2024-05-14T11:21:57.403074+0000","last_became_active":"2024-05-14T11:21:56.378402+0000","last_became_peered":"2024-05-14T11:21:56.378402+0000","last_unstale":"2024-05-14T11:21:57.403074+0000","last_undegraded":"2024-05-14T11:21:57.403074+0000","last_fullsized":"2024-05-14T11:21:57.403074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369688+0000","last_change":"2024-05-14T11:21:55.369688+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369688+0000","last_undegraded":"2024-05-14T11:21:55.369688+0000","last_fullsized":"2024-05-14T11:21:55.369688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402927+0000","last_change":"2024-05-14T11:21:56.392559+0000","last_active":"2024-05-14T11:21:57.402927+0000","last_peered":"2024-05-14T11:21:57.402927+0000","last_clean":"2024-05-14T11:21:57.402927+0000","last_became_active":"2024-05-14T11:21:56.391449+0000","last_became_peered":"2024-05-14T11:21:56.391449+0000","last_unstale":"2024-05-14T11:21:57.402927+0000","last_undegraded":"2024-05-14T11:21:57.402927+0000","last_fullsized":"2024-05-14T11:21:57.402927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459732+0000","last_change":"2024-05-14T11:21:56.378008+0000","last_active":"2024-05-14T11:21:57.459732+0000","last_peered":"2024-05-14T11:21:57.459732+0000","last_clean":"2024-05-14T11:21:57.459732+0000","last_became_active":"2024-05-14T11:21:56.377707+0000","last_became_peered":"2024-05-14T11:21:56.377707+0000","last_unstale":"2024-05-14T11:21:57.459732+0000","last_undegraded":"2024-05-14T11:21:57.459732+0000","last_fullsized":"2024-05-14T11:21:57.459732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461077+0000","last_change":"2024-05-14T11:21:56.391093+0000","last_active":"2024-05-14T11:21:57.461077+0000","last_peered":"2024-05-14T11:21:57.461077+0000","last_clean":"2024-05-14T11:21:57.461077+0000","last_became_active":"2024-05-14T11:21:56.390712+0000","last_became_peered":"2024-05-14T11:21:56.390712+0000","last_unstale":"2024-05-14T11:21:57.461077+0000","last_undegraded":"2024-05-14T11:21:57.461077+0000","last_fullsized":"2024-05-14T11:21:57.461077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460609+0000","last_change":"2024-05-14T11:21:56.398553+0000","last_active":"2024-05-14T11:21:57.460609+0000","last_peered":"2024-05-14T11:21:57.460609+0000","last_clean":"2024-05-14T11:21:57.460609+0000","last_became_active":"2024-05-14T11:21:56.396664+0000","last_became_peered":"2024-05-14T11:21:56.396664+0000","last_unstale":"2024-05-14T11:21:57.460609+0000","last_undegraded":"2024-05-14T11:21:57.460609+0000","last_fullsized":"2024-05-14T11:21:57.460609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:59:19.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386491+0000","last_change":"2024-05-14T11:21:56.385275+0000","last_active":"2024-05-14T11:21:57.386491+0000","last_peered":"2024-05-14T11:21:57.386491+0000","last_clean":"2024-05-14T11:21:57.386491+0000","last_became_active":"2024-05-14T11:21:56.384904+0000","last_became_peered":"2024-05-14T11:21:56.384904+0000","last_unstale":"2024-05-14T11:21:57.386491+0000","last_undegraded":"2024-05-14T11:21:57.386491+0000","last_fullsized":"2024-05-14T11:21:57.386491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:36.820614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459460+0000","last_change":"2024-05-14T11:21:56.382681+0000","last_active":"2024-05-14T11:21:57.459460+0000","last_peered":"2024-05-14T11:21:57.459460+0000","last_clean":"2024-05-14T11:21:57.459460+0000","last_became_active":"2024-05-14T11:21:56.381079+0000","last_became_peered":"2024-05-14T11:21:56.381079+0000","last_unstale":"2024-05-14T11:21:57.459460+0000","last_undegraded":"2024-05-14T11:21:57.459460+0000","last_fullsized":"2024-05-14T11:21:57.459460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.394964+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.393782+0000","last_became_peered":"2024-05-14T11:21:56.393782+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380908+0000","last_change":"2024-05-14T11:21:56.384798+0000","last_active":"2024-05-14T11:21:57.380908+0000","last_peered":"2024-05-14T11:21:57.380908+0000","last_clean":"2024-05-14T11:21:57.380908+0000","last_became_active":"2024-05-14T11:21:56.384447+0000","last_became_peered":"2024-05-14T11:21:56.384447+0000","last_unstale":"2024-05-14T11:21:57.380908+0000","last_undegraded":"2024-05-14T11:21:57.380908+0000","last_fullsized":"2024-05-14T11:21:57.380908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:12.540726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.391270+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.390667+0000","last_became_peered":"2024-05-14T11:21:56.390667+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380632+0000","last_change":"2024-05-14T11:21:56.379345+0000","last_active":"2024-05-14T11:21:57.380632+0000","last_peered":"2024-05-14T11:21:57.380632+0000","last_clean":"2024-05-14T11:21:57.380632+0000","last_became_active":"2024-05-14T11:21:56.379007+0000","last_became_peered":"2024-05-14T11:21:56.379007+0000","last_unstale":"2024-05-14T11:21:57.380632+0000","last_undegraded":"2024-05-14T11:21:57.380632+0000","last_fullsized":"2024-05-14T11:21:57.380632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:31.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459974+0000","last_change":"2024-05-14T11:21:56.384668+0000","last_active":"2024-05-14T11:21:57.459974+0000","last_peered":"2024-05-14T11:21:57.459974+0000","last_clean":"2024-05-14T11:21:57.459974+0000","last_became_active":"2024-05-14T11:21:56.382366+0000","last_became_peered":"2024-05-14T11:21:56.382366+0000","last_unstale":"2024-05-14T11:21:57.459974+0000","last_undegraded":"2024-05-14T11:21:57.459974+0000","last_fullsized":"2024-05-14T11:21:57.459974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460403+0000","last_change":"2024-05-14T11:21:56.389629+0000","last_active":"2024-05-14T11:21:57.460403+0000","last_peered":"2024-05-14T11:21:57.460403+0000","last_clean":"2024-05-14T11:21:57.460403+0000","last_became_active":"2024-05-14T11:21:56.389299+0000","last_became_peered":"2024-05-14T11:21:56.389299+0000","last_unstale":"2024-05-14T11:21:57.460403+0000","last_undegraded":"2024-05-14T11:21:57.460403+0000","last_fullsized":"2024-05-14T11:21:57.460403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:00:13.582984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459513+0000","last_change":"2024-05-14T11:21:56.387482+0000","last_active":"2024-05-14T11:21:57.459513+0000","last_peered":"2024-05-14T11:21:57.459513+0000","last_clean":"2024-05-14T11:21:57.459513+0000","last_became_active":"2024-05-14T11:21:56.385978+0000","last_became_peered":"2024-05-14T11:21:56.385978+0000","last_unstale":"2024-05-14T11:21:57.459513+0000","last_undegraded":"2024-05-14T11:21:57.459513+0000","last_fullsized":"2024-05-14T11:21:57.459513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460528+0000","last_change":"2024-05-14T11:21:56.386948+0000","last_active":"2024-05-14T11:21:57.460528+0000","last_peered":"2024-05-14T11:21:57.460528+0000","last_clean":"2024-05-14T11:21:57.460528+0000","last_became_active":"2024-05-14T11:21:56.385310+0000","last_became_peered":"2024-05-14T11:21:56.385310+0000","last_unstale":"2024-05-14T11:21:57.460528+0000","last_undegraded":"2024-05-14T11:21:57.460528+0000","last_fullsized":"2024-05-14T11:21:57.460528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:31:18.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461165+0000","last_change":"2024-05-14T11:21:56.400922+0000","last_active":"2024-05-14T11:21:57.461165+0000","last_peered":"2024-05-14T11:21:57.461165+0000","last_clean":"2024-05-14T11:21:57.461165+0000","last_became_active":"2024-05-14T11:21:56.400597+0000","last_became_peered":"2024-05-14T11:21:56.400597+0000","last_unstale":"2024-05-14T11:21:57.461165+0000","last_undegraded":"2024-05-14T11:21:57.461165+0000","last_fullsized":"2024-05-14T11:21:57.461165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:07.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403099+0000","last_change":"2024-05-14T11:21:56.397705+0000","last_active":"2024-05-14T11:21:57.403099+0000","last_peered":"2024-05-14T11:21:57.403099+0000","last_clean":"2024-05-14T11:21:57.403099+0000","last_became_active":"2024-05-14T11:21:56.397144+0000","last_became_peered":"2024-05-14T11:21:56.397144+0000","last_unstale":"2024-05-14T11:21:57.403099+0000","last_undegraded":"2024-05-14T11:21:57.403099+0000","last_fullsized":"2024-05-14T11:21:57.403099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461475+0000","last_change":"2024-05-14T11:21:56.390962+0000","last_active":"2024-05-14T11:21:57.461475+0000","last_peered":"2024-05-14T11:21:57.461475+0000","last_clean":"2024-05-14T11:21:57.461475+0000","last_became_active":"2024-05-14T11:21:56.390072+0000","last_became_peered":"2024-05-14T11:21:56.390072+0000","last_unstale":"2024-05-14T11:21:57.461475+0000","last_undegraded":"2024-05-14T11:21:57.461475+0000","last_fullsized":"2024-05-14T11:21:57.461475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459937+0000","last_change":"2024-05-14T11:21:56.379798+0000","last_active":"2024-05-14T11:21:57.459937+0000","last_peered":"2024-05-14T11:21:57.459937+0000","last_clean":"2024-05-14T11:21:57.459937+0000","last_became_active":"2024-05-14T11:21:56.378722+0000","last_became_peered":"2024-05-14T11:21:56.378722+0000","last_unstale":"2024-05-14T11:21:57.459937+0000","last_undegraded":"2024-05-14T11:21:57.459937+0000","last_fullsized":"2024-05-14T11:21:57.459937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460813+0000","last_change":"2024-05-14T11:21:56.391852+0000","last_active":"2024-05-14T11:21:57.460813+0000","last_peered":"2024-05-14T11:21:57.460813+0000","last_clean":"2024-05-14T11:21:57.460813+0000","last_became_active":"2024-05-14T11:21:56.391495+0000","last_became_peered":"2024-05-14T11:21:56.391495+0000","last_unstale":"2024-05-14T11:21:57.460813+0000","last_undegraded":"2024-05-14T11:21:57.460813+0000","last_fullsized":"2024-05-14T11:21:57.460813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460442+0000","last_change":"2024-05-14T11:21:56.390530+0000","last_active":"2024-05-14T11:21:57.460442+0000","last_peered":"2024-05-14T11:21:57.460442+0000","last_clean":"2024-05-14T11:21:57.460442+0000","last_became_active":"2024-05-14T11:21:56.390215+0000","last_became_peered":"2024-05-14T11:21:56.390215+0000","last_unstale":"2024-05-14T11:21:57.460442+0000","last_undegraded":"2024-05-14T11:21:57.460442+0000","last_fullsized":"2024-05-14T11:21:57.460442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:07.315380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387804+0000","last_change":"2024-05-14T11:21:56.391315+0000","last_active":"2024-05-14T11:21:57.387804+0000","last_peered":"2024-05-14T11:21:57.387804+0000","last_clean":"2024-05-14T11:21:57.387804+0000","last_became_active":"2024-05-14T11:21:56.390649+0000","last_became_peered":"2024-05-14T11:21:56.390649+0000","last_unstale":"2024-05-14T11:21:57.387804+0000","last_undegraded":"2024-05-14T11:21:57.387804+0000","last_fullsized":"2024-05-14T11:21:57.387804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402786+0000","last_change":"2024-05-14T11:21:56.394765+0000","last_active":"2024-05-14T11:21:57.402786+0000","last_peered":"2024-05-14T11:21:57.402786+0000","last_clean":"2024-05-14T11:21:57.402786+0000","last_became_active":"2024-05-14T11:21:56.394490+0000","last_became_peered":"2024-05-14T11:21:56.394490+0000","last_unstale":"2024-05-14T11:21:57.402786+0000","last_undegraded":"2024-05-14T11:21:57.402786+0000","last_fullsized":"2024-05-14T11:21:57.402786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.372405+0000","last_change":"2024-05-14T11:21:55.372405+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.372405+0000","last_undegraded":"2024-05-14T11:21:55.372405+0000","last_fullsized":"2024-05-14T11:21:55.372405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461831+0000","last_change":"2024-05-14T11:21:56.389580+0000","last_active":"2024-05-14T11:21:57.461831+0000","last_peered":"2024-05-14T11:21:57.461831+0000","last_clean":"2024-05-14T11:21:57.461831+0000","last_became_active":"2024-05-14T11:21:56.389252+0000","last_became_peered":"2024-05-14T11:21:56.389252+0000","last_unstale":"2024-05-14T11:21:57.461831+0000","last_undegraded":"2024-05-14T11:21:57.461831+0000","last_fullsized":"2024-05-14T11:21:57.461831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.381007+0000","last_change":"2024-05-14T11:21:56.378063+0000","last_active":"2024-05-14T11:21:57.381007+0000","last_peered":"2024-05-14T11:21:57.381007+0000","last_clean":"2024-05-14T11:21:57.381007+0000","last_became_active":"2024-05-14T11:21:56.377759+0000","last_became_peered":"2024-05-14T11:21:56.377759+0000","last_unstale":"2024-05-14T11:21:57.381007+0000","last_undegraded":"2024-05-14T11:21:57.381007+0000","last_fullsized":"2024-05-14T11:21:57.381007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:43.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460090+0000","last_change":"2024-05-14T11:21:56.398675+0000","last_active":"2024-05-14T11:21:57.460090+0000","last_peered":"2024-05-14T11:21:57.460090+0000","last_clean":"2024-05-14T11:21:57.460090+0000","last_became_active":"2024-05-14T11:21:56.392629+0000","last_became_peered":"2024-05-14T11:21:56.392629+0000","last_unstale":"2024-05-14T11:21:57.460090+0000","last_undegraded":"2024-05-14T11:21:57.460090+0000","last_fullsized":"2024-05-14T11:21:57.460090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386641+0000","last_change":"2024-05-14T11:21:56.371575+0000","last_active":"2024-05-14T11:21:57.386641+0000","last_peered":"2024-05-14T11:21:57.386641+0000","last_clean":"2024-05-14T11:21:57.386641+0000","last_became_active":"2024-05-14T11:21:56.371212+0000","last_became_peered":"2024-05-14T11:21:56.371212+0000","last_unstale":"2024-05-14T11:21:57.386641+0000","last_undegraded":"2024-05-14T11:21:57.386641+0000","last_fullsized":"2024-05-14T11:21:57.386641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:49:03.645090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460626+0000","last_change":"2024-05-14T11:21:56.394494+0000","last_active":"2024-05-14T11:21:57.460626+0000","last_peered":"2024-05-14T11:21:57.460626+0000","last_clean":"2024-05-14T11:21:57.460626+0000","last_became_active":"2024-05-14T11:21:56.394205+0000","last_became_peered":"2024-05-14T11:21:56.394205+0000","last_unstale":"2024-05-14T11:21:57.460626+0000","last_undegraded":"2024-05-14T11:21:57.460626+0000","last_fullsized":"2024-05-14T11:21:57.460626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370175+0000","last_change":"2024-05-14T11:21:55.370175+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370175+0000","last_undegraded":"2024-05-14T11:21:55.370175+0000","last_fullsized":"2024-05-14T11:21:55.370175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460844+0000","last_change":"2024-05-14T11:21:56.382783+0000","last_active":"2024-05-14T11:21:57.460844+0000","last_peered":"2024-05-14T11:21:57.460844+0000","last_clean":"2024-05-14T11:21:57.460844+0000","last_became_active":"2024-05-14T11:21:56.382496+0000","last_became_peered":"2024-05-14T11:21:56.382496+0000","last_unstale":"2024-05-14T11:21:57.460844+0000","last_undegraded":"2024-05-14T11:21:57.460844+0000","last_fullsized":"2024-05-14T11:21:57.460844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:44.636667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461113+0000","last_change":"2024-05-14T11:21:56.399015+0000","last_active":"2024-05-14T11:21:57.461113+0000","last_peered":"2024-05-14T11:21:57.461113+0000","last_clean":"2024-05-14T11:21:57.461113+0000","last_became_active":"2024-05-14T11:21:56.397801+0000","last_became_peered":"2024-05-14T11:21:56.397801+0000","last_unstale":"2024-05-14T11:21:57.461113+0000","last_undegraded":"2024-05-14T11:21:57.461113+0000","last_fullsized":"2024-05-14T11:21:57.461113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:03.545193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387341+0000","last_change":"2024-05-14T11:21:55.377300+0000","last_active":"2024-05-14T11:21:57.387341+0000","last_peered":"2024-05-14T11:21:57.387341+0000","last_clean":"2024-05-14T11:21:57.387341+0000","last_became_active":"2024-05-14T11:21:55.376999+0000","last_became_peered":"2024-05-14T11:21:55.376999+0000","last_unstale":"2024-05-14T11:21:57.387341+0000","last_undegraded":"2024-05-14T11:21:57.387341+0000","last_fullsized":"2024-05-14T11:21:57.387341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:14.148384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460907+0000","last_change":"2024-05-14T11:21:56.381804+0000","last_active":"2024-05-14T11:21:57.460907+0000","last_peered":"2024-05-14T11:21:57.460907+0000","last_clean":"2024-05-14T11:21:57.460907+0000","last_became_active":"2024-05-14T11:21:56.381554+0000","last_became_peered":"2024-05-14T11:21:56.381554+0000","last_unstale":"2024-05-14T11:21:57.460907+0000","last_undegraded":"2024-05-14T11:21:57.460907+0000","last_fullsized":"2024-05-14T11:21:57.460907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:37:23.266696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461538+0000","last_change":"2024-05-14T11:21:55.373916+0000","last_active":"2024-05-14T11:21:57.461538+0000","last_peered":"2024-05-14T11:21:57.461538+0000","last_clean":"2024-05-14T11:21:57.461538+0000","last_became_active":"2024-05-14T11:21:55.373461+0000","last_became_peered":"2024-05-14T11:21:55.373461+0000","last_unstale":"2024-05-14T11:21:57.461538+0000","last_undegraded":"2024-05-14T11:21:57.461538+0000","last_fullsized":"2024-05-14T11:21:57.461538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:01.508933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386940+0000","last_change":"2024-05-14T11:21:56.385784+0000","last_active":"2024-05-14T11:21:57.386940+0000","last_peered":"2024-05-14T11:21:57.386940+0000","last_clean":"2024-05-14T11:21:57.386940+0000","last_became_active":"2024-05-14T11:21:56.385460+0000","last_became_peered":"2024-05-14T11:21:56.385460+0000","last_unstale":"2024-05-14T11:21:57.386940+0000","last_undegraded":"2024-05-14T11:21:57.386940+0000","last_fullsized":"2024-05-14T11:21:57.386940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:02:04.422136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387331+0000","last_change":"2024-05-14T11:21:55.384239+0000","last_active":"2024-05-14T11:21:57.387331+0000","last_peered":"2024-05-14T11:21:57.387331+0000","last_clean":"2024-05-14T11:21:57.387331+0000","last_became_active":"2024-05-14T11:21:55.383906+0000","last_became_peered":"2024-05-14T11:21:55.383906+0000","last_unstale":"2024-05-14T11:21:57.387331+0000","last_undegraded":"2024-05-14T11:21:57.387331+0000","last_fullsized":"2024-05-14T11:21:57.387331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:11.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461213+0000","last_change":"2024-05-14T11:21:56.398729+0000","last_active":"2024-05-14T11:21:57.461213+0000","last_peered":"2024-05-14T11:21:57.461213+0000","last_clean":"2024-05-14T11:21:57.461213+0000","last_became_active":"2024-05-14T11:21:56.397051+0000","last_became_peered":"2024-05-14T11:21:56.397051+0000","last_unstale":"2024-05-14T11:21:57.461213+0000","last_undegraded":"2024-05-14T11:21:57.461213+0000","last_fullsized":"2024-05-14T11:21:57.461213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:22:03.622138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387307+0000","last_change":"2024-05-14T11:21:55.386498+0000","last_active":"2024-05-14T11:21:57.387307+0000","last_peered":"2024-05-14T11:21:57.387307+0000","last_clean":"2024-05-14T11:21:57.387307+0000","last_became_active":"2024-05-14T11:21:55.385760+0000","last_became_peered":"2024-05-14T11:21:55.385760+0000","last_unstale":"2024-05-14T11:21:57.387307+0000","last_undegraded":"2024-05-14T11:21:57.387307+0000","last_fullsized":"2024-05-14T11:21:57.387307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.837126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459652+0000","last_change":"2024-05-14T11:21:56.381076+0000","last_active":"2024-05-14T11:21:57.459652+0000","last_peered":"2024-05-14T11:21:57.459652+0000","last_clean":"2024-05-14T11:21:57.459652+0000","last_became_active":"2024-05-14T11:21:56.380328+0000","last_became_peered":"2024-05-14T11:21:56.380328+0000","last_unstale":"2024-05-14T11:21:57.459652+0000","last_undegraded":"2024-05-14T11:21:57.459652+0000","last_fullsized":"2024-05-14T11:21:57.459652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461212+0000","last_change":"2024-05-14T11:21:55.383653+0000","last_active":"2024-05-14T11:21:57.461212+0000","last_peered":"2024-05-14T11:21:57.461212+0000","last_clean":"2024-05-14T11:21:57.461212+0000","last_became_active":"2024-05-14T11:21:55.383294+0000","last_became_peered":"2024-05-14T11:21:55.383294+0000","last_unstale":"2024-05-14T11:21:57.461212+0000","last_undegraded":"2024-05-14T11:21:57.461212+0000","last_fullsized":"2024-05-14T11:21:57.461212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:14.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386959+0000","last_change":"2024-05-14T11:21:56.378909+0000","last_active":"2024-05-14T11:21:57.386959+0000","last_peered":"2024-05-14T11:21:57.386959+0000","last_clean":"2024-05-14T11:21:57.386959+0000","last_became_active":"2024-05-14T11:21:56.378614+0000","last_became_peered":"2024-05-14T11:21:56.378614+0000","last_unstale":"2024-05-14T11:21:57.386959+0000","last_undegraded":"2024-05-14T11:21:57.386959+0000","last_fullsized":"2024-05-14T11:21:57.386959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460014+0000","last_change":"2024-05-14T11:21:55.387444+0000","last_active":"2024-05-14T11:21:57.460014+0000","last_peered":"2024-05-14T11:21:57.460014+0000","last_clean":"2024-05-14T11:21:57.460014+0000","last_became_active":"2024-05-14T11:21:55.386992+0000","last_became_peered":"2024-05-14T11:21:55.386992+0000","last_unstale":"2024-05-14T11:21:57.460014+0000","last_undegraded":"2024-05-14T11:21:57.460014+0000","last_fullsized":"2024-05-14T11:21:57.460014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:34.670851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460786+0000","last_change":"2024-05-14T11:21:56.389934+0000","last_active":"2024-05-14T11:21:57.460786+0000","last_peered":"2024-05-14T11:21:57.460786+0000","last_clean":"2024-05-14T11:21:57.460786+0000","last_became_active":"2024-05-14T11:21:56.388204+0000","last_became_peered":"2024-05-14T11:21:56.388204+0000","last_unstale":"2024-05-14T11:21:57.460786+0000","last_undegraded":"2024-05-14T11:21:57.460786+0000","last_fullsized":"2024-05-14T11:21:57.460786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:42.638658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402951+0000","last_change":"2024-05-14T11:21:55.379234+0000","last_active":"2024-05-14T11:21:57.402951+0000","last_peered":"2024-05-14T11:21:57.402951+0000","last_clean":"2024-05-14T11:21:57.402951+0000","last_became_active":"2024-05-14T11:21:55.378924+0000","last_became_peered":"2024-05-14T11:21:55.378924+0000","last_unstale":"2024-05-14T11:21:57.402951+0000","last_undegraded":"2024-05-14T11:21:57.402951+0000","last_fullsized":"2024-05-14T11:21:57.402951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460137+0000","last_change":"2024-05-14T11:21:56.383194+0000","last_active":"2024-05-14T11:21:57.460137+0000","last_peered":"2024-05-14T11:21:57.460137+0000","last_clean":"2024-05-14T11:21:57.460137+0000","last_became_active":"2024-05-14T11:21:56.379071+0000","last_became_peered":"2024-05-14T11:21:56.379071+0000","last_unstale":"2024-05-14T11:21:57.460137+0000","last_undegraded":"2024-05-14T11:21:57.460137+0000","last_fullsized":"2024-05-14T11:21:57.460137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386225+0000","last_change":"2024-05-14T11:21:55.384145+0000","last_active":"2024-05-14T11:21:57.386225+0000","last_peered":"2024-05-14T11:21:57.386225+0000","last_clean":"2024-05-14T11:21:57.386225+0000","last_became_active":"2024-05-14T11:21:55.383834+0000","last_became_peered":"2024-05-14T11:21:55.383834+0000","last_unstale":"2024-05-14T11:21:57.386225+0000","last_undegraded":"2024-05-14T11:21:57.386225+0000","last_fullsized":"2024-05-14T11:21:57.386225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:31.347753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461043+0000","last_change":"2024-05-14T11:21:56.396531+0000","last_active":"2024-05-14T11:21:57.461043+0000","last_peered":"2024-05-14T11:21:57.461043+0000","last_clean":"2024-05-14T11:21:57.461043+0000","last_became_active":"2024-05-14T11:21:56.396223+0000","last_became_peered":"2024-05-14T11:21:56.396223+0000","last_unstale":"2024-05-14T11:21:57.461043+0000","last_undegraded":"2024-05-14T11:21:57.461043+0000","last_fullsized":"2024-05-14T11:21:57.461043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.385972+0000","last_change":"2024-05-14T11:21:55.379424+0000","last_active":"2024-05-14T11:21:55.385972+0000","last_peered":"2024-05-14T11:21:55.385972+0000","last_clean":"2024-05-14T11:21:55.385972+0000","last_became_active":"2024-05-14T11:21:55.378552+0000","last_became_peered":"2024-05-14T11:21:55.378552+0000","last_unstale":"2024-05-14T11:21:55.385972+0000","last_undegraded":"2024-05-14T11:21:55.385972+0000","last_fullsized":"2024-05-14T11:21:55.385972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:39.156471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387395+0000","last_change":"2024-05-14T11:21:56.392129+0000","last_active":"2024-05-14T11:21:57.387395+0000","last_peered":"2024-05-14T11:21:57.387395+0000","last_clean":"2024-05-14T11:21:57.387395+0000","last_became_active":"2024-05-14T11:21:56.391814+0000","last_became_peered":"2024-05-14T11:21:56.391814+0000","last_unstale":"2024-05-14T11:21:57.387395+0000","last_undegraded":"2024-05-14T11:21:57.387395+0000","last_fullsized":"2024-05-14T11:21:57.387395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:20.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403200+0000","last_change":"2024-05-14T11:21:55.388521+0000","last_active":"2024-05-14T11:21:57.403200+0000","last_peered":"2024-05-14T11:21:57.403200+0000","last_clean":"2024-05-14T11:21:57.403200+0000","last_became_active":"2024-05-14T11:21:55.387310+0000","last_became_peered":"2024-05-14T11:21:55.387310+0000","last_unstale":"2024-05-14T11:21:57.403200+0000","last_undegraded":"2024-05-14T11:21:57.403200+0000","last_fullsized":"2024-05-14T11:21:57.403200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403125+0000","last_change":"2024-05-14T11:21:56.379479+0000","last_active":"2024-05-14T11:21:57.403125+0000","last_peered":"2024-05-14T11:21:57.403125+0000","last_clean":"2024-05-14T11:21:57.403125+0000","last_became_active":"2024-05-14T11:21:56.379170+0000","last_became_peered":"2024-05-14T11:21:56.379170+0000","last_unstale":"2024-05-14T11:21:57.403125+0000","last_undegraded":"2024-05-14T11:21:57.403125+0000","last_fullsized":"2024-05-14T11:21:57.403125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:45.902619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461559+0000","last_change":"2024-05-14T11:21:55.379532+0000","last_active":"2024-05-14T11:21:57.461559+0000","last_peered":"2024-05-14T11:21:57.461559+0000","last_clean":"2024-05-14T11:21:57.461559+0000","last_became_active":"2024-05-14T11:21:55.379224+0000","last_became_peered":"2024-05-14T11:21:55.379224+0000","last_unstale":"2024-05-14T11:21:57.461559+0000","last_undegraded":"2024-05-14T11:21:57.461559+0000","last_fullsized":"2024-05-14T11:21:57.461559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:23:55.106701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459814+0000","last_change":"2024-05-14T11:21:56.400307+0000","last_active":"2024-05-14T11:21:57.459814+0000","last_peered":"2024-05-14T11:21:57.459814+0000","last_clean":"2024-05-14T11:21:57.459814+0000","last_became_active":"2024-05-14T11:21:56.399940+0000","last_became_peered":"2024-05-14T11:21:56.399940+0000","last_unstale":"2024-05-14T11:21:57.459814+0000","last_undegraded":"2024-05-14T11:21:57.459814+0000","last_fullsized":"2024-05-14T11:21:57.459814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461259+0000","last_change":"2024-05-14T11:21:55.378699+0000","last_active":"2024-05-14T11:21:57.461259+0000","last_peered":"2024-05-14T11:21:57.461259+0000","last_clean":"2024-05-14T11:21:57.461259+0000","last_became_active":"2024-05-14T11:21:55.378372+0000","last_became_peered":"2024-05-14T11:21:55.378372+0000","last_unstale":"2024-05-14T11:21:57.461259+0000","last_undegraded":"2024-05-14T11:21:57.461259+0000","last_fullsized":"2024-05-14T11:21:57.461259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:42.121073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459654+0000","last_change":"2024-05-14T11:21:56.380739+0000","last_active":"2024-05-14T11:21:57.459654+0000","last_peered":"2024-05-14T11:21:57.459654+0000","last_clean":"2024-05-14T11:21:57.459654+0000","last_became_active":"2024-05-14T11:21:56.377804+0000","last_became_peered":"2024-05-14T11:21:56.377804+0000","last_unstale":"2024-05-14T11:21:57.459654+0000","last_undegraded":"2024-05-14T11:21:57.459654+0000","last_fullsized":"2024-05-14T11:21:57.459654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386631+0000","last_change":"2024-05-14T11:21:55.380045+0000","last_active":"2024-05-14T11:21:57.386631+0000","last_peered":"2024-05-14T11:21:57.386631+0000","last_clean":"2024-05-14T11:21:57.386631+0000","last_became_active":"2024-05-14T11:21:55.378906+0000","last_became_peered":"2024-05-14T11:21:55.378906+0000","last_unstale":"2024-05-14T11:21:57.386631+0000","last_undegraded":"2024-05-14T11:21:57.386631+0000","last_fullsized":"2024-05-14T11:21:57.386631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:34.974768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460535+0000","last_change":"2024-05-14T11:21:56.388451+0000","last_active":"2024-05-14T11:21:57.460535+0000","last_peered":"2024-05-14T11:21:57.460535+0000","last_clean":"2024-05-14T11:21:57.460535+0000","last_became_active":"2024-05-14T11:21:56.388148+0000","last_became_peered":"2024-05-14T11:21:56.388148+0000","last_unstale":"2024-05-14T11:21:57.460535+0000","last_undegraded":"2024-05-14T11:21:57.460535+0000","last_fullsized":"2024-05-14T11:21:57.460535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460827+0000","last_change":"2024-05-14T11:21:55.379659+0000","last_active":"2024-05-14T11:21:57.460827+0000","last_peered":"2024-05-14T11:21:57.460827+0000","last_clean":"2024-05-14T11:21:57.460827+0000","last_became_active":"2024-05-14T11:21:55.379357+0000","last_became_peered":"2024-05-14T11:21:55.379357+0000","last_unstale":"2024-05-14T11:21:57.460827+0000","last_undegraded":"2024-05-14T11:21:57.460827+0000","last_fullsized":"2024-05-14T11:21:57.460827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:54.423695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380729+0000","last_change":"2024-05-14T11:21:56.390097+0000","last_active":"2024-05-14T11:21:57.380729+0000","last_peered":"2024-05-14T11:21:57.380729+0000","last_clean":"2024-05-14T11:21:57.380729+0000","last_became_active":"2024-05-14T11:21:56.389775+0000","last_became_peered":"2024-05-14T11:21:56.389775+0000","last_unstale":"2024-05-14T11:21:57.380729+0000","last_undegraded":"2024-05-14T11:21:57.380729+0000","last_fullsized":"2024-05-14T11:21:57.380729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:24.671464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459571+0000","last_change":"2024-05-14T11:21:55.384436+0000","last_active":"2024-05-14T11:21:57.459571+0000","last_peered":"2024-05-14T11:21:57.459571+0000","last_clean":"2024-05-14T11:21:57.459571+0000","last_became_active":"2024-05-14T11:21:55.384104+0000","last_became_peered":"2024-05-14T11:21:55.384104+0000","last_unstale":"2024-05-14T11:21:57.459571+0000","last_undegraded":"2024-05-14T11:21:57.459571+0000","last_fullsized":"2024-05-14T11:21:57.459571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:56.800584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380862+0000","last_change":"2024-05-14T11:21:56.393507+0000","last_active":"2024-05-14T11:21:57.380862+0000","last_peered":"2024-05-14T11:21:57.380862+0000","last_clean":"2024-05-14T11:21:57.380862+0000","last_became_active":"2024-05-14T11:21:56.393227+0000","last_became_peered":"2024-05-14T11:21:56.393227+0000","last_unstale":"2024-05-14T11:21:57.380862+0000","last_undegraded":"2024-05-14T11:21:57.380862+0000","last_fullsized":"2024-05-14T11:21:57.380862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:26.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459792+0000","last_change":"2024-05-14T11:21:55.386065+0000","last_active":"2024-05-14T11:21:57.459792+0000","last_peered":"2024-05-14T11:21:57.459792+0000","last_clean":"2024-05-14T11:21:57.459792+0000","last_became_active":"2024-05-14T11:21:55.385699+0000","last_became_peered":"2024-05-14T11:21:55.385699+0000","last_unstale":"2024-05-14T11:21:57.459792+0000","last_undegraded":"2024-05-14T11:21:57.459792+0000","last_fullsized":"2024-05-14T11:21:57.459792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:34:21.465908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461286+0000","last_change":"2024-05-14T11:21:56.396244+0000","last_active":"2024-05-14T11:21:57.461286+0000","last_peered":"2024-05-14T11:21:57.461286+0000","last_clean":"2024-05-14T11:21:57.461286+0000","last_became_active":"2024-05-14T11:21:56.395934+0000","last_became_peered":"2024-05-14T11:21:56.395934+0000","last_unstale":"2024-05-14T11:21:57.461286+0000","last_undegraded":"2024-05-14T11:21:57.461286+0000","last_fullsized":"2024-05-14T11:21:57.461286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:44.571245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387256+0000","last_change":"2024-05-14T11:21:55.376695+0000","last_active":"2024-05-14T11:21:57.387256+0000","last_peered":"2024-05-14T11:21:57.387256+0000","last_clean":"2024-05-14T11:21:57.387256+0000","last_became_active":"2024-05-14T11:21:55.375674+0000","last_became_peered":"2024-05-14T11:21:55.375674+0000","last_unstale":"2024-05-14T11:21:57.387256+0000","last_undegraded":"2024-05-14T11:21:57.387256+0000","last_fullsized":"2024-05-14T11:21:57.387256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:57.676773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386997+0000","last_change":"2024-05-14T11:21:56.394005+0000","last_active":"2024-05-14T11:21:57.386997+0000","last_peered":"2024-05-14T11:21:57.386997+0000","last_clean":"2024-05-14T11:21:57.386997+0000","last_became_active":"2024-05-14T11:21:56.393708+0000","last_became_peered":"2024-05-14T11:21:56.393708+0000","last_unstale":"2024-05-14T11:21:57.386997+0000","last_undegraded":"2024-05-14T11:21:57.386997+0000","last_fullsized":"2024-05-14T11:21:57.386997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387203+0000","last_change":"2024-05-14T11:21:55.378651+0000","last_active":"2024-05-14T11:21:57.387203+0000","last_peered":"2024-05-14T11:21:57.387203+0000","last_clean":"2024-05-14T11:21:57.387203+0000","last_became_active":"2024-05-14T11:21:55.378348+0000","last_became_peered":"2024-05-14T11:21:55.378348+0000","last_unstale":"2024-05-14T11:21:57.387203+0000","last_undegraded":"2024-05-14T11:21:57.387203+0000","last_fullsized":"2024-05-14T11:21:57.387203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:13.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461456+0000","last_change":"2024-05-14T11:21:56.389463+0000","last_active":"2024-05-14T11:21:57.461456+0000","last_peered":"2024-05-14T11:21:57.461456+0000","last_clean":"2024-05-14T11:21:57.461456+0000","last_became_active":"2024-05-14T11:21:56.389014+0000","last_became_peered":"2024-05-14T11:21:56.389014+0000","last_unstale":"2024-05-14T11:21:57.461456+0000","last_undegraded":"2024-05-14T11:21:57.461456+0000","last_fullsized":"2024-05-14T11:21:57.461456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461621+0000","last_change":"2024-05-14T11:21:55.383442+0000","last_active":"2024-05-14T11:21:57.461621+0000","last_peered":"2024-05-14T11:21:57.461621+0000","last_clean":"2024-05-14T11:21:57.461621+0000","last_became_active":"2024-05-14T11:21:55.383197+0000","last_became_peered":"2024-05-14T11:21:55.383197+0000","last_unstale":"2024-05-14T11:21:57.461621+0000","last_undegraded":"2024-05-14T11:21:57.461621+0000","last_fullsized":"2024-05-14T11:21:57.461621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:52:51.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460829+0000","last_change":"2024-05-14T11:21:56.397428+0000","last_active":"2024-05-14T11:21:57.460829+0000","last_peered":"2024-05-14T11:21:57.460829+0000","last_clean":"2024-05-14T11:21:57.460829+0000","last_became_active":"2024-05-14T11:21:56.397079+0000","last_became_peered":"2024-05-14T11:21:56.397079+0000","last_unstale":"2024-05-14T11:21:57.460829+0000","last_undegraded":"2024-05-14T11:21:57.460829+0000","last_fullsized":"2024-05-14T11:21:57.460829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:05.256399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388807+0000","last_change":"2024-05-14T11:21:55.381263+0000","last_active":"2024-05-14T11:21:55.388807+0000","last_peered":"2024-05-14T11:21:55.388807+0000","last_clean":"2024-05-14T11:21:55.388807+0000","last_became_active":"2024-05-14T11:21:55.380928+0000","last_became_peered":"2024-05-14T11:21:55.380928+0000","last_unstale":"2024-05-14T11:21:55.388807+0000","last_undegraded":"2024-05-14T11:21:55.388807+0000","last_fullsized":"2024-05-14T11:21:55.388807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:43.506348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.369616+0000","last_change":"2024-05-14T11:21:55.369616+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.369616+0000","last_undegraded":"2024-05-14T11:21:55.369616+0000","last_fullsized":"2024-05-14T11:21:55.369616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380740+0000","last_change":"2024-05-14T11:21:55.386139+0000","last_active":"2024-05-14T11:21:57.380740+0000","last_peered":"2024-05-14T11:21:57.380740+0000","last_clean":"2024-05-14T11:21:57.380740+0000","last_became_active":"2024-05-14T11:21:55.385435+0000","last_became_peered":"2024-05-14T11:21:55.385435+0000","last_unstale":"2024-05-14T11:21:57.380740+0000","last_undegraded":"2024-05-14T11:21:57.380740+0000","last_fullsized":"2024-05-14T11:21:57.380740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:00.748983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.370647+0000","last_change":"2024-05-14T11:21:55.370647+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.370647+0000","last_undegraded":"2024-05-14T11:21:55.370647+0000","last_fullsized":"2024-05-14T11:21:55.370647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.381246+0000","last_change":"2024-05-14T11:21:55.377847+0000","last_active":"2024-05-14T11:21:55.381246+0000","last_peered":"2024-05-14T11:21:55.381246+0000","last_clean":"2024-05-14T11:21:55.381246+0000","last_became_active":"2024-05-14T11:21:55.377479+0000","last_became_peered":"2024-05-14T11:21:55.377479+0000","last_unstale":"2024-05-14T11:21:55.381246+0000","last_undegraded":"2024-05-14T11:21:55.381246+0000","last_fullsized":"2024-05-14T11:21:55.381246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:40.925873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460650+0000","last_change":"2024-05-14T11:21:56.381486+0000","last_active":"2024-05-14T11:21:57.460650+0000","last_peered":"2024-05-14T11:21:57.460650+0000","last_clean":"2024-05-14T11:21:57.460650+0000","last_became_active":"2024-05-14T11:21:56.381191+0000","last_became_peered":"2024-05-14T11:21:56.381191+0000","last_unstale":"2024-05-14T11:21:57.460650+0000","last_undegraded":"2024-05-14T11:21:57.460650+0000","last_fullsized":"2024-05-14T11:21:57.460650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:11:28.255356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380491+0000","last_change":"2024-05-14T11:21:55.380084+0000","last_active":"2024-05-14T11:21:57.380491+0000","last_peered":"2024-05-14T11:21:57.380491+0000","last_clean":"2024-05-14T11:21:57.380491+0000","last_became_active":"2024-05-14T11:21:55.379631+0000","last_became_peered":"2024-05-14T11:21:55.379631+0000","last_unstale":"2024-05-14T11:21:57.380491+0000","last_undegraded":"2024-05-14T11:21:57.380491+0000","last_fullsized":"2024-05-14T11:21:57.380491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:11.076627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387177+0000","last_change":"2024-05-14T11:21:56.380163+0000","last_active":"2024-05-14T11:21:57.387177+0000","last_peered":"2024-05-14T11:21:57.387177+0000","last_clean":"2024-05-14T11:21:57.387177+0000","last_became_active":"2024-05-14T11:21:56.377500+0000","last_became_peered":"2024-05-14T11:21:56.377500+0000","last_unstale":"2024-05-14T11:21:57.387177+0000","last_undegraded":"2024-05-14T11:21:57.387177+0000","last_fullsized":"2024-05-14T11:21:57.387177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387601+0000","last_change":"2024-05-14T11:21:55.382014+0000","last_active":"2024-05-14T11:21:57.387601+0000","last_peered":"2024-05-14T11:21:57.387601+0000","last_clean":"2024-05-14T11:21:57.387601+0000","last_became_active":"2024-05-14T11:21:55.381670+0000","last_became_peered":"2024-05-14T11:21:55.381670+0000","last_unstale":"2024-05-14T11:21:57.387601+0000","last_undegraded":"2024-05-14T11:21:57.387601+0000","last_fullsized":"2024-05-14T11:21:57.387601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:25.098189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461127+0000","last_change":"2024-05-14T11:21:56.397506+0000","last_active":"2024-05-14T11:21:57.461127+0000","last_peered":"2024-05-14T11:21:57.461127+0000","last_clean":"2024-05-14T11:21:57.461127+0000","last_became_active":"2024-05-14T11:21:56.395519+0000","last_became_peered":"2024-05-14T11:21:56.395519+0000","last_unstale":"2024-05-14T11:21:57.461127+0000","last_undegraded":"2024-05-14T11:21:57.461127+0000","last_fullsized":"2024-05-14T11:21:57.461127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:26:54.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461633+0000","last_change":"2024-05-14T11:21:55.375676+0000","last_active":"2024-05-14T11:21:57.461633+0000","last_peered":"2024-05-14T11:21:57.461633+0000","last_clean":"2024-05-14T11:21:57.461633+0000","last_became_active":"2024-05-14T11:21:55.375356+0000","last_became_peered":"2024-05-14T11:21:55.375356+0000","last_unstale":"2024-05-14T11:21:57.461633+0000","last_undegraded":"2024-05-14T11:21:57.461633+0000","last_fullsized":"2024-05-14T11:21:57.461633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:11.395431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461112+0000","last_change":"2024-05-14T11:21:56.393934+0000","last_active":"2024-05-14T11:21:57.461112+0000","last_peered":"2024-05-14T11:21:57.461112+0000","last_clean":"2024-05-14T11:21:57.461112+0000","last_became_active":"2024-05-14T11:21:56.393609+0000","last_became_peered":"2024-05-14T11:21:56.393609+0000","last_unstale":"2024-05-14T11:21:57.461112+0000","last_undegraded":"2024-05-14T11:21:57.461112+0000","last_fullsized":"2024-05-14T11:21:57.461112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:22.581542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461639+0000","last_change":"2024-05-14T11:21:55.379420+0000","last_active":"2024-05-14T11:21:57.461639+0000","last_peered":"2024-05-14T11:21:57.461639+0000","last_clean":"2024-05-14T11:21:57.461639+0000","last_became_active":"2024-05-14T11:21:55.378985+0000","last_became_peered":"2024-05-14T11:21:55.378985+0000","last_unstale":"2024-05-14T11:21:57.461639+0000","last_undegraded":"2024-05-14T11:21:57.461639+0000","last_fullsized":"2024-05-14T11:21:57.461639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:30.781659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2024-05-14T11:21:55.371911+0000","last_change":"2024-05-14T11:21:55.371911+0000","last_active":"2024-05-14T11:21:55.349508+0000","last_peered":"2024-05-14T11:21:55.349508+0000","last_clean":"2024-05-14T11:21:55.349508+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-14T11:21:55.371911+0000","last_undegraded":"2024-05-14T11:21:55.371911+0000","last_fullsized":"2024-05-14T11:21:55.371911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387442+0000","last_change":"2024-05-14T11:21:55.374216+0000","last_active":"2024-05-14T11:21:57.387442+0000","last_peered":"2024-05-14T11:21:57.387442+0000","last_clean":"2024-05-14T11:21:57.387442+0000","last_became_active":"2024-05-14T11:21:55.373713+0000","last_became_peered":"2024-05-14T11:21:55.373713+0000","last_unstale":"2024-05-14T11:21:57.387442+0000","last_undegraded":"2024-05-14T11:21:57.387442+0000","last_fullsized":"2024-05-14T11:21:57.387442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:32:45.264705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380622+0000","last_change":"2024-05-14T11:21:56.383298+0000","last_active":"2024-05-14T11:21:57.380622+0000","last_peered":"2024-05-14T11:21:57.380622+0000","last_clean":"2024-05-14T11:21:57.380622+0000","last_became_active":"2024-05-14T11:21:56.381852+0000","last_became_peered":"2024-05-14T11:21:56.381852+0000","last_unstale":"2024-05-14T11:21:57.380622+0000","last_undegraded":"2024-05-14T11:21:57.380622+0000","last_fullsized":"2024-05-14T11:21:57.380622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:45:25.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460760+0000","last_change":"2024-05-14T11:21:55.379545+0000","last_active":"2024-05-14T11:21:57.460760+0000","last_peered":"2024-05-14T11:21:57.460760+0000","last_clean":"2024-05-14T11:21:57.460760+0000","last_became_active":"2024-05-14T11:21:55.378762+0000","last_became_peered":"2024-05-14T11:21:55.378762+0000","last_unstale":"2024-05-14T11:21:57.460760+0000","last_undegraded":"2024-05-14T11:21:57.460760+0000","last_fullsized":"2024-05-14T11:21:57.460760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:44.899842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402884+0000","last_change":"2024-05-14T11:21:56.384970+0000","last_active":"2024-05-14T11:21:57.402884+0000","last_peered":"2024-05-14T11:21:57.402884+0000","last_clean":"2024-05-14T11:21:57.402884+0000","last_became_active":"2024-05-14T11:21:56.384703+0000","last_became_peered":"2024-05-14T11:21:56.384703+0000","last_unstale":"2024-05-14T11:21:57.402884+0000","last_undegraded":"2024-05-14T11:21:57.402884+0000","last_fullsized":"2024-05-14T11:21:57.402884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T11:21:55.388829+0000","last_change":"2024-05-14T11:21:55.381768+0000","last_active":"2024-05-14T11:21:55.388829+0000","last_peered":"2024-05-14T11:21:55.388829+0000","last_clean":"2024-05-14T11:21:55.388829+0000","last_became_active":"2024-05-14T11:21:55.381485+0000","last_became_peered":"2024-05-14T11:21:55.381485+0000","last_unstale":"2024-05-14T11:21:55.388829+0000","last_undegraded":"2024-05-14T11:21:55.388829+0000","last_fullsized":"2024-05-14T11:21:55.388829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:38.207761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386457+0000","last_change":"2024-05-14T11:21:56.392100+0000","last_active":"2024-05-14T11:21:57.386457+0000","last_peered":"2024-05-14T11:21:57.386457+0000","last_clean":"2024-05-14T11:21:57.386457+0000","last_became_active":"2024-05-14T11:21:56.391699+0000","last_became_peered":"2024-05-14T11:21:56.391699+0000","last_unstale":"2024-05-14T11:21:57.386457+0000","last_undegraded":"2024-05-14T11:21:57.386457+0000","last_fullsized":"2024-05-14T11:21:57.386457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387086+0000","last_change":"2024-05-14T11:21:55.373928+0000","last_active":"2024-05-14T11:21:57.387086+0000","last_peered":"2024-05-14T11:21:57.387086+0000","last_clean":"2024-05-14T11:21:57.387086+0000","last_became_active":"2024-05-14T11:21:55.373638+0000","last_became_peered":"2024-05-14T11:21:55.373638+0000","last_unstale":"2024-05-14T11:21:57.387086+0000","last_undegraded":"2024-05-14T11:21:57.387086+0000","last_fullsized":"2024-05-14T11:21:57.387086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:35.515189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460703+0000","last_change":"2024-05-14T11:21:56.395619+0000","last_active":"2024-05-14T11:21:57.460703+0000","last_peered":"2024-05-14T11:21:57.460703+0000","last_clean":"2024-05-14T11:21:57.460703+0000","last_became_active":"2024-05-14T11:21:56.394829+0000","last_became_peered":"2024-05-14T11:21:56.394829+0000","last_unstale":"2024-05-14T11:21:57.460703+0000","last_undegraded":"2024-05-14T11:21:57.460703+0000","last_fullsized":"2024-05-14T11:21:57.460703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:36:28.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386239+0000","last_change":"2024-05-14T11:21:55.387084+0000","last_active":"2024-05-14T11:21:57.386239+0000","last_peered":"2024-05-14T11:21:57.386239+0000","last_clean":"2024-05-14T11:21:57.386239+0000","last_became_active":"2024-05-14T11:21:55.386834+0000","last_became_peered":"2024-05-14T11:21:55.386834+0000","last_unstale":"2024-05-14T11:21:57.386239+0000","last_undegraded":"2024-05-14T11:21:57.386239+0000","last_fullsized":"2024-05-14T11:21:57.386239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:17:10.801403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387496+0000","last_change":"2024-05-14T11:21:56.387037+0000","last_active":"2024-05-14T11:21:57.387496+0000","last_peered":"2024-05-14T11:21:57.387496+0000","last_clean":"2024-05-14T11:21:57.387496+0000","last_became_active":"2024-05-14T11:21:56.386496+0000","last_became_peered":"2024-05-14T11:21:56.386496+0000","last_unstale":"2024-05-14T11:21:57.387496+0000","last_undegraded":"2024-05-14T11:21:57.387496+0000","last_fullsized":"2024-05-14T11:21:57.387496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:21.878567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460199+0000","last_change":"2024-05-14T11:21:55.387153+0000","last_active":"2024-05-14T11:21:57.460199+0000","last_peered":"2024-05-14T11:21:57.460199+0000","last_clean":"2024-05-14T11:21:57.460199+0000","last_became_active":"2024-05-14T11:21:55.386815+0000","last_became_peered":"2024-05-14T11:21:55.386815+0000","last_unstale":"2024-05-14T11:21:57.460199+0000","last_undegraded":"2024-05-14T11:21:57.460199+0000","last_fullsized":"2024-05-14T11:21:57.460199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:44.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386383+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:21:57.386383+0000","last_peered":"2024-05-14T11:21:57.386383+0000","last_clean":"2024-05-14T11:21:57.386383+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:21:57.386383+0000","last_undegraded":"2024-05-14T11:21:57.386383+0000","last_fullsized":"2024-05-14T11:21:57.386383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1064313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":476105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":453363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":473959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1066575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:22:05.096 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:22:05.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.253+0000 7fbc85c00640 1 Processor -- start 2024-05-14T11:22:05.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.253+0000 7fbc85c00640 1 -- start start 2024-05-14T11:22:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc85c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc80166750 0x7fbc80168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc85c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8006ff20 con 0x7fbc8015db30 2024-05-14T11:22:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc85c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc80070470 con 0x7fbc800d1670 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc85c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8006d710 con 0x7fbc80166750 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc80166750 0x7fbc80168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbc8015db30 0x7fbc800aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37154/0 (socket says 172.21.15.116:37154) 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7ea00640 1 -- 172.21.15.116:0/2174211473 learned_addr learned my addr 172.21.15.116:0/2174211473 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7f400640 1 -- 172.21.15.116:0/2174211473 >> v1:172.21.15.149:6789/0 conn(0x7fbc800d1670 legacy=0x7fbc800d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7f400640 1 -- 172.21.15.116:0/2174211473 >> v1:172.21.15.116:6789/0 conn(0x7fbc8015db30 legacy=0x7fbc800aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.257+0000 7fbc7f400640 1 -- 172.21.15.116:0/2174211473 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc80070750 con 0x7fbc80166750 2024-05-14T11:22:05.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc7f400640 1 --2- 172.21.15.116:0/2174211473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc80166750 0x7fbc80168b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d90 tx=0x7fbc7801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41267f4f88731b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc7e000640 1 -- 172.21.15.116:0/2174211473 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780b2070 con 0x7fbc80166750 2024-05-14T11:22:05.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc7e000640 1 -- 172.21.15.116:0/2174211473 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc780a5710 con 0x7fbc80166750 2024-05-14T11:22:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc7e000640 1 -- 172.21.15.116:0/2174211473 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780a5ae0 con 0x7fbc80166750 2024-05-14T11:22:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- 172.21.15.116:0/2174211473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc80166750 msgr2=0x7fbc80168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2174211473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc80166750 0x7fbc80168b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d90 tx=0x7fbc7801c850 comp rx=0 tx=0).stop 2024-05-14T11:22:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- 172.21.15.116:0/2174211473 shutdown_connections 2024-05-14T11:22:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2174211473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc80166750 0x7fbc80168b70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- 172.21.15.116:0/2174211473 >> 172.21.15.116:0/2174211473 conn(0x7fbc800b4620 msgr2=0x7fbc800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- 172.21.15.116:0/2174211473 shutdown_connections 2024-05-14T11:22:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- 172.21.15.116:0/2174211473 wait complete. 2024-05-14T11:22:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 Processor -- start 2024-05-14T11:22:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.261+0000 7fbc85c00640 1 -- start start 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc800d1670 0x7fbc80074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc80166750 0x7fbc800cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8016bae0 con 0x7fbc8015db30 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc800ae870 con 0x7fbc800d1670 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8016bc60 con 0x7fbc80166750 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42576/0 (socket says 172.21.15.116:42576) 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 -- 172.21.15.116:0/2142819914 learned_addr learned my addr 172.21.15.116:0/2142819914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc7ea00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc800d1670 0x7fbc80074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc7f400640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc80166750 0x7fbc800cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 -- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc80166750 msgr2=0x7fbc800cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc80166750 0x7fbc800cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 -- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc800d1670 msgr2=0x7fbc80074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc800d1670 0x7fbc80074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 -- 172.21.15.116:0/2142819914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8009ae80 con 0x7fbc8015db30 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc76000640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c3f0 tx=0x7fbc7000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70017070 con 0x7fbc8015db30 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc700052d0 con 0x7fbc8015db30 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003820 con 0x7fbc8015db30 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc800b1000 con 0x7fbc8015db30 2024-05-14T11:22:05.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc800b15c0 con 0x7fbc8015db30 2024-05-14T11:22:05.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.265+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8006a9c0 con 0x7fbc8015db30 2024-05-14T11:22:05.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.269+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbc7001a070 con 0x7fbc8015db30 2024-05-14T11:22:05.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.269+0000 7fbc77400640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 0x7fbc5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.269+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fbc7008e820 con 0x7fbc8015db30 2024-05-14T11:22:05.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.269+0000 7fbc7ea00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 0x7fbc5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.269+0000 7fbc7ea00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 0x7fbc5c071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc68007fd0 tx=0x7fbc68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.273+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbc70005480 con 0x7fbc8015db30 2024-05-14T11:22:05.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.429+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc80073550 con 0x7fbc5c06f400 2024-05-14T11:22:05.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.437+0000 7fbc77400640 1 -- 172.21.15.116:0/2142819914 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7fbc80073550 con 0x7fbc5c06f400 2024-05-14T11:22:05.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:05.442 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 msgr2=0x7fbc5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 0x7fbc5c071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc68007fd0 tx=0x7fbc68009040 comp rx=0 tx=0).stop 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 msgr2=0x7fbc8009a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c3f0 tx=0x7fbc7000c8b0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 shutdown_connections 2024-05-14T11:22:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbc5c06f400 0x7fbc5c071910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc80166750 0x7fbc800cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc8015db30 0x7fbc8009a940 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 --2- 172.21.15.116:0/2142819914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc800d1670 0x7fbc80074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 >> 172.21.15.116:0/2142819914 conn(0x7fbc800b4620 msgr2=0x7fbc80159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 shutdown_connections 2024-05-14T11:22:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.441+0000 7fbc85c00640 1 -- 172.21.15.116:0/2142819914 wait complete. 2024-05-14T11:22:05.460 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-05-14T11:22:04.448260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258804,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715916,"statfs":{"total":773094113280,"available":772829097984,"internally_reserved":0,"allocated":9543680,"data_stored":4976668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.086249"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403074+0000","last_change":"2024-05-14T11:21:56.378698+0000","last_active":"2024-05-14T11:21:57.403074+0000","last_peered":"2024-05-14T11:21:57.403074+0000","last_clean":"2024-05-14T11:21:57.403074+0000","last_became_active":"2024-05-14T11:21:56.378402+0000","last_became_peered":"2024-05-14T11:21:56.378402+0000","last_unstale":"2024-05-14T11:21:57.403074+0000","last_undegraded":"2024-05-14T11:21:57.403074+0000","last_fullsized":"2024-05-14T11:21:57.403074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461028+0000","last_change":"2024-05-14T11:21:56.398616+0000","last_active":"2024-05-14T11:21:57.461028+0000","last_peered":"2024-05-14T11:21:57.461028+0000","last_clean":"2024-05-14T11:21:57.461028+0000","last_became_active":"2024-05-14T11:21:56.397969+0000","last_became_peered":"2024-05-14T11:21:56.397969+0000","last_unstale":"2024-05-14T11:21:57.461028+0000","last_undegraded":"2024-05-14T11:21:57.461028+0000","last_fullsized":"2024-05-14T11:21:57.461028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402927+0000","last_change":"2024-05-14T11:21:56.392559+0000","last_active":"2024-05-14T11:21:57.402927+0000","last_peered":"2024-05-14T11:21:57.402927+0000","last_clean":"2024-05-14T11:21:57.402927+0000","last_became_active":"2024-05-14T11:21:56.391449+0000","last_became_peered":"2024-05-14T11:21:56.391449+0000","last_unstale":"2024-05-14T11:21:57.402927+0000","last_undegraded":"2024-05-14T11:21:57.402927+0000","last_fullsized":"2024-05-14T11:21:57.402927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459732+0000","last_change":"2024-05-14T11:21:56.378008+0000","last_active":"2024-05-14T11:21:57.459732+0000","last_peered":"2024-05-14T11:21:57.459732+0000","last_clean":"2024-05-14T11:21:57.459732+0000","last_became_active":"2024-05-14T11:21:56.377707+0000","last_became_peered":"2024-05-14T11:21:56.377707+0000","last_unstale":"2024-05-14T11:21:57.459732+0000","last_undegraded":"2024-05-14T11:21:57.459732+0000","last_fullsized":"2024-05-14T11:21:57.459732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461077+0000","last_change":"2024-05-14T11:21:56.391093+0000","last_active":"2024-05-14T11:21:57.461077+0000","last_peered":"2024-05-14T11:21:57.461077+0000","last_clean":"2024-05-14T11:21:57.461077+0000","last_became_active":"2024-05-14T11:21:56.390712+0000","last_became_peered":"2024-05-14T11:21:56.390712+0000","last_unstale":"2024-05-14T11:21:57.461077+0000","last_undegraded":"2024-05-14T11:21:57.461077+0000","last_fullsized":"2024-05-14T11:21:57.461077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460609+0000","last_change":"2024-05-14T11:21:56.398553+0000","last_active":"2024-05-14T11:21:57.460609+0000","last_peered":"2024-05-14T11:21:57.460609+0000","last_clean":"2024-05-14T11:21:57.460609+0000","last_became_active":"2024-05-14T11:21:56.396664+0000","last_became_peered":"2024-05-14T11:21:56.396664+0000","last_unstale":"2024-05-14T11:21:57.460609+0000","last_undegraded":"2024-05-14T11:21:57.460609+0000","last_fullsized":"2024-05-14T11:21:57.460609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:59:19.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386491+0000","last_change":"2024-05-14T11:21:56.385275+0000","last_active":"2024-05-14T11:21:57.386491+0000","last_peered":"2024-05-14T11:21:57.386491+0000","last_clean":"2024-05-14T11:21:57.386491+0000","last_became_active":"2024-05-14T11:21:56.384904+0000","last_became_peered":"2024-05-14T11:21:56.384904+0000","last_unstale":"2024-05-14T11:21:57.386491+0000","last_undegraded":"2024-05-14T11:21:57.386491+0000","last_fullsized":"2024-05-14T11:21:57.386491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:36.820614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459460+0000","last_change":"2024-05-14T11:21:56.382681+0000","last_active":"2024-05-14T11:21:57.459460+0000","last_peered":"2024-05-14T11:21:57.459460+0000","last_clean":"2024-05-14T11:21:57.459460+0000","last_became_active":"2024-05-14T11:21:56.381079+0000","last_became_peered":"2024-05-14T11:21:56.381079+0000","last_unstale":"2024-05-14T11:21:57.459460+0000","last_undegraded":"2024-05-14T11:21:57.459460+0000","last_fullsized":"2024-05-14T11:21:57.459460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.394964+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.393782+0000","last_became_peered":"2024-05-14T11:21:56.393782+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380908+0000","last_change":"2024-05-14T11:21:56.384798+0000","last_active":"2024-05-14T11:21:57.380908+0000","last_peered":"2024-05-14T11:21:57.380908+0000","last_clean":"2024-05-14T11:21:57.380908+0000","last_became_active":"2024-05-14T11:21:56.384447+0000","last_became_peered":"2024-05-14T11:21:56.384447+0000","last_unstale":"2024-05-14T11:21:57.380908+0000","last_undegraded":"2024-05-14T11:21:57.380908+0000","last_fullsized":"2024-05-14T11:21:57.380908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:46:12.540726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461319+0000","last_change":"2024-05-14T11:21:56.391270+0000","last_active":"2024-05-14T11:21:57.461319+0000","last_peered":"2024-05-14T11:21:57.461319+0000","last_clean":"2024-05-14T11:21:57.461319+0000","last_became_active":"2024-05-14T11:21:56.390667+0000","last_became_peered":"2024-05-14T11:21:56.390667+0000","last_unstale":"2024-05-14T11:21:57.461319+0000","last_undegraded":"2024-05-14T11:21:57.461319+0000","last_fullsized":"2024-05-14T11:21:57.461319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380632+0000","last_change":"2024-05-14T11:21:56.379345+0000","last_active":"2024-05-14T11:21:57.380632+0000","last_peered":"2024-05-14T11:21:57.380632+0000","last_clean":"2024-05-14T11:21:57.380632+0000","last_became_active":"2024-05-14T11:21:56.379007+0000","last_became_peered":"2024-05-14T11:21:56.379007+0000","last_unstale":"2024-05-14T11:21:57.380632+0000","last_undegraded":"2024-05-14T11:21:57.380632+0000","last_fullsized":"2024-05-14T11:21:57.380632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:31:31.560352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459974+0000","last_change":"2024-05-14T11:21:56.384668+0000","last_active":"2024-05-14T11:21:57.459974+0000","last_peered":"2024-05-14T11:21:57.459974+0000","last_clean":"2024-05-14T11:21:57.459974+0000","last_became_active":"2024-05-14T11:21:56.382366+0000","last_became_peered":"2024-05-14T11:21:56.382366+0000","last_unstale":"2024-05-14T11:21:57.459974+0000","last_undegraded":"2024-05-14T11:21:57.459974+0000","last_fullsized":"2024-05-14T11:21:57.459974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460403+0000","last_change":"2024-05-14T11:21:56.389629+0000","last_active":"2024-05-14T11:21:57.460403+0000","last_peered":"2024-05-14T11:21:57.460403+0000","last_clean":"2024-05-14T11:21:57.460403+0000","last_became_active":"2024-05-14T11:21:56.389299+0000","last_became_peered":"2024-05-14T11:21:56.389299+0000","last_unstale":"2024-05-14T11:21:57.460403+0000","last_undegraded":"2024-05-14T11:21:57.460403+0000","last_fullsized":"2024-05-14T11:21:57.460403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:00:13.582984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459513+0000","last_change":"2024-05-14T11:21:56.387482+0000","last_active":"2024-05-14T11:21:57.459513+0000","last_peered":"2024-05-14T11:21:57.459513+0000","last_clean":"2024-05-14T11:21:57.459513+0000","last_became_active":"2024-05-14T11:21:56.385978+0000","last_became_peered":"2024-05-14T11:21:56.385978+0000","last_unstale":"2024-05-14T11:21:57.459513+0000","last_undegraded":"2024-05-14T11:21:57.459513+0000","last_fullsized":"2024-05-14T11:21:57.459513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460528+0000","last_change":"2024-05-14T11:21:56.386948+0000","last_active":"2024-05-14T11:21:57.460528+0000","last_peered":"2024-05-14T11:21:57.460528+0000","last_clean":"2024-05-14T11:21:57.460528+0000","last_became_active":"2024-05-14T11:21:56.385310+0000","last_became_peered":"2024-05-14T11:21:56.385310+0000","last_unstale":"2024-05-14T11:21:57.460528+0000","last_undegraded":"2024-05-14T11:21:57.460528+0000","last_fullsized":"2024-05-14T11:21:57.460528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:31:18.889545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461165+0000","last_change":"2024-05-14T11:21:56.400922+0000","last_active":"2024-05-14T11:21:57.461165+0000","last_peered":"2024-05-14T11:21:57.461165+0000","last_clean":"2024-05-14T11:21:57.461165+0000","last_became_active":"2024-05-14T11:21:56.400597+0000","last_became_peered":"2024-05-14T11:21:56.400597+0000","last_unstale":"2024-05-14T11:21:57.461165+0000","last_undegraded":"2024-05-14T11:21:57.461165+0000","last_fullsized":"2024-05-14T11:21:57.461165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:07.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403099+0000","last_change":"2024-05-14T11:21:56.397705+0000","last_active":"2024-05-14T11:21:57.403099+0000","last_peered":"2024-05-14T11:21:57.403099+0000","last_clean":"2024-05-14T11:21:57.403099+0000","last_became_active":"2024-05-14T11:21:56.397144+0000","last_became_peered":"2024-05-14T11:21:56.397144+0000","last_unstale":"2024-05-14T11:21:57.403099+0000","last_undegraded":"2024-05-14T11:21:57.403099+0000","last_fullsized":"2024-05-14T11:21:57.403099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461475+0000","last_change":"2024-05-14T11:21:56.390962+0000","last_active":"2024-05-14T11:21:57.461475+0000","last_peered":"2024-05-14T11:21:57.461475+0000","last_clean":"2024-05-14T11:21:57.461475+0000","last_became_active":"2024-05-14T11:21:56.390072+0000","last_became_peered":"2024-05-14T11:21:56.390072+0000","last_unstale":"2024-05-14T11:21:57.461475+0000","last_undegraded":"2024-05-14T11:21:57.461475+0000","last_fullsized":"2024-05-14T11:21:57.461475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459937+0000","last_change":"2024-05-14T11:21:56.379798+0000","last_active":"2024-05-14T11:21:57.459937+0000","last_peered":"2024-05-14T11:21:57.459937+0000","last_clean":"2024-05-14T11:21:57.459937+0000","last_became_active":"2024-05-14T11:21:56.378722+0000","last_became_peered":"2024-05-14T11:21:56.378722+0000","last_unstale":"2024-05-14T11:21:57.459937+0000","last_undegraded":"2024-05-14T11:21:57.459937+0000","last_fullsized":"2024-05-14T11:21:57.459937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:35.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460813+0000","last_change":"2024-05-14T11:21:56.391852+0000","last_active":"2024-05-14T11:21:57.460813+0000","last_peered":"2024-05-14T11:21:57.460813+0000","last_clean":"2024-05-14T11:21:57.460813+0000","last_became_active":"2024-05-14T11:21:56.391495+0000","last_became_peered":"2024-05-14T11:21:56.391495+0000","last_unstale":"2024-05-14T11:21:57.460813+0000","last_undegraded":"2024-05-14T11:21:57.460813+0000","last_fullsized":"2024-05-14T11:21:57.460813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460442+0000","last_change":"2024-05-14T11:21:56.390530+0000","last_active":"2024-05-14T11:21:57.460442+0000","last_peered":"2024-05-14T11:21:57.460442+0000","last_clean":"2024-05-14T11:21:57.460442+0000","last_became_active":"2024-05-14T11:21:56.390215+0000","last_became_peered":"2024-05-14T11:21:56.390215+0000","last_unstale":"2024-05-14T11:21:57.460442+0000","last_undegraded":"2024-05-14T11:21:57.460442+0000","last_fullsized":"2024-05-14T11:21:57.460442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:16:07.315380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387804+0000","last_change":"2024-05-14T11:21:56.391315+0000","last_active":"2024-05-14T11:21:57.387804+0000","last_peered":"2024-05-14T11:21:57.387804+0000","last_clean":"2024-05-14T11:21:57.387804+0000","last_became_active":"2024-05-14T11:21:56.390649+0000","last_became_peered":"2024-05-14T11:21:56.390649+0000","last_unstale":"2024-05-14T11:21:57.387804+0000","last_undegraded":"2024-05-14T11:21:57.387804+0000","last_fullsized":"2024-05-14T11:21:57.387804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:03:44.509125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402786+0000","last_change":"2024-05-14T11:21:56.394765+0000","last_active":"2024-05-14T11:21:57.402786+0000","last_peered":"2024-05-14T11:21:57.402786+0000","last_clean":"2024-05-14T11:21:57.402786+0000","last_became_active":"2024-05-14T11:21:56.394490+0000","last_became_peered":"2024-05-14T11:21:56.394490+0000","last_unstale":"2024-05-14T11:21:57.402786+0000","last_undegraded":"2024-05-14T11:21:57.402786+0000","last_fullsized":"2024-05-14T11:21:57.402786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461170+0000","last_change":"2024-05-14T11:21:56.395590+0000","last_active":"2024-05-14T11:21:57.461170+0000","last_peered":"2024-05-14T11:21:57.461170+0000","last_clean":"2024-05-14T11:21:57.461170+0000","last_became_active":"2024-05-14T11:21:56.395269+0000","last_became_peered":"2024-05-14T11:21:56.395269+0000","last_unstale":"2024-05-14T11:21:57.461170+0000","last_undegraded":"2024-05-14T11:21:57.461170+0000","last_fullsized":"2024-05-14T11:21:57.461170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:40:39.793828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461831+0000","last_change":"2024-05-14T11:21:56.389580+0000","last_active":"2024-05-14T11:21:57.461831+0000","last_peered":"2024-05-14T11:21:57.461831+0000","last_clean":"2024-05-14T11:21:57.461831+0000","last_became_active":"2024-05-14T11:21:56.389252+0000","last_became_peered":"2024-05-14T11:21:56.389252+0000","last_unstale":"2024-05-14T11:21:57.461831+0000","last_undegraded":"2024-05-14T11:21:57.461831+0000","last_fullsized":"2024-05-14T11:21:57.461831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.381007+0000","last_change":"2024-05-14T11:21:56.378063+0000","last_active":"2024-05-14T11:21:57.381007+0000","last_peered":"2024-05-14T11:21:57.381007+0000","last_clean":"2024-05-14T11:21:57.381007+0000","last_became_active":"2024-05-14T11:21:56.377759+0000","last_became_peered":"2024-05-14T11:21:56.377759+0000","last_unstale":"2024-05-14T11:21:57.381007+0000","last_undegraded":"2024-05-14T11:21:57.381007+0000","last_fullsized":"2024-05-14T11:21:57.381007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:49:43.364381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460090+0000","last_change":"2024-05-14T11:21:56.398675+0000","last_active":"2024-05-14T11:21:57.460090+0000","last_peered":"2024-05-14T11:21:57.460090+0000","last_clean":"2024-05-14T11:21:57.460090+0000","last_became_active":"2024-05-14T11:21:56.392629+0000","last_became_peered":"2024-05-14T11:21:56.392629+0000","last_unstale":"2024-05-14T11:21:57.460090+0000","last_undegraded":"2024-05-14T11:21:57.460090+0000","last_fullsized":"2024-05-14T11:21:57.460090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:36.210570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386641+0000","last_change":"2024-05-14T11:21:56.371575+0000","last_active":"2024-05-14T11:21:57.386641+0000","last_peered":"2024-05-14T11:21:57.386641+0000","last_clean":"2024-05-14T11:21:57.386641+0000","last_became_active":"2024-05-14T11:21:56.371212+0000","last_became_peered":"2024-05-14T11:21:56.371212+0000","last_unstale":"2024-05-14T11:21:57.386641+0000","last_undegraded":"2024-05-14T11:21:57.386641+0000","last_fullsized":"2024-05-14T11:21:57.386641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:49:03.645090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460626+0000","last_change":"2024-05-14T11:21:56.394494+0000","last_active":"2024-05-14T11:21:57.460626+0000","last_peered":"2024-05-14T11:21:57.460626+0000","last_clean":"2024-05-14T11:21:57.460626+0000","last_became_active":"2024-05-14T11:21:56.394205+0000","last_became_peered":"2024-05-14T11:21:56.394205+0000","last_unstale":"2024-05-14T11:21:57.460626+0000","last_undegraded":"2024-05-14T11:21:57.460626+0000","last_fullsized":"2024-05-14T11:21:57.460626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460503+0000","last_change":"2024-05-14T11:21:56.403467+0000","last_active":"2024-05-14T11:21:57.460503+0000","last_peered":"2024-05-14T11:21:57.460503+0000","last_clean":"2024-05-14T11:21:57.460503+0000","last_became_active":"2024-05-14T11:21:56.403189+0000","last_became_peered":"2024-05-14T11:21:56.403189+0000","last_unstale":"2024-05-14T11:21:57.460503+0000","last_undegraded":"2024-05-14T11:21:57.460503+0000","last_fullsized":"2024-05-14T11:21:57.460503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:35.977545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460844+0000","last_change":"2024-05-14T11:21:56.382783+0000","last_active":"2024-05-14T11:21:57.460844+0000","last_peered":"2024-05-14T11:21:57.460844+0000","last_clean":"2024-05-14T11:21:57.460844+0000","last_became_active":"2024-05-14T11:21:56.382496+0000","last_became_peered":"2024-05-14T11:21:56.382496+0000","last_unstale":"2024-05-14T11:21:57.460844+0000","last_undegraded":"2024-05-14T11:21:57.460844+0000","last_fullsized":"2024-05-14T11:21:57.460844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:44.636667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461113+0000","last_change":"2024-05-14T11:21:56.399015+0000","last_active":"2024-05-14T11:21:57.461113+0000","last_peered":"2024-05-14T11:21:57.461113+0000","last_clean":"2024-05-14T11:21:57.461113+0000","last_became_active":"2024-05-14T11:21:56.397801+0000","last_became_peered":"2024-05-14T11:21:56.397801+0000","last_unstale":"2024-05-14T11:21:57.461113+0000","last_undegraded":"2024-05-14T11:21:57.461113+0000","last_fullsized":"2024-05-14T11:21:57.461113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:03.545193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387341+0000","last_change":"2024-05-14T11:21:55.377300+0000","last_active":"2024-05-14T11:21:57.387341+0000","last_peered":"2024-05-14T11:21:57.387341+0000","last_clean":"2024-05-14T11:21:57.387341+0000","last_became_active":"2024-05-14T11:21:55.376999+0000","last_became_peered":"2024-05-14T11:21:55.376999+0000","last_unstale":"2024-05-14T11:21:57.387341+0000","last_undegraded":"2024-05-14T11:21:57.387341+0000","last_fullsized":"2024-05-14T11:21:57.387341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:14.148384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460907+0000","last_change":"2024-05-14T11:21:56.381804+0000","last_active":"2024-05-14T11:21:57.460907+0000","last_peered":"2024-05-14T11:21:57.460907+0000","last_clean":"2024-05-14T11:21:57.460907+0000","last_became_active":"2024-05-14T11:21:56.381554+0000","last_became_peered":"2024-05-14T11:21:56.381554+0000","last_unstale":"2024-05-14T11:21:57.460907+0000","last_undegraded":"2024-05-14T11:21:57.460907+0000","last_fullsized":"2024-05-14T11:21:57.460907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:37:23.266696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461538+0000","last_change":"2024-05-14T11:21:55.373916+0000","last_active":"2024-05-14T11:21:57.461538+0000","last_peered":"2024-05-14T11:21:57.461538+0000","last_clean":"2024-05-14T11:21:57.461538+0000","last_became_active":"2024-05-14T11:21:55.373461+0000","last_became_peered":"2024-05-14T11:21:55.373461+0000","last_unstale":"2024-05-14T11:21:57.461538+0000","last_undegraded":"2024-05-14T11:21:57.461538+0000","last_fullsized":"2024-05-14T11:21:57.461538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:12:01.508933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386940+0000","last_change":"2024-05-14T11:21:56.385784+0000","last_active":"2024-05-14T11:21:57.386940+0000","last_peered":"2024-05-14T11:21:57.386940+0000","last_clean":"2024-05-14T11:21:57.386940+0000","last_became_active":"2024-05-14T11:21:56.385460+0000","last_became_peered":"2024-05-14T11:21:56.385460+0000","last_unstale":"2024-05-14T11:21:57.386940+0000","last_undegraded":"2024-05-14T11:21:57.386940+0000","last_fullsized":"2024-05-14T11:21:57.386940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:02:04.422136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387331+0000","last_change":"2024-05-14T11:21:55.384239+0000","last_active":"2024-05-14T11:21:57.387331+0000","last_peered":"2024-05-14T11:21:57.387331+0000","last_clean":"2024-05-14T11:21:57.387331+0000","last_became_active":"2024-05-14T11:21:55.383906+0000","last_became_peered":"2024-05-14T11:21:55.383906+0000","last_unstale":"2024-05-14T11:21:57.387331+0000","last_undegraded":"2024-05-14T11:21:57.387331+0000","last_fullsized":"2024-05-14T11:21:57.387331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:11.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461213+0000","last_change":"2024-05-14T11:21:56.398729+0000","last_active":"2024-05-14T11:21:57.461213+0000","last_peered":"2024-05-14T11:21:57.461213+0000","last_clean":"2024-05-14T11:21:57.461213+0000","last_became_active":"2024-05-14T11:21:56.397051+0000","last_became_peered":"2024-05-14T11:21:56.397051+0000","last_unstale":"2024-05-14T11:21:57.461213+0000","last_undegraded":"2024-05-14T11:21:57.461213+0000","last_fullsized":"2024-05-14T11:21:57.461213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:22:03.622138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387307+0000","last_change":"2024-05-14T11:21:55.386498+0000","last_active":"2024-05-14T11:21:57.387307+0000","last_peered":"2024-05-14T11:21:57.387307+0000","last_clean":"2024-05-14T11:21:57.387307+0000","last_became_active":"2024-05-14T11:21:55.385760+0000","last_became_peered":"2024-05-14T11:21:55.385760+0000","last_unstale":"2024-05-14T11:21:57.387307+0000","last_undegraded":"2024-05-14T11:21:57.387307+0000","last_fullsized":"2024-05-14T11:21:57.387307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:04.837126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459652+0000","last_change":"2024-05-14T11:21:56.381076+0000","last_active":"2024-05-14T11:21:57.459652+0000","last_peered":"2024-05-14T11:21:57.459652+0000","last_clean":"2024-05-14T11:21:57.459652+0000","last_became_active":"2024-05-14T11:21:56.380328+0000","last_became_peered":"2024-05-14T11:21:56.380328+0000","last_unstale":"2024-05-14T11:21:57.459652+0000","last_undegraded":"2024-05-14T11:21:57.459652+0000","last_fullsized":"2024-05-14T11:21:57.459652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461212+0000","last_change":"2024-05-14T11:21:55.383653+0000","last_active":"2024-05-14T11:21:57.461212+0000","last_peered":"2024-05-14T11:21:57.461212+0000","last_clean":"2024-05-14T11:21:57.461212+0000","last_became_active":"2024-05-14T11:21:55.383294+0000","last_became_peered":"2024-05-14T11:21:55.383294+0000","last_unstale":"2024-05-14T11:21:57.461212+0000","last_undegraded":"2024-05-14T11:21:57.461212+0000","last_fullsized":"2024-05-14T11:21:57.461212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:14.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386959+0000","last_change":"2024-05-14T11:21:56.378909+0000","last_active":"2024-05-14T11:21:57.386959+0000","last_peered":"2024-05-14T11:21:57.386959+0000","last_clean":"2024-05-14T11:21:57.386959+0000","last_became_active":"2024-05-14T11:21:56.378614+0000","last_became_peered":"2024-05-14T11:21:56.378614+0000","last_unstale":"2024-05-14T11:21:57.386959+0000","last_undegraded":"2024-05-14T11:21:57.386959+0000","last_fullsized":"2024-05-14T11:21:57.386959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:54:00.401464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460014+0000","last_change":"2024-05-14T11:21:55.387444+0000","last_active":"2024-05-14T11:21:57.460014+0000","last_peered":"2024-05-14T11:21:57.460014+0000","last_clean":"2024-05-14T11:21:57.460014+0000","last_became_active":"2024-05-14T11:21:55.386992+0000","last_became_peered":"2024-05-14T11:21:55.386992+0000","last_unstale":"2024-05-14T11:21:57.460014+0000","last_undegraded":"2024-05-14T11:21:57.460014+0000","last_fullsized":"2024-05-14T11:21:57.460014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:34.670851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460786+0000","last_change":"2024-05-14T11:21:56.389934+0000","last_active":"2024-05-14T11:21:57.460786+0000","last_peered":"2024-05-14T11:21:57.460786+0000","last_clean":"2024-05-14T11:21:57.460786+0000","last_became_active":"2024-05-14T11:21:56.388204+0000","last_became_peered":"2024-05-14T11:21:56.388204+0000","last_unstale":"2024-05-14T11:21:57.460786+0000","last_undegraded":"2024-05-14T11:21:57.460786+0000","last_fullsized":"2024-05-14T11:21:57.460786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:29:42.638658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402951+0000","last_change":"2024-05-14T11:21:55.379234+0000","last_active":"2024-05-14T11:21:57.402951+0000","last_peered":"2024-05-14T11:21:57.402951+0000","last_clean":"2024-05-14T11:21:57.402951+0000","last_became_active":"2024-05-14T11:21:55.378924+0000","last_became_peered":"2024-05-14T11:21:55.378924+0000","last_unstale":"2024-05-14T11:21:57.402951+0000","last_undegraded":"2024-05-14T11:21:57.402951+0000","last_fullsized":"2024-05-14T11:21:57.402951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460137+0000","last_change":"2024-05-14T11:21:56.383194+0000","last_active":"2024-05-14T11:21:57.460137+0000","last_peered":"2024-05-14T11:21:57.460137+0000","last_clean":"2024-05-14T11:21:57.460137+0000","last_became_active":"2024-05-14T11:21:56.379071+0000","last_became_peered":"2024-05-14T11:21:56.379071+0000","last_unstale":"2024-05-14T11:21:57.460137+0000","last_undegraded":"2024-05-14T11:21:57.460137+0000","last_fullsized":"2024-05-14T11:21:57.460137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:03.882008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386225+0000","last_change":"2024-05-14T11:21:55.384145+0000","last_active":"2024-05-14T11:21:57.386225+0000","last_peered":"2024-05-14T11:21:57.386225+0000","last_clean":"2024-05-14T11:21:57.386225+0000","last_became_active":"2024-05-14T11:21:55.383834+0000","last_became_peered":"2024-05-14T11:21:55.383834+0000","last_unstale":"2024-05-14T11:21:57.386225+0000","last_undegraded":"2024-05-14T11:21:57.386225+0000","last_fullsized":"2024-05-14T11:21:57.386225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:00:31.347753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461043+0000","last_change":"2024-05-14T11:21:56.396531+0000","last_active":"2024-05-14T11:21:57.461043+0000","last_peered":"2024-05-14T11:21:57.461043+0000","last_clean":"2024-05-14T11:21:57.461043+0000","last_became_active":"2024-05-14T11:21:56.396223+0000","last_became_peered":"2024-05-14T11:21:56.396223+0000","last_unstale":"2024-05-14T11:21:57.461043+0000","last_undegraded":"2024-05-14T11:21:57.461043+0000","last_fullsized":"2024-05-14T11:21:57.461043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461311+0000","last_change":"2024-05-14T11:21:55.379424+0000","last_active":"2024-05-14T11:21:57.461311+0000","last_peered":"2024-05-14T11:21:57.461311+0000","last_clean":"2024-05-14T11:21:57.461311+0000","last_became_active":"2024-05-14T11:21:55.378552+0000","last_became_peered":"2024-05-14T11:21:55.378552+0000","last_unstale":"2024-05-14T11:21:57.461311+0000","last_undegraded":"2024-05-14T11:21:57.461311+0000","last_fullsized":"2024-05-14T11:21:57.461311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:25:02.879231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387395+0000","last_change":"2024-05-14T11:21:56.392129+0000","last_active":"2024-05-14T11:21:57.387395+0000","last_peered":"2024-05-14T11:21:57.387395+0000","last_clean":"2024-05-14T11:21:57.387395+0000","last_became_active":"2024-05-14T11:21:56.391814+0000","last_became_peered":"2024-05-14T11:21:56.391814+0000","last_unstale":"2024-05-14T11:21:57.387395+0000","last_undegraded":"2024-05-14T11:21:57.387395+0000","last_fullsized":"2024-05-14T11:21:57.387395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:20.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403200+0000","last_change":"2024-05-14T11:21:55.388521+0000","last_active":"2024-05-14T11:21:57.403200+0000","last_peered":"2024-05-14T11:21:57.403200+0000","last_clean":"2024-05-14T11:21:57.403200+0000","last_became_active":"2024-05-14T11:21:55.387310+0000","last_became_peered":"2024-05-14T11:21:55.387310+0000","last_unstale":"2024-05-14T11:21:57.403200+0000","last_undegraded":"2024-05-14T11:21:57.403200+0000","last_fullsized":"2024-05-14T11:21:57.403200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.403125+0000","last_change":"2024-05-14T11:21:56.379479+0000","last_active":"2024-05-14T11:21:57.403125+0000","last_peered":"2024-05-14T11:21:57.403125+0000","last_clean":"2024-05-14T11:21:57.403125+0000","last_became_active":"2024-05-14T11:21:56.379170+0000","last_became_peered":"2024-05-14T11:21:56.379170+0000","last_unstale":"2024-05-14T11:21:57.403125+0000","last_undegraded":"2024-05-14T11:21:57.403125+0000","last_fullsized":"2024-05-14T11:21:57.403125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:45.902619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461559+0000","last_change":"2024-05-14T11:21:55.379532+0000","last_active":"2024-05-14T11:21:57.461559+0000","last_peered":"2024-05-14T11:21:57.461559+0000","last_clean":"2024-05-14T11:21:57.461559+0000","last_became_active":"2024-05-14T11:21:55.379224+0000","last_became_peered":"2024-05-14T11:21:55.379224+0000","last_unstale":"2024-05-14T11:21:57.461559+0000","last_undegraded":"2024-05-14T11:21:57.461559+0000","last_fullsized":"2024-05-14T11:21:57.461559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:23:55.106701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459814+0000","last_change":"2024-05-14T11:21:56.400307+0000","last_active":"2024-05-14T11:21:57.459814+0000","last_peered":"2024-05-14T11:21:57.459814+0000","last_clean":"2024-05-14T11:21:57.459814+0000","last_became_active":"2024-05-14T11:21:56.399940+0000","last_became_peered":"2024-05-14T11:21:56.399940+0000","last_unstale":"2024-05-14T11:21:57.459814+0000","last_undegraded":"2024-05-14T11:21:57.459814+0000","last_fullsized":"2024-05-14T11:21:57.459814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:58:39.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461259+0000","last_change":"2024-05-14T11:21:55.378699+0000","last_active":"2024-05-14T11:21:57.461259+0000","last_peered":"2024-05-14T11:21:57.461259+0000","last_clean":"2024-05-14T11:21:57.461259+0000","last_became_active":"2024-05-14T11:21:55.378372+0000","last_became_peered":"2024-05-14T11:21:55.378372+0000","last_unstale":"2024-05-14T11:21:57.461259+0000","last_undegraded":"2024-05-14T11:21:57.461259+0000","last_fullsized":"2024-05-14T11:21:57.461259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:02:42.121073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459654+0000","last_change":"2024-05-14T11:21:56.380739+0000","last_active":"2024-05-14T11:21:57.459654+0000","last_peered":"2024-05-14T11:21:57.459654+0000","last_clean":"2024-05-14T11:21:57.459654+0000","last_became_active":"2024-05-14T11:21:56.377804+0000","last_became_peered":"2024-05-14T11:21:56.377804+0000","last_unstale":"2024-05-14T11:21:57.459654+0000","last_undegraded":"2024-05-14T11:21:57.459654+0000","last_fullsized":"2024-05-14T11:21:57.459654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386631+0000","last_change":"2024-05-14T11:21:55.380045+0000","last_active":"2024-05-14T11:21:57.386631+0000","last_peered":"2024-05-14T11:21:57.386631+0000","last_clean":"2024-05-14T11:21:57.386631+0000","last_became_active":"2024-05-14T11:21:55.378906+0000","last_became_peered":"2024-05-14T11:21:55.378906+0000","last_unstale":"2024-05-14T11:21:57.386631+0000","last_undegraded":"2024-05-14T11:21:57.386631+0000","last_fullsized":"2024-05-14T11:21:57.386631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:34.974768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460535+0000","last_change":"2024-05-14T11:21:56.388451+0000","last_active":"2024-05-14T11:21:57.460535+0000","last_peered":"2024-05-14T11:21:57.460535+0000","last_clean":"2024-05-14T11:21:57.460535+0000","last_became_active":"2024-05-14T11:21:56.388148+0000","last_became_peered":"2024-05-14T11:21:56.388148+0000","last_unstale":"2024-05-14T11:21:57.460535+0000","last_undegraded":"2024-05-14T11:21:57.460535+0000","last_fullsized":"2024-05-14T11:21:57.460535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460827+0000","last_change":"2024-05-14T11:21:55.379659+0000","last_active":"2024-05-14T11:21:57.460827+0000","last_peered":"2024-05-14T11:21:57.460827+0000","last_clean":"2024-05-14T11:21:57.460827+0000","last_became_active":"2024-05-14T11:21:55.379357+0000","last_became_peered":"2024-05-14T11:21:55.379357+0000","last_unstale":"2024-05-14T11:21:57.460827+0000","last_undegraded":"2024-05-14T11:21:57.460827+0000","last_fullsized":"2024-05-14T11:21:57.460827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:54.423695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380729+0000","last_change":"2024-05-14T11:21:56.390097+0000","last_active":"2024-05-14T11:21:57.380729+0000","last_peered":"2024-05-14T11:21:57.380729+0000","last_clean":"2024-05-14T11:21:57.380729+0000","last_became_active":"2024-05-14T11:21:56.389775+0000","last_became_peered":"2024-05-14T11:21:56.389775+0000","last_unstale":"2024-05-14T11:21:57.380729+0000","last_undegraded":"2024-05-14T11:21:57.380729+0000","last_fullsized":"2024-05-14T11:21:57.380729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:53:24.671464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459571+0000","last_change":"2024-05-14T11:21:55.384436+0000","last_active":"2024-05-14T11:21:57.459571+0000","last_peered":"2024-05-14T11:21:57.459571+0000","last_clean":"2024-05-14T11:21:57.459571+0000","last_became_active":"2024-05-14T11:21:55.384104+0000","last_became_peered":"2024-05-14T11:21:55.384104+0000","last_unstale":"2024-05-14T11:21:57.459571+0000","last_undegraded":"2024-05-14T11:21:57.459571+0000","last_fullsized":"2024-05-14T11:21:57.459571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:42:56.800584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380862+0000","last_change":"2024-05-14T11:21:56.393507+0000","last_active":"2024-05-14T11:21:57.380862+0000","last_peered":"2024-05-14T11:21:57.380862+0000","last_clean":"2024-05-14T11:21:57.380862+0000","last_became_active":"2024-05-14T11:21:56.393227+0000","last_became_peered":"2024-05-14T11:21:56.393227+0000","last_unstale":"2024-05-14T11:21:57.380862+0000","last_undegraded":"2024-05-14T11:21:57.380862+0000","last_fullsized":"2024-05-14T11:21:57.380862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:26.465849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"83'8","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.459792+0000","last_change":"2024-05-14T11:21:55.386065+0000","last_active":"2024-05-14T11:21:57.459792+0000","last_peered":"2024-05-14T11:21:57.459792+0000","last_clean":"2024-05-14T11:21:57.459792+0000","last_became_active":"2024-05-14T11:21:55.385699+0000","last_became_peered":"2024-05-14T11:21:55.385699+0000","last_unstale":"2024-05-14T11:21:57.459792+0000","last_undegraded":"2024-05-14T11:21:57.459792+0000","last_fullsized":"2024-05-14T11:21:57.459792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:34:21.465908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461286+0000","last_change":"2024-05-14T11:21:56.396244+0000","last_active":"2024-05-14T11:21:57.461286+0000","last_peered":"2024-05-14T11:21:57.461286+0000","last_clean":"2024-05-14T11:21:57.461286+0000","last_became_active":"2024-05-14T11:21:56.395934+0000","last_became_peered":"2024-05-14T11:21:56.395934+0000","last_unstale":"2024-05-14T11:21:57.461286+0000","last_undegraded":"2024-05-14T11:21:57.461286+0000","last_fullsized":"2024-05-14T11:21:57.461286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:21:44.571245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387256+0000","last_change":"2024-05-14T11:21:55.376695+0000","last_active":"2024-05-14T11:21:57.387256+0000","last_peered":"2024-05-14T11:21:57.387256+0000","last_clean":"2024-05-14T11:21:57.387256+0000","last_became_active":"2024-05-14T11:21:55.375674+0000","last_became_peered":"2024-05-14T11:21:55.375674+0000","last_unstale":"2024-05-14T11:21:57.387256+0000","last_undegraded":"2024-05-14T11:21:57.387256+0000","last_fullsized":"2024-05-14T11:21:57.387256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:57.676773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386997+0000","last_change":"2024-05-14T11:21:56.394005+0000","last_active":"2024-05-14T11:21:57.386997+0000","last_peered":"2024-05-14T11:21:57.386997+0000","last_clean":"2024-05-14T11:21:57.386997+0000","last_became_active":"2024-05-14T11:21:56.393708+0000","last_became_peered":"2024-05-14T11:21:56.393708+0000","last_unstale":"2024-05-14T11:21:57.386997+0000","last_undegraded":"2024-05-14T11:21:57.386997+0000","last_fullsized":"2024-05-14T11:21:57.386997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:41.928650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387203+0000","last_change":"2024-05-14T11:21:55.378651+0000","last_active":"2024-05-14T11:21:57.387203+0000","last_peered":"2024-05-14T11:21:57.387203+0000","last_clean":"2024-05-14T11:21:57.387203+0000","last_became_active":"2024-05-14T11:21:55.378348+0000","last_became_peered":"2024-05-14T11:21:55.378348+0000","last_unstale":"2024-05-14T11:21:57.387203+0000","last_undegraded":"2024-05-14T11:21:57.387203+0000","last_fullsized":"2024-05-14T11:21:57.387203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:13.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461456+0000","last_change":"2024-05-14T11:21:56.389463+0000","last_active":"2024-05-14T11:21:57.461456+0000","last_peered":"2024-05-14T11:21:57.461456+0000","last_clean":"2024-05-14T11:21:57.461456+0000","last_became_active":"2024-05-14T11:21:56.389014+0000","last_became_peered":"2024-05-14T11:21:56.389014+0000","last_unstale":"2024-05-14T11:21:57.461456+0000","last_undegraded":"2024-05-14T11:21:57.461456+0000","last_fullsized":"2024-05-14T11:21:57.461456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:32.340388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461621+0000","last_change":"2024-05-14T11:21:55.383442+0000","last_active":"2024-05-14T11:21:57.461621+0000","last_peered":"2024-05-14T11:21:57.461621+0000","last_clean":"2024-05-14T11:21:57.461621+0000","last_became_active":"2024-05-14T11:21:55.383197+0000","last_became_peered":"2024-05-14T11:21:55.383197+0000","last_unstale":"2024-05-14T11:21:57.461621+0000","last_undegraded":"2024-05-14T11:21:57.461621+0000","last_fullsized":"2024-05-14T11:21:57.461621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:52:51.947749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460829+0000","last_change":"2024-05-14T11:21:56.397428+0000","last_active":"2024-05-14T11:21:57.460829+0000","last_peered":"2024-05-14T11:21:57.460829+0000","last_clean":"2024-05-14T11:21:57.460829+0000","last_became_active":"2024-05-14T11:21:56.397079+0000","last_became_peered":"2024-05-14T11:21:56.397079+0000","last_unstale":"2024-05-14T11:21:57.460829+0000","last_undegraded":"2024-05-14T11:21:57.460829+0000","last_fullsized":"2024-05-14T11:21:57.460829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:05.256399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460499+0000","last_change":"2024-05-14T11:21:55.381263+0000","last_active":"2024-05-14T11:21:57.460499+0000","last_peered":"2024-05-14T11:21:57.460499+0000","last_clean":"2024-05-14T11:21:57.460499+0000","last_became_active":"2024-05-14T11:21:55.380928+0000","last_became_peered":"2024-05-14T11:21:55.380928+0000","last_unstale":"2024-05-14T11:21:57.460499+0000","last_undegraded":"2024-05-14T11:21:57.460499+0000","last_fullsized":"2024-05-14T11:21:57.460499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:48:34.959395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460695+0000","last_change":"2024-05-14T11:21:56.398290+0000","last_active":"2024-05-14T11:21:57.460695+0000","last_peered":"2024-05-14T11:21:57.460695+0000","last_clean":"2024-05-14T11:21:57.460695+0000","last_became_active":"2024-05-14T11:21:56.397954+0000","last_became_peered":"2024-05-14T11:21:56.397954+0000","last_unstale":"2024-05-14T11:21:57.460695+0000","last_undegraded":"2024-05-14T11:21:57.460695+0000","last_fullsized":"2024-05-14T11:21:57.460695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:28:12.674671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380740+0000","last_change":"2024-05-14T11:21:55.386139+0000","last_active":"2024-05-14T11:21:57.380740+0000","last_peered":"2024-05-14T11:21:57.380740+0000","last_clean":"2024-05-14T11:21:57.380740+0000","last_became_active":"2024-05-14T11:21:55.385435+0000","last_became_peered":"2024-05-14T11:21:55.385435+0000","last_unstale":"2024-05-14T11:21:57.380740+0000","last_undegraded":"2024-05-14T11:21:57.380740+0000","last_fullsized":"2024-05-14T11:21:57.380740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:00.748983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461061+0000","last_change":"2024-05-14T11:21:56.403567+0000","last_active":"2024-05-14T11:21:57.461061+0000","last_peered":"2024-05-14T11:21:57.461061+0000","last_clean":"2024-05-14T11:21:57.461061+0000","last_became_active":"2024-05-14T11:21:56.403222+0000","last_became_peered":"2024-05-14T11:21:56.403222+0000","last_unstale":"2024-05-14T11:21:57.461061+0000","last_undegraded":"2024-05-14T11:21:57.461061+0000","last_fullsized":"2024-05-14T11:21:57.461061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:26.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460664+0000","last_change":"2024-05-14T11:21:55.377847+0000","last_active":"2024-05-14T11:21:57.460664+0000","last_peered":"2024-05-14T11:21:57.460664+0000","last_clean":"2024-05-14T11:21:57.460664+0000","last_became_active":"2024-05-14T11:21:55.377479+0000","last_became_peered":"2024-05-14T11:21:55.377479+0000","last_unstale":"2024-05-14T11:21:57.460664+0000","last_undegraded":"2024-05-14T11:21:57.460664+0000","last_fullsized":"2024-05-14T11:21:57.460664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:40:31.337612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460650+0000","last_change":"2024-05-14T11:21:56.381486+0000","last_active":"2024-05-14T11:21:57.460650+0000","last_peered":"2024-05-14T11:21:57.460650+0000","last_clean":"2024-05-14T11:21:57.460650+0000","last_became_active":"2024-05-14T11:21:56.381191+0000","last_became_peered":"2024-05-14T11:21:56.381191+0000","last_unstale":"2024-05-14T11:21:57.460650+0000","last_undegraded":"2024-05-14T11:21:57.460650+0000","last_fullsized":"2024-05-14T11:21:57.460650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:11:28.255356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380491+0000","last_change":"2024-05-14T11:21:55.380084+0000","last_active":"2024-05-14T11:21:57.380491+0000","last_peered":"2024-05-14T11:21:57.380491+0000","last_clean":"2024-05-14T11:21:57.380491+0000","last_became_active":"2024-05-14T11:21:55.379631+0000","last_became_peered":"2024-05-14T11:21:55.379631+0000","last_unstale":"2024-05-14T11:21:57.380491+0000","last_undegraded":"2024-05-14T11:21:57.380491+0000","last_fullsized":"2024-05-14T11:21:57.380491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:11.076627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387177+0000","last_change":"2024-05-14T11:21:56.380163+0000","last_active":"2024-05-14T11:21:57.387177+0000","last_peered":"2024-05-14T11:21:57.387177+0000","last_clean":"2024-05-14T11:21:57.387177+0000","last_became_active":"2024-05-14T11:21:56.377500+0000","last_became_peered":"2024-05-14T11:21:56.377500+0000","last_unstale":"2024-05-14T11:21:57.387177+0000","last_undegraded":"2024-05-14T11:21:57.387177+0000","last_fullsized":"2024-05-14T11:21:57.387177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:46:40.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387601+0000","last_change":"2024-05-14T11:21:55.382014+0000","last_active":"2024-05-14T11:21:57.387601+0000","last_peered":"2024-05-14T11:21:57.387601+0000","last_clean":"2024-05-14T11:21:57.387601+0000","last_became_active":"2024-05-14T11:21:55.381670+0000","last_became_peered":"2024-05-14T11:21:55.381670+0000","last_unstale":"2024-05-14T11:21:57.387601+0000","last_undegraded":"2024-05-14T11:21:57.387601+0000","last_fullsized":"2024-05-14T11:21:57.387601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:25.098189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461127+0000","last_change":"2024-05-14T11:21:56.397506+0000","last_active":"2024-05-14T11:21:57.461127+0000","last_peered":"2024-05-14T11:21:57.461127+0000","last_clean":"2024-05-14T11:21:57.461127+0000","last_became_active":"2024-05-14T11:21:56.395519+0000","last_became_peered":"2024-05-14T11:21:56.395519+0000","last_unstale":"2024-05-14T11:21:57.461127+0000","last_undegraded":"2024-05-14T11:21:57.461127+0000","last_fullsized":"2024-05-14T11:21:57.461127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:26:54.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461633+0000","last_change":"2024-05-14T11:21:55.375676+0000","last_active":"2024-05-14T11:21:57.461633+0000","last_peered":"2024-05-14T11:21:57.461633+0000","last_clean":"2024-05-14T11:21:57.461633+0000","last_became_active":"2024-05-14T11:21:55.375356+0000","last_became_peered":"2024-05-14T11:21:55.375356+0000","last_unstale":"2024-05-14T11:21:57.461633+0000","last_undegraded":"2024-05-14T11:21:57.461633+0000","last_fullsized":"2024-05-14T11:21:57.461633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:11.395431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461112+0000","last_change":"2024-05-14T11:21:56.393934+0000","last_active":"2024-05-14T11:21:57.461112+0000","last_peered":"2024-05-14T11:21:57.461112+0000","last_clean":"2024-05-14T11:21:57.461112+0000","last_became_active":"2024-05-14T11:21:56.393609+0000","last_became_peered":"2024-05-14T11:21:56.393609+0000","last_unstale":"2024-05-14T11:21:57.461112+0000","last_undegraded":"2024-05-14T11:21:57.461112+0000","last_fullsized":"2024-05-14T11:21:57.461112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:22.581542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461639+0000","last_change":"2024-05-14T11:21:55.379420+0000","last_active":"2024-05-14T11:21:57.461639+0000","last_peered":"2024-05-14T11:21:57.461639+0000","last_clean":"2024-05-14T11:21:57.461639+0000","last_became_active":"2024-05-14T11:21:55.378985+0000","last_became_peered":"2024-05-14T11:21:55.378985+0000","last_unstale":"2024-05-14T11:21:57.461639+0000","last_undegraded":"2024-05-14T11:21:57.461639+0000","last_fullsized":"2024-05-14T11:21:57.461639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:30.781659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.461293+0000","last_change":"2024-05-14T11:21:56.399837+0000","last_active":"2024-05-14T11:21:57.461293+0000","last_peered":"2024-05-14T11:21:57.461293+0000","last_clean":"2024-05-14T11:21:57.461293+0000","last_became_active":"2024-05-14T11:21:56.399430+0000","last_became_peered":"2024-05-14T11:21:56.399430+0000","last_unstale":"2024-05-14T11:21:57.461293+0000","last_undegraded":"2024-05-14T11:21:57.461293+0000","last_fullsized":"2024-05-14T11:21:57.461293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:55.426472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387442+0000","last_change":"2024-05-14T11:21:55.374216+0000","last_active":"2024-05-14T11:21:57.387442+0000","last_peered":"2024-05-14T11:21:57.387442+0000","last_clean":"2024-05-14T11:21:57.387442+0000","last_became_active":"2024-05-14T11:21:55.373713+0000","last_became_peered":"2024-05-14T11:21:55.373713+0000","last_unstale":"2024-05-14T11:21:57.387442+0000","last_undegraded":"2024-05-14T11:21:57.387442+0000","last_fullsized":"2024-05-14T11:21:57.387442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:32:45.264705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.380622+0000","last_change":"2024-05-14T11:21:56.383298+0000","last_active":"2024-05-14T11:21:57.380622+0000","last_peered":"2024-05-14T11:21:57.380622+0000","last_clean":"2024-05-14T11:21:57.380622+0000","last_became_active":"2024-05-14T11:21:56.381852+0000","last_became_peered":"2024-05-14T11:21:56.381852+0000","last_unstale":"2024-05-14T11:21:57.380622+0000","last_undegraded":"2024-05-14T11:21:57.380622+0000","last_fullsized":"2024-05-14T11:21:57.380622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:45:25.027740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460760+0000","last_change":"2024-05-14T11:21:55.379545+0000","last_active":"2024-05-14T11:21:57.460760+0000","last_peered":"2024-05-14T11:21:57.460760+0000","last_clean":"2024-05-14T11:21:57.460760+0000","last_became_active":"2024-05-14T11:21:55.378762+0000","last_became_peered":"2024-05-14T11:21:55.378762+0000","last_unstale":"2024-05-14T11:21:57.460760+0000","last_undegraded":"2024-05-14T11:21:57.460760+0000","last_fullsized":"2024-05-14T11:21:57.460760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:16:44.899842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.402884+0000","last_change":"2024-05-14T11:21:56.384970+0000","last_active":"2024-05-14T11:21:57.402884+0000","last_peered":"2024-05-14T11:21:57.402884+0000","last_clean":"2024-05-14T11:21:57.402884+0000","last_became_active":"2024-05-14T11:21:56.384703+0000","last_became_peered":"2024-05-14T11:21:56.384703+0000","last_unstale":"2024-05-14T11:21:57.402884+0000","last_undegraded":"2024-05-14T11:21:57.402884+0000","last_fullsized":"2024-05-14T11:21:57.402884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:14.258034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460851+0000","last_change":"2024-05-14T11:21:55.381768+0000","last_active":"2024-05-14T11:21:57.460851+0000","last_peered":"2024-05-14T11:21:57.460851+0000","last_clean":"2024-05-14T11:21:57.460851+0000","last_became_active":"2024-05-14T11:21:55.381485+0000","last_became_peered":"2024-05-14T11:21:55.381485+0000","last_unstale":"2024-05-14T11:21:57.460851+0000","last_undegraded":"2024-05-14T11:21:57.460851+0000","last_fullsized":"2024-05-14T11:21:57.460851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:39:35.207793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386457+0000","last_change":"2024-05-14T11:21:56.392100+0000","last_active":"2024-05-14T11:21:57.386457+0000","last_peered":"2024-05-14T11:21:57.386457+0000","last_clean":"2024-05-14T11:21:57.386457+0000","last_became_active":"2024-05-14T11:21:56.391699+0000","last_became_peered":"2024-05-14T11:21:56.391699+0000","last_unstale":"2024-05-14T11:21:57.386457+0000","last_undegraded":"2024-05-14T11:21:57.386457+0000","last_fullsized":"2024-05-14T11:21:57.386457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:57:19.772221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387086+0000","last_change":"2024-05-14T11:21:55.373928+0000","last_active":"2024-05-14T11:21:57.387086+0000","last_peered":"2024-05-14T11:21:57.387086+0000","last_clean":"2024-05-14T11:21:57.387086+0000","last_became_active":"2024-05-14T11:21:55.373638+0000","last_became_peered":"2024-05-14T11:21:55.373638+0000","last_unstale":"2024-05-14T11:21:57.387086+0000","last_undegraded":"2024-05-14T11:21:57.387086+0000","last_fullsized":"2024-05-14T11:21:57.387086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:35.515189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460703+0000","last_change":"2024-05-14T11:21:56.395619+0000","last_active":"2024-05-14T11:21:57.460703+0000","last_peered":"2024-05-14T11:21:57.460703+0000","last_clean":"2024-05-14T11:21:57.460703+0000","last_became_active":"2024-05-14T11:21:56.394829+0000","last_became_peered":"2024-05-14T11:21:56.394829+0000","last_unstale":"2024-05-14T11:21:57.460703+0000","last_undegraded":"2024-05-14T11:21:57.460703+0000","last_fullsized":"2024-05-14T11:21:57.460703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:36:28.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386239+0000","last_change":"2024-05-14T11:21:55.387084+0000","last_active":"2024-05-14T11:21:57.386239+0000","last_peered":"2024-05-14T11:21:57.386239+0000","last_clean":"2024-05-14T11:21:57.386239+0000","last_became_active":"2024-05-14T11:21:55.386834+0000","last_became_peered":"2024-05-14T11:21:55.386834+0000","last_unstale":"2024-05-14T11:21:57.386239+0000","last_undegraded":"2024-05-14T11:21:57.386239+0000","last_fullsized":"2024-05-14T11:21:57.386239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:17:10.801403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.387496+0000","last_change":"2024-05-14T11:21:56.387037+0000","last_active":"2024-05-14T11:21:57.387496+0000","last_peered":"2024-05-14T11:21:57.387496+0000","last_clean":"2024-05-14T11:21:57.387496+0000","last_became_active":"2024-05-14T11:21:56.386496+0000","last_became_peered":"2024-05-14T11:21:56.386496+0000","last_unstale":"2024-05-14T11:21:57.387496+0000","last_undegraded":"2024-05-14T11:21:57.387496+0000","last_fullsized":"2024-05-14T11:21:57.387496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:55.349508+0000","last_clean_scrub_stamp":"2024-05-14T11:21:55.349508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:27:21.878567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.460199+0000","last_change":"2024-05-14T11:21:55.387153+0000","last_active":"2024-05-14T11:21:57.460199+0000","last_peered":"2024-05-14T11:21:57.460199+0000","last_clean":"2024-05-14T11:21:57.460199+0000","last_became_active":"2024-05-14T11:21:55.386815+0000","last_became_peered":"2024-05-14T11:21:55.386815+0000","last_unstale":"2024-05-14T11:21:57.460199+0000","last_undegraded":"2024-05-14T11:21:57.460199+0000","last_fullsized":"2024-05-14T11:21:57.460199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:21:54.346731+0000","last_clean_scrub_stamp":"2024-05-14T11:21:54.346731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:44.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-14T11:21:57.386383+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:21:57.386383+0000","last_peered":"2024-05-14T11:21:57.386383+0000","last_clean":"2024-05-14T11:21:57.386383+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:21:57.386383+0000","last_undegraded":"2024-05-14T11:21:57.386383+0000","last_fullsized":"2024-05-14T11:21:57.386383+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":1064313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":476105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":473889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":473959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1066575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:22:05.498 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:22:05.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 Processor -- start 2024-05-14T11:22:05.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 -- start start 2024-05-14T11:22:05.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd83406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83406f860 con 0x7fd8340d1730 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83406fae0 con 0x7fd8340cde70 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd83a000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd83406db80 con 0x7fd83406f450 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd8340d1730 0x7fd8340d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37156/0 (socket says 172.21.15.116:37156) 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832a00640 1 -- 172.21.15.116:0/1587550267 learned_addr learned my addr 172.21.15.116:0/1587550267 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd833400640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd83406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3719170723 0 0) 0x7fd820001120 con 0x7fd8340d1730 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd818003530 con 0x7fd8340d1730 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572185264 0 0) 0x7fd83406fae0 con 0x7fd8340cde70 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd820001120 con 0x7fd8340cde70 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 860574085 0 0) 0x7fd818003530 con 0x7fd8340d1730 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 msgr2=0x7fd83406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd83406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 >> v1:172.21.15.149:6789/0 conn(0x7fd8340cde70 legacy=0x7fd8341675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83406fdc0 con 0x7fd8340d1730 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fd8200018d0 con 0x7fd8340d1730 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 >> v1:172.21.15.116:6789/0 conn(0x7fd8340d1730 legacy=0x7fd8340d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd818003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd818004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.657+0000 7fd832000640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd818004da0 0x7fd818005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd83406db80 con 0x7fd8180041d0 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd81800bc30 con 0x7fd818004da0 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd81800be30 con 0x7fd83406f450 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd818004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd833400640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd818003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd833e00640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd818004da0 0x7fd818005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 -- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 msgr2=0x7fd818003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd83406f450 0x7fd818003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 -- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd818004da0 msgr2=0x7fd818005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd818004da0 0x7fd818005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 -- 172.21.15.116:0/1587550267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd818003530 con 0x7fd8180041d0 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832a00640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd818004810 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd82000c490 tx=0x7fd82001cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecfa58809bccf58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82000c0e0 con 0x7fd8180041d0 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200a9070 con 0x7fd8180041d0 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.661+0000 7fd832000640 1 -- 172.21.15.116:0/1587550267 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820005d40 con 0x7fd8180041d0 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- 172.21.15.116:0/1587550267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 msgr2=0x7fd818004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 --2- 172.21.15.116:0/1587550267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd818004810 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd82000c490 tx=0x7fd82001cae0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd833e00640 1 -- 172.21.15.116:0/1587550267 reap_dead start 2024-05-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- 172.21.15.116:0/1587550267 shutdown_connections 2024-05-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- 172.21.15.116:0/1587550267 >> 172.21.15.116:0/1587550267 conn(0x7fd8340b4720 msgr2=0x7fd8340b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- 172.21.15.116:0/1587550267 shutdown_connections 2024-05-14T11:22:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- 172.21.15.116:0/1587550267 wait complete. 2024-05-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 Processor -- start 2024-05-14T11:22:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.665+0000 7fd83a000640 1 -- start start 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd834156cc0 0x7fd83415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 0x7fd834160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd834070660 con 0x7fd8180041d0 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd834070380 con 0x7fd8341570d0 2024-05-14T11:22:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd83406d640 con 0x7fd834156cc0 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42582/0 (socket says 172.21.15.116:42582) 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 -- 172.21.15.116:0/701262371 learned_addr learned my addr 172.21.15.116:0/701262371 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd832a00640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd834156cc0 0x7fd83415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833e00640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 0x7fd834160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 -- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd834156cc0 msgr2=0x7fd83415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd834156cc0 0x7fd83415fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 -- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 msgr2=0x7fd834160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 0x7fd834160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 -- 172.21.15.116:0/701262371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83415e460 con 0x7fd8180041d0 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833e00640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 0x7fd834160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd833400640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd824011930 tx=0x7fd824011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400fa70 con 0x7fd8180041d0 2024-05-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82400fc10 con 0x7fd8180041d0 2024-05-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341608f0 con 0x7fd8180041d0 2024-05-14T11:22:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824014440 con 0x7fd8180041d0 2024-05-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd834154bf0 con 0x7fd8180041d0 2024-05-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.669+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83406a9c0 con 0x7fd8180041d0 2024-05-14T11:22:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.673+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd824012040 con 0x7fd8180041d0 2024-05-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.673+0000 7fd82b400640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 0x7fd814071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.673+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fd82408fa80 con 0x7fd8180041d0 2024-05-14T11:22:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.673+0000 7fd832a00640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 0x7fd814071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.673+0000 7fd832a00640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 0x7fd814071940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8200ab000 tx=0x7fd8200b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:05.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.677+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd824091e20 con 0x7fd8180041d0 2024-05-14T11:22:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.893+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd834069de0 con 0x7fd8180041d0 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.893+0000 7fd82b400640 1 -- 172.21.15.116:0/701262371 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fd824019030 con 0x7fd8180041d0 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:22:05.895 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:22:05.896 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 msgr2=0x7fd814071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 0x7fd814071940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8200ab000 tx=0x7fd8200b1040 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 msgr2=0x7fd83415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd824011930 tx=0x7fd824011df0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 shutdown_connections 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd81406f430 0x7fd814071940 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8341570d0 0x7fd834160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd834156cc0 0x7fd83415fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 --2- 172.21.15.116:0/701262371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8180041d0 0x7fd83415df20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 >> 172.21.15.116:0/701262371 conn(0x7fd8340b4720 msgr2=0x7fd8340c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:05.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 shutdown_connections 2024-05-14T11:22:05.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:05.897+0000 7fd83a000640 1 -- 172.21.15.116:0/701262371 wait complete. 2024-05-14T11:22:05.918 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:06.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 Processor -- start 2024-05-14T11:22:06.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 -- start start 2024-05-14T11:22:06.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4a58070610 0x7f4a58160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a580b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a5806f5c0 con 0x7f4a58070130 2024-05-14T11:22:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4a58161200 con 0x7f4a58070610 2024-05-14T11:22:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a5d200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a58161450 con 0x7f4a580b54b0 2024-05-14T11:22:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a55e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4a58070130 0x7f4a581605b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37166/0 (socket says 172.21.15.116:37166) 2024-05-14T11:22:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a55e00640 1 -- 172.21.15.116:0/3272249734 learned_addr learned my addr 172.21.15.116:0/3272249734 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a56800640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a580b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a4d400640 1 --2- 172.21.15.116:0/3272249734 >> v2:172.21.15.149:3300/0 conn(0x7f4a58070610 0x7f4a58160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.065+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709621825 0 0) 0x7f4a5806f5c0 con 0x7f4a58070130 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a3c003530 con 0x7f4a58070130 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4134141719 0 0) 0x7f4a3c003530 con 0x7f4a58070130 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 msgr2=0x7f4a580b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a580b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 >> v2:172.21.15.149:3300/0 conn(0x7f4a58070610 msgr2=0x7f4a58160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 --2- 172.21.15.116:0/3272249734 >> v2:172.21.15.149:3300/0 conn(0x7f4a58070610 0x7f4a58160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a580d1fb0 con 0x7f4a58070130 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f4a48001700 con 0x7f4a58070130 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 >> v1:172.21.15.116:6789/0 conn(0x7f4a58070130 legacy=0x7f4a581605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a3c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a3c004d50 0x7f4a3c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58161200 con 0x7f4a3c003fe0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58161450 con 0x7f4a3c004d50 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c00bad0 con 0x7f4a580b54b0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a55e00640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a3c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a56800640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a3c004d50 0x7f4a3c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 -- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 msgr2=0x7f4a3c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a580b54b0 0x7f4a3c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 -- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a3c004d50 msgr2=0x7f4a3c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a3c004d50 0x7f4a3c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a4d400640 1 -- 172.21.15.116:0/3272249734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c003530 con 0x7f4a3c003fe0 2024-05-14T11:22:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.069+0000 7f4a56800640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a3c004d50 0x7f4a3c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a4d400640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a3c004810 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4a58073a60 tx=0x7f4a4001c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d676b95ebf309ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400b1070 con 0x7f4a3c003fe0 2024-05-14T11:22:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400a4a00 con 0x7f4a3c003fe0 2024-05-14T11:22:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a55400640 1 -- 172.21.15.116:0/3272249734 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400096b0 con 0x7f4a3c003fe0 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- 172.21.15.116:0/3272249734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 msgr2=0x7f4a3c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 --2- 172.21.15.116:0/3272249734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a3c004810 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4a58073a60 tx=0x7f4a4001c6f0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a56800640 1 -- 172.21.15.116:0/3272249734 reap_dead start 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- 172.21.15.116:0/3272249734 shutdown_connections 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- 172.21.15.116:0/3272249734 >> 172.21.15.116:0/3272249734 conn(0x7f4a58071370 msgr2=0x7f4a58071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- 172.21.15.116:0/3272249734 shutdown_connections 2024-05-14T11:22:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- 172.21.15.116:0/3272249734 wait complete. 2024-05-14T11:22:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 Processor -- start 2024-05-14T11:22:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.073+0000 7f4a5d200640 1 -- start start 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 0x7f4a581e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 0x7f4a580cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a580b6260 con 0x7f4a3c003fe0 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a580b60e0 con 0x7f4a581e9f50 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4a580b63e0 con 0x7f4a58147b40 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42602/0 (socket says 172.21.15.116:42602) 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 -- 172.21.15.116:0/2111421521 learned_addr learned my addr 172.21.15.116:0/2111421521 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a56800640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 0x7f4a580cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a4d400640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 0x7f4a581e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 -- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 msgr2=0x7f4a581e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 0x7f4a581e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 -- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 msgr2=0x7f4a580cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 0x7f4a580cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 -- 172.21.15.116:0/2111421521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581ea360 con 0x7f4a3c003fe0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a4d400640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 0x7f4a581e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a56800640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 0x7f4a580cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a55e00640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4a48012490 tx=0x7f4a48012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4801c070 con 0x7f4a3c003fe0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a48014810 con 0x7f4a3c003fe0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48014c00 con 0x7f4a3c003fe0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a580c7e00 con 0x7f4a3c003fe0 2024-05-14T11:22:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a580ccf80 con 0x7f4a3c003fe0 2024-05-14T11:22:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.077+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5806a9c0 con 0x7f4a3c003fe0 2024-05-14T11:22:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.081+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a480153e0 con 0x7f4a3c003fe0 2024-05-14T11:22:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.081+0000 7f4a4ea00640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.081+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f4a48091e70 con 0x7f4a3c003fe0 2024-05-14T11:22:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.081+0000 7f4a4d400640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.081+0000 7f4a4d400640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 0x7f4a34071910 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a400ae000 tx=0x7f4a400046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.085+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4a581ea360 con 0x7f4a3c003fe0 2024-05-14T11:22:06.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.273+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a580c8d30 con 0x7f4a3c003fe0 2024-05-14T11:22:06.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.273+0000 7f4a4ea00640 1 -- 172.21.15.116:0/2111421521 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5189 (secure 0 0 0) 0x7f4a4805ec50 con 0x7f4a3c003fe0 2024-05-14T11:22:06.274 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:06.274 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:21:58.770840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6835/536312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":536312712},{"type":"v1","addr":"172.21.15.149:6835","nonce":536312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":6}]} 2024-05-14T11:22:06.274 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 61 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 msgr2=0x7f4a34071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 0x7f4a34071910 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a400ae000 tx=0x7f4a400046d0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 msgr2=0x7f4a580c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4a48012490 tx=0x7f4a48012950 comp rx=0 tx=0).stop 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 shutdown_connections 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a581e9f50 0x7f4a580cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4a58147b40 0x7f4a581e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 --2- 172.21.15.116:0/2111421521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a3c003fe0 0x7f4a580c7420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 >> 172.21.15.116:0/2111421521 conn(0x7f4a58071370 msgr2=0x7f4a5806d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 shutdown_connections 2024-05-14T11:22:06.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.277+0000 7f4a5d200640 1 -- 172.21.15.116:0/2111421521 wait complete. 2024-05-14T11:22:06.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:21:58.770840+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6825}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6825': {'gid': 6825, 'name': 'b', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6835/536312712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 536312712}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 536312712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6825, 'qdb_cluster': [6825]} 2024-05-14T11:22:06.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:22:06.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-14T11:22:06.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 Processor -- start 2024-05-14T11:22:06.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 -- start start 2024-05-14T11:22:06.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9034163fc0 0x7f9034156890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90341603f0 0x7f9034160800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90340b05d0 con 0x7f90340b50d0 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f90340b08b0 con 0x7f90341603f0 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903aa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90340b0b90 con 0x7f9034163fc0 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903a000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90341603f0 0x7f9034160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903a000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90341603f0 0x7f9034160800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42618/0 (socket says 172.21.15.116:42618) 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f903a000640 1 -- 172.21.15.116:0/4000136714 learned_addr learned my addr 172.21.15.116:0/4000136714 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.849+0000 7f9039600640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9034163fc0 0x7f9034156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903a000640 1 -- 172.21.15.116:0/4000136714 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9034163fc0 msgr2=0x7f9034156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903a000640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9034163fc0 0x7f9034156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903a000640 1 -- 172.21.15.116:0/4000136714 >> v1:172.21.15.149:6789/0 conn(0x7f90340b50d0 legacy=0x7f90340b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903a000640 1 -- 172.21.15.116:0/4000136714 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90340b0e70 con 0x7f90341603f0 2024-05-14T11:22:06.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903a000640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90341603f0 0x7f9034160800 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f902c009e50 tx=0x7f902c01d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0cb0eaf836da36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f9033e00640 1 -- 172.21.15.116:0/4000136714 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0a5770 con 0x7f90341603f0 2024-05-14T11:22:06.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f9033e00640 1 -- 172.21.15.116:0/4000136714 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c0a5910 con 0x7f90341603f0 2024-05-14T11:22:06.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f9033e00640 1 -- 172.21.15.116:0/4000136714 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0a5ce0 con 0x7f90341603f0 2024-05-14T11:22:06.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903aa00640 1 -- 172.21.15.116:0/4000136714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90341603f0 msgr2=0x7f9034160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.853+0000 7f903aa00640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90341603f0 0x7f9034160800 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f902c009e50 tx=0x7f902c01d400 comp rx=0 tx=0).stop 2024-05-14T11:22:06.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- 172.21.15.116:0/4000136714 shutdown_connections 2024-05-14T11:22:06.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90341603f0 0x7f9034160800 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 --2- 172.21.15.116:0/4000136714 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9034163fc0 0x7f9034156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- 172.21.15.116:0/4000136714 >> 172.21.15.116:0/4000136714 conn(0x7f90340b3e90 msgr2=0x7f90340d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- 172.21.15.116:0/4000136714 shutdown_connections 2024-05-14T11:22:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- 172.21.15.116:0/4000136714 wait complete. 2024-05-14T11:22:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 Processor -- start 2024-05-14T11:22:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- start start 2024-05-14T11:22:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90340b50d0 0x7f9034150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 0x7f90341456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90340af480 con 0x7f9034163fc0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9034160f50 con 0x7f90341603f0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.857+0000 7f903aa00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90340af600 con 0x7f90340b50d0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42634/0 (socket says 172.21.15.116:42634) 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 -- 172.21.15.116:0/1253551055 learned_addr learned my addr 172.21.15.116:0/1253551055 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9039600640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90340b50d0 0x7f9034150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9038c00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 0x7f90341456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 -- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90340b50d0 msgr2=0x7f9034150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90340b50d0 0x7f9034150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 -- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 msgr2=0x7f90341456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 0x7f90341456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 -- 172.21.15.116:0/1253551055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9034145be0 con 0x7f9034163fc0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9038c00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 0x7f90341456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903a000640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f902c0af040 tx=0x7f902c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9031600640 1 -- 172.21.15.116:0/1253551055 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0b8470 con 0x7f9034163fc0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9031600640 1 -- 172.21.15.116:0/1253551055 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c0b8610 con 0x7f9034163fc0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9031600640 1 -- 172.21.15.116:0/1253551055 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0b8a00 con 0x7f9034163fc0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9034146960 con 0x7f9034163fc0 2024-05-14T11:22:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9034146f20 con 0x7f9034163fc0 2024-05-14T11:22:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.861+0000 7f9031600640 1 -- 172.21.15.116:0/1253551055 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f902c0c0020 con 0x7f9034163fc0 2024-05-14T11:22:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9031600640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 0x7f9014071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9031600640 1 -- 172.21.15.116:0/1253551055 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f902c129bf0 con 0x7f9034163fc0 2024-05-14T11:22:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9039600640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 0x7f9014071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 Processor -- start 2024-05-14T11:22:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 -- start start 2024-05-14T11:22:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9039600640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 0x7f9014071910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9028009ac0 tx=0x7f9028009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d12e7c0 0x55650d668320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d668da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55650d505620 con 0x55650d516f80 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650d420740 con 0x55650d12e7c0 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55650d46e090 con 0x55650d5006b0 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42650/0 (socket says 172.21.15.116:42650) 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 -- 172.21.15.116:0/378291135 learned_addr learned my addr 172.21.15.116:0/378291135 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f903a000640 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d668da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9039600640 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d12e7c0 0x55650d668320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 -- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 msgr2=0x55650d668da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d668da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 -- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d12e7c0 msgr2=0x55650d668320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d12e7c0 0x55650d668320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 -- 172.21.15.116:0/378291135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650d6020a0 con 0x55650d516f80 2024-05-14T11:22:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.865+0000 7f9038c00640 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9020001530 tx=0x7f90200019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36ed93a773775d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f9025c00640 1 -- 172.21.15.116:0/378291135 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020019070 con 0x55650d516f80 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f9025c00640 1 -- 172.21.15.116:0/378291135 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90200052e0 con 0x55650d516f80 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f9025c00640 1 -- 172.21.15.116:0/378291135 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200069e0 con 0x55650d516f80 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- 172.21.15.116:0/378291135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 msgr2=0x55650d668860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9020001530 tx=0x7f90200019f0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- 172.21.15.116:0/378291135 shutdown_connections 2024-05-14T11:22:06.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d668da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d668860 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- 172.21.15.116:0/378291135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d12e7c0 0x55650d668320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- 172.21.15.116:0/378291135 >> 172.21.15.116:0/378291135 conn(0x55650d5ed340 msgr2=0x55650d61cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- 172.21.15.116:0/378291135 shutdown_connections 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- 172.21.15.116:0/378291135 wait complete. 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- start start 2024-05-14T11:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 Processor -- start 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d12e7c0 0x55650d623340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d5006b0 0x55650d623880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55650d46a050 con 0x55650d516f80 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650d426ec0 con 0x55650d5006b0 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55650d40dc90 con 0x55650d12e7c0 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.869+0000 7f904364b000 10 client.0 _ll_register_callbacks cb 0x55650d609e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42664/0 (socket says 172.21.15.116:42664) 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 -- 172.21.15.116:0/3003678337 learned_addr learned my addr 172.21.15.116:0/3003678337 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f9039600640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d12e7c0 0x55650d623340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f9038c00640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d5006b0 0x55650d623880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 -- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d12e7c0 msgr2=0x55650d623340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d12e7c0 0x55650d623340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 -- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d5006b0 msgr2=0x55650d623880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d5006b0 0x55650d623880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 -- 172.21.15.116:0/3003678337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650d5186f0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f903a000640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f902c127ef0 tx=0x7f902c0f6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c1ca51568ce76c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c0f5cd0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90200052e0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f902c0b8ba0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f902c009590 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 10 client.6984 fetch_fsmap learned FSMap version 61 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55650d5146a0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90200052e0 con 0x55650d516f80 2024-05-14T11:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f902c0f5420 con 0x55650d516f80 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 10 client.6984 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 10 client.6984 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 20 client.6984 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d1137d0 0x55650d6798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 4 client.6984 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 --> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650d406be0 con 0x55650d1137d0 2024-05-14T11:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f9039600640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d1137d0 0x55650d6798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 10 client.6984 ms_handle_connect on v2:172.21.15.149:6833/536312712 2024-05-14T11:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f9039600640 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d1137d0 0x55650d6798e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243ee6b22a3c6dee server_cookie=d0b5d2025f0962cc in_seq=0 out_seq=0 2024-05-14T11:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 1 -- 172.21.15.116:0/3003678337 <== mds.0 v2:172.21.15.149:6833/536312712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55650d406be0 con 0x55650d1137d0 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f901a000640 10 client.6984 handle_command_reply: tid=0 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 client.6984 shutdown 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 msgr2=0x55650d64d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.873+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f902c127ef0 tx=0x7f902c0f6790 comp rx=0 tx=0).stop 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 shutdown_connections 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d1137d0 0x55650d6798e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d5006b0 0x55650d623880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d516f80 0x55650d64d770 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f901a000640 0 client.6984 ms_handle_reset on v2:172.21.15.149:6833/536312712 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 --2- 172.21.15.116:0/3003678337 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d12e7c0 0x55650d623340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 >> 172.21.15.116:0/3003678337 conn(0x55650d5ed340 msgr2=0x55650d61d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 shutdown_connections 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 1 -- 172.21.15.116:0/3003678337 wait complete. 2024-05-14T11:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.877+0000 7f904364b000 20 client.6984 trim_cache size 0 max 16384 2024-05-14T11:22:06.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 Processor -- start 2024-05-14T11:22:06.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 -- start start 2024-05-14T11:22:06.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d64d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d61d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9034152900 con 0x55650d12e7c0 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f903414f7a0 con 0x55650d516f80 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.893+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55650d4117d0 con 0x55650d5006b0 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42668/0 (socket says 172.21.15.116:42668) 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 -- 172.21.15.116:0/875641744 learned_addr learned my addr 172.21.15.116:0/875641744 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9038c00640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d64d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f903a000640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d61d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 -- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 msgr2=0x55650d64d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d64d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 -- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 msgr2=0x55650d61d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d61d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 -- 172.21.15.116:0/875641744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650d447af0 con 0x55650d12e7c0 2024-05-14T11:22:06.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f903a000640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d61d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:06.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9039600640 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x55650d437c90 tx=0x7f9028016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ba5d2ec59178b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9025200640 1 -- 172.21.15.116:0/875641744 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028011e30 con 0x55650d12e7c0 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9025200640 1 -- 172.21.15.116:0/875641744 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9028030070 con 0x55650d12e7c0 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f9025200640 1 -- 172.21.15.116:0/875641744 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028003d10 con 0x55650d12e7c0 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- 172.21.15.116:0/875641744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 msgr2=0x55650d64ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x55650d437c90 tx=0x7f9028016000 comp rx=0 tx=0).stop 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- 172.21.15.116:0/875641744 shutdown_connections 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d61d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d64d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 --2- 172.21.15.116:0/875641744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d64ce30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- 172.21.15.116:0/875641744 >> 172.21.15.116:0/875641744 conn(0x55650d5ed340 msgr2=0x55650d64c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- 172.21.15.116:0/875641744 shutdown_connections 2024-05-14T11:22:06.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- 172.21.15.116:0/875641744 wait complete. 2024-05-14T11:22:06.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 -- start start 2024-05-14T11:22:06.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.897+0000 7f904364b000 1 Processor -- start 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d6db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d6957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55650d46cf80 con 0x55650d12e7c0 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55650d46a050 con 0x55650d516f80 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55650d528d80 con 0x55650d5006b0 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 10 client.0 _ll_register_callbacks cb 0x7f902c0f2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42672/0 (socket says 172.21.15.116:42672) 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 -- 172.21.15.116:0/1173573155 learned_addr learned my addr 172.21.15.116:0/1173573155 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9038c00640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d6db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f903a000640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d6957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 -- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 msgr2=0x55650d6db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d6db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 -- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 msgr2=0x55650d6957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d6957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 -- 172.21.15.116:0/1173573155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650d446e40 con 0x55650d12e7c0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f903a000640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d6957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9039600640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9028009540 tx=0x7f9028004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7c2a669997ba2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028005db0 con 0x55650d12e7c0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f902802b2c0 con 0x55650d12e7c0 2024-05-14T11:22:06.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9028030070 con 0x55650d12e7c0 2024-05-14T11:22:06.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9028045050 con 0x55650d12e7c0 2024-05-14T11:22:06.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 10 client.6990 fetch_fsmap learned FSMap version 61 2024-05-14T11:22:06.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55650d4ee4e0 con 0x55650d12e7c0 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9028030070 con 0x55650d12e7c0 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f9028048020 con 0x55650d12e7c0 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 10 client.6990 fetch_fsmap finished waiting for FSMap version 61 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 10 client.6990 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 20 client.6990 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d101830 0x55650d6e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 4 client.6990 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 --> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] -- command(tid 0: {"prefix": "status"}) -- 0x55650d47bca0 con 0x55650d101830 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9038c00640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d101830 0x55650d6e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f9038c00640 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d101830 0x55650d6e39e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8793556997c458 server_cookie=644c6da79a36180b in_seq=0 out_seq=0 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.901+0000 7f901a000640 10 client.6990 ms_handle_connect on v2:172.21.15.149:6833/536312712 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f901a000640 1 -- 172.21.15.116:0/1173573155 <== mds.0 v2:172.21.15.149:6833/536312712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55650d47bca0 con 0x55650d101830 2024-05-14T11:22:06.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f901a000640 10 client.6990 handle_command_reply: tid=0 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 client.6990 shutdown 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 msgr2=0x55650d6dab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9028009540 tx=0x7f9028004740 comp rx=0 tx=0).stop 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 shutdown_connections 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:6833/536312712,v1:172.21.15.149:6835/536312712] conn(0x55650d101830 0x55650d6e39e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55650d516f80 0x55650d6957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55650d5006b0 0x55650d6db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 --2- 172.21.15.116:0/1173573155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55650d12e7c0 0x55650d6dab70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f901a000640 0 client.6990 ms_handle_reset on v2:172.21.15.149:6833/536312712 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 >> 172.21.15.116:0/1173573155 conn(0x55650d5ed340 msgr2=0x55650d674ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 shutdown_connections 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 1 -- 172.21.15.116:0/1173573155 wait complete. 2024-05-14T11:22:06.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f904364b000 20 client.6990 trim_cache size 0 max 16384 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6825, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 10.546500203000001, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 61, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 84, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 84, 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 15.359899144 2024-05-14T11:22:06.907 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:06.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 msgr2=0x7f9014071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 0x7f9014071910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9028009ac0 tx=0x7f9028009190 comp rx=0 tx=0).stop 2024-05-14T11:22:06.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 msgr2=0x7f903414d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f902c0af040 tx=0x7f902c019740 comp rx=0 tx=0).stop 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 shutdown_connections 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f901406f400 0x7f9014071910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90341603f0 0x7f90341456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9034163fc0 0x7f903414d290 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 --2- 172.21.15.116:0/1253551055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90340b50d0 0x7f9034150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 >> 172.21.15.116:0/1253551055 conn(0x7f90340b3e90 msgr2=0x7f9034070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 shutdown_connections 2024-05-14T11:22:06.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:06.905+0000 7f903aa00640 1 -- 172.21.15.116:0/1253551055 wait complete. 2024-05-14T11:22:06.923 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:22:07.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.069+0000 7f3c25c00640 1 Processor -- start 2024-05-14T11:22:07.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.069+0000 7f3c25c00640 1 -- start start 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c25c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3c2006f730 0x7f3c2006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c25c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c20070080 con 0x7f3c200d1670 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c25c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2006cf10 con 0x7f3c200b0770 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c25c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3c20073b40 con 0x7f3c2006f730 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c1f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3c200d1670 0x7f3c20166f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37176/0 (socket says 172.21.15.116:37176) 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c1f400640 1 -- 172.21.15.116:0/3440646398 learned_addr learned my addr 172.21.15.116:0/3440646398 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c16000640 1 --2- 172.21.15.116:0/3440646398 >> v2:172.21.15.149:3300/0 conn(0x7f3c2006f730 0x7f3c2006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c1e000640 1 -- 172.21.15.116:0/3440646398 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3266895539 0 0) 0x7f3c180014a0 con 0x7f3c200d1670 2024-05-14T11:22:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c1e000640 1 -- 172.21.15.116:0/3440646398 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c04003530 con 0x7f3c200d1670 2024-05-14T11:22:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c16000640 1 -- 172.21.15.116:0/3440646398 >> v1:172.21.15.149:6789/0 conn(0x7f3c200b0770 legacy=0x7f3c2006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c16000640 1 -- 172.21.15.116:0/3440646398 >> v1:172.21.15.116:6789/0 conn(0x7f3c200d1670 legacy=0x7f3c20166f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.073+0000 7f3c16000640 1 -- 172.21.15.116:0/3440646398 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2006d1f0 con 0x7f3c2006f730 2024-05-14T11:22:07.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c16000640 1 --2- 172.21.15.116:0/3440646398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2006f730 0x7f3c2006fb40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c080097c0 tx=0x7f3c0801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a315a28d3671d83b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c1e000640 1 -- 172.21.15.116:0/3440646398 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080b0070 con 0x7f3c2006f730 2024-05-14T11:22:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c1e000640 1 -- 172.21.15.116:0/3440646398 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c080ab600 con 0x7f3c2006f730 2024-05-14T11:22:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c1e000640 1 -- 172.21.15.116:0/3440646398 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080ab9d0 con 0x7f3c2006f730 2024-05-14T11:22:07.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 -- 172.21.15.116:0/3440646398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2006f730 msgr2=0x7f3c2006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 --2- 172.21.15.116:0/3440646398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2006f730 0x7f3c2006fb40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c080097c0 tx=0x7f3c0801c410 comp rx=0 tx=0).stop 2024-05-14T11:22:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 -- 172.21.15.116:0/3440646398 shutdown_connections 2024-05-14T11:22:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 --2- 172.21.15.116:0/3440646398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c2006f730 0x7f3c2006fb40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 -- 172.21.15.116:0/3440646398 >> 172.21.15.116:0/3440646398 conn(0x7f3c200b4620 msgr2=0x7f3c200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 -- 172.21.15.116:0/3440646398 shutdown_connections 2024-05-14T11:22:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.077+0000 7f3c25c00640 1 -- 172.21.15.116:0/3440646398 wait complete. 2024-05-14T11:22:07.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 Processor -- start 2024-05-14T11:22:07.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- start start 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200b0770 0x7f3c20153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200d1670 0x7f3c2015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2016b050 con 0x7f3c2006f730 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2016aed0 con 0x7f3c200d1670 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2016b1d0 con 0x7f3c200b0770 2024-05-14T11:22:07.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42688/0 (socket says 172.21.15.116:42688) 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 -- 172.21.15.116:0/822663008 learned_addr learned my addr 172.21.15.116:0/822663008 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c16000640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200b0770 0x7f3c20153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1f400640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200d1670 0x7f3c2015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 -- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200b0770 msgr2=0x7f3c20153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200b0770 0x7f3c20153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 -- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200d1670 msgr2=0x7f3c2015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200d1670 0x7f3c2015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 -- 172.21.15.116:0/822663008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20154900 con 0x7f3c2006f730 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c1ea00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000c430 tx=0x7f3c1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10017070 con 0x7f3c2006f730 2024-05-14T11:22:07.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c100052d0 con 0x7f3c2006f730 2024-05-14T11:22:07.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c100054c0 con 0x7f3c2006f730 2024-05-14T11:22:07.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c20154be0 con 0x7f3c2006f730 2024-05-14T11:22:07.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.081+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2009d750 con 0x7f3c2006f730 2024-05-14T11:22:07.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2006a9c0 con 0x7f3c2006f730 2024-05-14T11:22:07.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c1001a070 con 0x7f3c2006f730 2024-05-14T11:22:07.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c17400640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 0x7f3bfc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:07.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f3c1008e7d0 con 0x7f3c2006f730 2024-05-14T11:22:07.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c16000640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 0x7f3bfc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:07.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.085+0000 7f3c16000640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 0x7f3bfc071910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3c080ad000 tx=0x7f3c08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:07.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.089+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3c1005ae40 con 0x7f3c2006f730 2024-05-14T11:22:07.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.281+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c2006f510 con 0x7f3c2006f730 2024-05-14T11:22:07.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.281+0000 7f3c17400640 1 -- 172.21.15.116:0/822663008 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f3c1005f140 con 0x7f3c2006f730 2024-05-14T11:22:07.281 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:22:07.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.281+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 msgr2=0x7f3bfc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.281+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 0x7f3bfc071910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3c080ad000 tx=0x7f3c08008040 comp rx=0 tx=0).stop 2024-05-14T11:22:07.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 msgr2=0x7f3c2009aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:07.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000c430 tx=0x7f3c1000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 shutdown_connections 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3bfc06f400 0x7f3bfc071910 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200d1670 0x7f3c2015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200b0770 0x7f3c20153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 --2- 172.21.15.116:0/822663008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c2006f730 0x7f3c2009aa40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 >> 172.21.15.116:0/822663008 conn(0x7f3c200b4620 msgr2=0x7f3c200afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 shutdown_connections 2024-05-14T11:22:07.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07.285+0000 7f3c25c00640 1 -- 172.21.15.116:0/822663008 wait complete. 2024-05-14T11:22:07.303 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:22:07.303 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:22:07.303 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:22:07.303 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:22:07.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:07.304 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:22:07.304 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:22:07.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:22:07.304 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:07.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.304 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:22:07.313 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:22:07.313 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:22:07.313 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:22:07.313 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.313 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.314 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42075sec preferred_lft 42075sec 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:07.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.316 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:22:07.372 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:22:07.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.374 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:22:07.426 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:22:07.427 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:07.427 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:07.427 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:07.427 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:22:07.427 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:22:07.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:22:07.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:22:07.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.566 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:22:07.575 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:22:07.575 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:22:07.575 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:07.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.575 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:22:07.638 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:22:07.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:22:07.638 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:07.687 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:07.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:22:07.687 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:07.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:22:07.798 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:22:07.852 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:22:07.853 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:07.854 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:07.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:07.855 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:22:07.906 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:22:07.906 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:22:07.949 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:22:07.969 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:22:08.020 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:22:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:08.022 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:08.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.023 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:22:08.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:22:08.053+0000 7fe38be48580 -1 init, newargv = 0x7fe37001b4e0 newargc=15 2024-05-14T11:22:08.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44171]: starting ceph client 2024-05-14T11:22:08.060 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:22:08.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44171]: starting fuse 2024-05-14T11:22:08.061 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:22:08.061 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:22:08.062 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:22:08.063 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:22:08.182 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.44171.asok 2024-05-14T11:22:08.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:22:08.193 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44171.asok 2024-05-14T11:22:08.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.193 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44171.asok status 2024-05-14T11:22:08.375 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:08.375 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:22:08.375 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:22:08.375 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:22:08.375 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "44171", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7005, 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 7005 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:22:08.376 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3455767889 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3455767889 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.7005 192.168.0.1:0/3455767889", 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/3455767889", 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 61, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 84, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:22:08.377 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:22:08.378 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:08.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.391 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:22:08.391 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:08.400 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:22:08.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:08.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.401 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:08.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.463 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-14T11:22:08.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.528 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-14T11:22:08.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.583 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-14T11:22:08.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:08.638 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-14T11:22:08.695 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:22:08.695 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:22:08.695 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:22:08.696 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:08.696 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-14T11:22:08.696 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:22:08.801 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:22:08.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:22:08.811 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:22:08.811 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:22:08.867 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:22:09.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.057+0000 7f183fe00640 1 Processor -- start 2024-05-14T11:22:09.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.057+0000 7f183fe00640 1 -- start start 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1838164220 0x7f18380d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f183806ce10 0x7f183816aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183fe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18380d1fc0 con 0x7f18380d1bb0 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f18381600a0 con 0x7f1838164220 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f183806f5d0 con 0x7f183806ce10 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1838164220 0x7f18380d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1838164220 0x7f18380d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37504/0 (socket says 172.21.15.116:37504) 2024-05-14T11:22:09.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 -- 172.21.15.116:0/3359079840 learned_addr learned my addr 172.21.15.116:0/3359079840 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183de00640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f183806ce10 0x7f183816aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 -- 172.21.15.116:0/3359079840 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f183806ce10 msgr2=0x7f183816aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f183806ce10 0x7f183816aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 -- 172.21.15.116:0/3359079840 >> v1:172.21.15.149:6789/0 conn(0x7f18380d1bb0 legacy=0x7f183806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 -- 172.21.15.116:0/3359079840 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183806f8b0 con 0x7f1838164220 2024-05-14T11:22:09.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.061+0000 7f183d400640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f18380d1670 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f182c009e50 tx=0x7f182c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92318f351e9de5d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f1837e00640 1 -- 172.21.15.116:0/3359079840 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0a5830 con 0x7f1838164220 2024-05-14T11:22:09.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f1837e00640 1 -- 172.21.15.116:0/3359079840 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c0a59d0 con 0x7f1838164220 2024-05-14T11:22:09.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f1837e00640 1 -- 172.21.15.116:0/3359079840 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c0a5da0 con 0x7f1838164220 2024-05-14T11:22:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- 172.21.15.116:0/3359079840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 msgr2=0x7f18380d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f18380d1670 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f182c009e50 tx=0x7f182c01c970 comp rx=0 tx=0).stop 2024-05-14T11:22:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- 172.21.15.116:0/3359079840 shutdown_connections 2024-05-14T11:22:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f183806ce10 0x7f183816aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 --2- 172.21.15.116:0/3359079840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f18380d1670 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- 172.21.15.116:0/3359079840 >> 172.21.15.116:0/3359079840 conn(0x7f18380b4620 msgr2=0x7f18380b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- 172.21.15.116:0/3359079840 shutdown_connections 2024-05-14T11:22:09.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- 172.21.15.116:0/3359079840 wait complete. 2024-05-14T11:22:09.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 Processor -- start 2024-05-14T11:22:09.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.065+0000 7f183fe00640 1 -- start start 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f183806ce10 0x7f1838157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18380d1bb0 0x7f1838157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f18380af2f0 con 0x7f1838164220 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1838073650 con 0x7f183806ce10 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1838164630 con 0x7f18380d1bb0 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37510/0 (socket says 172.21.15.116:37510) 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 -- 172.21.15.116:0/3253920261 learned_addr learned my addr 172.21.15.116:0/3253920261 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183ca00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18380d1bb0 0x7f1838157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183d400640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f183806ce10 0x7f1838157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 -- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18380d1bb0 msgr2=0x7f1838157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18380d1bb0 0x7f1838157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 -- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f183806ce10 msgr2=0x7f1838157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f183806ce10 0x7f1838157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 -- 172.21.15.116:0/3253920261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380af7a0 con 0x7f1838164220 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183de00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1830006e90 tx=0x7f183000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300125f0 con 0x7f1838164220 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1830019070 con 0x7f1838164220 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18380afa80 con 0x7f1838164220 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300129d0 con 0x7f1838164220 2024-05-14T11:22:09.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18380b0040 con 0x7f1838164220 2024-05-14T11:22:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.069+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183806a9c0 con 0x7f1838164220 2024-05-14T11:22:09.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.073+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1830012b70 con 0x7f1838164220 2024-05-14T11:22:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.073+0000 7f1835600640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 0x7f1818071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.073+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f183008f740 con 0x7f1838164220 2024-05-14T11:22:09.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.073+0000 7f183d400640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 0x7f1818071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.073+0000 7f183d400640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 0x7f1818071910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f182c0ae000 tx=0x7f182c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.077+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f183005bdd0 con 0x7f1838164220 2024-05-14T11:22:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.245+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18380af500 con 0x7f1838164220 2024-05-14T11:22:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.245+0000 7f1835600640 1 -- 172.21.15.116:0/3253920261 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) ==== 80+0+0 (secure 0 0 0) 0x7f18300600d0 con 0x7f1838164220 2024-05-14T11:22:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:22:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 msgr2=0x7f1818071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 0x7f1818071910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f182c0ae000 tx=0x7f182c0a7070 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 msgr2=0x7f1838072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1830006e90 tx=0x7f183000d9e0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 shutdown_connections 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f181806f400 0x7f1818071910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838164220 0x7f1838072a30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18380d1bb0 0x7f1838157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 --2- 172.21.15.116:0/3253920261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f183806ce10 0x7f1838157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 >> 172.21.15.116:0/3253920261 conn(0x7f18380b4620 msgr2=0x7f1838158530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 shutdown_connections 2024-05-14T11:22:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.249+0000 7f183fe00640 1 -- 172.21.15.116:0/3253920261 wait complete. 2024-05-14T11:22:09.268 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:22:09.268 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:22:09.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:22:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 Processor -- start 2024-05-14T11:22:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 -- start start 2024-05-14T11:22:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbbdc15db30 0x7fbbdc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 0x7fbbdc06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbdc06ff20 con 0x7fbbdc0d1670 2024-05-14T11:22:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fbbdc0ae100 con 0x7fbbdc15db30 2024-05-14T11:22:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.469+0000 7fbbe2800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc0ae870 con 0x7fbbdc06c700 2024-05-14T11:22:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 0x7fbbdc06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdaa00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fbbdc0d1670 0x7fbbdc0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:50118/0 (socket says 172.21.15.116:50118) 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbbdc15db30 0x7fbbdc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdaa00640 1 -- 172.21.15.116:0/1705784304 learned_addr learned my addr 172.21.15.116:0/1705784304 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 -- 172.21.15.116:0/1705784304 >> v2:172.21.15.149:3300/0 conn(0x7fbbdc15db30 msgr2=0x7fbbdc0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 --2- 172.21.15.116:0/1705784304 >> v2:172.21.15.149:3300/0 conn(0x7fbbdc15db30 0x7fbbdc0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbda000640 1 -- 172.21.15.116:0/1705784304 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951656512 0 0) 0x7fbbdc06ff20 con 0x7fbbdc0d1670 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 -- 172.21.15.116:0/1705784304 >> v1:172.21.15.149:6789/0 conn(0x7fbbdc0d1670 legacy=0x7fbbdc0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 -- 172.21.15.116:0/1705784304 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc070470 con 0x7fbbdc06c700 2024-05-14T11:22:09.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbdbe00640 1 --2- 172.21.15.116:0/1705784304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 0x7fbbdc06cb10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd401c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51fc3267c0dd332 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbda000640 1 -- 172.21.15.116:0/1705784304 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b2070 con 0x7fbbdc06c700 2024-05-14T11:22:09.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbda000640 1 -- 172.21.15.116:0/1705784304 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd40a5850 con 0x7fbbdc06c700 2024-05-14T11:22:09.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.473+0000 7fbbda000640 1 -- 172.21.15.116:0/1705784304 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40a5c20 con 0x7fbbdc06c700 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- 172.21.15.116:0/1705784304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 msgr2=0x7fbbdc06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 --2- 172.21.15.116:0/1705784304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 0x7fbbdc06cb10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd401c8a0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- 172.21.15.116:0/1705784304 shutdown_connections 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 --2- 172.21.15.116:0/1705784304 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc06c700 0x7fbbdc06cb10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 --2- 172.21.15.116:0/1705784304 >> v2:172.21.15.149:3300/0 conn(0x7fbbdc15db30 0x7fbbdc0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- 172.21.15.116:0/1705784304 >> 172.21.15.116:0/1705784304 conn(0x7fbbdc0b4620 msgr2=0x7fbbdc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- 172.21.15.116:0/1705784304 shutdown_connections 2024-05-14T11:22:09.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- 172.21.15.116:0/1705784304 wait complete. 2024-05-14T11:22:09.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 Processor -- start 2024-05-14T11:22:09.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.477+0000 7fbbe2800640 1 -- start start 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbdc0d1670 0x7fbbdc09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc15db30 0x7fbbdc167a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc16a770 con 0x7fbbdc06c700 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc15df40 con 0x7fbbdc0d1670 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc16a8f0 con 0x7fbbdc15db30 2024-05-14T11:22:09.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37524/0 (socket says 172.21.15.116:37524) 2024-05-14T11:22:09.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 -- 172.21.15.116:0/2759668258 learned_addr learned my addr 172.21.15.116:0/2759668258 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:09.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdaa00640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbdc0d1670 0x7fbbdc09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdbe00640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc15db30 0x7fbbdc167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 -- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc15db30 msgr2=0x7fbbdc167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc15db30 0x7fbbdc167a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 -- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbdc0d1670 msgr2=0x7fbbdc09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbdc0d1670 0x7fbbdc09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 -- 172.21.15.116:0/2759668258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc0cdac0 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbdb400640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e870 tx=0x7fbbcc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc017070 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbcc00c940 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00ccf0 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc0cdda0 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc0ce360 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.481+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc06a9c0 con 0x7fbbdc06c700 2024-05-14T11:22:09.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.485+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbbcc01a070 con 0x7fbbdc06c700 2024-05-14T11:22:09.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.485+0000 7fbbd3400640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 0x7fbbbc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:09.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.485+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7fbbcc08f6b0 con 0x7fbbdc06c700 2024-05-14T11:22:09.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.485+0000 7fbbdaa00640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 0x7fbbbc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:09.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.485+0000 7fbbdaa00640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 0x7fbbbc071940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc167a20 tx=0x7fbbc8007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:09.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.489+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbbcc05bd20 con 0x7fbbdc06c700 2024-05-14T11:22:09.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.657+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbdc15f4d0 con 0x7fbbdc06c700 2024-05-14T11:22:09.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.657+0000 7fbbd3400640 1 -- 172.21.15.116:0/2759668258 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) ==== 80+0+0 (secure 0 0 0) 0x7fbbcc060020 con 0x7fbbdc06c700 2024-05-14T11:22:09.658 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 msgr2=0x7fbbbc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 0x7fbbbc071940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc167a20 tx=0x7fbbc8007df0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 msgr2=0x7fbbdc0d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e870 tx=0x7fbbcc00ec70 comp rx=0 tx=0).stop 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 shutdown_connections 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbbbc06f430 0x7fbbbc071940 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbdc0d1670 0x7fbbdc09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbbdc15db30 0x7fbbdc167a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 --2- 172.21.15.116:0/2759668258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbdc06c700 0x7fbbdc0d0330 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:09.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 >> 172.21.15.116:0/2759668258 conn(0x7fbbdc0b4620 msgr2=0x7fbbdc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:09.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 shutdown_connections 2024-05-14T11:22:09.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:09.661+0000 7fbbe2800640 1 -- 172.21.15.116:0/2759668258 wait complete. 2024-05-14T11:22:09.679 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:22:09.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:09.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:09.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44171]: fuse finished with error 0 and tester_r 0 2024-05-14T11:22:09.700 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:22:12.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:22:12.651 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:22:15.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:22:15.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:15.802 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:22:15.813 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:16.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 Processor -- start 2024-05-14T11:22:16.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 -- start start 2024-05-14T11:22:16.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a280af420 0x7f5a28074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5a280d1d80 0x7f5a2806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2815d340 con 0x7f5a28074b70 2024-05-14T11:22:16.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5a2815d620 con 0x7f5a280d1d80 2024-05-14T11:22:16.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2815d900 con 0x7f5a280af420 2024-05-14T11:22:16.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2ca00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5a28074b70 0x7f5a280d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36688/0 (socket says 172.21.15.116:36688) 2024-05-14T11:22:16.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.017+0000 7f5a2ca00640 1 -- 172.21.15.116:0/156041391 learned_addr learned my addr 172.21.15.116:0/156041391 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2d400640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a280af420 0x7f5a28074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 --2- 172.21.15.116:0/156041391 >> v2:172.21.15.149:3300/0 conn(0x7f5a280d1d80 0x7f5a2806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181811669 0 0) 0x7f5a2815d340 con 0x7f5a28074b70 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a10003530 con 0x7f5a28074b70 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886216295 0 0) 0x7f5a10003530 con 0x7f5a28074b70 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a280af420 msgr2=0x7f5a28074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a280af420 0x7f5a28074630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 >> v2:172.21.15.149:3300/0 conn(0x7f5a280d1d80 msgr2=0x7f5a2806edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 --2- 172.21.15.116:0/156041391 >> v2:172.21.15.149:3300/0 conn(0x7f5a280d1d80 0x7f5a2806edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2815dbe0 con 0x7f5a28074b70 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f5a14001700 con 0x7f5a28074b70 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 >> v1:172.21.15.116:6789/0 conn(0x7f5a28074b70 legacy=0x7f5a280d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10003a10 0x7f5a10003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a10004810 0x7f5a10004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a10005160 0x7f5a1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2815d620 con 0x7f5a10005160 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2815d900 con 0x7f5a10004810 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1000c040 con 0x7f5a10003a10 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2d400640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a280af420 0x7f5a28074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a10005160 0x7f5a1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2ca00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a10004810 0x7f5a10004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2d400640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10003a10 0x7f5a10003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 -- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10003a10 msgr2=0x7f5a10003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10003a10 0x7f5a10003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 -- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a10004810 msgr2=0x7f5a10004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a10004810 0x7f5a10004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2de00640 1 -- 172.21.15.116:0/156041391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a10003530 con 0x7f5a10005160 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.021+0000 7f5a2d400640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10003a10 0x7f5a10003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2de00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a10005160 0x7f5a1000b9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009f90 tx=0x7f5a2001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeb01a307c7a1b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200b2070 con 0x7f5a10005160 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a200a5c90 con 0x7f5a10005160 2024-05-14T11:22:16.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a27e00640 1 -- 172.21.15.116:0/156041391 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20009c10 con 0x7f5a10005160 2024-05-14T11:22:16.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 -- 172.21.15.116:0/156041391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a10005160 msgr2=0x7f5a1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/156041391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a10005160 0x7f5a1000b9d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009f90 tx=0x7f5a2001cc80 comp rx=0 tx=0).stop 2024-05-14T11:22:16.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2de00640 1 -- 172.21.15.116:0/156041391 reap_dead start 2024-05-14T11:22:16.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 -- 172.21.15.116:0/156041391 shutdown_connections 2024-05-14T11:22:16.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 -- 172.21.15.116:0/156041391 >> 172.21.15.116:0/156041391 conn(0x7f5a280b4620 msgr2=0x7f5a280b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 -- 172.21.15.116:0/156041391 shutdown_connections 2024-05-14T11:22:16.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 -- 172.21.15.116:0/156041391 wait complete. 2024-05-14T11:22:16.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.025+0000 7f5a2fe00640 1 Processor -- start 2024-05-14T11:22:16.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- start start 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10005160 0x7f5a280d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 0x7f5a280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280af830 con 0x7f5a2814fa30 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2815d0d0 con 0x7f5a2814f0d0 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280af9b0 con 0x7f5a10005160 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37542/0 (socket says 172.21.15.116:37542) 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 -- 172.21.15.116:0/2026884518 learned_addr learned my addr 172.21.15.116:0/2026884518 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2d400640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10005160 0x7f5a280d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2ca00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 0x7f5a280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 -- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10005160 msgr2=0x7f5a280d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10005160 0x7f5a280d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 -- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 msgr2=0x7f5a280b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 0x7f5a280b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 -- 172.21.15.116:0/2026884518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280cf5b0 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2ca00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 0x7f5a280b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2de00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5a20002240 tx=0x7f5a20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200bd070 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a200a5a90 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a280cf890 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200b2040 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a280cfe50 con 0x7f5a2814fa30 2024-05-14T11:22:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.029+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a280cf320 con 0x7f5a2814fa30 2024-05-14T11:22:16.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.033+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5a200c0070 con 0x7f5a2814fa30 2024-05-14T11:22:16.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.033+0000 7f5a25600640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 0x7f5a08071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.033+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f5a20129600 con 0x7f5a2814fa30 2024-05-14T11:22:16.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.033+0000 7f5a2d400640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 0x7f5a08071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.033+0000 7f5a2d400640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 0x7f5a08071940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00b690 tx=0x7f5a1c013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.037+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5a20016990 con 0x7f5a2814fa30 2024-05-14T11:22:16.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a2806fe00 con 0x7f5a2814fa30 2024-05-14T11:22:16.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a25600640 1 -- 172.21.15.116:0/2026884518 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5189 (secure 0 0 0) 0x7f5a200f5c70 con 0x7f5a2814fa30 2024-05-14T11:22:16.229 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:16.229 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:21:58.770840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6825},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6825":{"gid":6825,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6835/536312712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":536312712},{"type":"v1","addr":"172.21.15.149:6835","nonce":536312712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825]},"id":6}]} 2024-05-14T11:22:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 61 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 msgr2=0x7f5a08071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 0x7f5a08071940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00b690 tx=0x7f5a1c013040 comp rx=0 tx=0).stop 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 msgr2=0x7f5a2814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5a20002240 tx=0x7f5a20009750 comp rx=0 tx=0).stop 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 shutdown_connections 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5a0806f430 0x7f5a08071940 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2814f0d0 0x7f5a280b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2814fa30 0x7f5a2814fe40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 --2- 172.21.15.116:0/2026884518 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5a10005160 0x7f5a280d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.229+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 >> 172.21.15.116:0/2026884518 conn(0x7f5a280b4620 msgr2=0x7f5a28155d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.233+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 shutdown_connections 2024-05-14T11:22:16.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.233+0000 7f5a2fe00640 1 -- 172.21.15.116:0/2026884518 wait complete. 2024-05-14T11:22:16.251 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-14T11:22:16.444 INFO:teuthology.orchestra.run.smithi149.stdout:{"message":"","return_code":0} 2024-05-14T11:22:16.445 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:22:16.445 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:22:16.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.593+0000 7f606de00640 1 Processor -- start 2024-05-14T11:22:16.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.593+0000 7f606de00640 1 -- start start 2024-05-14T11:22:16.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f606815db30 0x7f60680aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f60680d1670 0x7f60680d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6068161600 0x7f6068161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f60680ae100 con 0x7f6068161600 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f60680ae870 con 0x7f60680d1670 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f606de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f606806ff20 con 0x7f606815db30 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6068161600 0x7f6068161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6066a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f606815db30 0x7f60680aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6068161600 0x7f6068161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37554/0 (socket says 172.21.15.116:37554) 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 -- 172.21.15.116:0/3455012436 learned_addr learned my addr 172.21.15.116:0/3455012436 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6066000640 1 --2- 172.21.15.116:0/3455012436 >> v2:172.21.15.149:3300/0 conn(0x7f60680d1670 0x7f60680d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 -- 172.21.15.116:0/3455012436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f606815db30 msgr2=0x7f60680aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 --2- 172.21.15.116:0/3455012436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f606815db30 0x7f60680aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 -- 172.21.15.116:0/3455012436 >> v2:172.21.15.149:3300/0 conn(0x7f60680d1670 msgr2=0x7f60680d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 --2- 172.21.15.116:0/3455012436 >> v2:172.21.15.149:3300/0 conn(0x7f60680d1670 0x7f60680d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 -- 172.21.15.116:0/3455012436 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680d1fc0 con 0x7f6068161600 2024-05-14T11:22:16.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.597+0000 7f6067400640 1 --2- 172.21.15.116:0/3455012436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068161a10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f60680d2280 tx=0x7f606001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40b6969423fe8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f6065600640 1 -- 172.21.15.116:0/3455012436 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600a56d0 con 0x7f6068161600 2024-05-14T11:22:16.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f6065600640 1 -- 172.21.15.116:0/3455012436 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600a5870 con 0x7f6068161600 2024-05-14T11:22:16.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f6065600640 1 -- 172.21.15.116:0/3455012436 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600a5c40 con 0x7f6068161600 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- 172.21.15.116:0/3455012436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 msgr2=0x7f6068161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 --2- 172.21.15.116:0/3455012436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068161a10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f60680d2280 tx=0x7f606001c960 comp rx=0 tx=0).stop 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- 172.21.15.116:0/3455012436 shutdown_connections 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 --2- 172.21.15.116:0/3455012436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068161a10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 --2- 172.21.15.116:0/3455012436 >> v2:172.21.15.149:3300/0 conn(0x7f60680d1670 0x7f60680d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 --2- 172.21.15.116:0/3455012436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f606815db30 0x7f60680aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- 172.21.15.116:0/3455012436 >> 172.21.15.116:0/3455012436 conn(0x7f60680b4620 msgr2=0x7f60680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- 172.21.15.116:0/3455012436 shutdown_connections 2024-05-14T11:22:16.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- 172.21.15.116:0/3455012436 wait complete. 2024-05-14T11:22:16.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 Processor -- start 2024-05-14T11:22:16.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.601+0000 7f606de00640 1 -- start start 2024-05-14T11:22:16.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60680d1670 0x7f606809ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f606815db30 0x7f606814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6068162360 con 0x7f6068161600 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60680aefe0 con 0x7f606815db30 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f606815df40 con 0x7f60680d1670 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37568/0 (socket says 172.21.15.116:37568) 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 -- 172.21.15.116:0/2819972677 learned_addr learned my addr 172.21.15.116:0/2819972677 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6066000640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f606815db30 0x7f606814cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6066a00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60680d1670 0x7f606809ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 -- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60680d1670 msgr2=0x7f606809ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60680d1670 0x7f606809ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 -- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f606815db30 msgr2=0x7f606814cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f606815db30 0x7f606814cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 -- 172.21.15.116:0/2819972677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680cd9f0 con 0x7f6068161600 2024-05-14T11:22:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f6067400640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f60600af040 tx=0x7f6060019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600bc020 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600b8690 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060009510 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60680cdcd0 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606814d650 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.605+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606806a9c0 con 0x7f6068161600 2024-05-14T11:22:16.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.609+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60600bf020 con 0x7f6068161600 2024-05-14T11:22:16.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.609+0000 7f605f400640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 0x7f6048071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.609+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6101+0+0 (secure 0 0 0) 0x7f606012a1a0 con 0x7f6068161600 2024-05-14T11:22:16.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.609+0000 7f6066a00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 0x7f6048071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.609+0000 7f6066a00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 0x7f6048071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6058009a40 tx=0x7f6058009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.613+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f60600c6030 con 0x7f6068161600 2024-05-14T11:22:16.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.801+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f606814ec70 con 0x7f6068161600 2024-05-14T11:22:16.803 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:22:16.801+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:22:16.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.805+0000 7f605f400640 1 -- 172.21.15.116:0/2819972677 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f60680d1fc0 con 0x7f6068161600 2024-05-14T11:22:16.805 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:22:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 msgr2=0x7f6048071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 0x7f6048071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6058009a40 tx=0x7f6058009190 comp rx=0 tx=0).stop 2024-05-14T11:22:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 msgr2=0x7f6068160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f60600af040 tx=0x7f6060019700 comp rx=0 tx=0).stop 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 shutdown_connections 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f604806f400 0x7f6048071910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f606815db30 0x7f606814cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6068161600 0x7f6068160080 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 --2- 172.21.15.116:0/2819972677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60680d1670 0x7f606809ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 >> 172.21.15.116:0/2819972677 conn(0x7f60680b4620 msgr2=0x7f60680b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 shutdown_connections 2024-05-14T11:22:16.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.809+0000 7f606de00640 1 -- 172.21.15.116:0/2819972677 wait complete. 2024-05-14T11:22:16.816 INFO:tasks.ceph.mds.b.smithi149.stderr: -4119> 2024-05-14T11:21:58.335+0000 7fc49e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:22:16.828 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 Processor -- start 2024-05-14T11:22:16.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 -- start start 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a16343a570 0x55a1634c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1634c6890 0x55a1634c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a163398aa0 con 0x55a1633f55c0 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a16339cae0 con 0x55a1634c6890 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a16343ab90 con 0x55a16343a570 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37582/0 (socket says 172.21.15.116:37582) 2024-05-14T11:22:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 -- 172.21.15.116:0/4200889024 learned_addr learned my addr 172.21.15.116:0/4200889024 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04c00640 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1634c6890 0x55a1634c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c03800640 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a16343a570 0x55a1634c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 -- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a16343a570 msgr2=0x55a1634c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a16343a570 0x55a1634c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 -- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1634c6890 msgr2=0x55a1634c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1634c6890 0x55a1634c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 -- 172.21.15.116:0/4200889024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1634c9da0 con 0x55a1633f55c0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c04200640 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ad040 tx=0x7f6bf4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13768ba049b7d383 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.885+0000 7f6c02400640 1 -- 172.21.15.116:0/4200889024 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40af040 con 0x55a1633f55c0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c02400640 1 -- 172.21.15.116:0/4200889024 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf40b9070 con 0x55a1633f55c0 2024-05-14T11:22:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c02400640 1 -- 172.21.15.116:0/4200889024 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b4410 con 0x55a1633f55c0 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/4200889024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 msgr2=0x55a1634bd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ad040 tx=0x7f6bf4003030 comp rx=0 tx=0).stop 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/4200889024 shutdown_connections 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1634c6890 0x55a1634c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a16343a570 0x55a1634c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/4200889024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634bd0b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/4200889024 >> 172.21.15.116:0/4200889024 conn(0x55a1633dde10 msgr2=0x55a1634384d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/4200889024 shutdown_connections 2024-05-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/4200889024 wait complete. 2024-05-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 Processor -- start 2024-05-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- start start 2024-05-14T11:22:16.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634ba960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a16343a570 0x55a163437760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a1634c6890 0x55a16355ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a1634a6b30 con 0x55a1633f55c0 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a1634c98a0 con 0x55a16343a570 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c0b0dc080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a1634bfa50 con 0x55a1634c6890 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c04200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c04200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634ba960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37596/0 (socket says 172.21.15.116:37596) 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c04200640 1 -- 172.21.15.116:0/2519943499 learned_addr learned my addr 172.21.15.116:0/2519943499 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c03800640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a16343a570 0x55a163437760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.889+0000 7f6c04c00640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a1634c6890 0x55a16355ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 -- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a1634c6890 msgr2=0x55a16355ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a1634c6890 0x55a16355ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 -- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a16343a570 msgr2=0x55a163437760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a16343a570 0x55a163437760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 -- 172.21.15.116:0/2519943499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a163435ab0 con 0x55a1633f55c0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c04200640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634ba960 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400a500 tx=0x7f6bf4018150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 -- 172.21.15.116:0/2519943499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40af050 con 0x55a1633f55c0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 -- 172.21.15.116:0/2519943499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf40c5070 con 0x55a1633f55c0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 -- 172.21.15.116:0/2519943499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b4850 con 0x55a1633f55c0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a163435d50 con 0x55a1633f55c0 2024-05-14T11:22:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1634baea0 con 0x55a1633f55c0 2024-05-14T11:22:16.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 -- 172.21.15.116:0/2519943499 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6bf40b49f0 con 0x55a1633f55c0 2024-05-14T11:22:16.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6be406dde0 0x7f6be40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c01000640 1 -- 172.21.15.116:0/2519943499 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f6bf4129df0 con 0x55a1633f55c0 2024-05-14T11:22:16.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.893+0000 7f6c03800640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6be406dde0 0x7f6be40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55a163560450 0x55a163436490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e85) -- 0x55a1634369d0 con 0x55a163560450 2024-05-14T11:22:16.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c04c00640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55a163560450 0x55a163436490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c03800640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6be406dde0 0x7f6be40702f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0003f20 tx=0x7f6bf0003820 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c04c00640 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55a163560450 0x55a163436490 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c04c00640 1 -- 172.21.15.116:0/2519943499 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v85'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6bfc010610 con 0x55a163560450 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55a163560450 msgr2=0x55a163436490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55a163560450 0x55a163436490 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6be406dde0 msgr2=0x7f6be40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6be406dde0 0x7f6be40702f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0003f20 tx=0x7f6bf0003820 comp rx=0 tx=0).stop 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 msgr2=0x55a1634ba960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 --2- 172.21.15.116:0/2519943499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1633f55c0 0x55a1634ba960 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400a500 tx=0x7f6bf4018150 comp rx=0 tx=0).stop 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c04c00640 1 -- 172.21.15.116:0/2519943499 reap_dead start 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 shutdown_connections 2024-05-14T11:22:16.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 >> 172.21.15.116:0/2519943499 conn(0x55a1633dde10 msgr2=0x55a16341de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 shutdown_connections 2024-05-14T11:22:16.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.897+0000 7f6c0b0dc080 1 -- 172.21.15.116:0/2519943499 wait complete. 2024-05-14T11:22:16.903 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-14T11:22:16.934 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:22:16.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.961+0000 7f85ecdd4400 1 -- start start 2024-05-14T11:22:16.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.961+0000 7f85ecdd4400 1 Processor -- start 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 0x55da58c8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 0x55da58c8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca4180 con 0x55da57f82400 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca4300 con 0x55da57f81680 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55da58ca4480 con 0x55da57f81b00 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 0x55da58c8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ea400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 0x55da58c8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:49402/0 (socket says 172.21.15.116:49402) 2024-05-14T11:22:16.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eae00640 1 -- 172.21.15.116:0/2885946198 learned_addr learned my addr 172.21.15.116:0/2885946198 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 -- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 msgr2=0x55da58c8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 0x55da58c8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 -- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 msgr2=0x55da58c8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 -- 172.21.15.116:0/2885946198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da58c86b40 con 0x55da57f82400 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ea400640 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 0x55da58c8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eae00640 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85eb800640 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 0x55da58c8cb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55da58d21920 tx=0x55da57fc44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1e9925fd3976c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85e7c00640 1 -- 172.21.15.116:0/2885946198 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee04e0 con 0x55da57f82400 2024-05-14T11:22:16.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85e7c00640 1 -- 172.21.15.116:0/2885946198 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55da57ef4e00 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85e7c00640 1 -- 172.21.15.116:0/2885946198 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee0b60 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da58c86d20 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55da58c86f00 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 4 waiting for MDS map... 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85e7c00640 1 -- 172.21.15.116:0/2885946198 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x55da58dd4600 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85e7c00640 1 -- 172.21.15.116:0/2885946198 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55da57f01180 con 0x55da57f82400 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 4 Got MDS map 62 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 10 main: JournalTool::main 2024-05-14T11:22:16.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 4 main: JournalTool: connecting to RADOS... 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 Processor -- start 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.965+0000 7f85ecdd4400 1 -- start start 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca4600 con 0x55da57f80d80 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca4780 con 0x55da57f80480 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55da58ca4900 con 0x55da57f80900 2024-05-14T11:22:16.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37632/0 (socket says 172.21.15.116:37632) 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 -- 172.21.15.116:0/1048152209 learned_addr learned my addr 172.21.15.116:0/1048152209 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85eae00640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 -- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 msgr2=0x55da58c8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 -- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80480 msgr2=0x55da58c8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 -- 172.21.15.116:0/1048152209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da58c86d20 con 0x55da57f80d80 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85eae00640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ea400640 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55da58d21020 tx=0x55da57fc5ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbdbfd65e05fead server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85e5400640 1 -- 172.21.15.116:0/1048152209 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee09c0 con 0x55da57f80d80 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85e5400640 1 -- 172.21.15.116:0/1048152209 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55da57ef4c40 con 0x55da57f80d80 2024-05-14T11:22:16.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85e5400640 1 -- 172.21.15.116:0/1048152209 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee04e0 con 0x55da57f80d80 2024-05-14T11:22:16.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1048152209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 msgr2=0x55da58c8d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55da58d21020 tx=0x55da57fc5ec0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1048152209 shutdown_connections 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1048152209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1048152209 >> 172.21.15.116:0/1048152209 conn(0x55da57f81200 msgr2=0x55da581cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1048152209 shutdown_connections 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1048152209 wait complete. 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.969+0000 7f85ecdd4400 1 Processor -- start 2024-05-14T11:22:16.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- start start 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca5b00 con 0x55da57f80d80 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca5980 con 0x55da57f80900 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55da58ca5800 con 0x55da57f80480 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:49418/0 (socket says 172.21.15.116:49418) 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85eae00640 1 -- 172.21.15.116:0/1649292392 learned_addr learned my addr 172.21.15.116:0/1649292392 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 -- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 msgr2=0x55da58c8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 -- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 msgr2=0x55da58c8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 -- 172.21.15.116:0/1649292392 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da58c87c20 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85eae00640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ea400640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 0x55da58c8db80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55da58de6ed0 tx=0x55da57fc5e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85e3600640 1 -- 172.21.15.116:0/1649292392 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee0b60 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85e3600640 1 -- 172.21.15.116:0/1649292392 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55da57ef4a80 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da58c87e00 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85e3600640 1 -- 172.21.15.116:0/1649292392 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee0000 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55da58c861e0 con 0x55da57f80480 2024-05-14T11:22:16.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.973+0000 7f85ecdd4400 4 main: JournalTool: resolving pool 13 2024-05-14T11:22:16.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85e3600640 1 -- 172.21.15.116:0/1649292392 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55da581e8c00 con 0x55da57f80480 2024-05-14T11:22:16.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85e3600640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 0x55da58c8e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85e3600640 1 -- 172.21.15.116:0/1649292392 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x55da581fc600 con 0x55da57f80480 2024-05-14T11:22:16.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85ecdd4400 4 main: JournalTool: creating IoCtx.. 2024-05-14T11:22:16.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85ecdd4400 4 main: Executing for rank 0 2024-05-14T11:22:16.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85eae00640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 0x55da58c8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85eae00640 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 0x55da58c8e100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55da58de7aa0 tx=0x55da58debf20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85ecdd4400 1 -- start start 2024-05-14T11:22:16.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.977+0000 7f85ecdd4400 1 Processor -- start 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f82d00 0x55da58c8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f83600 0x55da58c8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca5500 con 0x55da57f82880 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55da58ca4780 con 0x55da57f82d00 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55da58ca4a80 con 0x55da57f83600 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37650/0 (socket says 172.21.15.116:37650) 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 learned_addr learned my addr 172.21.15.116:0/4260953474 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ea400640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f83600 0x55da58c8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eae00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f82d00 0x55da58c8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f83600 msgr2=0x55da58c8f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f83600 0x55da58c8f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f82d00 msgr2=0x55da58c8ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f82d00 0x55da58c8ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5833e780 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55da58297cb0 tx=0x55da57fc4900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7171579bcd46d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85dfa00640 1 -- 172.21.15.116:0/4260953474 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da57ee0340 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85dfa00640 1 -- 172.21.15.116:0/4260953474 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55da57ef5340 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85dfa00640 1 -- 172.21.15.116:0/4260953474 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55da582211e0 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5833e960 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55da5833eb40 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 4 waiting for MDS map... 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85dfa00640 1 -- 172.21.15.116:0/4260953474 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x55da58352200 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85dfa00640 1 -- 172.21.15.116:0/4260953474 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55da57f01500 con 0x55da57f82880 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 4 Got MDS map 62 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55da57f83a80 0x55da58c8f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x55da58355800 con 0x55da57f83a80 2024-05-14T11:22:16.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eae00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55da57f83a80 0x55da58c8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eae00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55da57f83a80 0x55da58c8f700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=3d61606d927eec0f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85eae00640 1 -- 172.21.15.116:0/4260953474 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55da57faa000 con 0x55da57f83a80 2024-05-14T11:22:16.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55da5836e000 0x55da58376000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x55da58355c00 con 0x55da5836e000 2024-05-14T11:22:16.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.981+0000 7f85ea400640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55da5836e000 0x55da58376000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ea400640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55da5836e000 0x55da58376000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=832889ec284df44c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ea400640 1 -- 172.21.15.116:0/4260953474 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55da57faa500 con 0x55da5836e000 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ddc00640 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85) -- 0x55da5837a000 con 0x55da5836e000 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ddc00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55da58371600 0x55da58377b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ddc00640 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85) -- 0x55da5837a400 con 0x55da58371600 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85ea400640 1 -- 172.21.15.116:0/4260953474 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55da57faa500 con 0x55da5836e000 2024-05-14T11:22:16.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55da58371600 0x55da58377b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.985+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55da58371600 0x55da58377b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e54629412d67ed1a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa780 con 0x55da58371600 2024-05-14T11:22:16.987 INFO:teuthology.orchestra.run.smithi116.stdout:old journal was 4208696~2030 2024-05-14T11:22:16.987 INFO:teuthology.orchestra.run.smithi116.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-05-14T11:22:16.987 INFO:teuthology.orchestra.run.smithi116.stdout:writing journal head 2024-05-14T11:22:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da5837ac00 con 0x55da5836e000 2024-05-14T11:22:16.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ea400640 1 -- 172.21.15.116:0/4260953474 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v85'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55da57faa500 con 0x55da5836e000 2024-05-14T11:22:16.989 INFO:teuthology.orchestra.run.smithi116.stdout:writing EResetJournal entry 2024-05-14T11:22:16.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da5837b000 con 0x55da58371600 2024-05-14T11:22:16.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55da5836e480 0x55da58376580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da5837b400 con 0x55da5836e480 2024-05-14T11:22:16.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da58355c00 con 0x55da57f83a80 2024-05-14T11:22:16.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55da5836e900 0x55da58376b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da581ff400 con 0x55da5836e900 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da581ff000 con 0x55da58371600 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55da5836ed80 0x55da58377080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da581fec00 con 0x55da5836ed80 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ea400640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55da5836e900 0x55da58376b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85eae00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55da5836e480 0x55da58376580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55da5836ed80 0x55da58377080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85ea400640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55da5836e900 0x55da58376b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=dfe83aed8f4c48f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85eae00640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55da5836e480 0x55da58376580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=f20f6e7dde0434da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.989+0000 7f85eb800640 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55da5836ed80 0x55da58377080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=1ad779a205876c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:16.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.993+0000 7f85eae00640 1 -- 172.21.15.116:0/4260953474 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(8 200.00000004 [delete] v85'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa000 con 0x55da57f83a80 2024-05-14T11:22:16.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.993+0000 7f85ea400640 1 -- 172.21.15.116:0/4260953474 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(9 200.00000005 [delete] v85'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa500 con 0x55da5836e900 2024-05-14T11:22:16.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.993+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(6 200.00000002 [delete] v85'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa780 con 0x55da58371600 2024-05-14T11:22:16.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.993+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(10 200.00000006 [delete] v85'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa780 con 0x55da58371600 2024-05-14T11:22:16.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(11 200.00000007 [delete] v85'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa780 con 0x55da5836ed80 2024-05-14T11:22:16.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85eae00640 1 -- 172.21.15.116:0/4260953474 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(7 200.00000003 [delete] v85'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55da57faa000 con 0x55da5836e480 2024-05-14T11:22:16.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ddc00640 1 -- 172.21.15.116:0/4260953474 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55da5837a800 con 0x55da58371600 2024-05-14T11:22:16.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85eb800640 1 -- 172.21.15.116:0/4260953474 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v85'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55da57faa780 con 0x55da58371600 2024-05-14T11:22:16.998 INFO:teuthology.orchestra.run.smithi116.stdout:done 2024-05-14T11:22:17.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55da5836e000 msgr2=0x55da58376000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55da5836e000 0x55da58376000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55da5836e900 msgr2=0x55da58376b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55da5836e900 0x55da58376b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55da58371600 msgr2=0x55da58377b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55da58371600 0x55da58377b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55da5836e480 msgr2=0x55da58376580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55da5836e480 0x55da58376580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55da57f83a80 msgr2=0x55da58c8f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55da57f83a80 0x55da58c8f700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55da5836ed80 msgr2=0x55da58377080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55da5836ed80 0x55da58377080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 msgr2=0x55da58c8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ea400640 1 -- 172.21.15.116:0/4260953474 reap_dead start 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55da58297cb0 tx=0x55da57fc4900 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 shutdown_connections 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/4260953474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82880 0x55da58c8e680 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 >> 172.21.15.116:0/4260953474 conn(0x55da57f80000 msgr2=0x55da581cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 shutdown_connections 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:16.997+0000 7f85ecdd4400 1 -- 172.21.15.116:0/4260953474 wait complete. 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 msgr2=0x55da58c8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 0x55da58c8e100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55da58de7aa0 tx=0x55da58debf20 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 msgr2=0x55da58c8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 0x55da58c8db80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55da58de6ed0 tx=0x55da57fc5e00 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 shutdown_connections 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55da57f83180 0x55da58c8e100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f80900 0x55da58c8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f80d80 0x55da58c8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/1649292392 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f80480 0x55da58c8db80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 >> 172.21.15.116:0/1649292392 conn(0x55da57f81200 msgr2=0x55da581cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 shutdown_connections 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/1649292392 wait complete. 2024-05-14T11:22:17.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 msgr2=0x55da58c8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 0x55da58c8cb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55da58d21920 tx=0x55da57fc44e0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 shutdown_connections 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da57f82400 0x55da58c8cb00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55da57f81b00 0x55da58c8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 --2- 172.21.15.116:0/2885946198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55da57f81680 0x55da58c8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 >> 172.21.15.116:0/2885946198 conn(0x55da57f81f80 msgr2=0x55da53ea9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 shutdown_connections 2024-05-14T11:22:17.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.001+0000 7f85ecdd4400 1 -- 172.21.15.116:0/2885946198 wait complete. 2024-05-14T11:22:17.006 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.102738, result: old journal was 4208696~2030 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-14T11:22:17.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:22:17.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 Processor -- start 2024-05-14T11:22:17.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 -- start start 2024-05-14T11:22:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac06f450 0x7febac06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1730 0x7febac0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac06f860 con 0x7febac0cce60 2024-05-14T11:22:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7febac06db80 con 0x7febac0d1730 2024-05-14T11:22:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febb0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7febac0d2080 con 0x7febac06f450 2024-05-14T11:22:17.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1730 0x7febac0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7febac0d1730 0x7febac0d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37656/0 (socket says 172.21.15.116:37656) 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba9e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac06f450 0x7febac06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 -- 172.21.15.116:0/1873565052 learned_addr learned my addr 172.21.15.116:0/1873565052 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7febaa800640 1 --1- 172.21.15.116:0/1873565052 >> v1:172.21.15.149:6789/0 conn(0x7febac0cce60 0x7febac0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:50152/0 (socket says 172.21.15.116:50152) 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 -- 172.21.15.116:0/1873565052 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac06f450 msgr2=0x7febac06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 --2- 172.21.15.116:0/1873565052 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac06f450 0x7febac06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 -- 172.21.15.116:0/1873565052 >> v1:172.21.15.149:6789/0 conn(0x7febac0cce60 legacy=0x7febac0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba9400640 1 -- 172.21.15.116:0/1873565052 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330660247 0 0) 0x7feba4001120 con 0x7febac0cce60 2024-05-14T11:22:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.157+0000 7feba1400640 1 -- 172.21.15.116:0/1873565052 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac06fae0 con 0x7febac0d1730 2024-05-14T11:22:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7feba1400640 1 --2- 172.21.15.116:0/1873565052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1730 0x7febac0d1b40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3173f600a62978 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7feba9400640 1 -- 172.21.15.116:0/1873565052 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac0ce190 con 0x7febac0d1730 2024-05-14T11:22:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7feba9400640 1 -- 172.21.15.116:0/1873565052 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb940b0070 con 0x7febac0d1730 2024-05-14T11:22:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7feba9400640 1 -- 172.21.15.116:0/1873565052 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940ab870 con 0x7febac0d1730 2024-05-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 -- 172.21.15.116:0/1873565052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1730 msgr2=0x7febac0d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 --2- 172.21.15.116:0/1873565052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1730 0x7febac0d1b40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9401c560 comp rx=0 tx=0).stop 2024-05-14T11:22:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 -- 172.21.15.116:0/1873565052 shutdown_connections 2024-05-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 --2- 172.21.15.116:0/1873565052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0d1730 0x7febac0d1b40 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 --2- 172.21.15.116:0/1873565052 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac06f450 0x7febac06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 -- 172.21.15.116:0/1873565052 >> 172.21.15.116:0/1873565052 conn(0x7febac0b4720 msgr2=0x7febac0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 -- 172.21.15.116:0/1873565052 shutdown_connections 2024-05-14T11:22:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.161+0000 7febb0c00640 1 -- 172.21.15.116:0/1873565052 wait complete. 2024-05-14T11:22:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 Processor -- start 2024-05-14T11:22:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 -- start start 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06f450 0x7febac159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 0x7febac158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7febac16a610 con 0x7febac0cce60 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7febac070660 con 0x7febac06f450 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febb0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7febac16a790 con 0x7febac0d1730 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37658/0 (socket says 172.21.15.116:37658) 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 -- 172.21.15.116:0/1032031106 learned_addr learned my addr 172.21.15.116:0/1032031106 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba9e00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06f450 0x7febac159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febaa800640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 0x7febac158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 -- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 msgr2=0x7febac158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 0x7febac158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 -- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06f450 msgr2=0x7febac159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06f450 0x7febac159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7feba1400640 1 -- 172.21.15.116:0/1032031106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac1f9530 con 0x7febac0cce60 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.165+0000 7febaa800640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 0x7febac158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba1400640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feb9401a190 tx=0x7feb9401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940ba070 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94004220 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac1fd8a0 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940ab920 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1f9a90 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac1fd5f0 con 0x7febac0cce60 2024-05-14T11:22:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb940abac0 con 0x7febac0cce60 2024-05-14T11:22:17.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 0x7feb88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7feb941295b0 con 0x7febac0cce60 2024-05-14T11:22:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.169+0000 7feba9e00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 0x7feb88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.173+0000 7feba9e00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 0x7feb88071940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7febac0c83b0 tx=0x7feb9c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:17.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.173+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7febac1f9530 con 0x7febac0cce60 2024-05-14T11:22:17.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.365+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7febac06a9c0 con 0x7febac0cce60 2024-05-14T11:22:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.805+0000 7feba2a00640 1 -- 172.21.15.116:0/1032031106 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7feb94016150 con 0x7febac0cce60 2024-05-14T11:22:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:22:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 msgr2=0x7feb88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 0x7feb88071940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7febac0c83b0 tx=0x7feb9c00b040 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 msgr2=0x7febac158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feb9401a190 tx=0x7feb9401ada0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 shutdown_connections 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb8806f430 0x7feb88071940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7febac0d1730 0x7febac158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febac0cce60 0x7febac158400 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 --2- 172.21.15.116:0/1032031106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febac06f450 0x7febac159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 >> 172.21.15.116:0/1032031106 conn(0x7febac0b4720 msgr2=0x7febac0b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 shutdown_connections 2024-05-14T11:22:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.809+0000 7febb0c00640 1 -- 172.21.15.116:0/1032031106 wait complete. 2024-05-14T11:22:17.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:17.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 Processor -- start 2024-05-14T11:22:17.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 -- start start 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f13b8070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f13b8071380 0x7f13b806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b806cc10 con 0x7f13b806c700 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f13b80b7390 con 0x7f13b8071380 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13be400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13b815eb40 con 0x7f13b80c5210 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f13b806c700 0x7f13b806de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34260/0 (socket says 172.21.15.116:34260) 2024-05-14T11:22:17.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6a00640 1 -- 172.21.15.116:0/1023977872 learned_addr learned my addr 172.21.15.116:0/1023977872 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> v2:172.21.15.149:3300/0 conn(0x7f13b8071380 0x7f13b806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b7400640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f13b8070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044083126 0 0) 0x7f13a8001120 con 0x7f13b806c700 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f139c003530 con 0x7f13b806c700 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4166222398 0 0) 0x7f139c003530 con 0x7f13b806c700 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 msgr2=0x7f13b8070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f13b8070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 >> v2:172.21.15.149:3300/0 conn(0x7f13b8071380 msgr2=0x7f13b806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:17.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 --2- 172.21.15.116:0/1023977872 >> v2:172.21.15.149:3300/0 conn(0x7f13b8071380 0x7f13b806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b815ee20 con 0x7f13b806c700 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f13a80018d0 con 0x7f13b806c700 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> v2:172.21.15.149:3300/0 conn(0x7f13b8071380 0x7f13b806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 >> v1:172.21.15.116:6789/0 conn(0x7f13b806c700 legacy=0x7f13b806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f139c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c003fe0 0x7f139c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.981+0000 7f13b6000640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c004d50 0x7f139c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80b7390 con 0x7f139c004d50 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13b815eb40 con 0x7f139c003fe0 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f139c00bb60 con 0x7f13b80c5210 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c004d50 0x7f139c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7400640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f139c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6a00640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c003fe0 0x7f139c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 -- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 msgr2=0x7f139c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:17.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80c5210 0x7f139c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 -- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c003fe0 msgr2=0x7f139c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c003fe0 0x7f139c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 -- 172.21.15.116:0/1023977872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c003530 con 0x7f139c004d50 2024-05-14T11:22:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b7e00640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c004d50 0x7f139c005160 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f13b80c8b70 tx=0x7f13b001cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b3e7e389ae3211 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0009de0 con 0x7f139c004d50 2024-05-14T11:22:17.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b00a5ce0 con 0x7f139c004d50 2024-05-14T11:22:17.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.985+0000 7f13b6000640 1 -- 172.21.15.116:0/1023977872 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003ba0 con 0x7f139c004d50 2024-05-14T11:22:17.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- 172.21.15.116:0/1023977872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c004d50 msgr2=0x7f139c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 --2- 172.21.15.116:0/1023977872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c004d50 0x7f139c005160 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f13b80c8b70 tx=0x7f13b001cde0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13b7e00640 1 -- 172.21.15.116:0/1023977872 reap_dead start 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- 172.21.15.116:0/1023977872 shutdown_connections 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- 172.21.15.116:0/1023977872 >> 172.21.15.116:0/1023977872 conn(0x7f13b80ce570 msgr2=0x7f13b80ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- 172.21.15.116:0/1023977872 shutdown_connections 2024-05-14T11:22:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- 172.21.15.116:0/1023977872 wait complete. 2024-05-14T11:22:17.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 Processor -- start 2024-05-14T11:22:17.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.989+0000 7f13be400640 1 -- start start 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 0x7f13b81f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80b6b20 0x7f13b81f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8071790 con 0x7f13b80c8b70 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13b80cc5c0 con 0x7f139c004d50 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13b8158490 con 0x7f13b80b6b20 2024-05-14T11:22:17.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46344/0 (socket says 172.21.15.116:46344) 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7e00640 1 -- 172.21.15.116:0/2047242519 learned_addr learned my addr 172.21.15.116:0/2047242519 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 0x7f13b81f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b6a00640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80b6b20 0x7f13b81f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 -- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80b6b20 msgr2=0x7f13b81f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80b6b20 0x7f13b81f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 -- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 msgr2=0x7f13b81f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 -- 172.21.15.116:0/2047242519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b80b1470 con 0x7f139c004d50 2024-05-14T11:22:17.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7e00640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13b7400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 0x7f13b81f5a60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f13a0011960 tx=0x7f13a0011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0018070 con 0x7f139c004d50 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a0004240 con 0x7f139c004d50 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b80b18e0 con 0x7f139c004d50 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a00059a0 con 0x7f139c004d50 2024-05-14T11:22:17.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b80d13a0 con 0x7f139c004d50 2024-05-14T11:22:17.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.993+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b806a9c0 con 0x7f139c004d50 2024-05-14T11:22:17.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.997+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13a001b070 con 0x7f139c004d50 2024-05-14T11:22:17.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.997+0000 7f13af400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:17.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.997+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f13a008f7c0 con 0x7f139c004d50 2024-05-14T11:22:17.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.997+0000 7f13b6a00640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:17.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:17.997+0000 7f13b6a00640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 0x7f1394071910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f13a80078f0 tx=0x7f13a80047f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:18.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.001+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f13a005be80 con 0x7f139c004d50 2024-05-14T11:22:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13b80b3200 con 0x7f139c004d50 2024-05-14T11:22:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13af400640 1 -- 172.21.15.116:0/2047242519 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5186 (secure 0 0 0) 0x7f13a0060180 con 0x7f139c004d50 2024-05-14T11:22:18.193 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:18.193 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:17.813387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-14T11:22:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 64 2024-05-14T11:22:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 msgr2=0x7f1394071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 0x7f1394071910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f13a80078f0 tx=0x7f13a80047f0 comp rx=0 tx=0).stop 2024-05-14T11:22:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 msgr2=0x7f13b81f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.193+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 0x7f13b81f5a60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f13a0011960 tx=0x7f13a0011d60 comp rx=0 tx=0).stop 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 shutdown_connections 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80c8b70 0x7f13b81f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b80b6b20 0x7f13b81f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 --2- 172.21.15.116:0/2047242519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004d50 0x7f13b81f5a60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 >> 172.21.15.116:0/2047242519 conn(0x7f13b80ce570 msgr2=0x7f13b80cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 shutdown_connections 2024-05-14T11:22:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:18.197+0000 7f13be400640 1 -- 172.21.15.116:0/2047242519 wait complete. 2024-05-14T11:22:18.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:22:17.813387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6769}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6769': {'gid': 6769, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1421948627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 1421948627}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1421948627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:22:18.617 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6769:up:replay 2024-05-14T11:22:19.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:19.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.769+0000 7f7f0c800640 1 Processor -- start 2024-05-14T11:22:19.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.769+0000 7f7f0c800640 1 -- start start 2024-05-14T11:22:19.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0c800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7f040d1670 0x7f7f040d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0c800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0406ff20 con 0x7f7f0406c700 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0c800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f040d1fc0 con 0x7f7f0415db30 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0c800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7f040ae100 con 0x7f7f040d1670 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7f0406c700 0x7f7f0406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34262/0 (socket says 172.21.15.116:34262) 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f0a800640 1 -- 172.21.15.116:0/2028459884 learned_addr learned my addr 172.21.15.116:0/2028459884 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f09400640 1 --2- 172.21.15.116:0/2028459884 >> v2:172.21.15.149:3300/0 conn(0x7f7f040d1670 0x7f7f040d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f08a00640 1 -- 172.21.15.116:0/2028459884 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508156107 0 0) 0x7f7f0406ff20 con 0x7f7f0406c700 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f08a00640 1 -- 172.21.15.116:0/2028459884 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eec003530 con 0x7f7f0406c700 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f09400640 1 -- 172.21.15.116:0/2028459884 >> v1:172.21.15.149:6789/0 conn(0x7f7f0415db30 legacy=0x7f7f040aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f09400640 1 -- 172.21.15.116:0/2028459884 >> v1:172.21.15.116:6789/0 conn(0x7f7f0406c700 legacy=0x7f7f0406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.773+0000 7f7f09400640 1 -- 172.21.15.116:0/2028459884 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f04070470 con 0x7f7f040d1670 2024-05-14T11:22:19.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f09400640 1 --2- 172.21.15.116:0/2028459884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f040d1a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f040ae280 tx=0x7f7ef401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31fe9ecdbf14e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:19.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f08a00640 1 -- 172.21.15.116:0/2028459884 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40b0070 con 0x7f7f040d1670 2024-05-14T11:22:19.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f08a00640 1 -- 172.21.15.116:0/2028459884 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef40ab600 con 0x7f7f040d1670 2024-05-14T11:22:19.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f08a00640 1 -- 172.21.15.116:0/2028459884 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40ab9d0 con 0x7f7f040d1670 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 -- 172.21.15.116:0/2028459884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 msgr2=0x7f7f040d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 --2- 172.21.15.116:0/2028459884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f040d1a80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f040ae280 tx=0x7f7ef401c380 comp rx=0 tx=0).stop 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 -- 172.21.15.116:0/2028459884 shutdown_connections 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 --2- 172.21.15.116:0/2028459884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f040d1a80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 -- 172.21.15.116:0/2028459884 >> 172.21.15.116:0/2028459884 conn(0x7f7f040b4620 msgr2=0x7f7f040b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 -- 172.21.15.116:0/2028459884 shutdown_connections 2024-05-14T11:22:19.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.777+0000 7f7f0c800640 1 -- 172.21.15.116:0/2028459884 wait complete. 2024-05-14T11:22:19.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 Processor -- start 2024-05-14T11:22:19.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- start start 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7f0406c700 0x7f7f040cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f04099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0416bf90 con 0x7f7f0415db30 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0415df40 con 0x7f7f040d1670 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0416c110 con 0x7f7f0406c700 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46350/0 (socket says 172.21.15.116:46350) 2024-05-14T11:22:19.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 -- 172.21.15.116:0/1928142213 learned_addr learned my addr 172.21.15.116:0/1928142213 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f09400640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f04099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f09e00640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7f0406c700 0x7f7f040cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 -- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7f0406c700 msgr2=0x7f7f040cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7f0406c700 0x7f7f040cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 -- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 msgr2=0x7f7f04099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f04099ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 -- 172.21.15.116:0/1928142213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f040ce600 con 0x7f7f0415db30 2024-05-14T11:22:19.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f09400640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f04099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0a800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000caa0 tx=0x7f7f0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000ac00 con 0x7f7f0415db30 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0000ada0 con 0x7f7f0415db30 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f040ce8e0 con 0x7f7f0415db30 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00012620 con 0x7f7f0415db30 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.781+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f040ceea0 con 0x7f7f0415db30 2024-05-14T11:22:19.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f040ce350 con 0x7f7f0415db30 2024-05-14T11:22:19.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7f000127c0 con 0x7f7f0415db30 2024-05-14T11:22:19.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7efe000640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 0x7f7ee8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:19.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f7f0008faa0 con 0x7f7f0415db30 2024-05-14T11:22:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7f09e00640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 0x7f7ee8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:19.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.785+0000 7f7f09e00640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 0x7f7ee8071940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c10 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.789+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7f0005c7a0 con 0x7f7f0415db30 2024-05-14T11:22:19.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.981+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f0406a9c0 con 0x7f7f0415db30 2024-05-14T11:22:19.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.981+0000 7f7efe000640 1 -- 172.21.15.116:0/1928142213 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5186 (secure 0 0 0) 0x7f7f0005c0a0 con 0x7f7f0415db30 2024-05-14T11:22:19.983 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:19.983 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:18.820960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-14T11:22:19.983 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 66 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 msgr2=0x7f7ee8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 0x7f7ee8071940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c10 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 msgr2=0x7f7f040cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000caa0 tx=0x7f7f0000cf60 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 shutdown_connections 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7ee806f430 0x7f7ee8071940 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f040d1670 0x7f7f04099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f0415db30 0x7f7f040cf6a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 --2- 172.21.15.116:0/1928142213 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7f0406c700 0x7f7f040cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:19.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 >> 172.21.15.116:0/1928142213 conn(0x7f7f040b4620 msgr2=0x7f7f04158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:19.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 shutdown_connections 2024-05-14T11:22:19.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:19.985+0000 7f7f0c800640 1 -- 172.21.15.116:0/1928142213 wait complete. 2024-05-14T11:22:20.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:22:18.820960+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6769}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6769': {'gid': 6769, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.149:6835/1421948627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 1421948627}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1421948627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:22:20.407 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6769:up:rejoin 2024-05-14T11:22:21.408 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:21.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.565+0000 7f5cb1000640 1 Processor -- start 2024-05-14T11:22:21.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.565+0000 7f5cb1000640 1 -- start start 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5cb1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5cac0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5cb1000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cac06ff20 con 0x7f5cac0d1670 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5cb1000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cac070470 con 0x7f5cac06c700 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5cb1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac0ae100 con 0x7f5cac15db30 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5cac0d1670 0x7f5cac0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34274/0 (socket says 172.21.15.116:34274) 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 -- 172.21.15.116:0/1790586598 learned_addr learned my addr 172.21.15.116:0/1790586598 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:21.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9e00640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5cac0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087554516 0 0) 0x7f5cac06ff20 con 0x7f5cac0d1670 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c90003530 con 0x7f5cac0d1670 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823508044 0 0) 0x7f5cac070470 con 0x7f5cac06c700 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cac06ff20 con 0x7f5cac06c700 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 797158288 0 0) 0x7f5c90003530 con 0x7f5cac0d1670 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 msgr2=0x7f5cac0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:21.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5cac0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 >> v1:172.21.15.149:6789/0 conn(0x7f5cac06c700 legacy=0x7f5cac06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac070750 con 0x7f5cac0d1670 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f5c94001700 con 0x7f5cac0d1670 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 >> v1:172.21.15.116:6789/0 conn(0x7f5cac0d1670 legacy=0x7f5cac0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c90003c70 0x7f5c900040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5c90004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5c9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0ae100 con 0x7f5c90004810 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900055d0 con 0x7f5c90003c70 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9000bf10 con 0x7f5cac15db30 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5c90004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca9e00640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c90003c70 0x7f5c900040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5caa800640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5c9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 -- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 msgr2=0x7f5c9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:21.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15db30 0x7f5c9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 -- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c90003c70 msgr2=0x7f5c900040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c90003c70 0x7f5c900040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.569+0000 7f5ca1400640 1 -- 172.21.15.116:0/1790586598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90003530 con 0x7f5c90004810 2024-05-14T11:22:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5ca1400640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5c90004c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0d2370 tx=0x7f5c9401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2c5abeed79eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94007c50 con 0x7f5c90004810 2024-05-14T11:22:21.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c940a9070 con 0x7f5c90004810 2024-05-14T11:22:21.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5ca9400640 1 -- 172.21.15.116:0/1790586598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94005d20 con 0x7f5c90004810 2024-05-14T11:22:21.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 -- 172.21.15.116:0/1790586598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 msgr2=0x7f5c90004c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 --2- 172.21.15.116:0/1790586598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5c90004c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0d2370 tx=0x7f5c9401ca60 comp rx=0 tx=0).stop 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5caa800640 1 -- 172.21.15.116:0/1790586598 reap_dead start 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 -- 172.21.15.116:0/1790586598 shutdown_connections 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 -- 172.21.15.116:0/1790586598 >> 172.21.15.116:0/1790586598 conn(0x7f5cac0b4620 msgr2=0x7f5cac0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 -- 172.21.15.116:0/1790586598 shutdown_connections 2024-05-14T11:22:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.573+0000 7f5cb1000640 1 -- 172.21.15.116:0/1790586598 wait complete. 2024-05-14T11:22:21.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 Processor -- start 2024-05-14T11:22:21.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 -- start start 2024-05-14T11:22:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 0x7f5cac14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 0x7f5cac0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0af030 con 0x7f5c90004810 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0aeeb0 con 0x7f5cac14e000 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5cb1000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac169f80 con 0x7f5cac15ca40 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46376/0 (socket says 172.21.15.116:46376) 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 -- 172.21.15.116:0/4221563324 learned_addr learned my addr 172.21.15.116:0/4221563324 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca1400640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 0x7f5cac14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5caa800640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 0x7f5cac0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 -- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 msgr2=0x7f5cac14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 0x7f5cac14dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 -- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 msgr2=0x7f5cac0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 0x7f5cac0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca9e00640 1 -- 172.21.15.116:0/4221563324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac14e410 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5ca1400640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 0x7f5cac14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.577+0000 7f5caa800640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 0x7f5cac0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca9e00640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0119d0 tx=0x7f5c9c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00fb50 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00fcf0 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00fea0 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac148ec0 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac0b1630 con 0x7f5c90004810 2024-05-14T11:22:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac06a9c0 con 0x7f5c90004810 2024-05-14T11:22:21.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c9c012040 con 0x7f5c90004810 2024-05-14T11:22:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.581+0000 7f5ca2a00640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 0x7f5c88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.585+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f5c9c08fad0 con 0x7f5c90004810 2024-05-14T11:22:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.585+0000 7f5ca1400640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 0x7f5c88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.585+0000 7f5ca1400640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 0x7f5c88071940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5c940ab000 tx=0x7f5c940a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.589+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5cac14e410 con 0x7f5c90004810 2024-05-14T11:22:21.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.777+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cac15ea90 con 0x7f5c90004810 2024-05-14T11:22:21.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.777+0000 7f5ca2a00640 1 -- 172.21.15.116:0/4221563324 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5193 (secure 0 0 0) 0x7f5c9c019030 con 0x7f5c90004810 2024-05-14T11:22:21.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:21.777 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:20.818975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6769,"qdb_cluster":[6769]},"id":6}]} 2024-05-14T11:22:21.777 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 67 2024-05-14T11:22:21.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.777+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 msgr2=0x7f5c88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 0x7f5c88071940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5c940ab000 tx=0x7f5c940a7070 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 msgr2=0x7f5cac148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0119d0 tx=0x7f5c9c011e90 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 shutdown_connections 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5c8806f430 0x7f5c88071940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cac14e000 0x7f5cac0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5cac15ca40 0x7f5cac14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 --2- 172.21.15.116:0/4221563324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90004810 0x7f5cac148550 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 >> 172.21.15.116:0/4221563324 conn(0x7f5cac0b4620 msgr2=0x7f5cac15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 shutdown_connections 2024-05-14T11:22:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:21.781+0000 7f5cb1000640 1 -- 172.21.15.116:0/4221563324 wait complete. 2024-05-14T11:22:22.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:22:20.818975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6769}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6769': {'gid': 6769, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.149:6835/1421948627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 1421948627}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1421948627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6769, 'qdb_cluster': [6769]} 2024-05-14T11:22:22.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:22:22.203 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-14T11:22:22.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.349+0000 7f9502c00640 1 Processor -- start 2024-05-14T11:22:22.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 -- start start 2024-05-14T11:22:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f94fc0b0770 0x7f94fc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94fc15fc00 0x7f94fc160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f94fc06db20 con 0x7f94fc15fc00 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f94fc070080 con 0x7f94fc0b0770 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9502c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94fc06cc40 con 0x7f94fc06f730 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94fc15fc00 0x7f94fc160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94fc15fc00 0x7f94fc160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46392/0 (socket says 172.21.15.116:46392) 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 -- 172.21.15.116:0/1950545115 learned_addr learned my addr 172.21.15.116:0/1950545115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f94fb400640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1950545115 >> v2:172.21.15.149:3300/0 conn(0x7f94fc0b0770 0x7f94fc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 -- 172.21.15.116:0/1950545115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 msgr2=0x7f94fc06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc06fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 -- 172.21.15.116:0/1950545115 >> v2:172.21.15.149:3300/0 conn(0x7f94fc0b0770 msgr2=0x7f94fc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 --2- 172.21.15.116:0/1950545115 >> v2:172.21.15.149:3300/0 conn(0x7f94fc0b0770 0x7f94fc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.353+0000 7f9500c00640 1 -- 172.21.15.116:0/1950545115 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc06cf10 con 0x7f94fc15fc00 2024-05-14T11:22:22.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9500c00640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc15fc00 0x7f94fc160010 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f94fc071f20 tx=0x7f94f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499498abcf153625 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f94faa00640 1 -- 172.21.15.116:0/1950545115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b0070 con 0x7f94fc15fc00 2024-05-14T11:22:22.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f94faa00640 1 -- 172.21.15.116:0/1950545115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f40ab600 con 0x7f94fc15fc00 2024-05-14T11:22:22.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f94faa00640 1 -- 172.21.15.116:0/1950545115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40ab9d0 con 0x7f94fc15fc00 2024-05-14T11:22:22.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 -- 172.21.15.116:0/1950545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc15fc00 msgr2=0x7f94fc160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc15fc00 0x7f94fc160010 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f94fc071f20 tx=0x7f94f401c490 comp rx=0 tx=0).stop 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 -- 172.21.15.116:0/1950545115 shutdown_connections 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc15fc00 0x7f94fc160010 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 --2- 172.21.15.116:0/1950545115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 --2- 172.21.15.116:0/1950545115 >> v2:172.21.15.149:3300/0 conn(0x7f94fc0b0770 0x7f94fc06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 -- 172.21.15.116:0/1950545115 >> 172.21.15.116:0/1950545115 conn(0x7f94fc0b4620 msgr2=0x7f94fc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 -- 172.21.15.116:0/1950545115 shutdown_connections 2024-05-14T11:22:22.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.357+0000 7f9502c00640 1 -- 172.21.15.116:0/1950545115 wait complete. 2024-05-14T11:22:22.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 Processor -- start 2024-05-14T11:22:22.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 -- start start 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94fc15fc00 0x7f94fc157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc160960 con 0x7f94fc0b0770 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc06dfb0 con 0x7f94fc15fc00 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9502c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f94fc168fa0 con 0x7f94fc06f730 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46404/0 (socket says 172.21.15.116:46404) 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 -- 172.21.15.116:0/1736685430 learned_addr learned my addr 172.21.15.116:0/1736685430 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f9500c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94fc15fc00 0x7f94fc157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 -- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 msgr2=0x7f94fc154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 -- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94fc15fc00 msgr2=0x7f94fc157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94fc15fc00 0x7f94fc157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 -- 172.21.15.116:0/1736685430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc153690 con 0x7f94fc0b0770 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94fb400640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f94e800e870 tx=0x7f94e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94f3e00640 1 -- 172.21.15.116:0/1736685430 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8017070 con 0x7f94fc0b0770 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94f3e00640 1 -- 172.21.15.116:0/1736685430 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e800c940 con 0x7f94fc0b0770 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.361+0000 7f94f3e00640 1 -- 172.21.15.116:0/1736685430 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e800ccf0 con 0x7f94fc0b0770 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc153b90 con 0x7f94fc0b0770 2024-05-14T11:22:22.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc154150 con 0x7f94fc0b0770 2024-05-14T11:22:22.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f94f3e00640 1 -- 172.21.15.116:0/1736685430 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f94e801a070 con 0x7f94fc0b0770 2024-05-14T11:22:22.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f94f3e00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 0x7f94dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f94f3e00640 1 -- 172.21.15.116:0/1736685430 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f94e808f300 con 0x7f94fc0b0770 2024-05-14T11:22:22.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 0x7f94dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.365+0000 7f950a231000 1 Processor -- start 2024-05-14T11:22:22.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- start start 2024-05-14T11:22:22.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 0x7f94dc071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0d1240 tx=0x7f94ec00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a25619860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25619da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555a254b6620 con 0x555a250df7c0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555a253d1740 con 0x555a254b16b0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555a2541f090 con 0x555a254c7f80 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46408/0 (socket says 172.21.15.116:46408) 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 -- 172.21.15.116:0/1711080301 learned_addr learned my addr 172.21.15.116:0/1711080301 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fb400640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a25619860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f9500c00640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25619da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 -- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 msgr2=0x555a25619860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a25619860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 -- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 msgr2=0x555a25619da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25619da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 -- 172.21.15.116:0/1711080301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a255b30a0 con 0x555a250df7c0 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f9500c00640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25619da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f94ec017740 tx=0x7f94ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5522250394d0db6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94e7e00640 1 -- 172.21.15.116:0/1711080301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec027070 con 0x555a250df7c0 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94e7e00640 1 -- 172.21.15.116:0/1711080301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec015a60 con 0x555a250df7c0 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f94e7e00640 1 -- 172.21.15.116:0/1711080301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec01b070 con 0x555a250df7c0 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- 172.21.15.116:0/1711080301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 msgr2=0x555a25619320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f94ec017740 tx=0x7f94ec014000 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- 172.21.15.116:0/1711080301 shutdown_connections 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25619da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a25619860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 --2- 172.21.15.116:0/1711080301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a250df7c0 0x555a25619320 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- 172.21.15.116:0/1711080301 >> 172.21.15.116:0/1711080301 conn(0x555a2559e340 msgr2=0x555a255cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- 172.21.15.116:0/1711080301 shutdown_connections 2024-05-14T11:22:22.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.369+0000 7f950a231000 1 -- 172.21.15.116:0/1711080301 wait complete. 2024-05-14T11:22:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 -- start start 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 Processor -- start 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a250df7c0 0x555a2561f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a255fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555a2541b050 con 0x555a254b16b0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555a253d7ec0 con 0x555a250df7c0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555a253bec90 con 0x555a254c7f80 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f950a231000 10 client.0 _ll_register_callbacks cb 0x555a255bae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46414/0 (socket says 172.21.15.116:46414) 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 -- 172.21.15.116:0/1299329625 learned_addr learned my addr 172.21.15.116:0/1299329625 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a250df7c0 0x555a2561f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f9500c00640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a255fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 -- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 msgr2=0x555a255fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a255fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 -- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a250df7c0 msgr2=0x555a2561f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a250df7c0 0x555a2561f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 -- 172.21.15.116:0/1299329625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a254c96f0 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94fb400640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c520 tx=0x7f94e80579a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947381fc84ea2c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8021040 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.373+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f94e805ae10 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ec015a60 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f94e805b4c0 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 10 client.7095 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x555a254c56a0 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec015a60 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f94e8024060 con 0x555a254b16b0 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 10 client.7095 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 10 client.7095 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 20 client.7095 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250c47d0 0x555a255fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 4 client.7095 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a253b7be0 con 0x555a250c47d0 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250c47d0 0x555a255fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250c47d0 0x555a255fe490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be4b38b97fef5ec server_cookie=4f6b34a6d0f00556 in_seq=0 out_seq=0 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 10 client.7095 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 1 -- 172.21.15.116:0/1299329625 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a253b7be0 con 0x555a250c47d0 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 10 client.7095 handle_command_reply: tid=0 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 client.7095 shutdown 2024-05-14T11:22:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 msgr2=0x555a2561fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c520 tx=0x7f94e80579a0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 shutdown_connections 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250c47d0 0x555a255fe490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a254c7f80 0x555a255fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561fec0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f94e4c00640 0 client.7095 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 --2- 172.21.15.116:0/1299329625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a250df7c0 0x555a2561f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 >> 172.21.15.116:0/1299329625 conn(0x555a2559e340 msgr2=0x555a255ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 shutdown_connections 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.377+0000 7f950a231000 1 -- 172.21.15.116:0/1299329625 wait complete. 2024-05-14T11:22:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.381+0000 7f950a231000 20 client.7095 trim_cache size 0 max 16384 2024-05-14T11:22:22.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 Processor -- start 2024-05-14T11:22:22.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 -- start start 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a256874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25687a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc15f540 con 0x555a254c7f80 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc204fe0 con 0x555a254b16b0 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.397+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555a2541e3a0 con 0x555a250df7c0 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46426/0 (socket says 172.21.15.116:46426) 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 -- 172.21.15.116:0/2156362783 learned_addr learned my addr 172.21.15.116:0/2156362783 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f94fbe00640 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a256874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f94fb400640 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25687a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 -- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 msgr2=0x555a256874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a256874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 -- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 msgr2=0x555a25687a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25687a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 -- 172.21.15.116:0/2156362783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a253f8af0 con 0x555a254c7f80 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f9500c00640 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f94f4008010 tx=0x7f94f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976164f318d1ee10 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f94e7400640 1 -- 172.21.15.116:0/2156362783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b0040 con 0x555a254c7f80 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f94e7400640 1 -- 172.21.15.116:0/2156362783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f40ab790 con 0x555a254c7f80 2024-05-14T11:22:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f94e7400640 1 -- 172.21.15.116:0/2156362783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40abb40 con 0x555a254c7f80 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- 172.21.15.116:0/2156362783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 msgr2=0x555a25687f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f94f4008010 tx=0x7f94f4004740 comp rx=0 tx=0).stop 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- 172.21.15.116:0/2156362783 shutdown_connections 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254b16b0 0x555a25687a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254c7f80 0x555a25687f40 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 --2- 172.21.15.116:0/2156362783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a256874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- 172.21.15.116:0/2156362783 >> 172.21.15.116:0/2156362783 conn(0x555a2559e340 msgr2=0x555a256257d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- 172.21.15.116:0/2156362783 shutdown_connections 2024-05-14T11:22:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- 172.21.15.116:0/2156362783 wait complete. 2024-05-14T11:22:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 -- start start 2024-05-14T11:22:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.401+0000 7f950a231000 1 Processor -- start 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a2561d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254c7f80 0x555a25648480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555a253b9220 con 0x555a254b16b0 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555a253f53a0 con 0x555a254c7f80 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555a254d9d80 con 0x555a250df7c0 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 10 client.0 _ll_register_callbacks cb 0x7f94ec009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46428/0 (socket says 172.21.15.116:46428) 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 -- 172.21.15.116:0/1899366801 learned_addr learned my addr 172.21.15.116:0/1899366801 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a2561d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f9500c00640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254c7f80 0x555a25648480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 -- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 msgr2=0x555a2561d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a2561d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 -- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254c7f80 msgr2=0x555a25648480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254c7f80 0x555a25648480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 -- 172.21.15.116:0/1899366801 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a253f7e40 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fb400640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f94e805b660 tx=0x7f94e808d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8372c792bb4ffa39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8021040 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f94e8057520 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f40ab790 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f94e8058000 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 10 client.7101 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x555a2549f4e0 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f40ab790 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f94e805a680 con 0x555a254b16b0 2024-05-14T11:22:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 10 client.7101 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 10 client.7101 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 20 client.7101 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250b2830 0x555a25648e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 4 client.7101 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "status"}) -- 0x555a2542cca0 con 0x555a250b2830 2024-05-14T11:22:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250b2830 0x555a25648e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94e4c00640 10 client.7101 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.405+0000 7f94fbe00640 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250b2830 0x555a25648e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356c4fd95e2c1b78 server_cookie=da128d3d87a6bc74 in_seq=0 out_seq=0 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f94e4c00640 1 -- 172.21.15.116:0/1899366801 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x555a2542cca0 con 0x555a250b2830 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f94e4c00640 10 client.7101 handle_command_reply: tid=0 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 client.7101 shutdown 2024-05-14T11:22:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 msgr2=0x555a2561de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f94e805b660 tx=0x7f94e808d480 comp rx=0 tx=0).stop 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 shutdown_connections 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a254c7f80 0x555a25648480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x555a250b2830 0x555a25648e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555a254b16b0 0x555a2561de50 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 --2- 172.21.15.116:0/1899366801 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555a250df7c0 0x555a2561d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f94e4c00640 0 client.7101 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 >> 172.21.15.116:0/1899366801 conn(0x555a2559e340 msgr2=0x555a256257d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 shutdown_connections 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 1 -- 172.21.15.116:0/1899366801 wait complete. 2024-05-14T11:22:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f950a231000 20 client.7101 trim_cache size 0 max 16384 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 6769, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.591835347, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 67, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 85, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 85, 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 5.4695400010000004 2024-05-14T11:22:22.411 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 msgr2=0x7f94dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 0x7f94dc071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0d1240 tx=0x7f94ec00b040 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 msgr2=0x7f94fc150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f94e800e870 tx=0x7f94e800ec70 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 shutdown_connections 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f94dc06f400 0x7f94dc071910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94fc15fc00 0x7f94fc157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94fc0b0770 0x7f94fc150880 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 --2- 172.21.15.116:0/1736685430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f94fc06f730 0x7f94fc154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 >> 172.21.15.116:0/1736685430 conn(0x7f94fc0b4620 msgr2=0x7f94fc15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 shutdown_connections 2024-05-14T11:22:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.409+0000 7f9502c00640 1 -- 172.21.15.116:0/1736685430 wait complete. 2024-05-14T11:22:22.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:22:22.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 Processor -- start 2024-05-14T11:22:22.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 -- start start 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc94406f730 0x7fc94406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc944072a30 0x7fc944072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc944070080 con 0x7fc9440b0770 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc94406db20 con 0x7fc94406f730 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc949a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc94406cf10 con 0x7fc944072a30 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc94406f730 0x7fc94406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc94406f730 0x7fc94406fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46430/0 (socket says 172.21.15.116:46430) 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 -- 172.21.15.116:0/4153143728 learned_addr learned my addr 172.21.15.116:0/4153143728 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc943400640 1 --2- 172.21.15.116:0/4153143728 >> v2:172.21.15.149:3300/0 conn(0x7fc944072a30 0x7fc944072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 -- 172.21.15.116:0/4153143728 >> v2:172.21.15.149:3300/0 conn(0x7fc944072a30 msgr2=0x7fc944072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 --2- 172.21.15.116:0/4153143728 >> v2:172.21.15.149:3300/0 conn(0x7fc944072a30 0x7fc944072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 -- 172.21.15.116:0/4153143728 >> v1:172.21.15.149:6789/0 conn(0x7fc9440b0770 legacy=0x7fc94406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.577+0000 7fc93a000640 1 -- 172.21.15.116:0/4153143728 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94406d1f0 con 0x7fc94406f730 2024-05-14T11:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc93a000640 1 --2- 172.21.15.116:0/4153143728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc94406f730 0x7fc94406fb40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc9440729c0 tx=0x7fc93401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f79bd5bc51f154 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc942000640 1 -- 172.21.15.116:0/4153143728 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9340b1070 con 0x7fc94406f730 2024-05-14T11:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc942000640 1 -- 172.21.15.116:0/4153143728 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9340a8050 con 0x7fc94406f730 2024-05-14T11:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc942000640 1 -- 172.21.15.116:0/4153143728 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9340ac860 con 0x7fc94406f730 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 -- 172.21.15.116:0/4153143728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc94406f730 msgr2=0x7fc94406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 --2- 172.21.15.116:0/4153143728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc94406f730 0x7fc94406fb40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc9440729c0 tx=0x7fc93401c460 comp rx=0 tx=0).stop 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 -- 172.21.15.116:0/4153143728 shutdown_connections 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 --2- 172.21.15.116:0/4153143728 >> v2:172.21.15.149:3300/0 conn(0x7fc944072a30 0x7fc944072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 --2- 172.21.15.116:0/4153143728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc94406f730 0x7fc94406fb40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 -- 172.21.15.116:0/4153143728 >> 172.21.15.116:0/4153143728 conn(0x7fc9440b4620 msgr2=0x7fc9440b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 -- 172.21.15.116:0/4153143728 shutdown_connections 2024-05-14T11:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.581+0000 7fc949a00640 1 -- 172.21.15.116:0/4153143728 wait complete. 2024-05-14T11:22:22.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 Processor -- start 2024-05-14T11:22:22.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- start start 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc94406f730 0x7fc944157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 0x7fc94415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc944073500 con 0x7fc9440b0770 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc944073380 con 0x7fc944072a30 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc944073680 con 0x7fc94406f730 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46444/0 (socket says 172.21.15.116:46444) 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 -- 172.21.15.116:0/1396847005 learned_addr learned my addr 172.21.15.116:0/1396847005 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc93a000640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 0x7fc94415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc942a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc94406f730 0x7fc944157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 -- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc94406f730 msgr2=0x7fc944157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc94406f730 0x7fc944157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 -- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 msgr2=0x7fc94415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 0x7fc94415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 -- 172.21.15.116:0/1396847005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94409c1e0 con 0x7fc9440b0770 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc93a000640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 0x7fc94415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc943400640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009de0 tx=0x7fc93c0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c019070 con 0x7fc9440b0770 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c003ba0 con 0x7fc9440b0770 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9440ce400 con 0x7fc9440b0770 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c007e60 con 0x7fc9440b0770 2024-05-14T11:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.585+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94409bc70 con 0x7fc9440b0770 2024-05-14T11:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94409bf30 con 0x7fc9440b0770 2024-05-14T11:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc93c01c070 con 0x7fc9440b0770 2024-05-14T11:22:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc93b400640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7fc93c08f300 con 0x7fc9440b0770 2024-05-14T11:22:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc942a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.589+0000 7fc942a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 0x7fc920071910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9440d1240 tx=0x7fc92c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.593+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc93c011070 con 0x7fc9440b0770 2024-05-14T11:22:22.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc94406a9c0 con 0x7fc9440b0770 2024-05-14T11:22:22.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc93b400640 1 -- 172.21.15.116:0/1396847005 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fc93c05c000 con 0x7fc9440b0770 2024-05-14T11:22:22.789 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 msgr2=0x7fc920071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 0x7fc920071910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9440d1240 tx=0x7fc92c009040 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 msgr2=0x7fc94415fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009de0 tx=0x7fc93c0075c0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 shutdown_connections 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc944072a30 0x7fc94415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9440b0770 0x7fc94415fd00 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 --2- 172.21.15.116:0/1396847005 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc94406f730 0x7fc944157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 >> 172.21.15.116:0/1396847005 conn(0x7fc9440b4620 msgr2=0x7fc9440b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 shutdown_connections 2024-05-14T11:22:22.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:22.789+0000 7fc949a00640 1 -- 172.21.15.116:0/1396847005 wait complete. 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:22:22.810 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:22.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:22.811 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:22:22.817 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:22:22.819+0000 7f7714a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:22:22.819 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:22:22.819 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:22:22.821 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42060sec preferred_lft 42060sec 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:22:22.822 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:22:22.823 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:22:22.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:22.824 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:22:22.884 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:22:22.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:22.886 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:22.938 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:22:22.939 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:22:22.939 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:22:23.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:22:23.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:22:23.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.072 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:22:23.082 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:22:23.082 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:22:23.082 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:23.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.082 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:22:23.142 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:22:23.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:22:23.142 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:23.190 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:22:23.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:22:23.191 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:23.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:22:23.302 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:22:23.352 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:22:23.353 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:23.354 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:23.355 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:23.355 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:23.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.356 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:22:23.406 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:22:23.406 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:22:23.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:22:23.469 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:22:23.520 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:22:23.520 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:22:23.521 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:22:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:23.523 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:22:23.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.524 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:22:23.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:22:23.553+0000 7fcf0cb0f580 -1 init, newargv = 0x7fcf00001410 newargc=15 2024-05-14T11:22:23.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44606]: starting ceph client 2024-05-14T11:22:23.562 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:22:23.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44606]: starting fuse 2024-05-14T11:22:23.563 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:22:23.563 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:22:23.564 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:22:23.565 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:22:23.565 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:22:23.565 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:22:23.691 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.44606.asok 2024-05-14T11:22:23.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:22:23.702 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44606.asok 2024-05-14T11:22:23.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.702 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44606.asok status 2024-05-14T11:22:23.879 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "44606", 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7116, 2024-05-14T11:22:23.880 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 7116 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3855307054 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3855307054 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:23.881 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.7116 192.168.0.1:0/3855307054", 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/3855307054", 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 67, 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 85, 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:22:23.882 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:23.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.895 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:22:23.895 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:23.905 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:22:23.905 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:23.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.905 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:22:23.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:23.966 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-14T11:22:24.024 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:22:24.023+0000 7f770fa00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-14T11:22:24.025 INFO:teuthology.orchestra.run.smithi116.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-14T11:22:24.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:22:24.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:24.026 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-14T11:22:24.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 Processor -- start 2024-05-14T11:22:24.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 -- start start 2024-05-14T11:22:24.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 0x7fda180cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda180d14b0 0x7fda180d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda180d1e00 con 0x7fda1806c700 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fda180d00d0 con 0x7fda180d14b0 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1d800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fda1806fa90 con 0x7fda1815cd00 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda180d14b0 0x7fda180d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda180d14b0 0x7fda180d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46460/0 (socket says 172.21.15.116:46460) 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 -- 172.21.15.116:0/1328204191 learned_addr learned my addr 172.21.15.116:0/1328204191 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda1ce00640 1 --1- 172.21.15.116:0/1328204191 >> v1:172.21.15.116:6789/0 conn(0x7fda1806c700 0x7fda1806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34282/0 (socket says 172.21.15.116:34282) 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17e00640 1 --2- 172.21.15.116:0/1328204191 >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 0x7fda180cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 -- 172.21.15.116:0/1328204191 >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 msgr2=0x7fda180cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 --2- 172.21.15.116:0/1328204191 >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 0x7fda180cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 -- 172.21.15.116:0/1328204191 >> v1:172.21.15.116:6789/0 conn(0x7fda1806c700 legacy=0x7fda1806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17400640 1 -- 172.21.15.116:0/1328204191 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1806fd70 con 0x7fda180d14b0 2024-05-14T11:22:24.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.217+0000 7fda17e00640 1 --2- 172.21.15.116:0/1328204191 >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 0x7fda180cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:24.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda17400640 1 --2- 172.21.15.116:0/1328204191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda180d14b0 0x7fda180d18c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fda18071d00 tx=0x7fda0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249ea23bc2e0008f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda16a00640 1 -- 172.21.15.116:0/1328204191 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040aa650 con 0x7fda180d14b0 2024-05-14T11:22:24.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda16a00640 1 -- 172.21.15.116:0/1328204191 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda040af070 con 0x7fda180d14b0 2024-05-14T11:22:24.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda16a00640 1 -- 172.21.15.116:0/1328204191 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040aaa10 con 0x7fda180d14b0 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 -- 172.21.15.116:0/1328204191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda180d14b0 msgr2=0x7fda180d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 --2- 172.21.15.116:0/1328204191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda180d14b0 0x7fda180d18c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fda18071d00 tx=0x7fda0401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 -- 172.21.15.116:0/1328204191 shutdown_connections 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 --2- 172.21.15.116:0/1328204191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda180d14b0 0x7fda180d18c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 --2- 172.21.15.116:0/1328204191 >> v2:172.21.15.149:3300/0 conn(0x7fda1815cd00 0x7fda180cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 -- 172.21.15.116:0/1328204191 >> 172.21.15.116:0/1328204191 conn(0x7fda180b43a0 msgr2=0x7fda180b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 -- 172.21.15.116:0/1328204191 shutdown_connections 2024-05-14T11:22:24.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.221+0000 7fda1d800640 1 -- 172.21.15.116:0/1328204191 wait complete. 2024-05-14T11:22:24.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 Processor -- start 2024-05-14T11:22:24.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 -- start start 2024-05-14T11:22:24.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 0x7fda18155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda180d14b0 0x7fda18157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda180b0560 con 0x7fda1815cd00 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda180b03e0 con 0x7fda1806c700 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda18167f60 con 0x7fda180d14b0 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46474/0 (socket says 172.21.15.116:46474) 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 -- 172.21.15.116:0/2368403767 learned_addr learned my addr 172.21.15.116:0/2368403767 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda17e00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 0x7fda18155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda17400640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda180d14b0 0x7fda18157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 -- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda180d14b0 msgr2=0x7fda18157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda180d14b0 0x7fda18157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 -- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 msgr2=0x7fda18155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 0x7fda18155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 -- 172.21.15.116:0/2368403767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda18156e80 con 0x7fda1815cd00 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda17e00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 0x7fda18155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:24.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1ce00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c760 tx=0x7fda1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda0fe00640 1 -- 172.21.15.116:0/2368403767 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10017070 con 0x7fda1815cd00 2024-05-14T11:22:24.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda0fe00640 1 -- 172.21.15.116:0/2368403767 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1000a5e0 con 0x7fda1815cd00 2024-05-14T11:22:24.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda18157160 con 0x7fda1815cd00 2024-05-14T11:22:24.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.225+0000 7fda0fe00640 1 -- 172.21.15.116:0/2368403767 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1000a9d0 con 0x7fda1815cd00 2024-05-14T11:22:24.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda18157720 con 0x7fda1815cd00 2024-05-14T11:22:24.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda0fe00640 1 -- 172.21.15.116:0/2368403767 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda1001a070 con 0x7fda1815cd00 2024-05-14T11:22:24.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda0fe00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 0x7fd9f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda0fe00640 1 -- 172.21.15.116:0/2368403767 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7fda1008ea50 con 0x7fda1815cd00 2024-05-14T11:22:24.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda263fe000 1 Processor -- start 2024-05-14T11:22:24.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda17e00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 0x7fd9f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.229+0000 7fda263fe000 1 -- start start 2024-05-14T11:22:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda17e00640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 0x7fd9f8071910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fda0800a9b0 tx=0x7fda08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d47130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b722be3a80 con 0x55b7227cd570 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b722b4c1c0 con 0x55b722bdeb10 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b722b04d60 con 0x55b722bf57a0 2024-05-14T11:22:24.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46478/0 (socket says 172.21.15.116:46478) 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 -- 172.21.15.116:0/1228676121 learned_addr learned my addr 172.21.15.116:0/1228676121 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda17400640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d47130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda17e00640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 -- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 msgr2=0x55b722d46bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d46bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 -- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 msgr2=0x55b722d47130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d47130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 -- 172.21.15.116:0/1228676121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b722b098c0 con 0x55b7227cd570 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda17400640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d47130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda17e00640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d46bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda1ce00640 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fda10057660 tx=0x7fda1005ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab76f4dc6fb25a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda0cc00640 1 -- 172.21.15.116:0/1228676121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1005f370 con 0x55b7227cd570 2024-05-14T11:22:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda0cc00640 1 -- 172.21.15.116:0/1228676121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda10063e20 con 0x55b7227cd570 2024-05-14T11:22:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda0cc00640 1 -- 172.21.15.116:0/1228676121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10063bd0 con 0x55b7227cd570 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- 172.21.15.116:0/1228676121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 msgr2=0x55b722d47670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fda10057660 tx=0x7fda1005ab80 comp rx=0 tx=0).stop 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- 172.21.15.116:0/1228676121 shutdown_connections 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d47130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d47670 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 --2- 172.21.15.116:0/1228676121 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d46bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- 172.21.15.116:0/1228676121 >> 172.21.15.116:0/1228676121 conn(0x55b72280c750 msgr2=0x55b722cfb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- 172.21.15.116:0/1228676121 shutdown_connections 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.233+0000 7fda263fe000 1 -- 172.21.15.116:0/1228676121 wait complete. 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- start start 2024-05-14T11:22:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 Processor -- start 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b7227cd570 0x55b722d475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b722b16cb0 con 0x55b722bf57a0 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b722c075a0 con 0x55b722bdeb10 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b722b08140 con 0x55b7227cd570 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 10 client.0 _ll_register_callbacks cb 0x55b722ce8520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46492/0 (socket says 172.21.15.116:46492) 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 -- 172.21.15.116:0/3657323827 learned_addr learned my addr 172.21.15.116:0/3657323827 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda17400640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda17e00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b7227cd570 0x55b722d475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 -- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b7227cd570 msgr2=0x55b722d475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b7227cd570 0x55b722d475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 -- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 msgr2=0x55b722d079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d079a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 -- 172.21.15.116:0/3657323827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b722af6920 con 0x55b722bf57a0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda1ce00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fda10057660 tx=0x7fda10001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31775cc2612258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1005f370 con 0x55b722bf57a0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7fda10071ca0 con 0x55b722bf57a0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda10063e20 con 0x55b722bf57a0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fda10071ea0 con 0x55b722bf57a0 2024-05-14T11:22:24.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 10 client.7131 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b722b04fb0 con 0x55b722bf57a0 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda10063e20 con 0x55b722bf57a0 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fda1005d070 con 0x55b722bf57a0 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 10 client.7131 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 10 client.7131 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 20 client.7131 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227f1ac0 0x55b722d08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 4 client.7131 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.237+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b722ae4f90 con 0x55b7227f1ac0 2024-05-14T11:22:24.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda17e00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227f1ac0 0x55b722d08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda17e00640 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227f1ac0 0x55b722d08c80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf463d0bbb8d9adc server_cookie=fca2f16ee8a96c5d in_seq=0 out_seq=0 2024-05-14T11:22:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda01600640 10 client.7131 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda01600640 1 -- 172.21.15.116:0/3657323827 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b722ae4f90 con 0x55b7227f1ac0 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda01600640 10 client.7131 handle_command_reply: tid=0 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 client.7131 shutdown 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 msgr2=0x55b722d07ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fda10057660 tx=0x7fda10001260 comp rx=0 tx=0).stop 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 shutdown_connections 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227f1ac0 0x55b722d08c80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bf57a0 0x55b722d07ee0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 --2- 172.21.15.116:0/3657323827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b7227cd570 0x55b722d475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda01600640 0 client.7131 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 >> 172.21.15.116:0/3657323827 conn(0x55b72280c750 msgr2=0x55b722cfb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 shutdown_connections 2024-05-14T11:22:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 1 -- 172.21.15.116:0/3657323827 wait complete. 2024-05-14T11:22:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.241+0000 7fda263fe000 20 client.7131 trim_cache size 0 max 16384 2024-05-14T11:22:24.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.257+0000 7fda263fe000 1 Processor -- start 2024-05-14T11:22:24.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.257+0000 7fda263fe000 1 -- start start 2024-05-14T11:22:24.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722cfa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda182056f0 con 0x55b7227cd570 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda18073530 con 0x55b722bdeb10 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda18073b50 con 0x55b722bf57a0 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46502/0 (socket says 172.21.15.116:46502) 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 -- 172.21.15.116:0/3542243527 learned_addr learned my addr 172.21.15.116:0/3542243527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda1ce00640 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722cfa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17400640 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 -- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 msgr2=0x55b722cfa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722cfa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 -- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 msgr2=0x55b722d2a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d2a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 -- 172.21.15.116:0/3542243527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b722bcd0d0 con 0x55b7227cd570 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda17e00640 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fda0800f5b0 tx=0x7fda08015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d720f00e8a603cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda03e00640 1 -- 172.21.15.116:0/3542243527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0802f070 con 0x55b7227cd570 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda03e00640 1 -- 172.21.15.116:0/3542243527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda08014d70 con 0x55b7227cd570 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda03e00640 1 -- 172.21.15.116:0/3542243527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08029450 con 0x55b7227cd570 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- 172.21.15.116:0/3542243527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 msgr2=0x55b722d29d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fda0800f5b0 tx=0x7fda08015000 comp rx=0 tx=0).stop 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- 172.21.15.116:0/3542243527 shutdown_connections 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b722bdeb10 0x55b722d2a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722cfa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 --2- 172.21.15.116:0/3542243527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7227cd570 0x55b722d29d90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- 172.21.15.116:0/3542243527 >> 172.21.15.116:0/3542243527 conn(0x55b72280c750 msgr2=0x55b722d5d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- 172.21.15.116:0/3542243527 shutdown_connections 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.261+0000 7fda263fe000 1 -- 172.21.15.116:0/3542243527 wait complete. 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- start start 2024-05-14T11:22:24.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 Processor -- start 2024-05-14T11:22:24.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7227cd570 0x55b722cfe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d75e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b722b507f0 con 0x55b722bdeb10 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b722b4b4d0 con 0x55b7227cd570 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b722b595c0 con 0x55b722bf57a0 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 10 client.0 _ll_register_callbacks cb 0x7fda1005aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46512/0 (socket says 172.21.15.116:46512) 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 -- 172.21.15.116:0/3428785900 learned_addr learned my addr 172.21.15.116:0/3428785900 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17e00640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7227cd570 0x55b722cfe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda1ce00640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d75e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 -- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 msgr2=0x55b722d75e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d75e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 -- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7227cd570 msgr2=0x55b722cfe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7227cd570 0x55b722cfe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 -- 172.21.15.116:0/3428785900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b722ba3bc0 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda17400640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fda0401a120 tx=0x7fda0401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9f14ba97981f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda01600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040c2070 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7fda040bd480 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda08014d70 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fda040cd050 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 10 client.7137 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.265+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b7228053b0 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda08014d70 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fda040d0020 con 0x55b722bdeb10 2024-05-14T11:22:24.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 10 client.7137 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 10 client.7137 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 20 client.7137 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227dfb20 0x55b722db9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 4 client.7137 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55b722b59dd0 con 0x55b7227dfb20 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda17e00640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227dfb20 0x55b722db9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 10 client.7137 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda17e00640 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227dfb20 0x55b722db9bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbbe3fbb1fbf82d server_cookie=29852cf4a680ca10 in_seq=0 out_seq=0 2024-05-14T11:22:24.272 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:22:24.271+0000 7f770fa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0) n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561742febb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-14T11:22:24.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 1 -- 172.21.15.116:0/3428785900 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b722b59dd0 con 0x55b7227dfb20 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 10 client.7137 handle_command_reply: tid=0 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 client.7137 shutdown 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 msgr2=0x55b722d75930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fda0401a120 tx=0x7fda0401ad30 comp rx=0 tx=0).stop 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 shutdown_connections 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55b7227dfb20 0x55b722db9bd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b722bf57a0 0x55b722d75e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b722bdeb10 0x55b722d75930 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 --2- 172.21.15.116:0/3428785900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7227cd570 0x55b722cfe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda01600640 0 client.7137 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 >> 172.21.15.116:0/3428785900 conn(0x55b72280c750 msgr2=0x55b722d28d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 shutdown_connections 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 1 -- 172.21.15.116:0/3428785900 wait complete. 2024-05-14T11:22:24.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda263fe000 20 client.7137 trim_cache size 0 max 16384 2024-05-14T11:22:24.274 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:24.274 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:22:24.274 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "d4e1b3bb-654b-4416-89bf-43bb35fabe19", 2024-05-14T11:22:24.274 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:22:24.274 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 msgr2=0x7fd9f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 0x7fd9f8071910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fda0800a9b0 tx=0x7fda08008040 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 msgr2=0x7fda18158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fda1000c760 tx=0x7fda1000cc20 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 shutdown_connections 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd9f806f400 0x7fd9f8071910 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda1815cd00 0x7fda18158260 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda180d14b0 0x7fda18157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 --2- 172.21.15.116:0/2368403767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda1806c700 0x7fda18155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 >> 172.21.15.116:0/2368403767 conn(0x7fda180b43a0 msgr2=0x7fda180af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 shutdown_connections 2024-05-14T11:22:24.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.269+0000 7fda1d800640 1 -- 172.21.15.116:0/2368403767 wait complete. 2024-05-14T11:22:24.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:24.290 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:22:24.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.437+0000 7f5d86400640 1 Processor -- start 2024-05-14T11:22:24.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 -- start start 2024-05-14T11:22:24.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d8015db30 0x7f5d800aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5d8006c700 0x7f5d8006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8006ff20 con 0x7f5d800d1670 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5d800ae100 con 0x7f5d8015db30 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d86400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5d800ae870 con 0x7f5d8006c700 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d8015db30 0x7f5d800aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d76a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5d800d1670 0x7f5d800d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34292/0 (socket says 172.21.15.116:34292) 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d76a00640 1 -- 172.21.15.116:0/51631595 learned_addr learned my addr 172.21.15.116:0/51631595 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/51631595 >> v2:172.21.15.149:3300/0 conn(0x7f5d8006c700 0x7f5d8006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7f400640 1 -- 172.21.15.116:0/51631595 >> v2:172.21.15.149:3300/0 conn(0x7f5d8006c700 msgr2=0x7f5d8006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7f400640 1 --2- 172.21.15.116:0/51631595 >> v2:172.21.15.149:3300/0 conn(0x7f5d8006c700 0x7f5d8006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7f400640 1 -- 172.21.15.116:0/51631595 >> v1:172.21.15.116:6789/0 conn(0x7f5d800d1670 legacy=0x7f5d800d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7f400640 1 -- 172.21.15.116:0/51631595 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d80070470 con 0x7f5d8015db30 2024-05-14T11:22:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.441+0000 7f5d7ea00640 1 -- 172.21.15.116:0/51631595 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910654485 0 0) 0x7f5d8006ff20 con 0x7f5d800d1670 2024-05-14T11:22:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d7f400640 1 --2- 172.21.15.116:0/51631595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d8015db30 0x7f5d800aeaa0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5d801626a0 tx=0x7f5d7001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de70b4ddfc4906 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d7ea00640 1 -- 172.21.15.116:0/51631595 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700b2070 con 0x7f5d8015db30 2024-05-14T11:22:24.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d7ea00640 1 -- 172.21.15.116:0/51631595 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d700a5810 con 0x7f5d8015db30 2024-05-14T11:22:24.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d7ea00640 1 -- 172.21.15.116:0/51631595 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700a5be0 con 0x7f5d8015db30 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 -- 172.21.15.116:0/51631595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d8015db30 msgr2=0x7f5d800aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 --2- 172.21.15.116:0/51631595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d8015db30 0x7f5d800aeaa0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f5d801626a0 tx=0x7f5d7001c860 comp rx=0 tx=0).stop 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 -- 172.21.15.116:0/51631595 shutdown_connections 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 --2- 172.21.15.116:0/51631595 >> v2:172.21.15.149:3300/0 conn(0x7f5d8006c700 0x7f5d8006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 --2- 172.21.15.116:0/51631595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d8015db30 0x7f5d800aeaa0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 -- 172.21.15.116:0/51631595 >> 172.21.15.116:0/51631595 conn(0x7f5d800b4620 msgr2=0x7f5d800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 -- 172.21.15.116:0/51631595 shutdown_connections 2024-05-14T11:22:24.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.445+0000 7f5d86400640 1 -- 172.21.15.116:0/51631595 wait complete. 2024-05-14T11:22:24.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 Processor -- start 2024-05-14T11:22:24.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 -- start start 2024-05-14T11:22:24.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d8006c700 0x7f5d801535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d8015db30 0x7f5d80166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8016a770 con 0x7f5d800d1670 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8015df40 con 0x7f5d8015db30 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d86400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8016a8f0 con 0x7f5d8006c700 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46532/0 (socket says 172.21.15.116:46532) 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 -- 172.21.15.116:0/3096213018 learned_addr learned my addr 172.21.15.116:0/3096213018 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d8015db30 0x7f5d80166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d8006c700 0x7f5d801535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 -- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d8006c700 msgr2=0x7f5d801535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d8006c700 0x7f5d801535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 -- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d8015db30 msgr2=0x7f5d80166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d8015db30 0x7f5d80166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 -- 172.21.15.116:0/3096213018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d800ced40 con 0x7f5d800d1670 2024-05-14T11:22:24.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800c460 tx=0x7f5d6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d77e00640 1 -- 172.21.15.116:0/3096213018 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d680074e0 con 0x7f5d800d1670 2024-05-14T11:22:24.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.449+0000 7f5d77e00640 1 -- 172.21.15.116:0/3096213018 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d680047f0 con 0x7f5d800d1670 2024-05-14T11:22:24.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.453+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d80153e90 con 0x7f5d800d1670 2024-05-14T11:22:24.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.453+0000 7f5d77e00640 1 -- 172.21.15.116:0/3096213018 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68004be0 con 0x7f5d800d1670 2024-05-14T11:22:24.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.453+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d80154450 con 0x7f5d800d1670 2024-05-14T11:22:24.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.453+0000 7f5d77e00640 1 -- 172.21.15.116:0/3096213018 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d68011070 con 0x7f5d800d1670 2024-05-14T11:22:24.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d77e00640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 0x7f5d5c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d77e00640 1 -- 172.21.15.116:0/3096213018 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f5d6808ef40 con 0x7f5d800d1670 2024-05-14T11:22:24.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 0x7f5d5c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 Processor -- start 2024-05-14T11:22:24.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 -- start start 2024-05-14T11:22:24.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 0x7f5d5c071940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af000 tx=0x7f5d700a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b8799c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b8799c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557b877a1100 con 0x557b874223e0 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557b87759ca0 con 0x557b87835120 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557b8779d0c0 con 0x557b87822f00 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46546/0 (socket says 172.21.15.116:46546) 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 -- 172.21.15.116:0/3268503742 learned_addr learned my addr 172.21.15.116:0/3268503742 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b8799c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b8799c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 -- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 msgr2=0x557b8799c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b8799c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 -- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 msgr2=0x557b8799c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b8799c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 -- 172.21.15.116:0/3268503742 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8775e800 con 0x557b874223e0 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b8799c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:24.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b8799c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:24.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007870 tx=0x7f5d78007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefce89680240648 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d6fe00640 1 -- 172.21.15.116:0/3268503742 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78018070 con 0x557b874223e0 2024-05-14T11:22:24.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d6fe00640 1 -- 172.21.15.116:0/3268503742 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d78010a40 con 0x557b874223e0 2024-05-14T11:22:24.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.457+0000 7f5d6fe00640 1 -- 172.21.15.116:0/3268503742 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78010e30 con 0x557b874223e0 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- 172.21.15.116:0/3268503742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 msgr2=0x557b8799cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007870 tx=0x7f5d78007d30 comp rx=0 tx=0).stop 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- 172.21.15.116:0/3268503742 shutdown_connections 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b8799cdd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b8799c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3268503742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b8799c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- 172.21.15.116:0/3268503742 >> 172.21.15.116:0/3268503742 conn(0x557b87462400 msgr2=0x557b8797f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- 172.21.15.116:0/3268503742 shutdown_connections 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- 172.21.15.116:0/3268503742 wait complete. 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- start start 2024-05-14T11:22:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 Processor -- start 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87955f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557b8786c2a0 con 0x557b874223e0 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557b8785cf40 con 0x557b87835120 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557b8775d080 con 0x557b87822f00 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d8db70000 10 client.0 _ll_register_callbacks cb 0x557b8793dc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46554/0 (socket says 172.21.15.116:46554) 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 -- 172.21.15.116:0/101276343 learned_addr learned my addr 172.21.15.116:0/101276343 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d76a00640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87955f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 -- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 msgr2=0x557b879a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 -- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 msgr2=0x557b87955f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87955f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.461+0000 7f5d7f400640 1 -- 172.21.15.116:0/101276343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8774b860 con 0x557b874223e0 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d7f400640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5d700b0040 tx=0x7f5d7001b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacb729dfa79bea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700b2040 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f5d700ad780 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d78010a40 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f5d700ad980 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 10 client.7152 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557b87759ef0 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d78010a40 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f5d700adb20 con 0x557b874223e0 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 10 client.7152 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 10 client.7152 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 20 client.7152 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:24.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87446930 0x557b87956cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 4 client.7152 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:24.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b87739df0 con 0x557b87446930 2024-05-14T11:22:24.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d76a00640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87446930 0x557b87956cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d76a00640 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87446930 0x557b87956cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f48b053fd3eb87e server_cookie=2fe0b38dd2d64575 in_seq=0 out_seq=0 2024-05-14T11:22:24.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 10 client.7152 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 1 -- 172.21.15.116:0/101276343 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b87739df0 con 0x557b87446930 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 10 client.7152 handle_command_reply: tid=0 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 client.7152 shutdown 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 msgr2=0x557b879a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5d700b0040 tx=0x7f5d7001b0a0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 shutdown_connections 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87955f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87446930 0x557b87956cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 --2- 172.21.15.116:0/101276343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a2dd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d6cc00640 0 client.7152 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 >> 172.21.15.116:0/101276343 conn(0x557b87462400 msgr2=0x557b8797f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 shutdown_connections 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.465+0000 7f5d8db70000 1 -- 172.21.15.116:0/101276343 wait complete. 2024-05-14T11:22:24.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.469+0000 7f5d8db70000 20 client.7152 trim_cache size 0 max 16384 2024-05-14T11:22:24.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 Processor -- start 2024-05-14T11:22:24.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 -- start start 2024-05-14T11:22:24.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b874223e0 0x557b87a0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87a0b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d802051e0 con 0x557b87822f00 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80073750 con 0x557b87835120 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5d80204be0 con 0x557b874223e0 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46564/0 (socket says 172.21.15.116:46564) 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d76a00640 1 -- 172.21.15.116:0/3608657618 learned_addr learned my addr 172.21.15.116:0/3608657618 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b874223e0 0x557b87a0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.485+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87a0b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 -- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b874223e0 msgr2=0x557b87a0ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b874223e0 0x557b87a0ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 -- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 msgr2=0x557b87a0b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87a0b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 -- 172.21.15.116:0/3608657618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b87954010 con 0x557b87822f00 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d7fe00640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87a0b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5d6805b4d0 tx=0x7f5d6808d0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb6e8b086070b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d6f400640 1 -- 172.21.15.116:0/3608657618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6805f8f0 con 0x557b87822f00 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d6f400640 1 -- 172.21.15.116:0/3608657618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d68064310 con 0x557b87822f00 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d6f400640 1 -- 172.21.15.116:0/3608657618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6808d300 con 0x557b87822f00 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- 172.21.15.116:0/3608657618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 msgr2=0x557b87a0b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5d6805b4d0 tx=0x7f5d6808d0c0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- 172.21.15.116:0/3608657618 shutdown_connections 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b87a0b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b87822f00 0x557b87a0b270 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3608657618 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b874223e0 0x557b87a0ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- 172.21.15.116:0/3608657618 >> 172.21.15.116:0/3608657618 conn(0x557b87462400 msgr2=0x557b87954ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- 172.21.15.116:0/3608657618 shutdown_connections 2024-05-14T11:22:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- 172.21.15.116:0/3608657618 wait complete. 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 -- start start 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.489+0000 7f5d8db70000 1 Processor -- start 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b879cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557b877a5730 con 0x557b874223e0 2024-05-14T11:22:24.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557b877a0410 con 0x557b87835120 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557b877ae500 con 0x557b87822f00 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 10 client.0 _ll_register_callbacks cb 0x7f5d70009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46580/0 (socket says 172.21.15.116:46580) 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 -- 172.21.15.116:0/3879152844 learned_addr learned my addr 172.21.15.116:0/3879152844 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 -- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 msgr2=0x557b879cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 -- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 msgr2=0x557b879cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b879cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 -- 172.21.15.116:0/3879152844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b87822340 con 0x557b874223e0 2024-05-14T11:22:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d7f400640 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5d7001cd40 tx=0x7f5d70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47579d1c74e7300 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700b2040 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 6187+0+0 (secure 0 0 0) 0x7f5d700e2cc0 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d68064310 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f5d700a5900 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 10 client.7158 fetch_fsmap learned FSMap version 67 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557b8773d8e0 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d68064310 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f5d700e9050 con 0x557b874223e0 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 10 client.7158 fetch_fsmap finished waiting for FSMap version 67 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 10 client.7158 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 20 client.7158 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87434990 0x557b879cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 4 client.7158 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:22:24.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b877aed10 con 0x557b87434990 2024-05-14T11:22:24.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87434990 0x557b879cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 10 client.7158 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d76a00640 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87434990 0x557b879cc350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4211d312b82ac537 server_cookie=2d8282c8f0a75b15 in_seq=0 out_seq=0 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 1 -- 172.21.15.116:0/3879152844 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x557b877aed10 con 0x557b87434990 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 10 client.7158 handle_command_reply: tid=0 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 client.7158 shutdown 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 msgr2=0x557b879a7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5d7001cd40 tx=0x7f5d70009750 comp rx=0 tx=0).stop 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 shutdown_connections 2024-05-14T11:22:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b87835120 0x557b879cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x557b87434990 0x557b879cc350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557b87822f00 0x557b879cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 --2- 172.21.15.116:0/3879152844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557b874223e0 0x557b879a7900 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d6cc00640 0 client.7158 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 >> 172.21.15.116:0/3879152844 conn(0x557b87462400 msgr2=0x557b879b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 shutdown_connections 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 1 -- 172.21.15.116:0/3879152844 wait complete. 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.493+0000 7f5d8db70000 20 client.7158 trim_cache size 0 max 16384 2024-05-14T11:22:24.498 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "d4e1b3bb-654b-4416-89bf-43bb35fabe19": { 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir", 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "d4e1b3bb-654b-4416-89bf-43bb35fabe19", 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive" 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:24.499 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 msgr2=0x7f5d5c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 0x7f5d5c071940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af000 tx=0x7f5d700a7070 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 msgr2=0x7f5d80151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800c460 tx=0x7f5d6800c860 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 shutdown_connections 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5d5c06f430 0x7f5d5c071940 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d8015db30 0x7f5d80166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d800d1670 0x7f5d80151900 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 --2- 172.21.15.116:0/3096213018 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d8006c700 0x7f5d801535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 >> 172.21.15.116:0/3096213018 conn(0x7f5d800b4620 msgr2=0x7f5d800b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 shutdown_connections 2024-05-14T11:22:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:24.497+0000 7f5d86400640 1 -- 172.21.15.116:0/3096213018 wait complete. 2024-05-14T11:22:24.514 INFO:tasks.cephfs.filesystem:scrub status for tag:d4e1b3bb-654b-4416-89bf-43bb35fabe19 - {'path': '/dir', 'tag': 'd4e1b3bb-654b-4416-89bf-43bb35fabe19', 'options': 'recursive'} 2024-05-14T11:22:27.838 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:22:27.837+0000 7f7ed0800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-14T11:22:42.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:22:42.703 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:22:54.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:54.516 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:22:54.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 Processor -- start 2024-05-14T11:22:54.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 -- start start 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32a006f730 0x7f32a006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f32a0072a30 0x7f32a0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a0070080 con 0x7f32a00b0770 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f32a006db20 con 0x7f32a006f730 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f32a4e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f32a006cf10 con 0x7f32a0072a30 2024-05-14T11:22:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32a006f730 0x7f32a006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f32a006f730 0x7f32a006fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60252/0 (socket says 172.21.15.116:60252) 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 -- 172.21.15.116:0/2300064097 learned_addr learned my addr 172.21.15.116:0/2300064097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329e800640 1 --2- 172.21.15.116:0/2300064097 >> v2:172.21.15.149:3300/0 conn(0x7f32a0072a30 0x7f32a0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 -- 172.21.15.116:0/2300064097 >> v2:172.21.15.149:3300/0 conn(0x7f32a0072a30 msgr2=0x7f32a0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 --2- 172.21.15.116:0/2300064097 >> v2:172.21.15.149:3300/0 conn(0x7f32a0072a30 0x7f32a0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 -- 172.21.15.116:0/2300064097 >> v1:172.21.15.149:6789/0 conn(0x7f32a00b0770 legacy=0x7f32a006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.676+0000 7f329d400640 1 -- 172.21.15.116:0/2300064097 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a006d1f0 con 0x7f32a006f730 2024-05-14T11:22:54.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f329d400640 1 --2- 172.21.15.116:0/2300064097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a006f730 0x7f32a006fb40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f32a00729c0 tx=0x7f328c01d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce6a28f5fdacb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f329ca00640 1 -- 172.21.15.116:0/2300064097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0b3070 con 0x7f32a006f730 2024-05-14T11:22:54.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f329ca00640 1 -- 172.21.15.116:0/2300064097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328c0a6860 con 0x7f32a006f730 2024-05-14T11:22:54.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f329ca00640 1 -- 172.21.15.116:0/2300064097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c00b040 con 0x7f32a006f730 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 -- 172.21.15.116:0/2300064097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a006f730 msgr2=0x7f32a006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 --2- 172.21.15.116:0/2300064097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a006f730 0x7f32a006fb40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f32a00729c0 tx=0x7f328c01d830 comp rx=0 tx=0).stop 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 -- 172.21.15.116:0/2300064097 shutdown_connections 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 --2- 172.21.15.116:0/2300064097 >> v2:172.21.15.149:3300/0 conn(0x7f32a0072a30 0x7f32a0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 --2- 172.21.15.116:0/2300064097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a006f730 0x7f32a006fb40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 -- 172.21.15.116:0/2300064097 >> 172.21.15.116:0/2300064097 conn(0x7f32a00b4620 msgr2=0x7f32a00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 -- 172.21.15.116:0/2300064097 shutdown_connections 2024-05-14T11:22:54.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.680+0000 7f32a4e00640 1 -- 172.21.15.116:0/2300064097 wait complete. 2024-05-14T11:22:54.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 Processor -- start 2024-05-14T11:22:54.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 -- start start 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 0x7f32a015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32a0072a30 0x7f32a0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0073500 con 0x7f32a00b0770 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0073380 con 0x7f32a006f730 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f32a4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0073680 con 0x7f32a0072a30 2024-05-14T11:22:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f329e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f329de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 0x7f32a015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f329d400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32a0072a30 0x7f32a0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f329e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60260/0 (socket says 172.21.15.116:60260) 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.684+0000 7f329e800640 1 -- 172.21.15.116:0/3375205452 learned_addr learned my addr 172.21.15.116:0/3375205452 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 -- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32a0072a30 msgr2=0x7f32a0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32a0072a30 0x7f32a0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 -- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 msgr2=0x7f32a015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 0x7f32a015e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 -- 172.21.15.116:0/3375205452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a01577b0 con 0x7f32a00b0770 2024-05-14T11:22:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329de00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 0x7f32a015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f329e800640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3298009cb0 tx=0x7f329800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f3296000640 1 -- 172.21.15.116:0/3375205452 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298018020 con 0x7f32a00b0770 2024-05-14T11:22:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f3296000640 1 -- 172.21.15.116:0/3375205452 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3298014480 con 0x7f32a00b0770 2024-05-14T11:22:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f3296000640 1 -- 172.21.15.116:0/3375205452 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298014870 con 0x7f32a00b0770 2024-05-14T11:22:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a0157a90 con 0x7f32a00b0770 2024-05-14T11:22:54.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a0158050 con 0x7f32a00b0770 2024-05-14T11:22:54.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.688+0000 7f3296000640 1 -- 172.21.15.116:0/3375205452 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f329801b020 con 0x7f32a00b0770 2024-05-14T11:22:54.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f3296000640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 0x7f3280071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f3296000640 1 -- 172.21.15.116:0/3375205452 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6303+0+0 (secure 0 0 0) 0x7f3298090110 con 0x7f32a00b0770 2024-05-14T11:22:54.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 0x7f3280071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 Processor -- start 2024-05-14T11:22:54.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 -- start start 2024-05-14T11:22:54.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 0x7f3280071910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3290007fd0 tx=0x7f3290009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a48d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a48ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56087a292100 con 0x56087a326120 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56087a24aca0 con 0x560879f133e0 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56087a28e0c0 con 0x56087a313f00 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60262/0 (socket says 172.21.15.116:60262) 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 -- 172.21.15.116:0/360244384 learned_addr learned my addr 172.21.15.116:0/360244384 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329d400640 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a48d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329e800640 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a48ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 -- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 msgr2=0x56087a48d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a48d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 -- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 msgr2=0x56087a48ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a48ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 -- 172.21.15.116:0/360244384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087a24f800 con 0x56087a326120 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f329de00640 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f329000fd40 tx=0x7f3290012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092fcd5e1df9f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f328aa00640 1 -- 172.21.15.116:0/360244384 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290025070 con 0x56087a326120 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f328aa00640 1 -- 172.21.15.116:0/360244384 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329001b050 con 0x56087a326120 2024-05-14T11:22:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.692+0000 7f328aa00640 1 -- 172.21.15.116:0/360244384 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290020610 con 0x56087a326120 2024-05-14T11:22:54.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- 172.21.15.116:0/360244384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 msgr2=0x56087a48d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f329000fd40 tx=0x7f3290012000 comp rx=0 tx=0).stop 2024-05-14T11:22:54.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- 172.21.15.116:0/360244384 shutdown_connections 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a48ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a48d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- 172.21.15.116:0/360244384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a48d350 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- 172.21.15.116:0/360244384 >> 172.21.15.116:0/360244384 conn(0x560879f53400 msgr2=0x56087a470310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- 172.21.15.116:0/360244384 shutdown_connections 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- 172.21.15.116:0/360244384 wait complete. 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- start start 2024-05-14T11:22:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 Processor -- start 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a313f00 0x56087a494310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a326120 0x56087a446ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56087a35d2a0 con 0x560879f133e0 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56087a34df40 con 0x56087a313f00 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56087a24e080 con 0x56087a326120 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f32ac57e000 10 client.0 _ll_register_callbacks cb 0x56087a42ec90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60268/0 (socket says 172.21.15.116:60268) 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 -- 172.21.15.116:0/2759984749 learned_addr learned my addr 172.21.15.116:0/2759984749 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329d400640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a313f00 0x56087a494310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329e800640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a326120 0x56087a446ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 -- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a326120 msgr2=0x56087a446ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a326120 0x56087a446ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 -- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a313f00 msgr2=0x56087a494310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a313f00 0x56087a494310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.696+0000 7f329de00640 1 -- 172.21.15.116:0/2759984749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087a23c860 con 0x560879f133e0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f329de00640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x56087a269230 tx=0x7f3290000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a563f337f465e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:54.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290025050 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329001b050 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6303+0+0 (secure 0 0 0) 0x7f329002a430 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3290037050 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 10 client.7170 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56087a24aef0 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329001b050 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f329003a020 con 0x560879f133e0 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 10 client.7170 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 10 client.7170 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 20 client.7170 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f37930 0x56087a447d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 4 client.7170 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087a22adf0 con 0x560879f37930 2024-05-14T11:22:54.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f329d400640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f37930 0x56087a447d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f329d400640 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f37930 0x56087a447d30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be3091f1d4cd466 server_cookie=a449c5583d51c925 in_seq=0 out_seq=0 2024-05-14T11:22:54.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 10 client.7170 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 1 -- 172.21.15.116:0/2759984749 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56087a22adf0 con 0x560879f37930 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 10 client.7170 handle_command_reply: tid=0 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 client.7170 shutdown 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 msgr2=0x56087a493dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x56087a269230 tx=0x7f3290000ee0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 shutdown_connections 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a313f00 0x56087a494310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f37930 0x56087a447d30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a326120 0x56087a446ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 --2- 172.21.15.116:0/2759984749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a493dd0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 >> 172.21.15.116:0/2759984749 conn(0x560879f53400 msgr2=0x56087a470200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f327f400640 0 client.7170 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 shutdown_connections 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 1 -- 172.21.15.116:0/2759984749 wait complete. 2024-05-14T11:22:54.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.700+0000 7f32ac57e000 20 client.7170 trim_cache size 0 max 16384 2024-05-14T11:22:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 Processor -- start 2024-05-14T11:22:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 -- start start 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 0x56087a4fc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0203140 con 0x560879f133e0 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32a009e600 con 0x56087a326120 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0202b20 con 0x56087a313f00 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60284/0 (socket says 172.21.15.116:60284) 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 -- 172.21.15.116:0/22145432 learned_addr learned my addr 172.21.15.116:0/22145432 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329e800640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 0x56087a4fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329d400640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 -- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 msgr2=0x56087a4fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 -- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 msgr2=0x56087a4fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 0x56087a4fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 -- 172.21.15.116:0/22145432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087a445070 con 0x560879f133e0 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329e800640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 0x56087a4fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f329de00640 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f329000fc60 tx=0x7f3290027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb351391e65ebf22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f328a000640 1 -- 172.21.15.116:0/22145432 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290025040 con 0x560879f133e0 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f328a000640 1 -- 172.21.15.116:0/22145432 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329001b070 con 0x560879f133e0 2024-05-14T11:22:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.720+0000 7f328a000640 1 -- 172.21.15.116:0/22145432 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900327b0 con 0x560879f133e0 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- 172.21.15.116:0/22145432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 msgr2=0x56087a4fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f329000fc60 tx=0x7f3290027000 comp rx=0 tx=0).stop 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- 172.21.15.116:0/22145432 shutdown_connections 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56087a326120 0x56087a4fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- 172.21.15.116:0/22145432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560879f133e0 0x56087a4fb700 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- 172.21.15.116:0/22145432 >> 172.21.15.116:0/22145432 conn(0x560879f53400 msgr2=0x56087a46f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- 172.21.15.116:0/22145432 shutdown_connections 2024-05-14T11:22:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- 172.21.15.116:0/22145432 wait complete. 2024-05-14T11:22:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- start start 2024-05-14T11:22:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 Processor -- start 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a4ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56087a296730 con 0x56087a326120 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56087a291410 con 0x560879f133e0 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56087a29f500 con 0x56087a313f00 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f32ac57e000 10 client.0 _ll_register_callbacks cb 0x7f3290017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60286/0 (socket says 172.21.15.116:60286) 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 -- 172.21.15.116:0/1033069137 learned_addr learned my addr 172.21.15.116:0/1033069137 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329d400640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329de00640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a4ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 -- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 msgr2=0x56087a4ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 -- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 msgr2=0x56087a4ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a4ff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.724+0000 7f329e800640 1 -- 172.21.15.116:0/1033069137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087a313340 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f329e800640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f329800c990 tx=0x7f3298061f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4085632ce9f1fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329805c750 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6303+0+0 (secure 0 0 0) 0x7f3298088df0 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329001b070 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3298060a50 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 10 client.7176 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x56087a22e8e0 con 0x56087a326120 2024-05-14T11:22:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329001b070 con 0x56087a326120 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f32980580c0 con 0x56087a326120 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 10 client.7176 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 10 client.7176 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 20 client.7176 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f25990 0x56087a498810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 4 client.7176 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56087a29fd10 con 0x560879f25990 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f329de00640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f25990 0x56087a498810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f329de00640 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f25990 0x56087a498810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866004c9fa3715c5 server_cookie=e4092e4c9d5c43a8 in_seq=0 out_seq=0 2024-05-14T11:22:54.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 10 client.7176 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 1 -- 172.21.15.116:0/1033069137 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56087a29fd10 con 0x560879f25990 2024-05-14T11:22:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 10 client.7176 handle_command_reply: tid=0 2024-05-14T11:22:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 client.7176 shutdown 2024-05-14T11:22:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 msgr2=0x56087a497bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f329800c990 tx=0x7f3298061f20 comp rx=0 tx=0).stop 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 shutdown_connections 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560879f25990 0x56087a498810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087a326120 0x56087a497bc0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56087a313f00 0x56087a4ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 --2- 172.21.15.116:0/1033069137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560879f133e0 0x56087a4ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 >> 172.21.15.116:0/1033069137 conn(0x560879f53400 msgr2=0x56087a46f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f327f400640 0 client.7176 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 shutdown_connections 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 1 -- 172.21.15.116:0/1033069137 wait complete. 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32ac57e000 20 client.7176 trim_cache size 0 max 16384 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:54.733 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:22:54.734 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:22:54.734 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:54.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 msgr2=0x7f3280071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 0x7f3280071910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3290007fd0 tx=0x7f3290009040 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 msgr2=0x7f32a01559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3298009cb0 tx=0x7f329800c490 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 shutdown_connections 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f328006f400 0x7f3280071910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a00b0770 0x7f32a01559b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f32a0072a30 0x7f32a0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 --2- 172.21.15.116:0/3375205452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a006f730 0x7f32a015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 >> 172.21.15.116:0/3375205452 conn(0x7f32a00b4620 msgr2=0x7f32a00b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 shutdown_connections 2024-05-14T11:22:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.728+0000 7f32a4e00640 1 -- 172.21.15.116:0/3375205452 wait complete. 2024-05-14T11:22:54.750 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:22:54.750 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:22:54.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 Processor -- start 2024-05-14T11:22:54.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 -- start start 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154806f730 0x7f154806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1548070080 con 0x7f15480b0770 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154806cf10 con 0x7f1548072a30 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154f400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f154806db20 con 0x7f154806f730 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f153fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154806f730 0x7f154806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f153fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154806f730 0x7f154806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60296/0 (socket says 172.21.15.116:60296) 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f153fe00640 1 -- 172.21.15.116:0/349555708 learned_addr learned my addr 172.21.15.116:0/349555708 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.896+0000 7f154ca00640 1 --1- 172.21.15.116:0/349555708 >> v1:172.21.15.116:6789/0 conn(0x7f15480b0770 0x7f154806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60926/0 (socket says 172.21.15.116:60926) 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f153fe00640 1 -- 172.21.15.116:0/349555708 >> v1:172.21.15.149:6789/0 conn(0x7f1548072a30 legacy=0x7f1548072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f153fe00640 1 -- 172.21.15.116:0/349555708 >> v1:172.21.15.116:6789/0 conn(0x7f15480b0770 legacy=0x7f154806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f153fe00640 1 -- 172.21.15.116:0/349555708 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154806d1f0 con 0x7f154806f730 2024-05-14T11:22:54.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f1547e00640 1 -- 172.21.15.116:0/349555708 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145276488 0 0) 0x7f1538001120 con 0x7f15480b0770 2024-05-14T11:22:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f153fe00640 1 --2- 172.21.15.116:0/349555708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154806f730 0x7f154806fb40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f153001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b19c70a064f0aef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f1547e00640 1 -- 172.21.15.116:0/349555708 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15300ab620 con 0x7f154806f730 2024-05-14T11:22:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f1547e00640 1 -- 172.21.15.116:0/349555708 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15300b0070 con 0x7f154806f730 2024-05-14T11:22:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f1547e00640 1 -- 172.21.15.116:0/349555708 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15300ab9e0 con 0x7f154806f730 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f154f400640 1 -- 172.21.15.116:0/349555708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154806f730 msgr2=0x7f154806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.900+0000 7f154f400640 1 --2- 172.21.15.116:0/349555708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154806f730 0x7f154806fb40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1530009870 tx=0x7f153001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 -- 172.21.15.116:0/349555708 shutdown_connections 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 --2- 172.21.15.116:0/349555708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154806f730 0x7f154806fb40 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 -- 172.21.15.116:0/349555708 >> 172.21.15.116:0/349555708 conn(0x7f15480b4620 msgr2=0x7f15480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 -- 172.21.15.116:0/349555708 shutdown_connections 2024-05-14T11:22:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 -- 172.21.15.116:0/349555708 wait complete. 2024-05-14T11:22:54.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 Processor -- start 2024-05-14T11:22:54.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.904+0000 7f154f400640 1 -- start start 2024-05-14T11:22:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f154806f730 0x7f15480d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1548072a30 0x7f154809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1548073ae0 con 0x7f15480b0770 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1548073960 con 0x7f1548072a30 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f154816b0d0 con 0x7f154806f730 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60310/0 (socket says 172.21.15.116:60310) 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 -- 172.21.15.116:0/1840913856 learned_addr learned my addr 172.21.15.116:0/1840913856 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154ca00640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f154806f730 0x7f15480d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f153fe00640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1548072a30 0x7f154809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 -- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f154806f730 msgr2=0x7f15480d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f154806f730 0x7f15480d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 -- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1548072a30 msgr2=0x7f154809a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1548072a30 0x7f154809a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 -- 172.21.15.116:0/1840913856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15480cfb70 con 0x7f15480b0770 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154d400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f154000ba10 tx=0x7f154000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540019070 con 0x7f15480b0770 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154000cb40 con 0x7f15480b0770 2024-05-14T11:22:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15480cfe50 con 0x7f15480b0770 2024-05-14T11:22:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540014440 con 0x7f15480b0770 2024-05-14T11:22:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154815e260 con 0x7f15480b0770 2024-05-14T11:22:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.908+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154806a9c0 con 0x7f15480b0770 2024-05-14T11:22:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.912+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f154001c070 con 0x7f15480b0770 2024-05-14T11:22:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.912+0000 7f1545600640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 0x7f1524071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.912+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6303+0+0 (secure 0 0 0) 0x7f154008efe0 con 0x7f15480b0770 2024-05-14T11:22:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.912+0000 7f154ca00640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 0x7f1524071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.912+0000 7f154ca00640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 0x7f1524071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1538000f70 tx=0x7f1538007e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:54.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:54.920+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f154005b6c0 con 0x7f15480b0770 2024-05-14T11:22:55.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.116+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f154806f510 con 0x7f15480b0770 2024-05-14T11:22:55.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.116+0000 7f1545600640 1 -- 172.21.15.116:0/1840913856 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5193 (secure 0 0 0) 0x7f154005f9c0 con 0x7f15480b0770 2024-05-14T11:22:55.119 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:55.119 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:27.819353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6769,"qdb_cluster":[6769]},"id":6}]} 2024-05-14T11:22:55.119 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 msgr2=0x7f1524071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 0x7f1524071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1538000f70 tx=0x7f1538007e10 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 msgr2=0x7f1548154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f154000ba10 tx=0x7f154000bed0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 shutdown_connections 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f152406f400 0x7f1524071910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1548072a30 0x7f154809a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15480b0770 0x7f1548154310 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 --2- 172.21.15.116:0/1840913856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f154806f730 0x7f15480d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 >> 172.21.15.116:0/1840913856 conn(0x7f15480b4620 msgr2=0x7f154815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 shutdown_connections 2024-05-14T11:22:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.120+0000 7f154f400640 1 -- 172.21.15.116:0/1840913856 wait complete. 2024-05-14T11:22:55.141 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-14T11:22:55.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 Processor -- start 2024-05-14T11:22:55.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 -- start start 2024-05-14T11:22:55.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 0x7fb69c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69c06ff20 con 0x7fb69c0d1670 2024-05-14T11:22:55.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69c070470 con 0x7fb69c06c700 2024-05-14T11:22:55.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb6a0a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb69c0ae100 con 0x7fb69c15db30 2024-05-14T11:22:55.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb691400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb69c0d1670 0x7fb69c0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60938/0 (socket says 172.21.15.116:60938) 2024-05-14T11:22:55.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.288+0000 7fb691400640 1 -- 172.21.15.116:0/2885981533 learned_addr learned my addr 172.21.15.116:0/2885981533 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699e00640 1 --2- 172.21.15.116:0/2885981533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 0x7fb69c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65890023 0 0) 0x7fb69c06ff20 con 0x7fb69c0d1670 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb680003530 con 0x7fb69c0d1670 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7696007 0 0) 0x7fb69c070470 con 0x7fb69c06c700 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb69c06ff20 con 0x7fb69c06c700 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699e00640 1 -- 172.21.15.116:0/2885981533 >> v1:172.21.15.149:6789/0 conn(0x7fb69c06c700 legacy=0x7fb69c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 593476858 0 0) 0x7fb680003530 con 0x7fb69c0d1670 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699e00640 1 -- 172.21.15.116:0/2885981533 >> v1:172.21.15.116:6789/0 conn(0x7fb69c0d1670 legacy=0x7fb69c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699e00640 1 -- 172.21.15.116:0/2885981533 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69c070750 con 0x7fb69c15db30 2024-05-14T11:22:55.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699e00640 1 --2- 172.21.15.116:0/2885981533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 0x7fb69c0aeaa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6840097f0 tx=0x7fb68401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c612f864f6effc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.292+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840b0070 con 0x7fb69c15db30 2024-05-14T11:22:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6840ab400 con 0x7fb69c15db30 2024-05-14T11:22:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb699400640 1 -- 172.21.15.116:0/2885981533 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c001700 con 0x7fb69c15db30 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- 172.21.15.116:0/2885981533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 msgr2=0x7fb69c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/2885981533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 0x7fb69c0aeaa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6840097f0 tx=0x7fb68401c370 comp rx=0 tx=0).stop 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- 172.21.15.116:0/2885981533 shutdown_connections 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/2885981533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c15db30 0x7fb69c0aeaa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- 172.21.15.116:0/2885981533 >> 172.21.15.116:0/2885981533 conn(0x7fb69c0b4620 msgr2=0x7fb69c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- 172.21.15.116:0/2885981533 shutdown_connections 2024-05-14T11:22:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- 172.21.15.116:0/2885981533 wait complete. 2024-05-14T11:22:55.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 Processor -- start 2024-05-14T11:22:55.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.296+0000 7fb6a0a00640 1 -- start start 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb69c06c700 0x7fb69c145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c0d1670 0x7fb69c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c16bf90 con 0x7fb69c15db30 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c15df40 con 0x7fb69c06c700 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb69c16c110 con 0x7fb69c0d1670 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60326/0 (socket says 172.21.15.116:60326) 2024-05-14T11:22:55.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 -- 172.21.15.116:0/148089686 learned_addr learned my addr 172.21.15.116:0/148089686 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb699e00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb69c06c700 0x7fb69c145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb691400640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c0d1670 0x7fb69c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 -- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c0d1670 msgr2=0x7fb69c151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c0d1670 0x7fb69c151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 -- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb69c06c700 msgr2=0x7fb69c145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb69c06c700 0x7fb69c145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 -- 172.21.15.116:0/148089686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69c144bd0 con 0x7fb69c15db30 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb69a800640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c760 tx=0x7fb69400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb692a00640 1 -- 172.21.15.116:0/148089686 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694003b80 con 0x7fb69c15db30 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb692a00640 1 -- 172.21.15.116:0/148089686 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb694003d60 con 0x7fb69c15db30 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69c0cdd80 con 0x7fb69c15db30 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb692a00640 1 -- 172.21.15.116:0/148089686 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69400aa90 con 0x7fb69c15db30 2024-05-14T11:22:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.300+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69c0ce340 con 0x7fb69c15db30 2024-05-14T11:22:55.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb692a00640 1 -- 172.21.15.116:0/148089686 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb69400ac30 con 0x7fb69c15db30 2024-05-14T11:22:55.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb692a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 0x7fb678071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb692a00640 1 -- 172.21.15.116:0/148089686 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7fb69408f2c0 con 0x7fb69c15db30 2024-05-14T11:22:55.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb699e00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 0x7fb678071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 Processor -- start 2024-05-14T11:22:55.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 -- start start 2024-05-14T11:22:55.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb699e00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 0x7fb678071940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6840ad000 tx=0x7fb6840a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e3782b91a0 con 0x55e3783402d0 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e378271b80 con 0x55e377f3a580 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55e3782b5160 con 0x55e378350bd0 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60338/0 (socket says 172.21.15.116:60338) 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb691400640 1 -- 172.21.15.116:0/3071474090 learned_addr learned my addr 172.21.15.116:0/3071474090 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.304+0000 7fb699e00640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb69a800640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 -- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 msgr2=0x55e3784b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 -- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 msgr2=0x55e3784b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 -- 172.21.15.116:0/3071474090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3782c5630 con 0x55e3783402d0 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb69a800640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb691400640 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001250 tx=0x7fb68c006eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca9338afe4abee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb68a600640 1 -- 172.21.15.116:0/3071474090 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c017020 con 0x55e3783402d0 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb68a600640 1 -- 172.21.15.116:0/3071474090 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c00fa50 con 0x55e3783402d0 2024-05-14T11:22:55.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb68a600640 1 -- 172.21.15.116:0/3071474090 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00fe00 con 0x55e3783402d0 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3071474090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 msgr2=0x55e3784b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001250 tx=0x7fb68c006eb0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3071474090 shutdown_connections 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e3784b43b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3071474090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3071474090 >> 172.21.15.116:0/3071474090 conn(0x55e377f79760 msgr2=0x55e378496e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3071474090 shutdown_connections 2024-05-14T11:22:55.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3071474090 wait complete. 2024-05-14T11:22:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 -- start start 2024-05-14T11:22:55.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.308+0000 7fb6a80ef000 1 Processor -- start 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e377f3a580 0x55e37846fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e378350bd0 0x55e378466b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e378258ab0 con 0x55e3783402d0 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e37826df20 con 0x55e378350bd0 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55e378318da0 con 0x55e377f3a580 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 10 client.0 _ll_register_callbacks cb 0x55e378454f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60346/0 (socket says 172.21.15.116:60346) 2024-05-14T11:22:55.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 -- 172.21.15.116:0/826102690 learned_addr learned my addr 172.21.15.116:0/826102690 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb699e00640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e377f3a580 0x55e37846fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb69a800640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e378350bd0 0x55e378466b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 -- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e377f3a580 msgr2=0x55e37846fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e377f3a580 0x55e37846fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 -- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e378350bd0 msgr2=0x55e378466b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e378350bd0 0x55e378466b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 -- 172.21.15.116:0/826102690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e378277ce0 con 0x55e3783402d0 2024-05-14T11:22:55.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb691400640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb68c006ef0 tx=0x7fb68c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41329a793daddb73 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c017040 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7fb68c027680 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c00fa50 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb68c02f050 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 10 client.7197 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55e378263740 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c00fa50 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fb68c032020 con 0x55e3783402d0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 10 client.7197 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 10 client.7197 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 20 client.7197 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f5ead0 0x55e378467900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 4 client.7197 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e378251e60 con 0x55e377f5ead0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb699e00640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f5ead0 0x55e378467900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb699e00640 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f5ead0 0x55e378467900 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a36e2cf6a04080 server_cookie=7bf596ad811a1e0c in_seq=0 out_seq=0 2024-05-14T11:22:55.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.312+0000 7fb67ea00640 10 client.7197 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb67ea00640 1 -- 172.21.15.116:0/826102690 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e378251e60 con 0x55e377f5ead0 2024-05-14T11:22:55.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb67ea00640 10 client.7197 handle_command_reply: tid=0 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 client.7197 shutdown 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 msgr2=0x55e378466650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb68c006ef0 tx=0x7fb68c005800 comp rx=0 tx=0).stop 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 shutdown_connections 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e378350bd0 0x55e378466b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f5ead0 0x55e378467900 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e378466650 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/826102690 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e377f3a580 0x55e37846fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb67ea00640 0 client.7197 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 >> 172.21.15.116:0/826102690 conn(0x55e377f79760 msgr2=0x55e378495640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 shutdown_connections 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 1 -- 172.21.15.116:0/826102690 wait complete. 2024-05-14T11:22:55.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.316+0000 7fb6a80ef000 20 client.7197 trim_cache size 0 max 16384 2024-05-14T11:22:55.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.328+0000 7fb6a80ef000 1 Processor -- start 2024-05-14T11:22:55.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.328+0000 7fb6a80ef000 1 -- start start 2024-05-14T11:22:55.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 0x55e3784ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c074010 con 0x55e378350bd0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c0cfd90 con 0x55e377f3a580 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb69c15fea0 con 0x55e3783402d0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60358/0 (socket says 172.21.15.116:60358) 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 -- 172.21.15.116:0/3624051386 learned_addr learned my addr 172.21.15.116:0/3624051386 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb699e00640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb691400640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 0x55e3784ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 -- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 msgr2=0x55e3784ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 0x55e3784ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 -- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 msgr2=0x55e3784ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 -- 172.21.15.116:0/3624051386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e378271dd0 con 0x55e378350bd0 2024-05-14T11:22:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb691400640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 0x55e3784ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb69a800640 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb69405b800 tx=0x7fb694012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10723670b2dfa08a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb689c00640 1 -- 172.21.15.116:0/3624051386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69408b070 con 0x55e378350bd0 2024-05-14T11:22:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb689c00640 1 -- 172.21.15.116:0/3624051386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69405b870 con 0x55e378350bd0 2024-05-14T11:22:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb689c00640 1 -- 172.21.15.116:0/3624051386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69408f0c0 con 0x55e378350bd0 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3624051386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 msgr2=0x55e37850b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb69405b800 tx=0x7fb694012000 comp rx=0 tx=0).stop 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3624051386 shutdown_connections 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e378350bd0 0x55e37850b600 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e3783402d0 0x55e3784ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/3624051386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e3784ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3624051386 >> 172.21.15.116:0/3624051386 conn(0x55e377f79760 msgr2=0x55e3784c4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3624051386 shutdown_connections 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- 172.21.15.116:0/3624051386 wait complete. 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 -- start start 2024-05-14T11:22:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.332+0000 7fb6a80ef000 1 Processor -- start 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e37852ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e3782625f0 con 0x55e3783402d0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3782b3e80 con 0x55e377f3a580 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55e378274f60 con 0x55e378350bd0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 10 client.0 _ll_register_callbacks cb 0x7fb68c0068f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60360/0 (socket says 172.21.15.116:60360) 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 -- 172.21.15.116:0/1366346785 learned_addr learned my addr 172.21.15.116:0/1366346785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb699e00640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e37852ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb69a800640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 -- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 msgr2=0x55e3784e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 -- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 msgr2=0x55e37852ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e37852ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 -- 172.21.15.116:0/1366346785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e378339c40 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb691400640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb68c006ef0 tx=0x7fb68c001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83dfdab5abd94c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c017030 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7fb68c027c40 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb69405b870 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb68c045050 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 10 client.7203 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55e377f723c0 con 0x55e3783402d0 2024-05-14T11:22:55.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69405b870 con 0x55e3783402d0 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fb68c048020 con 0x55e3783402d0 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 10 client.7203 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 10 client.7203 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 20 client.7203 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f4cb30 0x55e3784e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 4 client.7203 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e3782c6db0 con 0x55e377f4cb30 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.336+0000 7fb699e00640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f4cb30 0x55e3784e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb699e00640 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f4cb30 0x55e3784e6ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25c5f3a1372206 server_cookie=a9e50cb151b6f99f in_seq=0 out_seq=0 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb67ea00640 10 client.7203 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb67ea00640 1 -- 172.21.15.116:0/1366346785 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x55e3782c6db0 con 0x55e377f4cb30 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb67ea00640 10 client.7203 handle_command_reply: tid=0 2024-05-14T11:22:55.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 client.7203 shutdown 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 msgr2=0x55e37852b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb68c006ef0 tx=0x7fb68c001760 comp rx=0 tx=0).stop 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 shutdown_connections 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x55e377f4cb30 0x55e3784e6ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e378350bd0 0x55e3784e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e3783402d0 0x55e37852b2b0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb67ea00640 0 client.7203 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 --2- 172.21.15.116:0/1366346785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e377f3a580 0x55e37852ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 >> 172.21.15.116:0/1366346785 conn(0x55e377f79760 msgr2=0x55e3784ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 shutdown_connections 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 1 -- 172.21.15.116:0/1366346785 wait complete. 2024-05-14T11:22:55.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a80ef000 20 client.7203 trim_cache size 0 max 16384 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "backtrace", 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 759149869, 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627776, 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir" 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "backtrace", 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 1526738102, 2024-05-14T11:22:55.344 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/testdir/hardlink" 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "dir_frag", 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 2480712067, 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627776, 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "frag": "*", 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir" 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:55.345 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 msgr2=0x7fb678071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 0x7fb678071940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6840ad000 tx=0x7fb6840a6000 comp rx=0 tx=0).stop 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 msgr2=0x7fb69c151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c760 tx=0x7fb69400cc20 comp rx=0 tx=0).stop 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 shutdown_connections 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb67806f430 0x7fb678071940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb69c15db30 0x7fb69c151690 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb69c0d1670 0x7fb69c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 --2- 172.21.15.116:0/148089686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb69c06c700 0x7fb69c145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 >> 172.21.15.116:0/148089686 conn(0x7fb69c0b4620 msgr2=0x7fb69c0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 shutdown_connections 2024-05-14T11:22:55.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.340+0000 7fb6a0a00640 1 -- 172.21.15.116:0/148089686 wait complete. 2024-05-14T11:22:55.360 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:22:55.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.504+0000 7fe03e200640 1 Processor -- start 2024-05-14T11:22:55.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.504+0000 7fe03e200640 1 -- start start 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe038156fa0 0x7fe03806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0380d1670 0x7fe0380d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe03806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe0381573b0 con 0x7fe0380d1670 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe038161120 con 0x7fe038156fa0 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe03e200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380d1fc0 con 0x7fe03806e9e0 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0380d1670 0x7fe0380d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0380d1670 0x7fe0380d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60376/0 (socket says 172.21.15.116:60376) 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 -- 172.21.15.116:0/327778489 learned_addr learned my addr 172.21.15.116:0/327778489 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe037400640 1 --2- 172.21.15.116:0/327778489 >> v2:172.21.15.149:3300/0 conn(0x7fe038156fa0 0x7fe03806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe037e00640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe03806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 -- 172.21.15.116:0/327778489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 msgr2=0x7fe03806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe03806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 -- 172.21.15.116:0/327778489 >> v2:172.21.15.149:3300/0 conn(0x7fe038156fa0 msgr2=0x7fe03806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 --2- 172.21.15.116:0/327778489 >> v2:172.21.15.149:3300/0 conn(0x7fe038156fa0 0x7fe03806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe036a00640 1 -- 172.21.15.116:0/327778489 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe038160510 con 0x7fe0380d1670 2024-05-14T11:22:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.508+0000 7fe037e00640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe03806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe036a00640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0380d1670 0x7fe0380d1a80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe038071f20 tx=0x7fe02401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e1afdc383e7db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe036000640 1 -- 172.21.15.116:0/327778489 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240aa610 con 0x7fe0380d1670 2024-05-14T11:22:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe036000640 1 -- 172.21.15.116:0/327778489 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0240af070 con 0x7fe0380d1670 2024-05-14T11:22:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe036000640 1 -- 172.21.15.116:0/327778489 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240aa9d0 con 0x7fe0380d1670 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- 172.21.15.116:0/327778489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0380d1670 msgr2=0x7fe0380d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0380d1670 0x7fe0380d1a80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe038071f20 tx=0x7fe02401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- 172.21.15.116:0/327778489 shutdown_connections 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0380d1670 0x7fe0380d1a80 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 --2- 172.21.15.116:0/327778489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe03806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 --2- 172.21.15.116:0/327778489 >> v2:172.21.15.149:3300/0 conn(0x7fe038156fa0 0x7fe03806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- 172.21.15.116:0/327778489 >> 172.21.15.116:0/327778489 conn(0x7fe0380b4620 msgr2=0x7fe0380b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- 172.21.15.116:0/327778489 shutdown_connections 2024-05-14T11:22:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- 172.21.15.116:0/327778489 wait complete. 2024-05-14T11:22:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 Processor -- start 2024-05-14T11:22:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.512+0000 7fe03e200640 1 -- start start 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe0380748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 0x7fe038072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe038070520 con 0x7fe038156fa0 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe03806f740 con 0x7fe0380d1670 2024-05-14T11:22:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380706a0 con 0x7fe03806e9e0 2024-05-14T11:22:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60390/0 (socket says 172.21.15.116:60390) 2024-05-14T11:22:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe0380748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 -- 172.21.15.116:0/24523285 learned_addr learned my addr 172.21.15.116:0/24523285 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe036a00640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 0x7fe038072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 -- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 msgr2=0x7fe0380748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe0380748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 -- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 msgr2=0x7fe038072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 0x7fe038072fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 -- 172.21.15.116:0/24523285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe038073800 con 0x7fe038156fa0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe036a00640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 0x7fe038072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe037e00640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe03000d800 tx=0x7fe03000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03000b6c0 con 0x7fe038156fa0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe030019070 con 0x7fe038156fa0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03000baa0 con 0x7fe038156fa0 2024-05-14T11:22:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe038073ae0 con 0x7fe038156fa0 2024-05-14T11:22:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0380740a0 con 0x7fe038156fa0 2024-05-14T11:22:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.516+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03806a9c0 con 0x7fe038156fa0 2024-05-14T11:22:55.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.520+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe03000bc40 con 0x7fe038156fa0 2024-05-14T11:22:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.520+0000 7fe02f400640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 0x7fe018071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.520+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7fe03008f940 con 0x7fe038156fa0 2024-05-14T11:22:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.520+0000 7fe037400640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 0x7fe018071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.520+0000 7fe037400640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 0x7fe018071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe0380b0d60 tx=0x7fe02800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.524+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe03005bef0 con 0x7fe038156fa0 2024-05-14T11:22:55.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe038073590 con 0x7fe038156fa0 2024-05-14T11:22:55.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe02f400640 1 -- 172.21.15.116:0/24523285 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fe0300601f0 con 0x7fe038156fa0 2024-05-14T11:22:55.716 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:22:55.716 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 msgr2=0x7fe018071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 0x7fe018071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe0380b0d60 tx=0x7fe02800b040 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 msgr2=0x7fe038144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe03000d800 tx=0x7fe03000dcc0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 shutdown_connections 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe01806f400 0x7fe018071910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0380d1670 0x7fe038072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe038156fa0 0x7fe038144470 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 --2- 172.21.15.116:0/24523285 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe03806e9e0 0x7fe0380748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 >> 172.21.15.116:0/24523285 conn(0x7fe0380b4620 msgr2=0x7fe0380b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 shutdown_connections 2024-05-14T11:22:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.716+0000 7fe03e200640 1 -- 172.21.15.116:0/24523285 wait complete. 2024-05-14T11:22:55.738 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-14T11:22:55.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:55.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-14T11:22:55.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 Processor -- start 2024-05-14T11:22:55.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 -- start start 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070640 0x7ff59006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff590070a50 0x7ff59006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5901421f0 0x7ff590142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff59015dc90 con 0x7ff5901421f0 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff59015d360 con 0x7ff590070a50 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff597200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff59015d640 con 0x7ff590070640 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5901421f0 0x7ff590142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5901421f0 0x7ff590142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60392/0 (socket says 172.21.15.116:60392) 2024-05-14T11:22:55.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 -- 172.21.15.116:0/399614202 learned_addr learned my addr 172.21.15.116:0/399614202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff587400640 1 --2- 172.21.15.116:0/399614202 >> v2:172.21.15.149:3300/0 conn(0x7ff590070a50 0x7ff59006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff58fe00640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070640 0x7ff59006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 -- 172.21.15.116:0/399614202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070640 msgr2=0x7ff59006e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070640 0x7ff59006e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 -- 172.21.15.116:0/399614202 >> v2:172.21.15.149:3300/0 conn(0x7ff590070a50 msgr2=0x7ff59006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 --2- 172.21.15.116:0/399614202 >> v2:172.21.15.149:3300/0 conn(0x7ff590070a50 0x7ff59006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.888+0000 7ff595200640 1 -- 172.21.15.116:0/399614202 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59015d920 con 0x7ff5901421f0 2024-05-14T11:22:55.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff595200640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5901421f0 0x7ff590142600 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff5900d0e10 tx=0x7ff58801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332634795610e2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff58f400640 1 -- 172.21.15.116:0/399614202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a5790 con 0x7ff5901421f0 2024-05-14T11:22:55.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff58f400640 1 -- 172.21.15.116:0/399614202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5880a5930 con 0x7ff5901421f0 2024-05-14T11:22:55.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff58f400640 1 -- 172.21.15.116:0/399614202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a5d00 con 0x7ff5901421f0 2024-05-14T11:22:55.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 -- 172.21.15.116:0/399614202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5901421f0 msgr2=0x7ff590142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5901421f0 0x7ff590142600 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff5900d0e10 tx=0x7ff58801c9f0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 -- 172.21.15.116:0/399614202 shutdown_connections 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5901421f0 0x7ff590142600 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 --2- 172.21.15.116:0/399614202 >> v2:172.21.15.149:3300/0 conn(0x7ff590070a50 0x7ff59006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 --2- 172.21.15.116:0/399614202 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070640 0x7ff59006e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 -- 172.21.15.116:0/399614202 >> 172.21.15.116:0/399614202 conn(0x7ff5900cde80 msgr2=0x7ff5900ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 -- 172.21.15.116:0/399614202 shutdown_connections 2024-05-14T11:22:55.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.892+0000 7ff597200640 1 -- 172.21.15.116:0/399614202 wait complete. 2024-05-14T11:22:55.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 Processor -- start 2024-05-14T11:22:55.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 -- start start 2024-05-14T11:22:55.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070a50 0x7ff59013e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 0x7ff59013eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5900cd980 con 0x7ff590070640 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5900cd800 con 0x7ff5901421f0 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff597200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff59006d7a0 con 0x7ff590070a50 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60402/0 (socket says 172.21.15.116:60402) 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff58fe00640 1 -- 172.21.15.116:0/595812004 learned_addr learned my addr 172.21.15.116:0/595812004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff587400640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070a50 0x7ff59013e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.896+0000 7ff595200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 0x7ff59013eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 -- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070a50 msgr2=0x7ff59013e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070a50 0x7ff59013e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 -- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 msgr2=0x7ff59013eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 0x7ff59013eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 -- 172.21.15.116:0/595812004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff590143300 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff595200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 0x7ff59013eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58fe00640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d5c0 tx=0x7ff58000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 -- 172.21.15.116:0/595812004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580004240 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 -- 172.21.15.116:0/595812004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5800043e0 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff590145c60 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 -- 172.21.15.116:0/595812004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580003840 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff590167e20 con 0x7ff590070640 2024-05-14T11:22:55.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 -- 172.21.15.116:0/595812004 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff5800039e0 con 0x7ff590070640 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 0x7ff56c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.900+0000 7ff58cc00640 1 -- 172.21.15.116:0/595812004 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7ff58008f6d0 con 0x7ff590070640 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 0x7ff56c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 Processor -- start 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- start start 2024-05-14T11:22:55.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 0x7ff56c071940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff5901457b0 tx=0x7ff578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a522c0 0x565195ba2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195ba38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565195a40960 con 0x565195a3b9f0 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5651959a8200 con 0x565195a522c0 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x565195960da0 con 0x565195629630 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60404/0 (socket says 172.21.15.116:60404) 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 -- 172.21.15.116:0/2023344792 learned_addr learned my addr 172.21.15.116:0/2023344792 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff58fe00640 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a522c0 0x565195ba2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff595200640 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195ba38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 -- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 msgr2=0x565195ba38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195ba38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 -- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a522c0 msgr2=0x565195ba2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a522c0 0x565195ba2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 -- 172.21.15.116:0/2023344792 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565195965900 con 0x565195a3b9f0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff587400640 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff578007720 tx=0x7ff578013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7895576547b218 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff584c00640 1 -- 172.21.15.116:0/2023344792 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57801c050 con 0x565195a3b9f0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff584c00640 1 -- 172.21.15.116:0/2023344792 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578026070 con 0x565195a3b9f0 2024-05-14T11:22:55.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff584c00640 1 -- 172.21.15.116:0/2023344792 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578016e10 con 0x565195a3b9f0 2024-05-14T11:22:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- 172.21.15.116:0/2023344792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 msgr2=0x565195ba3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff578007720 tx=0x7ff578013000 comp rx=0 tx=0).stop 2024-05-14T11:22:55.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- 172.21.15.116:0/2023344792 shutdown_connections 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195ba38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195ba3380 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 --2- 172.21.15.116:0/2023344792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a522c0 0x565195ba2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.904+0000 7ff59e900000 1 -- 172.21.15.116:0/2023344792 >> 172.21.15.116:0/2023344792 conn(0x565195a5f760 msgr2=0x565195b57a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- 172.21.15.116:0/2023344792 shutdown_connections 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- 172.21.15.116:0/2023344792 wait complete. 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- start start 2024-05-14T11:22:55.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 Processor -- start 2024-05-14T11:22:55.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195b5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 0x565195b657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565195982df0 con 0x565195a3b9f0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565195964180 con 0x565195629630 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5651959a7510 con 0x565195a522c0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff59e900000 10 client.0 _ll_register_callbacks cb 0x565195a5b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60416/0 (socket says 172.21.15.116:60416) 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 -- 172.21.15.116:0/247239848 learned_addr learned my addr 172.21.15.116:0/247239848 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff58fe00640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195b5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff595200640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 0x565195b657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 -- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 msgr2=0x565195b657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 0x565195b657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 -- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 msgr2=0x565195b5ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195b5ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 -- 172.21.15.116:0/247239848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565195952960 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff595200640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 0x565195b657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff587400640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff578008010 tx=0x7ff578004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6e3902cb0ac9b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff57d600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578004020 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.908+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7ff5780041c0 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff578026070 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff57801c020 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 10 client.7221 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x565195960ff0 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578026070 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7ff5780217f0 con 0x565195a3b9f0 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 10 client.7221 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 10 client.7221 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 20 client.7221 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519564db80 0x565195ba9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 4 client.7221 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:55.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565195940ef0 con 0x56519564db80 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff58fe00640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519564db80 0x565195ba9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff58fe00640 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519564db80 0x565195ba9e00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59f9c78ad37ed78 server_cookie=9d73f5d4da537b9f in_seq=0 out_seq=0 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 10 client.7221 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 1 -- 172.21.15.116:0/247239848 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565195940ef0 con 0x56519564db80 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 10 client.7221 handle_command_reply: tid=0 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 client.7221 shutdown 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 msgr2=0x565195b65270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff578008010 tx=0x7ff578004740 comp rx=0 tx=0).stop 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 shutdown_connections 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519564db80 0x565195ba9e00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a522c0 0x565195b657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a3b9f0 0x565195b65270 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 --2- 172.21.15.116:0/247239848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195b5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff57d600640 0 client.7221 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 >> 172.21.15.116:0/247239848 conn(0x565195a5f760 msgr2=0x565195b581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 shutdown_connections 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 1 -- 172.21.15.116:0/247239848 wait complete. 2024-05-14T11:22:55.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.912+0000 7ff59e900000 20 client.7221 trim_cache size 0 max 16384 2024-05-14T11:22:55.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 Processor -- start 2024-05-14T11:22:55.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 -- start start 2024-05-14T11:22:55.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195c10e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a3b9f0 0x565195c11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5902057a0 con 0x565195a522c0 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901692e0 con 0x565195629630 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff590205150 con 0x565195a3b9f0 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60426/0 (socket says 172.21.15.116:60426) 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 -- 172.21.15.116:0/540920466 learned_addr learned my addr 172.21.15.116:0/540920466 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff58fe00640 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195c10e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff587400640 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a3b9f0 0x565195c11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 -- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a3b9f0 msgr2=0x565195c11350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a3b9f0 0x565195c11350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 -- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 msgr2=0x565195c10e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195c10e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.932+0000 7ff595200640 1 -- 172.21.15.116:0/540920466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565195a29970 con 0x565195a522c0 2024-05-14T11:22:55.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff595200640 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff588009e50 tx=0x7ff5880a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baac2d3e10ee9a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff57fe00640 1 -- 172.21.15.116:0/540920466 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588009190 con 0x565195a522c0 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff57fe00640 1 -- 172.21.15.116:0/540920466 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5880a5790 con 0x565195a522c0 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff57fe00640 1 -- 172.21.15.116:0/540920466 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a5b40 con 0x565195a522c0 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- 172.21.15.116:0/540920466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 msgr2=0x565195c11890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff588009e50 tx=0x7ff5880a7070 comp rx=0 tx=0).stop 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- 172.21.15.116:0/540920466 shutdown_connections 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195c11890 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195a3b9f0 0x565195c11350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 --2- 172.21.15.116:0/540920466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195629630 0x565195c10e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- 172.21.15.116:0/540920466 >> 172.21.15.116:0/540920466 conn(0x565195a5f760 msgr2=0x565195ba8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- 172.21.15.116:0/540920466 shutdown_connections 2024-05-14T11:22:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- 172.21.15.116:0/540920466 wait complete. 2024-05-14T11:22:55.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 -- start start 2024-05-14T11:22:55.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.936+0000 7ff59e900000 1 Processor -- start 2024-05-14T11:22:55.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195c15660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 0x565195c15ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5651959b5600 con 0x565195a522c0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5651959ac830 con 0x565195a3b9f0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56519595d140 con 0x565195629630 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 10 client.0 _ll_register_callbacks cb 0x7ff580057ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60436/0 (socket says 172.21.15.116:60436) 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 -- 172.21.15.116:0/1561223954 learned_addr learned my addr 172.21.15.116:0/1561223954 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff587400640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 0x565195c15ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff58fe00640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195c15660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 -- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 msgr2=0x565195c15660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195c15660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 -- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 msgr2=0x565195c15ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 0x565195c15ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 -- 172.21.15.116:0/1561223954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651959449e0 con 0x565195a522c0 2024-05-14T11:22:55.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff587400640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 0x565195c15ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff58fe00640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195c15660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff595200640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff588009e50 tx=0x7ff58801b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7141e9caf7b23219 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880c1420 con 0x565195a522c0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7ff5880c15c0 con 0x565195a522c0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5880a5790 con 0x565195a522c0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff5880d0050 con 0x565195a522c0 2024-05-14T11:22:55.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 10 client.7227 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x565195661470 con 0x565195a522c0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5880a5790 con 0x565195a522c0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7ff5880d3020 con 0x565195a522c0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 10 client.7227 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 10 client.7227 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 20 client.7227 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519563bbe0 0x565195baf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 4 client.7227 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x5651959b5e10 con 0x56519563bbe0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff58fe00640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519563bbe0 0x565195baf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff58fe00640 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519563bbe0 0x565195baf2c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea757c9de4e1c43 server_cookie=8535434bc2d76a81 in_seq=0 out_seq=0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 10 client.7227 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 1 -- 172.21.15.116:0/1561223954 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651959b5e10 con 0x56519563bbe0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 10 client.7227 handle_command_reply: tid=0 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 client.7227 shutdown 2024-05-14T11:22:55.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 msgr2=0x565195bae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff588009e50 tx=0x7ff58801b230 comp rx=0 tx=0).stop 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 shutdown_connections 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565195a3b9f0 0x565195c15ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x56519563bbe0 0x565195baf2c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565195a522c0 0x565195bae900 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 --2- 172.21.15.116:0/1561223954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x565195629630 0x565195c15660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff57d600640 0 client.7227 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.940+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 >> 172.21.15.116:0/1561223954 conn(0x565195a5f760 msgr2=0x565195b64c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 shutdown_connections 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff59e900000 1 -- 172.21.15.116:0/1561223954 wait complete. 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff59e900000 20 client.7227 trim_cache size 0 max 16384 2024-05-14T11:22:55.945 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:55.946 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:22:55.946 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "dc9ec28c-3709-4f88-bcba-850a840896d3", 2024-05-14T11:22:55.946 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:22:55.946 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:55.947 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:22:55.942+0000 7f770fa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)->f(v1 m2024-05-14T11:22:08.584531+0000) n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)->n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561742febb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 msgr2=0x7ff56c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 0x7ff56c071940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff5901457b0 tx=0x7ff578008040 comp rx=0 tx=0).stop 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 msgr2=0x7ff59013b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d5c0 tx=0x7ff58000da80 comp rx=0 tx=0).stop 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 shutdown_connections 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff56c06f430 0x7ff56c071940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5901421f0 0x7ff59013eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff590070a50 0x7ff59013e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 --2- 172.21.15.116:0/595812004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff590070640 0x7ff59013b140 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:55.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 >> 172.21.15.116:0/595812004 conn(0x7ff5900cde80 msgr2=0x7ff5901671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:55.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 shutdown_connections 2024-05-14T11:22:55.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:55.944+0000 7ff597200640 1 -- 172.21.15.116:0/595812004 wait complete. 2024-05-14T11:22:55.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:22:55.964 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:22:56.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 Processor -- start 2024-05-14T11:22:56.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 -- start start 2024-05-14T11:22:56.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7dec0d1670 0x7f7dec0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec06c700 0x7f7dec06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dec06ff20 con 0x7f7dec15db30 2024-05-14T11:22:56.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7dec0ae100 con 0x7f7dec0d1670 2024-05-14T11:22:56.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7df1c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec0ae870 con 0x7f7dec06c700 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7dec0d1670 0x7f7dec0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7deaa00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7dec15db30 0x7f7dec0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60954/0 (socket says 172.21.15.116:60954) 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7deaa00640 1 -- 172.21.15.116:0/3765828499 learned_addr learned my addr 172.21.15.116:0/3765828499 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7dec0d1670 0x7f7dec0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60446/0 (socket says 172.21.15.116:60446) 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7deb400640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec06c700 0x7f7dec06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 -- 172.21.15.116:0/3765828499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec06c700 msgr2=0x7f7dec06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec06c700 0x7f7dec06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 -- 172.21.15.116:0/3765828499 >> v1:172.21.15.116:6789/0 conn(0x7f7dec15db30 legacy=0x7f7dec0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7de2000640 1 -- 172.21.15.116:0/3765828499 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec0d1fc0 con 0x7f7dec0d1670 2024-05-14T11:22:56.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.112+0000 7f7dea000640 1 -- 172.21.15.116:0/3765828499 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209530007 0 0) 0x7f7dec06ff20 con 0x7f7dec15db30 2024-05-14T11:22:56.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7de2000640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec0d1670 0x7f7dec0d1a80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009d90 tx=0x7f7dd401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155929c8cc9544 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7dea000640 1 -- 172.21.15.116:0/3765828499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40b2070 con 0x7f7dec0d1670 2024-05-14T11:22:56.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7dea000640 1 -- 172.21.15.116:0/3765828499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd40a57c0 con 0x7f7dec0d1670 2024-05-14T11:22:56.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7dea000640 1 -- 172.21.15.116:0/3765828499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40a5b90 con 0x7f7dec0d1670 2024-05-14T11:22:56.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 -- 172.21.15.116:0/3765828499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec0d1670 msgr2=0x7f7dec0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec0d1670 0x7f7dec0d1a80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009d90 tx=0x7f7dd401c810 comp rx=0 tx=0).stop 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 -- 172.21.15.116:0/3765828499 shutdown_connections 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec06c700 0x7f7dec06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3765828499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec0d1670 0x7f7dec0d1a80 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 -- 172.21.15.116:0/3765828499 >> 172.21.15.116:0/3765828499 conn(0x7f7dec0b4620 msgr2=0x7f7dec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 -- 172.21.15.116:0/3765828499 shutdown_connections 2024-05-14T11:22:56.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.116+0000 7f7df1c00640 1 -- 172.21.15.116:0/3765828499 wait complete. 2024-05-14T11:22:56.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 Processor -- start 2024-05-14T11:22:56.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 -- start start 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7dec06c700 0x7f7dec15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec0d1670 0x7f7dec14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec16a770 con 0x7f7dec15db30 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec15df40 con 0x7f7dec06c700 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7df1c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec16a8f0 con 0x7f7dec0d1670 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60450/0 (socket says 172.21.15.116:60450) 2024-05-14T11:22:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7deb400640 1 -- 172.21.15.116:0/3126671902 learned_addr learned my addr 172.21.15.116:0/3126671902 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7deaa00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7dec06c700 0x7f7dec15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.120+0000 7f7de2000640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec0d1670 0x7f7dec14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 -- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec0d1670 msgr2=0x7f7dec14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec0d1670 0x7f7dec14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 -- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7dec06c700 msgr2=0x7f7dec15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7dec06c700 0x7f7dec15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 -- 172.21.15.116:0/3126671902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec14eb80 con 0x7f7dec15db30 2024-05-14T11:22:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7deb400640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d500 tx=0x7f7de400d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7de3400640 1 -- 172.21.15.116:0/3126671902 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de40125f0 con 0x7f7dec15db30 2024-05-14T11:22:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7de3400640 1 -- 172.21.15.116:0/3126671902 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4019070 con 0x7f7dec15db30 2024-05-14T11:22:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec15f8e0 con 0x7f7dec15db30 2024-05-14T11:22:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7de3400640 1 -- 172.21.15.116:0/3126671902 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de40129d0 con 0x7f7dec15db30 2024-05-14T11:22:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1569e0 con 0x7f7dec15db30 2024-05-14T11:22:56.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7de3400640 1 -- 172.21.15.116:0/3126671902 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7de4012b70 con 0x7f7dec15db30 2024-05-14T11:22:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.124+0000 7f7de3400640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 0x7f7dc8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7de3400640 1 -- 172.21.15.116:0/3126671902 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f7de408f540 con 0x7f7dec15db30 2024-05-14T11:22:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deaa00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 0x7f7dc8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 Processor -- start 2024-05-14T11:22:56.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 -- start start 2024-05-14T11:22:56.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deaa00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 0x7f7dc8071910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc000f70 tx=0x7f7ddc007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2b3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5617a293f100 con 0x5617a25c03e0 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5617a28f7ca0 con 0x5617a29d3120 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5617a293b0c0 con 0x5617a29c0f00 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60466/0 (socket says 172.21.15.116:60466) 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 -- 172.21.15.116:0/3410962980 learned_addr learned my addr 172.21.15.116:0/3410962980 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deaa00640 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2b3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7de2000640 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 -- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 msgr2=0x5617a2b3a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b3a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 -- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 msgr2=0x5617a2b3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2b3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 -- 172.21.15.116:0/3410962980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a28fc800 con 0x5617a25c03e0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7deb400640 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7de4057b80 tx=0x7f7de405b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a8bfc6aaed6966 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7ddb400640 1 -- 172.21.15.116:0/3410962980 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4064170 con 0x5617a25c03e0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7ddb400640 1 -- 172.21.15.116:0/3410962980 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4064820 con 0x5617a25c03e0 2024-05-14T11:22:56.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.128+0000 7f7ddb400640 1 -- 172.21.15.116:0/3410962980 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de408f310 con 0x5617a25c03e0 2024-05-14T11:22:56.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- 172.21.15.116:0/3410962980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 msgr2=0x5617a2b3add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7de4057b80 tx=0x7f7de405b580 comp rx=0 tx=0).stop 2024-05-14T11:22:56.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- 172.21.15.116:0/3410962980 shutdown_connections 2024-05-14T11:22:56.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b3add0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b3a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- 172.21.15.116:0/3410962980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2b3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- 172.21.15.116:0/3410962980 >> 172.21.15.116:0/3410962980 conn(0x5617a2600400 msgr2=0x5617a2b1d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- 172.21.15.116:0/3410962980 shutdown_connections 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- 172.21.15.116:0/3410962980 wait complete. 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- start start 2024-05-14T11:22:56.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 Processor -- start 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2af3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5617a2a0a2a0 con 0x5617a25c03e0 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5617a29faf40 con 0x5617a29d3120 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5617a28fb080 con 0x5617a29c0f00 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7df9396000 10 client.0 _ll_register_callbacks cb 0x5617a2adbc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60468/0 (socket says 172.21.15.116:60468) 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 -- 172.21.15.116:0/496959310 learned_addr learned my addr 172.21.15.116:0/496959310 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:56.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deb400640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2af3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 -- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 msgr2=0x5617a2b41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7de2000640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 -- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 msgr2=0x5617a2af3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2af3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 -- 172.21.15.116:0/496959310 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a28e9860 con 0x5617a25c03e0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deb400640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2af3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7de2000640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7deaa00640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007150 tx=0x7f7ddc001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b549cce9ab33c0ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.132+0000 7f7dcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc004ce0 con 0x5617a25c03e0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f7ddc018a00 con 0x5617a25c03e0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4064820 con 0x5617a25c03e0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f7ddc018c00 con 0x5617a25c03e0 2024-05-14T11:22:56.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 10 client.7242 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5617a28f7ef0 con 0x5617a25c03e0 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4064820 con 0x5617a25c03e0 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f7ddc0128d0 con 0x5617a25c03e0 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 10 client.7242 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 10 client.7242 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 20 client.7242 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25e4930 0x5617a2af4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 4 client.7242 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a28d7df0 con 0x5617a25e4930 2024-05-14T11:22:56.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7de2000640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25e4930 0x5617a2af4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7de2000640 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25e4930 0x5617a2af4d00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738e0301929861b5 server_cookie=5d40b03a8ee2884f in_seq=0 out_seq=0 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 10 client.7242 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 1 -- 172.21.15.116:0/496959310 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617a28d7df0 con 0x5617a25e4930 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 10 client.7242 handle_command_reply: tid=0 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 client.7242 shutdown 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 msgr2=0x5617a2b40dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007150 tx=0x7f7ddc001090 comp rx=0 tx=0).stop 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 shutdown_connections 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29d3120 0x5617a2af3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25e4930 0x5617a2af4d00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29c0f00 0x5617a2b41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 --2- 172.21.15.116:0/496959310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a25c03e0 0x5617a2b40dd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7dcfe00640 0 client.7242 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 >> 172.21.15.116:0/496959310 conn(0x5617a2600400 msgr2=0x5617a2b1d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 shutdown_connections 2024-05-14T11:22:56.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 1 -- 172.21.15.116:0/496959310 wait complete. 2024-05-14T11:22:56.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.136+0000 7f7df9396000 20 client.7242 trim_cache size 0 max 16384 2024-05-14T11:22:56.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 Processor -- start 2024-05-14T11:22:56.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 -- start start 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 0x5617a2ba84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29c0f00 0x5617a2ba8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec205280 con 0x5617a29d3120 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec155f30 con 0x5617a29c0f00 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec204d60 con 0x5617a25c03e0 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60470/0 (socket says 172.21.15.116:60470) 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 -- 172.21.15.116:0/84097087 learned_addr learned my addr 172.21.15.116:0/84097087 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7de2000640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29c0f00 0x5617a2ba8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deaa00640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 0x5617a2ba84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 -- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 msgr2=0x5617a2ba84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 0x5617a2ba84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 -- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29c0f00 msgr2=0x5617a2ba8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29c0f00 0x5617a2ba8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 -- 172.21.15.116:0/84097087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a2af2070 con 0x5617a29d3120 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deaa00640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 0x5617a2ba84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.156+0000 7f7deb400640 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7de4020040 tx=0x7f7de408e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259e387e56a5784e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7ddaa00640 1 -- 172.21.15.116:0/84097087 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4064170 con 0x5617a29d3120 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7ddaa00640 1 -- 172.21.15.116:0/84097087 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4064820 con 0x5617a29d3120 2024-05-14T11:22:56.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7ddaa00640 1 -- 172.21.15.116:0/84097087 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de408f310 con 0x5617a29d3120 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- 172.21.15.116:0/84097087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 msgr2=0x5617a2ba8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7de4020040 tx=0x7f7de408e000 comp rx=0 tx=0).stop 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- 172.21.15.116:0/84097087 shutdown_connections 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a29c0f00 0x5617a2ba8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29d3120 0x5617a2ba8f70 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- 172.21.15.116:0/84097087 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a25c03e0 0x5617a2ba84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- 172.21.15.116:0/84097087 >> 172.21.15.116:0/84097087 conn(0x5617a2600400 msgr2=0x5617a2b1c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- 172.21.15.116:0/84097087 shutdown_connections 2024-05-14T11:22:56.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- 172.21.15.116:0/84097087 wait complete. 2024-05-14T11:22:56.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- start start 2024-05-14T11:22:56.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 Processor -- start 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29c0f00 0x5617a2bace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29d3120 0x5617a2b44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5617a2943730 con 0x5617a29c0f00 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5617a293e410 con 0x5617a25c03e0 2024-05-14T11:22:56.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5617a294c500 con 0x5617a29d3120 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.160+0000 7f7df9396000 10 client.0 _ll_register_callbacks cb 0x7f7ddc007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:49434/0 (socket says 172.21.15.116:49434) 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 -- 172.21.15.116:0/4124233149 learned_addr learned my addr 172.21.15.116:0/4124233149 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7de2000640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29c0f00 0x5617a2bace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deb400640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29d3120 0x5617a2b44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 -- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29d3120 msgr2=0x5617a2b44bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29d3120 0x5617a2b44bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 -- 172.21.15.116:0/4124233149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29c0f00 msgr2=0x5617a2bace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29c0f00 0x5617a2bace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 -- 172.21.15.116:0/4124233149 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a29c0340 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7deaa00640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc008f70 tx=0x7f7ddc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b770a011287a02ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc03a060 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f7ddc018170 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4064820 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f7ddc040050 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 10 client.6943 fetch_fsmap learned FSMap version 68 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5617a28db8e0 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4064820 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f7ddc043020 con 0x5617a25c03e0 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 10 client.6943 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:22:56.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 10 client.6943 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 20 client.6943 populate_metadata read hostname 'smithi116' 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25d2990 0x5617a2b45710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 4 client.6943 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617a294cd10 con 0x5617a25d2990 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7de2000640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25d2990 0x5617a2b45710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7de2000640 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25d2990 0x5617a2b45710 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c4edab3e8eed1f server_cookie=a44fb447484d2741 in_seq=0 out_seq=0 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 10 client.6943 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 1 -- 172.21.15.116:0/4124233149 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5617a294cd10 con 0x5617a25d2990 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 10 client.6943 handle_command_reply: tid=0 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 client.6943 shutdown 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 msgr2=0x5617a2bac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc008f70 tx=0x7f7ddc014000 comp rx=0 tx=0).stop 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 shutdown_connections 2024-05-14T11:22:56.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5617a25d2990 0x5617a2b45710 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5617a29d3120 0x5617a2b44bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5617a29c0f00 0x5617a2bace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 --2- 172.21.15.116:0/4124233149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a25c03e0 0x5617a2bac920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7dcfe00640 0 client.6943 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 >> 172.21.15.116:0/4124233149 conn(0x5617a2600400 msgr2=0x5617a2b1c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 shutdown_connections 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 1 -- 172.21.15.116:0/4124233149 wait complete. 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.164+0000 7f7df9396000 20 client.6943 trim_cache size 0 max 16384 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:22:56.169 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: "dc9ec28c-3709-4f88-bcba-850a840896d3": { 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir", 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "dc9ec28c-3709-4f88-bcba-850a840896d3", 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,repair" 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:22:56.170 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 msgr2=0x7f7dc8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 0x7f7dc8071910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc000f70 tx=0x7f7ddc007df0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 msgr2=0x7f7dec0740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d500 tx=0x7f7de400d9c0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 shutdown_connections 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7dc806f400 0x7f7dc8071910 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dec15db30 0x7f7dec0740a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7dec0d1670 0x7f7dec14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 --2- 172.21.15.116:0/3126671902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7dec06c700 0x7f7dec15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 >> 172.21.15.116:0/3126671902 conn(0x7f7dec0b4620 msgr2=0x7f7dec0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:22:56.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 shutdown_connections 2024-05-14T11:22:56.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:22:56.168+0000 7f7df1c00640 1 -- 172.21.15.116:0/3126671902 wait complete. 2024-05-14T11:22:56.186 INFO:tasks.cephfs.filesystem:scrub status for tag:dc9ec28c-3709-4f88-bcba-850a840896d3 - {'path': '/dir', 'tag': 'dc9ec28c-3709-4f88-bcba-850a840896d3', 'options': 'recursive,repair'} 2024-05-14T11:23:13.077 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:23:13.081 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:23:26.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:26.187 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:23:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 Processor -- start 2024-05-14T11:23:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 -- start start 2024-05-14T11:23:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2d840b0770 0x7f2d8406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d84072a30 0x7f2d84072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8406cc40 con 0x7f2d8406f730 2024-05-14T11:23:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8406db20 con 0x7f2d840b0770 2024-05-14T11:23:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d89c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8406cf10 con 0x7f2d84072a30 2024-05-14T11:23:26.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d82a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2d840b0770 0x7f2d8406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d82a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2d840b0770 0x7f2d8406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:43638/0 (socket says 172.21.15.116:43638) 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d82a00640 1 -- 172.21.15.116:0/2999533893 learned_addr learned my addr 172.21.15.116:0/2999533893 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.348+0000 7f2d83400640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d84072a30 0x7f2d84072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d82a00640 1 -- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d84072a30 msgr2=0x7f2d84072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d82a00640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d84072a30 0x7f2d84072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d82a00640 1 -- 172.21.15.116:0/2999533893 >> v1:172.21.15.149:6789/0 conn(0x7f2d8406f730 legacy=0x7f2d8406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d82a00640 1 -- 172.21.15.116:0/2999533893 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8406d1f0 con 0x7f2d840b0770 2024-05-14T11:23:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d82a00640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d8406c700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009870 tx=0x7f2d7401d5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af25d73ffe18b959 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d81600640 1 -- 172.21.15.116:0/2999533893 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740ab680 con 0x7f2d840b0770 2024-05-14T11:23:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d81600640 1 -- 172.21.15.116:0/2999533893 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d740b0070 con 0x7f2d840b0770 2024-05-14T11:23:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.352+0000 7f2d81600640 1 -- 172.21.15.116:0/2999533893 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740aba40 con 0x7f2d840b0770 2024-05-14T11:23:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 msgr2=0x7f2d8406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d8406c700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009870 tx=0x7f2d7401d5b0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- 172.21.15.116:0/2999533893 shutdown_connections 2024-05-14T11:23:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d84072a30 0x7f2d84072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2999533893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d8406c700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- 172.21.15.116:0/2999533893 >> 172.21.15.116:0/2999533893 conn(0x7f2d840b4620 msgr2=0x7f2d840b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- 172.21.15.116:0/2999533893 shutdown_connections 2024-05-14T11:23:26.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- 172.21.15.116:0/2999533893 wait complete. 2024-05-14T11:23:26.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 Processor -- start 2024-05-14T11:23:26.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.356+0000 7f2d89c00640 1 -- start start 2024-05-14T11:23:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d8406f730 0x7f2d8415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d841537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84073500 con 0x7f2d84072a30 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84073380 con 0x7f2d840b0770 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2d84073680 con 0x7f2d8406f730 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45990/0 (socket says 172.21.15.116:45990) 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 -- 172.21.15.116:0/2456671217 learned_addr learned my addr 172.21.15.116:0/2456671217 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82a00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d8406f730 0x7f2d8415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d83400640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d841537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 -- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d8406f730 msgr2=0x7f2d8415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d8406f730 0x7f2d8415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 -- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 msgr2=0x7f2d841537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d841537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 -- 172.21.15.116:0/2456671217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8415e910 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d82000640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c3c0 tx=0x7f2d7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d7aa00640 1 -- 172.21.15.116:0/2456671217 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70017070 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d7aa00640 1 -- 172.21.15.116:0/2456671217 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d700052d0 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8415e630 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d7aa00640 1 -- 172.21.15.116:0/2456671217 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d700054c0 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8409c610 con 0x7f2d84072a30 2024-05-14T11:23:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.360+0000 7f2d7aa00640 1 -- 172.21.15.116:0/2456671217 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2d7001a070 con 0x7f2d84072a30 2024-05-14T11:23:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d7aa00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d7aa00640 1 -- 172.21.15.116:0/2456671217 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f2d7008e860 con 0x7f2d84072a30 2024-05-14T11:23:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 Processor -- start 2024-05-14T11:23:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 -- start start 2024-05-14T11:23:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 0x7f2d64071940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d740af000 tx=0x7f2d74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 0x56451384b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x56451384bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564513650100 con 0x5645136e4120 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564513608ca0 con 0x5645132d13e0 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56451364c0c0 con 0x5645136d1f00 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45998/0 (socket says 172.21.15.116:45998) 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 -- 172.21.15.116:0/617528716 learned_addr learned my addr 172.21.15.116:0/617528716 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82000640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 0x56451384b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d83400640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x56451384bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 -- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 msgr2=0x56451384b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 0x56451384b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 -- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 msgr2=0x56451384bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x56451384bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 -- 172.21.15.116:0/617528716 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451360d800 con 0x5645136e4120 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82000640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 0x56451384b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d83400640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x56451384bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d82a00640 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ad040 tx=0x7f2d740a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4b980ee6ad774a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d6f400640 1 -- 172.21.15.116:0/617528716 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740c7070 con 0x5645136e4120 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d6f400640 1 -- 172.21.15.116:0/617528716 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d740b0040 con 0x5645136e4120 2024-05-14T11:23:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.364+0000 7f2d6f400640 1 -- 172.21.15.116:0/617528716 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d740b5b00 con 0x5645136e4120 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- 172.21.15.116:0/617528716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 msgr2=0x56451384b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ad040 tx=0x7f2d740a8000 comp rx=0 tx=0).stop 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- 172.21.15.116:0/617528716 shutdown_connections 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x56451384bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136d1f00 0x56451384b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- 172.21.15.116:0/617528716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x56451384b350 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- 172.21.15.116:0/617528716 >> 172.21.15.116:0/617528716 conn(0x564513311400 msgr2=0x56451382e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- 172.21.15.116:0/617528716 shutdown_connections 2024-05-14T11:23:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- 172.21.15.116:0/617528716 wait complete. 2024-05-14T11:23:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- start start 2024-05-14T11:23:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 Processor -- start 2024-05-14T11:23:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645132d13e0 0x564513851dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645136d1f00 0x564513852310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56451371b2a0 con 0x5645136e4120 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56451370bf40 con 0x5645136d1f00 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56451360c080 con 0x5645132d13e0 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d912a4000 10 client.0 _ll_register_callbacks cb 0x5645137ecc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d83400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d83400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46008/0 (socket says 172.21.15.116:46008) 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d83400640 1 -- 172.21.15.116:0/4094509431 learned_addr learned my addr 172.21.15.116:0/4094509431 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645132d13e0 0x564513851dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.368+0000 7f2d82000640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645136d1f00 0x564513852310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 -- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645132d13e0 msgr2=0x564513851dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645132d13e0 0x564513851dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 -- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645136d1f00 msgr2=0x564513852310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645136d1f00 0x564513852310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 -- 172.21.15.116:0/4094509431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645135fa860 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d83400640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007d70 tx=0x7f2d7c012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3115958e3bc5dc57 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c01c020 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f2d7c018670 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d740b0040 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2d7c026050 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 10 client.7257 fetch_fsmap learned FSMap version 68 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x564513608ef0 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d740b0040 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2d7c0039a0 con 0x5645136e4120 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 10 client.7257 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 10 client.7257 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 20 client.7257 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132f5930 0x564513805d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 4 client.7257 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:23:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645135e8df0 con 0x5645132f5930 2024-05-14T11:23:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132f5930 0x564513805d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132f5930 0x564513805d90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a4240561e39c0d server_cookie=68739833584dcb20 in_seq=0 out_seq=0 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 10 client.7257 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 1 -- 172.21.15.116:0/4094509431 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645135e8df0 con 0x5645132f5930 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 10 client.7257 handle_command_reply: tid=0 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 client.7257 shutdown 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 msgr2=0x564513804ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007d70 tx=0x7f2d7c012610 comp rx=0 tx=0).stop 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 shutdown_connections 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645136d1f00 0x564513852310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132f5930 0x564513805d90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136e4120 0x564513804ff0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4094509431 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645132d13e0 0x564513851dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 >> 172.21.15.116:0/4094509431 conn(0x564513311400 msgr2=0x56451382e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.372+0000 7f2d63e00640 0 client.7257 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.376+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 shutdown_connections 2024-05-14T11:23:26.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.376+0000 7f2d912a4000 1 -- 172.21.15.116:0/4094509431 wait complete. 2024-05-14T11:23:26.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.376+0000 7f2d912a4000 20 client.7257 trim_cache size 0 max 16384 2024-05-14T11:23:26.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 Processor -- start 2024-05-14T11:23:26.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 -- start start 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x5645138ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84200e80 con 0x5645136d1f00 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8409d7f0 con 0x5645132d13e0 2024-05-14T11:23:26.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2d84200860 con 0x5645136e4120 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46024/0 (socket says 172.21.15.116:46024) 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d82000640 1 -- 172.21.15.116:0/1614203367 learned_addr learned my addr 172.21.15.116:0/1614203367 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d82a00640 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.392+0000 7f2d83400640 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x5645138ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 -- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 msgr2=0x5645138ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x5645138ba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 -- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 msgr2=0x5645138b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 -- 172.21.15.116:0/1614203367 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564513803070 con 0x5645136d1f00 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d82000640 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2d70057380 tx=0x7f2d70007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b716ab4b8653e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d6ea00640 1 -- 172.21.15.116:0/1614203367 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7005f090 con 0x5645136d1f00 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d6ea00640 1 -- 172.21.15.116:0/1614203367 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d70063b40 con 0x5645136d1f00 2024-05-14T11:23:26.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d6ea00640 1 -- 172.21.15.116:0/1614203367 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70059ff0 con 0x5645136d1f00 2024-05-14T11:23:26.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- 172.21.15.116:0/1614203367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 msgr2=0x5645138b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2d70057380 tx=0x7f2d70007f90 comp rx=0 tx=0).stop 2024-05-14T11:23:26.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- 172.21.15.116:0/1614203367 shutdown_connections 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x5645138ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138b9c40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 --2- 172.21.15.116:0/1614203367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- 172.21.15.116:0/1614203367 >> 172.21.15.116:0/1614203367 conn(0x564513311400 msgr2=0x56451382d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- 172.21.15.116:0/1614203367 shutdown_connections 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- 172.21.15.116:0/1614203367 wait complete. 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 -- start start 2024-05-14T11:23:26.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.396+0000 7f2d912a4000 1 Processor -- start 2024-05-14T11:23:26.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x564513855bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564513654730 con 0x5645136d1f00 2024-05-14T11:23:26.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56451364f410 con 0x5645132d13e0 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56451365d500 con 0x5645136e4120 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 10 client.0 _ll_register_callbacks cb 0x7f2d7c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46040/0 (socket says 172.21.15.116:46040) 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 -- 172.21.15.116:0/4125542039 learned_addr learned my addr 172.21.15.116:0/4125542039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d83400640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x564513855bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 -- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 msgr2=0x564513855bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x564513855bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 -- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 msgr2=0x5645138bdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138bdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 -- 172.21.15.116:0/4125542039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645136d1340 con 0x5645136d1f00 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82000640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2d70056dc0 tx=0x7f2d70000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d00dfc274fca0b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70089070 con 0x5645136d1f00 2024-05-14T11:23:26.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f2d7007bb80 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d70063b40 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2d70083050 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 10 client.7263 fetch_fsmap learned FSMap version 68 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5645135ec8e0 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d70063b40 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f2d70056700 con 0x5645136d1f00 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 10 client.7263 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 10 client.7263 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 20 client.7263 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132e3990 0x5645138565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 4 client.7263 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56451365dd10 con 0x5645132e3990 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132e3990 0x5645138565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 10 client.7263 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d82a00640 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132e3990 0x5645138565b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c37b6c3aacfe9a server_cookie=e16b0bcb6ffe0309 in_seq=0 out_seq=0 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 1 -- 172.21.15.116:0/4125542039 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56451365dd10 con 0x5645132e3990 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 10 client.7263 handle_command_reply: tid=0 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 client.7263 shutdown 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 msgr2=0x5645138be280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2d70056dc0 tx=0x7f2d70000bd0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 shutdown_connections 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x5645132e3990 0x5645138565b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5645136e4120 0x564513855bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5645136d1f00 0x5645138be280 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d63e00640 0 client.7263 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 --2- 172.21.15.116:0/4125542039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5645132d13e0 0x5645138bdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 >> 172.21.15.116:0/4125542039 conn(0x564513311400 msgr2=0x56451382d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 shutdown_connections 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 1 -- 172.21.15.116:0/4125542039 wait complete. 2024-05-14T11:23:26.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.400+0000 7f2d912a4000 20 client.7263 trim_cache size 0 max 16384 2024-05-14T11:23:26.406 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:23:26.406 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:23:26.406 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:23:26.406 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 msgr2=0x7f2d64071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 0x7f2d64071940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d740af000 tx=0x7f2d74008040 comp rx=0 tx=0).stop 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 msgr2=0x7f2d84158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c3c0 tx=0x7f2d7000c880 comp rx=0 tx=0).stop 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 shutdown_connections 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d6406f430 0x7f2d64071940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d840b0770 0x7f2d841537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d84072a30 0x7f2d84158230 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 --2- 172.21.15.116:0/2456671217 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d8406f730 0x7f2d8415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 >> 172.21.15.116:0/2456671217 conn(0x7f2d840b4620 msgr2=0x7f2d84155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 shutdown_connections 2024-05-14T11:23:26.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.404+0000 7f2d89c00640 1 -- 172.21.15.116:0/2456671217 wait complete. 2024-05-14T11:23:26.422 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:23:26.422 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:23:26.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.568+0000 7fc087400640 1 Processor -- start 2024-05-14T11:23:26.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.568+0000 7fc087400640 1 -- start start 2024-05-14T11:23:26.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc087400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc08006f730 0x7fc08006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc087400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc080070080 con 0x7fc0800b0770 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc087400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc08006cf10 con 0x7fc080072a30 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc087400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc08006db20 con 0x7fc08006f730 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc084a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc0800b0770 0x7fc08006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50198/0 (socket says 172.21.15.116:50198) 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc084a00640 1 -- 172.21.15.116:0/2504585971 learned_addr learned my addr 172.21.15.116:0/2504585971 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07fe00640 1 --2- 172.21.15.116:0/2504585971 >> v2:172.21.15.116:3300/0 conn(0x7fc08006f730 0x7fc08006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07f400640 1 -- 172.21.15.116:0/2504585971 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293289062 0 0) 0x7fc070001120 con 0x7fc0800b0770 2024-05-14T11:23:26.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07f400640 1 -- 172.21.15.116:0/2504585971 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc064003530 con 0x7fc0800b0770 2024-05-14T11:23:26.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07fe00640 1 -- 172.21.15.116:0/2504585971 >> v1:172.21.15.149:6789/0 conn(0x7fc080072a30 legacy=0x7fc080072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07fe00640 1 -- 172.21.15.116:0/2504585971 >> v1:172.21.15.116:6789/0 conn(0x7fc0800b0770 legacy=0x7fc08006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.572+0000 7fc07fe00640 1 -- 172.21.15.116:0/2504585971 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08006d1f0 con 0x7fc08006f730 2024-05-14T11:23:26.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc07fe00640 1 --2- 172.21.15.116:0/2504585971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc08006f730 0x7fc08006fb40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097c0 tx=0x7fc06c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b36bbdf5ff60aac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc07f400640 1 -- 172.21.15.116:0/2504585971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b2070 con 0x7fc08006f730 2024-05-14T11:23:26.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc07f400640 1 -- 172.21.15.116:0/2504585971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c0a9070 con 0x7fc08006f730 2024-05-14T11:23:26.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc07f400640 1 -- 172.21.15.116:0/2504585971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ad860 con 0x7fc08006f730 2024-05-14T11:23:26.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 -- 172.21.15.116:0/2504585971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc08006f730 msgr2=0x7fc08006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 --2- 172.21.15.116:0/2504585971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc08006f730 0x7fc08006fb40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0097c0 tx=0x7fc06c01c3e0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 -- 172.21.15.116:0/2504585971 shutdown_connections 2024-05-14T11:23:26.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 --2- 172.21.15.116:0/2504585971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc08006f730 0x7fc08006fb40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 -- 172.21.15.116:0/2504585971 >> 172.21.15.116:0/2504585971 conn(0x7fc0800b4620 msgr2=0x7fc0800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 -- 172.21.15.116:0/2504585971 shutdown_connections 2024-05-14T11:23:26.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.576+0000 7fc087400640 1 -- 172.21.15.116:0/2504585971 wait complete. 2024-05-14T11:23:26.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 Processor -- start 2024-05-14T11:23:26.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- start start 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 0x7fc080167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 0x7fc080157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc080073ae0 con 0x7fc0800b0770 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc080073960 con 0x7fc08015ef70 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc08016b0d0 con 0x7fc080072a30 2024-05-14T11:23:26.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46066/0 (socket says 172.21.15.116:46066) 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 -- 172.21.15.116:0/2729358523 learned_addr learned my addr 172.21.15.116:0/2729358523 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 -- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 msgr2=0x7fc080167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc084a00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 0x7fc080167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc085400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 0x7fc080157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 0x7fc080167020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 -- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 msgr2=0x7fc080157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 0x7fc080157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 -- 172.21.15.116:0/2729358523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc080158740 con 0x7fc0800b0770 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc085400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 0x7fc080157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc084a00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 0x7fc080167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07fe00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0b0040 tx=0x7fc06c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b2070 con 0x7fc0800b0770 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c0bc070 con 0x7fc0800b0770 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c003c50 con 0x7fc0800b0770 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc080143f60 con 0x7fc0800b0770 2024-05-14T11:23:26.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc080155550 con 0x7fc0800b0770 2024-05-14T11:23:26.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.580+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080167700 con 0x7fc0800b0770 2024-05-14T11:23:26.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.584+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc06c0ab070 con 0x7fc0800b0770 2024-05-14T11:23:26.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.584+0000 7fc07cc00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 0x7fc060071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.584+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7fc06c1291b0 con 0x7fc0800b0770 2024-05-14T11:23:26.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.584+0000 7fc084a00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 0x7fc060071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.584+0000 7fc084a00640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 0x7fc060071910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc070001420 tx=0x7fc070009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.588+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc06c0c6020 con 0x7fc0800b0770 2024-05-14T11:23:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.780+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08006a9c0 con 0x7fc0800b0770 2024-05-14T11:23:26.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.780+0000 7fc07cc00640 1 -- 172.21.15.116:0/2729358523 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5193 (secure 0 0 0) 0x7fc06c016090 con 0x7fc0800b0770 2024-05-14T11:23:26.782 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:26.782 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:27.819353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6769,"qdb_cluster":[6769]},"id":6}]} 2024-05-14T11:23:26.783 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 msgr2=0x7fc060071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 0x7fc060071910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc070001420 tx=0x7fc070009040 comp rx=0 tx=0).stop 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 msgr2=0x7fc080146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0b0040 tx=0x7fc06c01ac20 comp rx=0 tx=0).stop 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 shutdown_connections 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc06006f400 0x7fc060071910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc08015ef70 0x7fc080157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0800b0770 0x7fc080146820 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 --2- 172.21.15.116:0/2729358523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc080072a30 0x7fc080167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 >> 172.21.15.116:0/2729358523 conn(0x7fc0800b4620 msgr2=0x7fc0800b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 shutdown_connections 2024-05-14T11:23:26.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.784+0000 7fc087400640 1 -- 172.21.15.116:0/2729358523 wait complete. 2024-05-14T11:23:26.805 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-14T11:23:26.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 Processor -- start 2024-05-14T11:23:26.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 -- start start 2024-05-14T11:23:26.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f79e815db30 0x7f79e80aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e806ff20 con 0x7f79e806c700 2024-05-14T11:23:26.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e8070470 con 0x7f79e80d1670 2024-05-14T11:23:26.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.952+0000 7f79ecc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f79e80ae100 con 0x7f79e815db30 2024-05-14T11:23:26.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e6800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f79e806c700 0x7f79e806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50204/0 (socket says 172.21.15.116:50204) 2024-05-14T11:23:26.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e6800640 1 -- 172.21.15.116:0/3312038165 learned_addr learned my addr 172.21.15.116:0/3312038165 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5e00640 1 --2- 172.21.15.116:0/3312038165 >> v2:172.21.15.149:3300/0 conn(0x7f79e815db30 0x7f79e80aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5400640 1 -- 172.21.15.116:0/3312038165 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2567558746 0 0) 0x7f79e806ff20 con 0x7f79e806c700 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5400640 1 -- 172.21.15.116:0/3312038165 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79cc003530 con 0x7f79e806c700 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5e00640 1 -- 172.21.15.116:0/3312038165 >> v1:172.21.15.149:6789/0 conn(0x7f79e80d1670 legacy=0x7f79e80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5e00640 1 -- 172.21.15.116:0/3312038165 >> v1:172.21.15.116:6789/0 conn(0x7f79e806c700 legacy=0x7f79e806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5e00640 1 -- 172.21.15.116:0/3312038165 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e8070750 con 0x7f79e815db30 2024-05-14T11:23:26.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.956+0000 7f79e5e00640 1 --2- 172.21.15.116:0/3312038165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e80aeaa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ae280 tx=0x7f79d801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547cb9273ece192e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79e5400640 1 -- 172.21.15.116:0/3312038165 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80b0070 con 0x7f79e815db30 2024-05-14T11:23:26.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79e5400640 1 -- 172.21.15.116:0/3312038165 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d80ab600 con 0x7f79e815db30 2024-05-14T11:23:26.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79e5400640 1 -- 172.21.15.116:0/3312038165 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80ab9d0 con 0x7f79e815db30 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- 172.21.15.116:0/3312038165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 msgr2=0x7f79e80aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 --2- 172.21.15.116:0/3312038165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e80aeaa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f79e80ae280 tx=0x7f79d801c410 comp rx=0 tx=0).stop 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- 172.21.15.116:0/3312038165 shutdown_connections 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 --2- 172.21.15.116:0/3312038165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e80aeaa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- 172.21.15.116:0/3312038165 >> 172.21.15.116:0/3312038165 conn(0x7f79e80b4620 msgr2=0x7f79e80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- 172.21.15.116:0/3312038165 shutdown_connections 2024-05-14T11:23:26.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- 172.21.15.116:0/3312038165 wait complete. 2024-05-14T11:23:26.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 Processor -- start 2024-05-14T11:23:26.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.960+0000 7f79ecc00640 1 -- start start 2024-05-14T11:23:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 0x7f79e8156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e8141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f79e816bf90 con 0x7f79e80d1670 2024-05-14T11:23:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79e815df40 con 0x7f79e815db30 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f79e816c110 con 0x7f79e806c700 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46070/0 (socket says 172.21.15.116:46070) 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 -- 172.21.15.116:0/655381558 learned_addr learned my addr 172.21.15.116:0/655381558 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79e6800640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e8141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79e5e00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 0x7f79e8156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 -- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 msgr2=0x7f79e8156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 0x7f79e8156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 -- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 msgr2=0x7f79e8141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e8141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 -- 172.21.15.116:0/655381558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e8142090 con 0x7f79e80d1670 2024-05-14T11:23:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79e5e00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 0x7f79e8156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dd400640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c3c0 tx=0x7f79d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dea00640 1 -- 172.21.15.116:0/655381558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0017070 con 0x7f79e80d1670 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dea00640 1 -- 172.21.15.116:0/655381558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d00052d0 con 0x7f79e80d1670 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e8142370 con 0x7f79e80d1670 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79dea00640 1 -- 172.21.15.116:0/655381558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00054c0 con 0x7f79e80d1670 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.964+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e8142930 con 0x7f79e80d1670 2024-05-14T11:23:26.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79dea00640 1 -- 172.21.15.116:0/655381558 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79d001a070 con 0x7f79e80d1670 2024-05-14T11:23:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79dea00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 0x7f79c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79dea00640 1 -- 172.21.15.116:0/655381558 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f79d008e8a0 con 0x7f79e80d1670 2024-05-14T11:23:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 0x7f79c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 Processor -- start 2024-05-14T11:23:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 -- start start 2024-05-14T11:23:26.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 0x7f79c4071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f79d80ad000 tx=0x7f79d80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f20b2d0 0x560e3f37f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f37f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e3f1841a0 con 0x560e3f21bbd0 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560e3f13cb80 con 0x560e3f20b2d0 2024-05-14T11:23:26.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x560e3f180160 con 0x560e3ee05580 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46074/0 (socket says 172.21.15.116:46074) 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 -- 172.21.15.116:0/1939917582 learned_addr learned my addr 172.21.15.116:0/1939917582 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79dd400640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f20b2d0 0x560e3f37f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e6800640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f37f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 -- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 msgr2=0x560e3f37f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f37f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 -- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f20b2d0 msgr2=0x560e3f37f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f20b2d0 0x560e3f37f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e5e00640 1 -- 172.21.15.116:0/1939917582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3f190630 con 0x560e3f21bbd0 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.968+0000 7f79e6800640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f37f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79e5e00640 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009130 tx=0x7f79d8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bca6ced3e52e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79d6600640 1 -- 172.21.15.116:0/1939917582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80b0040 con 0x560e3f21bbd0 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79d6600640 1 -- 172.21.15.116:0/1939917582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d80c5070 con 0x560e3f21bbd0 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79d6600640 1 -- 172.21.15.116:0/1939917582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80b6640 con 0x560e3f21bbd0 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- 172.21.15.116:0/1939917582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 msgr2=0x560e3f37ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009130 tx=0x7f79d8004740 comp rx=0 tx=0).stop 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- 172.21.15.116:0/1939917582 shutdown_connections 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f20b2d0 0x560e3f37f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f37f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 --2- 172.21.15.116:0/1939917582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f21bbd0 0x560e3f37ee70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- 172.21.15.116:0/1939917582 >> 172.21.15.116:0/1939917582 conn(0x560e3ee44760 msgr2=0x560e3f361e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- 172.21.15.116:0/1939917582 shutdown_connections 2024-05-14T11:23:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- 172.21.15.116:0/1939917582 wait complete. 2024-05-14T11:23:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 -- start start 2024-05-14T11:23:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.972+0000 7f79f4219000 1 Processor -- start 2024-05-14T11:23:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f38a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f21bbd0 0x560e3f3382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e3f123ab0 con 0x560e3f20b2d0 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560e3f138f20 con 0x560e3f21bbd0 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x560e3f1e3da0 con 0x560e3ee05580 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 10 client.0 _ll_register_callbacks cb 0x560e3f31ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46080/0 (socket says 172.21.15.116:46080) 2024-05-14T11:23:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 -- 172.21.15.116:0/2721422809 learned_addr learned my addr 172.21.15.116:0/2721422809 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79e5e00640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f38a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79e6800640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f21bbd0 0x560e3f3382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 -- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 msgr2=0x560e3f38a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f38a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 -- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f21bbd0 msgr2=0x560e3f3382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f21bbd0 0x560e3f3382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 -- 172.21.15.116:0/2721422809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3f142ce0 con 0x560e3f20b2d0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79dd400640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f79d0056e50 tx=0x7f79d005aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f91f93d72bdfd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0079060 con 0x560e3f20b2d0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f79d005a230 con 0x560e3f20b2d0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d80c5070 con 0x560e3f20b2d0 2024-05-14T11:23:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f79d008c050 con 0x560e3f20b2d0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 10 client.7284 fetch_fsmap learned FSMap version 68 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560e3f12e740 con 0x560e3f20b2d0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d80c5070 con 0x560e3f20b2d0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f79d0056740 con 0x560e3f20b2d0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 10 client.7284 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 10 client.7284 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 20 client.7284 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee29ad0 0x560e3f339070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 4 client.7284 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3f11ce60 con 0x560e3ee29ad0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79e5e00640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee29ad0 0x560e3f339070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 10 client.7284 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79e5e00640 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee29ad0 0x560e3f339070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceefb7447c166c0e server_cookie=3b8ced35d41ff3fe in_seq=0 out_seq=0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 1 -- 172.21.15.116:0/2721422809 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e3f11ce60 con 0x560e3ee29ad0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79caa00640 10 client.7284 handle_command_reply: tid=0 2024-05-14T11:23:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.976+0000 7f79f4219000 1 client.7284 shutdown 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 msgr2=0x560e3f38ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f79d0056e50 tx=0x7f79d005aa10 comp rx=0 tx=0).stop 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 shutdown_connections 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3f21bbd0 0x560e3f3382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee29ad0 0x560e3f339070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f38ae80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 --2- 172.21.15.116:0/2721422809 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f38a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79caa00640 0 client.7284 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 >> 172.21.15.116:0/2721422809 conn(0x560e3ee44760 msgr2=0x560e3f360640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 shutdown_connections 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 1 -- 172.21.15.116:0/2721422809 wait complete. 2024-05-14T11:23:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.980+0000 7f79f4219000 20 client.7284 trim_cache size 0 max 16384 2024-05-14T11:23:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.992+0000 7f79f4219000 1 Processor -- start 2024-05-14T11:23:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.992+0000 7f79f4219000 1 -- start start 2024-05-14T11:23:26.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f39aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3f20b2d0 0x560e3f38c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8074640 con 0x560e3ee05580 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79e809def0 con 0x560e3ee44760 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f79e8073ff0 con 0x560e3f20b2d0 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46084/0 (socket says 172.21.15.116:46084) 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 -- 172.21.15.116:0/1910670838 learned_addr learned my addr 172.21.15.116:0/1910670838 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79e5e00640 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f39aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79e6800640 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3f20b2d0 0x560e3f38c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 -- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3f20b2d0 msgr2=0x560e3f38c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3f20b2d0 0x560e3f38c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 -- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 msgr2=0x560e3f39aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f39aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 -- 172.21.15.116:0/1910670838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3f13cdd0 con 0x560e3ee05580 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79dd400640 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f79d00573c0 tx=0x7f79d0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb23bfc3437fe0a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79d5c00640 1 -- 172.21.15.116:0/1910670838 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0071070 con 0x560e3ee05580 2024-05-14T11:23:26.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79d5c00640 1 -- 172.21.15.116:0/1910670838 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d007a020 con 0x560e3ee05580 2024-05-14T11:23:26.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79d5c00640 1 -- 172.21.15.116:0/1910670838 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d007be00 con 0x560e3ee05580 2024-05-14T11:23:26.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- 172.21.15.116:0/1910670838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 msgr2=0x560e3f39b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f79d00573c0 tx=0x7f79d0014070 comp rx=0 tx=0).stop 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- 172.21.15.116:0/1910670838 shutdown_connections 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3f20b2d0 0x560e3f38c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3ee05580 0x560e3f39b020 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 --2- 172.21.15.116:0/1910670838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f39aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- 172.21.15.116:0/1910670838 >> 172.21.15.116:0/1910670838 conn(0x560e3f391390 msgr2=0x560e3f337cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- 172.21.15.116:0/1910670838 shutdown_connections 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- 172.21.15.116:0/1910670838 wait complete. 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 -- start start 2024-05-14T11:23:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:26.996+0000 7f79f4219000 1 Processor -- start 2024-05-14T11:23:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f3f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f3b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e3f12d5f0 con 0x560e3f20b2d0 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560e3f17ee80 con 0x560e3ee44760 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x560e3f13ff60 con 0x560e3ee05580 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 10 client.0 _ll_register_callbacks cb 0x7f79d005ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f3b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79dd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f3f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:51096/0 (socket says 172.21.15.116:51096) 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f3b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46094/0 (socket says 172.21.15.116:46094) 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 -- 172.21.15.116:0/3435095343 learned_addr learned my addr 172.21.15.116:0/3435095343 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 -- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 msgr2=0x560e3f3f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f3f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 -- 172.21.15.116:0/3435095343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 msgr2=0x560e3f3b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f3b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 -- 172.21.15.116:0/3435095343 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x560e3f204c40 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79dd400640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f3f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79e5e00640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79d801c8f0 tx=0x7f79d80046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5455c3b1c3f0048f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3301/0 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80d8060 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mon.2 v2:172.21.15.149:3301/0 2 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f79d80d3600 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d007a020 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f79d80de050 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 10 client.6341 fetch_fsmap learned FSMap version 68 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560e3ee3d3c0 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mon.2 v2:172.21.15.149:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d007a020 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mon.2 v2:172.21.15.149:3301/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f79d80e1020 con 0x560e3ee05580 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 10 client.6341 fetch_fsmap finished waiting for FSMap version 68 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 10 client.6341 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:23:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 20 client.6341 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee43180 0x560e3f3ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 4 client.6341 mds_command: new command op to 6769 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.000+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 --> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560e3f191db0 con 0x560e3ee43180 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79dd400640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee43180 0x560e3f3ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79dd400640 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee43180 0x560e3f3ff850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07bccebbda976f9 server_cookie=9262d6dae487530 in_seq=0 out_seq=0 2024-05-14T11:23:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79caa00640 10 client.6341 ms_handle_connect on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:27.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79caa00640 1 -- 172.21.15.116:0/3435095343 <== mds.0 v2:172.21.15.149:6833/1421948627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x560e3f191db0 con 0x560e3ee43180 2024-05-14T11:23:27.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79caa00640 10 client.6341 handle_command_reply: tid=0 2024-05-14T11:23:27.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 client.6341 shutdown 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 msgr2=0x560e3f3f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79d801c8f0 tx=0x7f79d80046d0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 shutdown_connections 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e3ee44760 0x560e3f3f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:6833/1421948627,v1:172.21.15.149:6835/1421948627] conn(0x560e3ee43180 0x560e3f3ff850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e3f20b2d0 0x560e3f3b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 --2- 172.21.15.116:0/3435095343 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x560e3ee05580 0x560e3f3f69e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 >> 172.21.15.116:0/3435095343 conn(0x560e3f391390 msgr2=0x560e3f399ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79caa00640 0 client.6341 ms_handle_reset on v2:172.21.15.149:6833/1421948627 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 shutdown_connections 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 1 -- 172.21.15.116:0/3435095343 wait complete. 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79f4219000 20 client.6341 trim_cache size 0 max 16384 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:27.008 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: "damage_type": "backtrace", 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 1526738102, 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 1099511627777, 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/testdir/hardlink" 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:23:27.009 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 msgr2=0x7f79c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 0x7f79c4071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f79d80ad000 tx=0x7f79d80a6000 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 msgr2=0x7f79e8141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c3c0 tx=0x7f79d000c880 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 shutdown_connections 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f79c406f400 0x7f79c4071910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79e815db30 0x7f79e8141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80d1670 0x7f79e8141310 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 --2- 172.21.15.116:0/655381558 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79e806c700 0x7f79e8156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 >> 172.21.15.116:0/655381558 conn(0x7f79e80b4620 msgr2=0x7f79e8159210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 shutdown_connections 2024-05-14T11:23:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.004+0000 7f79ecc00640 1 -- 172.21.15.116:0/655381558 wait complete. 2024-05-14T11:23:27.024 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:23:27.024 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:23:27.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:23:27.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.220+0000 7f7ea0600640 1 Processor -- start 2024-05-14T11:23:27.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.220+0000 7f7ea0600640 1 -- start start 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7ea0600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e98156fa0 0x7f7e9806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7ea0600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7e980af380 0x7f7e980af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7ea0600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e98160510 con 0x7f7e980d1670 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7ea0600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7e981573b0 con 0x7f7e98156fa0 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7ea0600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7e98161120 con 0x7f7e980af380 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e98156fa0 0x7f7e9806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e98156fa0 0x7f7e9806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46110/0 (socket says 172.21.15.116:46110) 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 -- 172.21.15.116:0/3145976196 learned_addr learned my addr 172.21.15.116:0/3145976196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9e600640 1 --2- 172.21.15.116:0/3145976196 >> v2:172.21.15.149:3300/0 conn(0x7f7e980af380 0x7f7e980af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 -- 172.21.15.116:0/3145976196 >> v2:172.21.15.149:3300/0 conn(0x7f7e980af380 msgr2=0x7f7e980af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 --2- 172.21.15.116:0/3145976196 >> v2:172.21.15.149:3300/0 conn(0x7f7e980af380 0x7f7e980af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 -- 172.21.15.116:0/3145976196 >> v1:172.21.15.149:6789/0 conn(0x7f7e980d1670 legacy=0x7f7e980d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.224+0000 7f7e9dc00640 1 -- 172.21.15.116:0/3145976196 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981607f0 con 0x7f7e98156fa0 2024-05-14T11:23:27.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7e9dc00640 1 --2- 172.21.15.116:0/3145976196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e98156fa0 0x7f7e9806ffe0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7e98074e70 tx=0x7f7e8c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d510e87e3ddf5b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7e97e00640 1 -- 172.21.15.116:0/3145976196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0aa650 con 0x7f7e98156fa0 2024-05-14T11:23:27.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7e97e00640 1 -- 172.21.15.116:0/3145976196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8c0af070 con 0x7f7e98156fa0 2024-05-14T11:23:27.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7e97e00640 1 -- 172.21.15.116:0/3145976196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0aaa10 con 0x7f7e98156fa0 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 -- 172.21.15.116:0/3145976196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e98156fa0 msgr2=0x7f7e9806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 --2- 172.21.15.116:0/3145976196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e98156fa0 0x7f7e9806ffe0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7e98074e70 tx=0x7f7e8c01c4f0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 -- 172.21.15.116:0/3145976196 shutdown_connections 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 --2- 172.21.15.116:0/3145976196 >> v2:172.21.15.149:3300/0 conn(0x7f7e980af380 0x7f7e980af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 --2- 172.21.15.116:0/3145976196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e98156fa0 0x7f7e9806ffe0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 -- 172.21.15.116:0/3145976196 >> 172.21.15.116:0/3145976196 conn(0x7f7e980b4620 msgr2=0x7f7e980b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 -- 172.21.15.116:0/3145976196 shutdown_connections 2024-05-14T11:23:27.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.228+0000 7f7ea0600640 1 -- 172.21.15.116:0/3145976196 wait complete. 2024-05-14T11:23:27.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 Processor -- start 2024-05-14T11:23:27.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- start start 2024-05-14T11:23:27.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e980af380 0x7f7e980ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e98156fa0 0x7f7e9815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980b0660 con 0x7f7e980d1670 2024-05-14T11:23:27.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980b04e0 con 0x7f7e98156fa0 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7e98169810 con 0x7f7e980af380 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46124/0 (socket says 172.21.15.116:46124) 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 -- 172.21.15.116:0/1667083095 learned_addr learned my addr 172.21.15.116:0/1667083095 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9e600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e98156fa0 0x7f7e9815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9dc00640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e980af380 0x7f7e980ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 -- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e980af380 msgr2=0x7f7e980ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e980af380 0x7f7e980ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 -- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e98156fa0 msgr2=0x7f7e9815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e98156fa0 0x7f7e9815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 -- 172.21.15.116:0/1667083095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981546d0 con 0x7f7e980d1670 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e9d200640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c3f0 tx=0x7f7e8400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84017070 con 0x7f7e980d1670 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e840052d0 con 0x7f7e980d1670 2024-05-14T11:23:27.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84003820 con 0x7f7e980d1670 2024-05-14T11:23:27.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e981f9980 con 0x7f7e980d1670 2024-05-14T11:23:27.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.232+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981f9f40 con 0x7f7e980d1670 2024-05-14T11:23:27.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9806a9c0 con 0x7f7e980d1670 2024-05-14T11:23:27.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e8401a070 con 0x7f7e980d1670 2024-05-14T11:23:27.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7e95600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 0x7f7e78071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f7e8408e7a0 con 0x7f7e980d1670 2024-05-14T11:23:27.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7e9dc00640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 0x7f7e78071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.236+0000 7f7e9dc00640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 0x7f7e78071940 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ae000 tx=0x7f7e8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.240+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7e84005480 con 0x7f7e980d1670 2024-05-14T11:23:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.408+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7e981f9710 con 0x7f7e980d1670 2024-05-14T11:23:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.408+0000 7f7e95600640 1 -- 172.21.15.116:0/1667083095 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v87) ==== 80+0+122 (secure 0 0 0) 0x7f7e84063420 con 0x7f7e980d1670 2024-05-14T11:23:27.409 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6835/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:23:27.409 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6833/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:23:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:23:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 msgr2=0x7f7e78071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 0x7f7e78071940 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ae000 tx=0x7f7e8c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 msgr2=0x7f7e9814f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c3f0 tx=0x7f7e8400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 shutdown_connections 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e7806f430 0x7f7e78071940 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e98156fa0 0x7f7e9815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e980d1670 0x7f7e9814f750 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 --2- 172.21.15.116:0/1667083095 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e980af380 0x7f7e980ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 >> 172.21.15.116:0/1667083095 conn(0x7f7e980b4620 msgr2=0x7f7e9815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 shutdown_connections 2024-05-14T11:23:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.412+0000 7f7ea0600640 1 -- 172.21.15.116:0/1667083095 wait complete. 2024-05-14T11:23:27.431 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:23:27.431 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:23:27.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:23:27.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 Processor -- start 2024-05-14T11:23:27.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 -- start start 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 0x7f60fc15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc0d28e0 con 0x7f60fc0b9020 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60fc0b9430 con 0x7f60fc0d24d0 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.628+0000 7f6102400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0d2040 con 0x7f60fc0d09c0 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fbe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f60fc0b9020 0x7f60fc0d02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51470/0 (socket says 172.21.15.116:51470) 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fbe00640 1 -- 172.21.15.116:0/2607285488 learned_addr learned my addr 172.21.15.116:0/2607285488 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f6101a00640 1 --2- 172.21.15.116:0/2607285488 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 0x7f60fc15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612010775 0 0) 0x7f60e80014a0 con 0x7f60fc0b9020 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e0003530 con 0x7f60fc0b9020 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47066770 0 0) 0x7f60fc0b9430 con 0x7f60fc0d24d0 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e80014a0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f6101a00640 1 -- 172.21.15.116:0/2607285488 >> v1:172.21.15.149:6789/0 conn(0x7f60fc0d24d0 legacy=0x7f60fc0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f6101a00640 1 -- 172.21.15.116:0/2607285488 >> v1:172.21.15.116:6789/0 conn(0x7f60fc0b9020 legacy=0x7f60fc0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f6101a00640 1 -- 172.21.15.116:0/2607285488 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0d1430 con 0x7f60fc0d09c0 2024-05-14T11:23:27.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.632+0000 7f6101a00640 1 --2- 172.21.15.116:0/2607285488 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 0x7f60fc15f2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0d21c0 tx=0x7f60f401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da70da8ee49cab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f40b1070 con 0x7f60fc0d09c0 2024-05-14T11:23:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f40ac410 con 0x7f60fc0d09c0 2024-05-14T11:23:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f60fb400640 1 -- 172.21.15.116:0/2607285488 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f40ac7c0 con 0x7f60fc0d09c0 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- 172.21.15.116:0/2607285488 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 msgr2=0x7f60fc15f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 --2- 172.21.15.116:0/2607285488 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 0x7f60fc15f2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0d21c0 tx=0x7f60f401c4c0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- 172.21.15.116:0/2607285488 shutdown_connections 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 --2- 172.21.15.116:0/2607285488 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0d09c0 0x7f60fc15f2a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- 172.21.15.116:0/2607285488 >> 172.21.15.116:0/2607285488 conn(0x7f60fc0b51f0 msgr2=0x7f60fc0b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- 172.21.15.116:0/2607285488 shutdown_connections 2024-05-14T11:23:27.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- 172.21.15.116:0/2607285488 wait complete. 2024-05-14T11:23:27.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 Processor -- start 2024-05-14T11:23:27.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.636+0000 7f6102400640 1 -- start start 2024-05-14T11:23:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 0x7f60fc0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0ca3f0 0x7f60fc0ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc16b9d0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc16b850 con 0x7f60fc0ca3f0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc16bb50 con 0x7f60fc0b9020 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49052/0 (socket says 172.21.15.116:49052) 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 -- 172.21.15.116:0/2512194806 learned_addr learned my addr 172.21.15.116:0/2512194806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6101a00640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0ca3f0 0x7f60fc0ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6101000640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 0x7f60fc0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 -- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 msgr2=0x7f60fc0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 0x7f60fc0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 -- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0ca3f0 msgr2=0x7f60fc0ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0ca3f0 0x7f60fc0ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 -- 172.21.15.116:0/2512194806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0cad40 con 0x7f60fc0d24d0 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6101000640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 0x7f60fc0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60fbe00640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f60e8001fb0 tx=0x7f60e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8012400 con 0x7f60fc0d24d0 2024-05-14T11:23:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e80125a0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc0ccdd0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8012950 con 0x7f60fc0d24d0 2024-05-14T11:23:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1670f0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.640+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc050880 con 0x7f60fc0d24d0 2024-05-14T11:23:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.644+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60e8012af0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.644+0000 7f60f8c00640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:27.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.644+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f60e808fec0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.644+0000 7f6101000640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:27.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.644+0000 7f6101000640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f60ec001620 tx=0x7f60ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:27.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.648+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f60fc0cad40 con 0x7f60fc0d24d0 2024-05-14T11:23:27.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.816+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60fc15f340 con 0x7f60fc0d24d0 2024-05-14T11:23:27.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.816+0000 7f60f8c00640 1 -- 172.21.15.116:0/2512194806 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v87) ==== 80+0+122 (secure 0 0 0) 0x7f60fc0d28e0 con 0x7f60fc0d24d0 2024-05-14T11:23:27.818 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6835/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:23:27.818 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6833/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:23:27.818 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 msgr2=0x7f60dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f60ec001620 tx=0x7f60ec009040 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 msgr2=0x7f60fc14f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f60e8001fb0 tx=0x7f60e800a430 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 shutdown_connections 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60dc06f400 0x7f60dc071910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc0ca3f0 0x7f60fc0ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60fc0d24d0 0x7f60fc14f610 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 --2- 172.21.15.116:0/2512194806 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60fc0b9020 0x7f60fc0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 >> 172.21.15.116:0/2512194806 conn(0x7f60fc0b51f0 msgr2=0x7f60fc15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 shutdown_connections 2024-05-14T11:23:27.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:27.820+0000 7f6102400640 1 -- 172.21.15.116:0/2512194806 wait complete. 2024-05-14T11:23:27.839 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:23:27.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:27.840 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:27.857 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[44606]: fuse finished with error 0 and tester_r 0 2024-05-14T11:23:27.860 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:23:33.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:23:33.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:33.962 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:23:33.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:23:34.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 Processor -- start 2024-05-14T11:23:34.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 -- start start 2024-05-14T11:23:34.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2a9415db30 0x7f2a940aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9406ff20 con 0x7f2a94161600 2024-05-14T11:23:34.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9406d710 con 0x7f2a9415db30 2024-05-14T11:23:34.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a9b400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2a940ae100 con 0x7f2a940d1670 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a98a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2a9415db30 0x7f2a940aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a8be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:40798/0 (socket says 172.21.15.116:40798) 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a99400640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f2a94161600 0x7f2a94161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:49784/0 (socket says 172.21.15.116:49784) 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a98a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2a9415db30 0x7f2a940aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59182/0 (socket says 172.21.15.116:59182) 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.172+0000 7f2a8be00640 1 -- 172.21.15.116:0/847492647 learned_addr learned my addr 172.21.15.116:0/847492647 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:34.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a98a00640 1 -- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 msgr2=0x7f2a940d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:34.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a98a00640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a98a00640 1 -- 172.21.15.116:0/847492647 >> v1:172.21.15.149:6789/0 conn(0x7f2a94161600 legacy=0x7f2a94161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a98a00640 1 -- 172.21.15.116:0/847492647 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a940d1fc0 con 0x7f2a9415db30 2024-05-14T11:23:34.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a93e00640 1 -- 172.21.15.116:0/847492647 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319671724 0 0) 0x7f2a9406ff20 con 0x7f2a94161600 2024-05-14T11:23:34.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a8be00640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:34.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a98a00640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a9415db30 0x7f2a940aeaa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a840097c0 tx=0x7f2a8401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2ab92217173d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a93e00640 1 -- 172.21.15.116:0/847492647 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840b0070 con 0x7f2a9415db30 2024-05-14T11:23:34.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.176+0000 7f2a93e00640 1 -- 172.21.15.116:0/847492647 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a840ab600 con 0x7f2a9415db30 2024-05-14T11:23:34.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a93e00640 1 -- 172.21.15.116:0/847492647 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840ab9d0 con 0x7f2a9415db30 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a9415db30 msgr2=0x7f2a940aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a9415db30 0x7f2a940aeaa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a840097c0 tx=0x7f2a8401c410 comp rx=0 tx=0).stop 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- 172.21.15.116:0/847492647 shutdown_connections 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a940d1670 0x7f2a940d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 --2- 172.21.15.116:0/847492647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a9415db30 0x7f2a940aeaa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- 172.21.15.116:0/847492647 >> 172.21.15.116:0/847492647 conn(0x7f2a940b4620 msgr2=0x7f2a940b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- 172.21.15.116:0/847492647 shutdown_connections 2024-05-14T11:23:34.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- 172.21.15.116:0/847492647 wait complete. 2024-05-14T11:23:34.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 Processor -- start 2024-05-14T11:23:34.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.180+0000 7f2a9b400640 1 -- start start 2024-05-14T11:23:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 0x7f2a94166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 0x7f2a94167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9415df40 con 0x7f2a940d1670 2024-05-14T11:23:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2a940aefe0 con 0x7f2a94161600 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9416af80 con 0x7f2a9415db30 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49062/0 (socket says 172.21.15.116:49062) 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 -- 172.21.15.116:0/1704905872 learned_addr learned my addr 172.21.15.116:0/1704905872 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a99400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 0x7f2a94167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a8be00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 0x7f2a94166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 -- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 msgr2=0x7f2a94166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 0x7f2a94166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 -- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 msgr2=0x7f2a94167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 0x7f2a94167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 -- 172.21.15.116:0/1704905872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a94074250 con 0x7f2a940d1670 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a8be00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 0x7f2a94166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a99400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 0x7f2a94167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:34.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a98a00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2a8401a040 tx=0x7f2a8401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840b0050 con 0x7f2a940d1670 2024-05-14T11:23:34.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a840bc070 con 0x7f2a940d1670 2024-05-14T11:23:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840b5830 con 0x7f2a940d1670 2024-05-14T11:23:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a94074530 con 0x7f2a940d1670 2024-05-14T11:23:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a94145160 con 0x7f2a940d1670 2024-05-14T11:23:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.184+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9406a9c0 con 0x7f2a940d1670 2024-05-14T11:23:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.188+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2a840b59d0 con 0x7f2a940d1670 2024-05-14T11:23:34.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.188+0000 7f2a91600640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 0x7f2a70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.188+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f2a84129c10 con 0x7f2a940d1670 2024-05-14T11:23:34.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.188+0000 7f2a8be00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 0x7f2a70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.188+0000 7f2a8be00640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 0x7f2a70071910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a9415d3e0 tx=0x7f2a7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.192+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2a840c6020 con 0x7f2a940d1670 2024-05-14T11:23:34.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.384+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a94168540 con 0x7f2a940d1670 2024-05-14T11:23:34.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.384+0000 7f2a91600640 1 -- 172.21.15.116:0/1704905872 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5193 (secure 0 0 0) 0x7f2a940d1fc0 con 0x7f2a940d1670 2024-05-14T11:23:34.387 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:34.387 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:22:27.819353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6769":{"gid":6769,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.149:6835/1421948627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":1421948627},{"type":"v1","addr":"172.21.15.149:6835","nonce":1421948627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6769,"qdb_cluster":[6769]},"id":6}]} 2024-05-14T11:23:34.387 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 msgr2=0x7f2a70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 0x7f2a70071910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a9415d3e0 tx=0x7f2a7c008040 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 msgr2=0x7f2a940736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2a8401a040 tx=0x7f2a8401ac50 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 shutdown_connections 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2a7006f400 0x7f2a70071910 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a94161600 0x7f2a94167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2a9415db30 0x7f2a94166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 --2- 172.21.15.116:0/1704905872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a940d1670 0x7f2a940736e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 >> 172.21.15.116:0/1704905872 conn(0x7f2a940b4620 msgr2=0x7f2a9415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 shutdown_connections 2024-05-14T11:23:34.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.388+0000 7f2a9b400640 1 -- 172.21.15.116:0/1704905872 wait complete. 2024-05-14T11:23:34.408 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-14T11:23:34.599 INFO:teuthology.orchestra.run.smithi149.stdout:{"message":"","return_code":0} 2024-05-14T11:23:34.599 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:23:34.599 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:23:34.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.744+0000 7f1f85a00640 1 Processor -- start 2024-05-14T11:23:34.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.744+0000 7f1f85a00640 1 -- start start 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f85a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1f8015db30 0x7f1f800aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f85a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f8006c700 0x7f1f8006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f85a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8006ff20 con 0x7f1f800d1670 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f85a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1f800ae100 con 0x7f1f8006c700 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f85a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1f800ae870 con 0x7f1f8015db30 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f8006c700 0x7f1f8006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f76000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1f800d1670 0x7f1f800d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51478/0 (socket says 172.21.15.116:51478) 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f76000640 1 -- 172.21.15.116:0/4175336443 learned_addr learned my addr 172.21.15.116:0/4175336443 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7ea00640 1 --2- 172.21.15.116:0/4175336443 >> v2:172.21.15.149:3300/0 conn(0x7f1f8015db30 0x7f1f800aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7f400640 1 -- 172.21.15.116:0/4175336443 >> v2:172.21.15.149:3300/0 conn(0x7f1f8015db30 msgr2=0x7f1f800aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7f400640 1 --2- 172.21.15.116:0/4175336443 >> v2:172.21.15.149:3300/0 conn(0x7f1f8015db30 0x7f1f800aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7f400640 1 -- 172.21.15.116:0/4175336443 >> v1:172.21.15.116:6789/0 conn(0x7f1f800d1670 legacy=0x7f1f800d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.748+0000 7f1f7f400640 1 -- 172.21.15.116:0/4175336443 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f80070470 con 0x7f1f8006c700 2024-05-14T11:23:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f7f400640 1 --2- 172.21.15.116:0/4175336443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f8006cb10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1f80071f20 tx=0x7f1f7801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20292f1e5f049cf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f7e000640 1 -- 172.21.15.116:0/4175336443 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780b2070 con 0x7f1f8006c700 2024-05-14T11:23:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f7e000640 1 -- 172.21.15.116:0/4175336443 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f780a5810 con 0x7f1f8006c700 2024-05-14T11:23:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f7e000640 1 -- 172.21.15.116:0/4175336443 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780a5be0 con 0x7f1f8006c700 2024-05-14T11:23:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 -- 172.21.15.116:0/4175336443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 msgr2=0x7f1f8006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4175336443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f8006cb10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1f80071f20 tx=0x7f1f7801c860 comp rx=0 tx=0).stop 2024-05-14T11:23:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 -- 172.21.15.116:0/4175336443 shutdown_connections 2024-05-14T11:23:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4175336443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f8006cb10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4175336443 >> v2:172.21.15.149:3300/0 conn(0x7f1f8015db30 0x7f1f800aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 -- 172.21.15.116:0/4175336443 >> 172.21.15.116:0/4175336443 conn(0x7f1f800b4620 msgr2=0x7f1f800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 -- 172.21.15.116:0/4175336443 shutdown_connections 2024-05-14T11:23:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.752+0000 7f1f85a00640 1 -- 172.21.15.116:0/4175336443 wait complete. 2024-05-14T11:23:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 Processor -- start 2024-05-14T11:23:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- start start 2024-05-14T11:23:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f800d1670 0x7f1f80160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 0x7f1f80160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8016a770 con 0x7f1f8006c700 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8015df40 con 0x7f1f800d1670 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8016a8f0 con 0x7f1f8015db30 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49094/0 (socket says 172.21.15.116:49094) 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 -- 172.21.15.116:0/4287325987 learned_addr learned my addr 172.21.15.116:0/4287325987 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f76000640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f800d1670 0x7f1f80160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7f400640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 0x7f1f80160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 -- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 msgr2=0x7f1f80160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 0x7f1f80160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 -- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f800d1670 msgr2=0x7f1f80160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f800d1670 0x7f1f80160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 -- 172.21.15.116:0/4287325987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8015feb0 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7f400640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 0x7f1f80160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f7ea00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e870 tx=0x7f1f7000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017070 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7000c940 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8015f610 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000ccf0 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.756+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8015fbd0 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8006a9c0 con 0x7f1f8006c700 2024-05-14T11:23:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f7001a070 con 0x7f1f8006c700 2024-05-14T11:23:34.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f77400640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 0x7f1f5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6303+0+0 (secure 0 0 0) 0x7f1f7008f3c0 con 0x7f1f8006c700 2024-05-14T11:23:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f76000640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 0x7f1f5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.760+0000 7f1f76000640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 0x7f1f5c071910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f80073060 tx=0x7f1f68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.764+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1f7005b980 con 0x7f1f8006c700 2024-05-14T11:23:34.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.956+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1f80159c60 con 0x7f1f8006c700 2024-05-14T11:23:34.956 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:23:34.956+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:23:34.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f77400640 1 -- 172.21.15.116:0/4287325987 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7f1f7005fc80 con 0x7f1f8006c700 2024-05-14T11:23:34.959 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:23:34.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 msgr2=0x7f1f5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 0x7f1f5c071910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f80073060 tx=0x7f1f68009040 comp rx=0 tx=0).stop 2024-05-14T11:23:34.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 msgr2=0x7f1f800cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e870 tx=0x7f1f7000ec70 comp rx=0 tx=0).stop 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 shutdown_connections 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5c06f400 0x7f1f5c071910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f800d1670 0x7f1f80160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f8015db30 0x7f1f80160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 --2- 172.21.15.116:0/4287325987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f8006c700 0x7f1f800cd7e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:34.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 >> 172.21.15.116:0/4287325987 conn(0x7f1f800b4620 msgr2=0x7f1f800b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:34.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 shutdown_connections 2024-05-14T11:23:34.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:34.960+0000 7f1f85a00640 1 -- 172.21.15.116:0/4287325987 wait complete. 2024-05-14T11:23:34.972 INFO:tasks.ceph.mds.b.smithi149.stderr: -4698> 2024-05-14T11:22:22.819+0000 7f7714a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:23:34.973 INFO:tasks.ceph.mds.b.smithi149.stderr: -4348> 2024-05-14T11:22:24.023+0000 7f770fa00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-14T11:23:34.974 INFO:tasks.ceph.mds.b.smithi149.stderr: -4182> 2024-05-14T11:22:24.271+0000 7f770fa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0) n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561742febb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-14T11:23:34.977 INFO:tasks.ceph.mds.b.smithi149.stderr: -2398> 2024-05-14T11:22:55.942+0000 7f770fa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)->f(v1 m2024-05-14T11:22:08.584531+0000) n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)->n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561742febb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-14T11:22:08.584531+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-14T11:22:08.696499+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-14T11:23:34.982 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-14T11:23:35.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 -- start start 2024-05-14T11:23:35.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 Processor -- start 2024-05-14T11:23:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76b680 0x55bbbe4d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 0x55bbbe4d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0180 con 0x55bbbd76bb00 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0300 con 0x55bbbd76bf80 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0480 con 0x55bbbd76b680 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49120/0 (socket says 172.21.15.116:49120) 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 -- 172.21.15.116:0/3970698405 learned_addr learned my addr 172.21.15.116:0/3970698405 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff625000640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76b680 0x55bbbe4d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff625a00640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 0x55bbbe4d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 -- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76b680 msgr2=0x55bbbe4d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76b680 0x55bbbe4d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 -- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 msgr2=0x55bbbe4d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 0x55bbbe4d4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff624600640 1 -- 172.21.15.116:0/3970698405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb96ee1e0 con 0x55bbbd76bb00 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.040+0000 7ff625a00640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 0x55bbbe4d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff624600640 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x55bbbe565380 tx=0x55bbbd813e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266cb5b9bc6b1ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff621e00640 1 -- 172.21.15.116:0/3970698405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e9c0 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff621e00640 1 -- 172.21.15.116:0/3970698405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bbbd7421c0 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff621e00640 1 -- 172.21.15.116:0/3970698405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e4e0 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb96ef2c0 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55bbb96ee3c0 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 4 waiting for MDS map... 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff621e00640 1 -- 172.21.15.116:0/3970698405 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x55bbbe624200 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff621e00640 1 -- 172.21.15.116:0/3970698405 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55bbbd74f180 con 0x55bbbd76bb00 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 4 Got MDS map 69 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 4 datascan.main: Using metadata pool output 2024-05-14T11:23:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 Processor -- start 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- start start 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0600 con 0x55bbbd76ad80 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0780 con 0x55bbbd76a900 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bbbe4e0900 con 0x55bbbd76c880 2024-05-14T11:23:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49122/0 (socket says 172.21.15.116:49122) 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 -- 172.21.15.116:0/3445609507 learned_addr learned my addr 172.21.15.116:0/3445609507 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff624600640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625000640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 -- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 msgr2=0x55bbbe4d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 -- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 msgr2=0x55bbbe4d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 -- 172.21.15.116:0/3445609507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb96ee1e0 con 0x55bbbd76ad80 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff624600640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625000640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.044+0000 7ff625a00640 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55bbbe4eb170 tx=0x55bbbe63b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadc72877257d23e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff61f600640 1 -- 172.21.15.116:0/3445609507 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e9c0 con 0x55bbbd76ad80 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff61f600640 1 -- 172.21.15.116:0/3445609507 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bbbd742c40 con 0x55bbbd76ad80 2024-05-14T11:23:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff61f600640 1 -- 172.21.15.116:0/3445609507 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e4e0 con 0x55bbbd76ad80 2024-05-14T11:23:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3445609507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 msgr2=0x55bbbe4d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55bbbe4eb170 tx=0x55bbbe63b800 comp rx=0 tx=0).stop 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3445609507 shutdown_connections 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3445609507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3445609507 >> 172.21.15.116:0/3445609507 conn(0x55bbbd76b200 msgr2=0x55bbbda18000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3445609507 shutdown_connections 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3445609507 wait complete. 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 Processor -- start 2024-05-14T11:23:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- start start 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e1c80 con 0x55bbbd76a900 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe4e1b00 con 0x55bbbd76ad80 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff626e3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bbbe4e1980 con 0x55bbbd76c880 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49138/0 (socket says 172.21.15.116:49138) 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 learned_addr learned my addr 172.21.15.116:0/2031053035 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 msgr2=0x55bbbe4d5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76c880 0x55bbbe4d5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76ad80 msgr2=0x55bbbe4d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.048+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76ad80 0x55bbbe4d5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb96ef4a0 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x55bbbe4ebe30 tx=0x55bbbd812540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 -- 172.21.15.116:0/2031053035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e000 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 -- 172.21.15.116:0/2031053035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bbbd742a80 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bbb96efa40 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 -- 172.21.15.116:0/2031053035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bbbd72e9c0 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb96efc20 con 0x55bbbd76a900 2024-05-14T11:23:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 4 datascan.init: resolving metadata pool 13 2024-05-14T11:23:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 -- 172.21.15.116:0/2031053035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55bbbda36c00 con 0x55bbbd76a900 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 0x55bbbe4d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff61d800640 1 -- 172.21.15.116:0/2031053035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x55bbbda25400 con 0x55bbbd76a900 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 0x55bbbe4d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55bbbd76cd00 0x55bbbe4d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4ac00 con 0x55bbbd76cd00 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55bbbd76cd00 0x55bbbe4d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.052+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 0x55bbbe4d6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55bbbd815080 tx=0x55bbbdacce40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55bbbd76cd00 0x55bbbe4d6680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff61ec00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55bbbdb46000 0x55bbbdb4a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54000 con 0x55bbbdb46000 2024-05-14T11:23:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55bbbdb46000 0x55bbbdb4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55bbbdb46000 0x55bbbdb4a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff61f600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bbbdb46480 0x55bbbdb4a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.056+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54400 con 0x55bbbdb46480 2024-05-14T11:23:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bbbdb46480 0x55bbbdb4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bbbdb46480 0x55bbbdb4a580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54800 con 0x55bbbdb46000 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55000 con 0x55bbbdb46000 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff61ec00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55bbbdb46900 0x55bbbdb4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54c00 con 0x55bbbdb46900 2024-05-14T11:23:35.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.060+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55bbbdb46900 0x55bbbdb4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55bbbdb46900 0x55bbbdb4ab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(6 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff61f600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 0x55bbbdb4b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55400 con 0x55bbbdb47200 2024-05-14T11:23:35.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 0x55bbbdb4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.064+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 0x55bbbdb4b600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54000 con 0x55bbbdb46480 2024-05-14T11:23:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff61f600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55bbbdb47680 0x55bbbdb4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54800 con 0x55bbbdb47680 2024-05-14T11:23:35.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55bbbdb47680 0x55bbbdb4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff625000640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55bbbdb47680 0x55bbbdb4bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb47680 2024-05-14T11:23:35.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.068+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4a800 con 0x55bbbdb46480 2024-05-14T11:23:35.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61f600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55bbbdb47b00 0x55bbbdb4c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55000 con 0x55bbbdb47b00 2024-05-14T11:23:35.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55bbbdb47b00 0x55bbbdb4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff624600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55bbbdb47b00 0x55bbbdb4c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4a400 con 0x55bbbd76cd00 2024-05-14T11:23:35.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55800 con 0x55bbbdb46000 2024-05-14T11:23:35.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4ac00 con 0x55bbbdb47200 2024-05-14T11:23:35.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54c00 con 0x55bbbdb46000 2024-05-14T11:23:35.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe627000 con 0x55bbbdb46480 2024-05-14T11:23:35.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55000 con 0x55bbbdb47680 2024-05-14T11:23:35.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb47680 2024-05-14T11:23:35.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.072+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626c00 con 0x55bbbdb47b00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54400 con 0x55bbbd76cd00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626800 con 0x55bbbdb47b00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff61f600640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55bbbdb47f80 0x55bbbdb4c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4bc00 con 0x55bbbdb47f80 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55bbbdb47f80 0x55bbbdb4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.076+0000 7ff625a00640 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55bbbdb47f80 0x55bbbdb4c680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54000 con 0x55bbbdb47b00 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(22 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72000 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(23 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54800 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72400 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55c00 con 0x55bbbdb47b00 2024-05-14T11:23:35.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(26 [pgnls start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72800 con 0x55bbbdb46900 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(27 [pgnls start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54400 con 0x55bbbd76cd00 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72c00 con 0x55bbbdb47b00 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54000 con 0x55bbbdb46900 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73000 con 0x55bbbdb47200 2024-05-14T11:23:35.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(31 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.080+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55c00 con 0x55bbbdb47f80 2024-05-14T11:23:35.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(32 [pgnls start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72800 con 0x55bbbdb46900 2024-05-14T11:23:35.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(33 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55400 con 0x55bbbdb47f80 2024-05-14T11:23:35.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(34 [pgnls start_epoch 88 out=49b] v84'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4a400 con 0x55bbbdb47b00 2024-05-14T11:23:35.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 7 ==== osd_op_reply(35 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626c00 con 0x55bbbdb46480 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4ac00 con 0x55bbbdb47f80 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(37 [pgnls start_epoch 88 out=49b] v84'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626800 con 0x55bbbdb47680 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb47680 2024-05-14T11:23:35.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4bc00 con 0x55bbbdb47680 2024-05-14T11:23:35.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb47680 2024-05-14T11:23:35.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72000 con 0x55bbbdb47200 2024-05-14T11:23:35.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55800 con 0x55bbbdb46000 2024-05-14T11:23:35.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 9 ==== osd_op_reply(41 [pgnls start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.084+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73400 con 0x55bbbd76cd00 2024-05-14T11:23:35.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 5 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54c00 con 0x55bbbdb47200 2024-05-14T11:23:35.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe627000 con 0x55bbbdb47b00 2024-05-14T11:23:35.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 8 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72400 con 0x55bbbdb47680 2024-05-14T11:23:35.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.6 v2:172.21.15.149:6800/3938301688 5 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb47680 2024-05-14T11:23:35.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626c00 con 0x55bbbdb47b00 2024-05-14T11:23:35.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 9 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73000 con 0x55bbbdb46000 2024-05-14T11:23:35.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 10 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626800 con 0x55bbbdb46900 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 5 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72000 con 0x55bbbdb46000 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 11 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55000 con 0x55bbbdb47200 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4ac00 con 0x55bbbd76cd00 2024-05-14T11:23:35.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 6 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4a800 con 0x55bbbd76cd00 2024-05-14T11:23:35.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.2 v2:172.21.15.116:6817/2784504541 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbd76cd00 2024-05-14T11:23:35.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54800 con 0x55bbbdb46000 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.088+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 12 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72c00 con 0x55bbbdb47b00 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 10 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55c00 con 0x55bbbdb46480 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72800 con 0x55bbbdb47b00 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 11 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55400 con 0x55bbbdb47f80 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v84'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55bbbda4b000 con 0x55bbbdb47f80 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff626e3d8c0 10 datascan.operator(): handling object 1099511627777.0 2024-05-14T11:23:35.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.092+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb55800 con 0x55bbbdb47f80 2024-05-14T11:23:35.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(59 10000000001.00000000 [call] v88'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb54c00 con 0x55bbbdb47f80 2024-05-14T11:23:35.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v84'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47f80 2024-05-14T11:23:35.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73000 con 0x55bbbdb46480 2024-05-14T11:23:35.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(61 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870780 con 0x55bbbdb46480 2024-05-14T11:23:35.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73400 con 0x55bbbdb47b00 2024-05-14T11:23:35.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.4 v2:172.21.15.149:6809/3313182122 12 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb47b00 2024-05-14T11:23:35.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb72000 con 0x55bbbdb47200 2024-05-14T11:23:35.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb47200 2024-05-14T11:23:35.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe627000 con 0x55bbbdb46000 2024-05-14T11:23:35.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 13 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61f600640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbdb73800 con 0x55bbbdb46000 2024-05-14T11:23:35.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625a00640 1 -- 172.21.15.116:0/2031053035 <== osd.0 v2:172.21.15.116:6808/772722056 14 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870500 con 0x55bbbdb46000 2024-05-14T11:23:35.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff61ec00640 1 -- 172.21.15.116:0/2031053035 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55bbbe626c00 con 0x55bbbdb46900 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff624600640 1 -- 172.21.15.116:0/2031053035 <== osd.3 v2:172.21.15.116:6816/1815220698 6 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bbbd870000 con 0x55bbbdb46900 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55bbbdb46000 msgr2=0x55bbbdb4a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55bbbdb46000 0x55bbbdb4a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bbbdb46480 msgr2=0x55bbbdb4a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bbbdb46480 0x55bbbdb4a580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55bbbd76cd00 msgr2=0x55bbbe4d6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55bbbd76cd00 0x55bbbe4d6680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55bbbdb46900 msgr2=0x55bbbdb4ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55bbbdb46900 0x55bbbdb4ab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55bbbdb47b00 msgr2=0x55bbbdb4c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55bbbdb47b00 0x55bbbdb4c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55bbbdb47f80 msgr2=0x55bbbdb4c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55bbbdb47f80 0x55bbbdb4c680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55bbbdb47680 msgr2=0x55bbbdb4bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55bbbdb47680 0x55bbbdb4bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 reap_dead start 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 msgr2=0x55bbbdb4b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 0x55bbbdb4b600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff625000640 1 -- 172.21.15.116:0/2031053035 reap_dead start 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 msgr2=0x55bbbe4d6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 0x55bbbe4d6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55bbbd815080 tx=0x55bbbdacce40 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 msgr2=0x55bbbe4d5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x55bbbe4ebe30 tx=0x55bbbd812540 comp rx=0 tx=0).stop 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 shutdown_connections 2024-05-14T11:23:35.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55bbbdb47200 0x55bbbdb4b600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bbbd76da80 0x55bbbe4d6100 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/2031053035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76a900 0x55bbbe4d5600 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.096+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 >> 172.21.15.116:0/2031053035 conn(0x55bbbd76b200 msgr2=0x55bbbda18000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 shutdown_connections 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/2031053035 wait complete. 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 msgr2=0x55bbbe4d4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x55bbbe565380 tx=0x55bbbd813e00 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 shutdown_connections 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbbd76bf80 0x55bbbe4d4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbbd76bb00 0x55bbbe4d4000 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 --2- 172.21.15.116:0/3970698405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bbbd76b680 0x55bbbe4d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 >> 172.21.15.116:0/3970698405 conn(0x55bbbd76c400 msgr2=0x55bbb96f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 shutdown_connections 2024-05-14T11:23:35.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.100+0000 7ff626e3d8c0 1 -- 172.21.15.116:0/3970698405 wait complete. 2024-05-14T11:23:35.103 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:23:35.105 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.122950, result: 2024-05-14T11:23:35.105 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-14T11:23:35.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.160+0000 7f921c84a8c0 1 -- start start 2024-05-14T11:23:35.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.160+0000 7f921c84a8c0 1 Processor -- start 2024-05-14T11:23:35.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 0x5651c32ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2565b00 0x5651c32ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5651c32ec180 con 0x5651c2565f80 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5651c32ec300 con 0x5651c2565680 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5651c32ec480 con 0x5651c2565b00 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49152/0 (socket says 172.21.15.116:49152) 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921aa00640 1 -- 172.21.15.116:0/1347011728 learned_addr learned my addr 172.21.15.116:0/1347011728 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 0x5651c32ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9219600640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2565b00 0x5651c32ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 -- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2565b00 msgr2=0x5651c32ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2565b00 0x5651c32ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 -- 172.21.15.116:0/1347011728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 msgr2=0x5651c32ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 -- 172.21.15.116:0/1347011728 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651be4e81e0 con 0x5651c2565680 2024-05-14T11:23:35.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921aa00640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921a000640 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 0x5651c32ce580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5651c335f2c0 tx=0x5651c260dec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad92320afbe0d769 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9216e00640 1 -- 172.21.15.116:0/1347011728 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c25289c0 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9216e00640 1 -- 172.21.15.116:0/1347011728 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5651c253ce00 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9216e00640 1 -- 172.21.15.116:0/1347011728 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c25284e0 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651be4e94a0 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5651be4e83c0 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 4 waiting for MDS map... 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9216e00640 1 -- 172.21.15.116:0/1347011728 <== mon.1 v2:172.21.15.149:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x5651c341e200 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f9216e00640 1 -- 172.21.15.116:0/1347011728 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x5651c2549180 con 0x5651c2565680 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 4 Got MDS map 69 2024-05-14T11:23:35.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 4 datascan.main: Using metadata pool output 2024-05-14T11:23:35.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:23:35.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 Processor -- start 2024-05-14T11:23:35.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.164+0000 7f921c84a8c0 1 -- start start 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5651c32ec600 con 0x5651c2566880 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5651c32ec780 con 0x5651c2564d80 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5651c32ec900 con 0x5651c2564900 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9219600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59228/0 (socket says 172.21.15.116:59228) 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9219600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49162/0 (socket says 172.21.15.116:49162) 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 -- 172.21.15.116:0/2630061492 learned_addr learned my addr 172.21.15.116:0/2630061492 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921a000640 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 -- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564900 msgr2=0x5651c32cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 -- 172.21.15.116:0/2630061492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 msgr2=0x5651c32cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 -- 172.21.15.116:0/2630061492 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651be4e81e0 con 0x5651c2564d80 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9219600640 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:35.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921aa00640 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5651c25d5380 tx=0x5651c25db800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea47b4aa6a689f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9214600640 1 -- 172.21.15.116:0/2630061492 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c25289c0 con 0x5651c2564d80 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9214600640 1 -- 172.21.15.116:0/2630061492 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5651c253ca80 con 0x5651c2564d80 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f9214600640 1 -- 172.21.15.116:0/2630061492 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c25284e0 con 0x5651c2564d80 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 msgr2=0x5651c32cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5651c25d5380 tx=0x5651c25db800 comp rx=0 tx=0).stop 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- 172.21.15.116:0/2630061492 shutdown_connections 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/2630061492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- 172.21.15.116:0/2630061492 >> 172.21.15.116:0/2630061492 conn(0x5651c2565200 msgr2=0x5651c280e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- 172.21.15.116:0/2630061492 shutdown_connections 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 -- 172.21.15.116:0/2630061492 wait complete. 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.168+0000 7f921c84a8c0 1 Processor -- start 2024-05-14T11:23:35.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- start start 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5651c32edc80 con 0x5651c2564900 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5651c32edb00 con 0x5651c2566880 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5651c32ed980 con 0x5651c2564d80 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49172/0 (socket says 172.21.15.116:49172) 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 learned_addr learned my addr 172.21.15.116:0/1072360104 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564d80 msgr2=0x5651c32cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2566880 msgr2=0x5651c32cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651be4e92c0 con 0x5651c2564900 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2564d80 0x5651c32cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2566880 0x5651c32cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x5651c2838000 tx=0x5651c260c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f9212800640 1 -- 172.21.15.116:0/1072360104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c2528000 con 0x5651c2564900 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f9212800640 1 -- 172.21.15.116:0/1072360104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5651c253d340 con 0x5651c2564900 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f9212800640 1 -- 172.21.15.116:0/1072360104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651c25289c0 con 0x5651c2564900 2024-05-14T11:23:35.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651be4e9a40 con 0x5651c2564900 2024-05-14T11:23:35.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651be4e9c20 con 0x5651c2564900 2024-05-14T11:23:35.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.172+0000 7f921c84a8c0 4 datascan.init: resolving metadata pool 13 2024-05-14T11:23:35.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9212800640 1 -- 172.21.15.116:0/1072360104 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x5651c2834c00 con 0x5651c2564900 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9212800640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 0x5651c32d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9212800640 1 -- 172.21.15.116:0/1072360104 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x5651c2817400 con 0x5651c2564900 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 0x5651c32d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5651c2566d00 0x5651c32d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c2566d00 2024-05-14T11:23:35.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5651c2566d00 0x5651c32d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 0x5651c32d0100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5651c25d4ea0 tx=0x5651c260df20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5651c2566d00 0x5651c32d0680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5651c2567180 0x5651c32d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c2567180 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5651c2567180 0x5651c32d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5651c2567180 0x5651c32d0c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5651c2567600 0x5651c32d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.176+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b400 con 0x5651c2567600 2024-05-14T11:23:35.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5651c2567600 0x5651c32d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5651c2567600 0x5651c32d1180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c294a000 con 0x5651c2566d00 2024-05-14T11:23:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9214600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5651c294a000 0x5651c2952000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c294a000 2024-05-14T11:23:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5651c294a000 0x5651c2952000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5651c294a000 0x5651c2952000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c2566d00 2024-05-14T11:23:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c2566d00 2024-05-14T11:23:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9213c00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5651c294a480 0x5651c2952580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294a480 2024-05-14T11:23:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5651c294a480 0x5651c2952580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.180+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5651c294a480 0x5651c2952580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5651c294a900 0x5651c2952b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c294a900 2024-05-14T11:23:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5651c294a900 0x5651c2952b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921a000640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5651c294a900 0x5651c2952b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c294a000 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b000 con 0x5651c2567180 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c294a000 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5651c294ad80 0x5651c2953080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b400 con 0x5651c294ad80 2024-05-14T11:23:35.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5651c294ad80 0x5651c2953080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9219600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5651c294ad80 0x5651c2953080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c2567600 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.184+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c2566d00 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c294a900 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c2566d00 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c294a000 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b400 con 0x5651c2567180 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a400 con 0x5651c294ad80 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c2567600 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c294ad80 2024-05-14T11:23:35.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9214600640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5651c294b200 0x5651c2953600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294b200 2024-05-14T11:23:35.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5651c294b200 0x5651c2953600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.188+0000 7f921aa00640 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5651c294b200 0x5651c2953600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c294ad80 2024-05-14T11:23:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c2566d00 2024-05-14T11:23:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c2566d00 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c2566d00 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b400 con 0x5651c294ad80 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c294a480 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c2567600 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a400 con 0x5651c294ad80 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c341d000 con 0x5651c294a480 2024-05-14T11:23:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c294a900 2024-05-14T11:23:35.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.192+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c294b200 2024-05-14T11:23:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e000 con 0x5651c294a480 2024-05-14T11:23:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b000 con 0x5651c294b200 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 88 out=49b] v84'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e400 con 0x5651c294ad80 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295bc00 con 0x5651c294a000 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e800 con 0x5651c294b200 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 88 out=49b] v84'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c2567180 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296ec00 con 0x5651c2567180 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c294a900 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(42 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f000 con 0x5651c2566d00 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.196+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c2567600 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f400 con 0x5651c294a900 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c341d000 con 0x5651c294ad80 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e400 con 0x5651c2567180 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b400 con 0x5651c294ad80 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e000 con 0x5651c2566d00 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a400 con 0x5651c294a480 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f800 con 0x5651c2566d00 2024-05-14T11:23:35.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c294a900 2024-05-14T11:23:35.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296fc00 con 0x5651c2567600 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f400 con 0x5651c2567600 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b400 con 0x5651c2566d00 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.200+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296fc00 con 0x5651c294ad80 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294a000 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(57 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c294ad80 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c294b200 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 88 out=81b] v88'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b800 con 0x5651c2567600 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e800 con 0x5651c2566d00 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c294a000 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296ec00 con 0x5651c2566d00 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c2566d00 2024-05-14T11:23:35.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e400 con 0x5651c294a480 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a000 con 0x5651c294a900 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(66 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f000 con 0x5651c294a000 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 8 ==== osd_op_reply(67 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.204+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c2567180 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e000 con 0x5651c294a000 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 9 ==== osd_op_reply(69 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294ad80 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a400 con 0x5651c2567600 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296fc00 con 0x5651c2566d00 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b400 con 0x5651c294a900 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 8 ==== osd_op_reply(73 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e800 con 0x5651c2566d00 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c341d000 con 0x5651c294a000 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296ec00 con 0x5651c2567180 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b000 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f000 con 0x5651c2567600 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 10 ==== osd_op_reply(78 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.208+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f800 con 0x5651c294b200 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c2566d00 2024-05-14T11:23:35.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295bc00 con 0x5651c2566d00 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c2566d00 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b400 con 0x5651c294ad80 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c294a480 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c2567600 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982000 con 0x5651c294ad80 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294a480 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982400 con 0x5651c294a900 2024-05-14T11:23:35.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 9 ==== osd_op_reply(90 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b000 con 0x5651c294b200 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982800 con 0x5651c294a480 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.212+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a800 con 0x5651c294b200 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 88 out=49b] v84'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982c00 con 0x5651c294ad80 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b800 con 0x5651c294a000 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 11 ==== osd_op_reply(95 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983000 con 0x5651c294b200 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 88 out=49b] v84'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c341d000 con 0x5651c2567180 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983400 con 0x5651c2567180 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e000 con 0x5651c294a900 2024-05-14T11:23:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 10 ==== osd_op_reply(99 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983800 con 0x5651c2566d00 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 88 out=49b] v84'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296fc00 con 0x5651c2567600 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 12 ==== osd_op_reply(101 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983c00 con 0x5651c294a900 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 11 ==== osd_op_reply(102 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.216+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e800 con 0x5651c294ad80 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a800 con 0x5651c2567180 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982800 con 0x5651c294ad80 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b800 con 0x5651c2566d00 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982400 con 0x5651c294a480 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296e400 con 0x5651c2566d00 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283a400 con 0x5651c294a900 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 12 ==== osd_op_reply(109 [pgnls-filter start_epoch 88 out=49b] v84'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296ec00 con 0x5651c2567600 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 88 out=49b] v84'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b400 con 0x5651c2567600 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.2 v2:172.21.15.116:6817/2784504541 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 88 out=49b] v84'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c2567600 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f800 con 0x5651c2566d00 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283ac00 con 0x5651c294ad80 2024-05-14T11:23:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f400 con 0x5651c294a000 2024-05-14T11:23:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.220+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 12 ==== osd_op_reply(114 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b000 con 0x5651c294ad80 2024-05-14T11:23:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 88 out=49b] v84'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c296f000 con 0x5651c294b200 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 88 out=81b] v88'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 10 datascan.operator(): handling object 10000000001.0 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c298c000 con 0x5651c294b200 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-14T11:23:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c298c400 con 0x5651c294a000 2024-05-14T11:23:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 13 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-14T11:23:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c298c800 con 0x5651c2567180 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.6 v2:172.21.15.149:6800/3938301688 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x5651c266a500 con 0x5651c2567180 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.get_frag_of: frag is * 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c298cc00 con 0x5651c294a000 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 14 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983000 con 0x5651c294a000 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 15 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 20 datascan.read_dentry: file_head not found in result 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-14T11:23:35.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.224+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x5651c2983400 con 0x5651c294a000 2024-05-14T11:23:35.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 16 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v88'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982400 con 0x5651c294b200 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.5 v2:172.21.15.149:6824/4173680411 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 88 out=49b] v84'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294b200 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295ac00 con 0x5651c294a000 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.1 v2:172.21.15.116:6800/4228261718 17 ==== osd_op_reply(124 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294a000 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c283b800 con 0x5651c294ad80 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.4 v2:172.21.15.149:6809/3313182122 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 88 out=49b] v84'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c294ad80 2024-05-14T11:23:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295a400 con 0x5651c294a900 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921a000640 1 -- 172.21.15.116:0/1072360104 <== osd.7 v2:172.21.15.149:6808/2875938804 13 ==== osd_op_reply(126 [pgnls-filter start_epoch 88 out=49b] v84'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a780 con 0x5651c294a900 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982000 con 0x5651c2566d00 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9213c00640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c295b000 con 0x5651c2566d00 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 <== osd.0 v2:172.21.15.116:6808/772722056 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 88 out=49b] v84'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a000 con 0x5651c2566d00 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9214600640 1 -- 172.21.15.116:0/1072360104 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x5651c2982c00 con 0x5651c294a480 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921aa00640 1 -- 172.21.15.116:0/1072360104 <== osd.3 v2:172.21.15.116:6816/1815220698 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 88 out=49b] v84'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5651c266a500 con 0x5651c294a480 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5651c2566d00 msgr2=0x5651c32d0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5651c2566d00 0x5651c32d0680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5651c294a000 msgr2=0x5651c2952000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5651c294a000 0x5651c2952000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5651c2567600 msgr2=0x5651c32d1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5651c2567600 0x5651c32d1180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5651c294a480 msgr2=0x5651c2952580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5651c294a480 0x5651c2952580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5651c294ad80 msgr2=0x5651c2953080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5651c294ad80 0x5651c2953080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5651c294b200 msgr2=0x5651c2953600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5651c294b200 0x5651c2953600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5651c2567180 msgr2=0x5651c32d0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5651c2567180 0x5651c32d0c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5651c294a900 msgr2=0x5651c2952b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5651c294a900 0x5651c2952b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 reap_dead start 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 reap_dead start 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f9219600640 1 -- 172.21.15.116:0/1072360104 reap_dead start 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 msgr2=0x5651c32d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 0x5651c32d0100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5651c25d4ea0 tx=0x5651c260df20 comp rx=0 tx=0).stop 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 msgr2=0x5651c32cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x5651c2838000 tx=0x5651c260c900 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 shutdown_connections 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5651c2567a80 0x5651c32d0100 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1072360104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2564900 0x5651c32cf600 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.228+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 >> 172.21.15.116:0/1072360104 conn(0x5651c2565200 msgr2=0x5651c280e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 shutdown_connections 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1072360104 wait complete. 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 msgr2=0x5651c32ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 0x5651c32ce580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5651c335f2c0 tx=0x5651c260dec0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 shutdown_connections 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5651c2565f80 0x5651c32ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5651c2565b00 0x5651c32ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 --2- 172.21.15.116:0/1347011728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5651c2565680 0x5651c32ce580 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 >> 172.21.15.116:0/1347011728 conn(0x5651c2566400 msgr2=0x5651be4f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 shutdown_connections 2024-05-14T11:23:35.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.232+0000 7f921c84a8c0 1 -- 172.21.15.116:0/1347011728 wait complete. 2024-05-14T11:23:35.237 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.131865, result: 2024-05-14T11:23:35.237 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-14T11:23:35.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.288+0000 7fcbb42548c0 1 -- start start 2024-05-14T11:23:35.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.288+0000 7fcbb42548c0 1 Processor -- start 2024-05-14T11:23:35.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fdb00 0x55b698166580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fdf80 0x55b698166000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6973ea900 con 0x55b6973fd680 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b69816e000 con 0x55b6973fdf80 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b69816e180 con 0x55b6973fdb00 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49200/0 (socket says 172.21.15.116:49200) 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 -- 172.21.15.116:0/3755025452 learned_addr learned my addr 172.21.15.116:0/3755025452 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fdb00 0x55b698166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb2200640 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fdf80 0x55b698166000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 -- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fdb00 msgr2=0x55b698166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fdb00 0x55b698166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 -- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fdf80 msgr2=0x55b698166000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fdf80 0x55b698166000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 -- 172.21.15.116:0/3755025452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6933801e0 con 0x55b6973fd680 2024-05-14T11:23:35.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb1800640 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55b6981f7b60 tx=0x55b6974a4a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788b45b11ced7c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbae600640 1 -- 172.21.15.116:0/3755025452 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c09c0 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbae600640 1 -- 172.21.15.116:0/3755025452 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b6973d41c0 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbae600640 1 -- 172.21.15.116:0/3755025452 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c04e0 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6933814a0 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b6933803c0 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 4 waiting for MDS map... 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbae600640 1 -- 172.21.15.116:0/3755025452 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x55b6982b4200 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbae600640 1 -- 172.21.15.116:0/3755025452 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55b6973e1180 con 0x55b6973fd680 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 4 Got MDS map 69 2024-05-14T11:23:35.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 4 datascan.main: Using metadata pool output 2024-05-14T11:23:35.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:23:35.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 Processor -- start 2024-05-14T11:23:35.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.292+0000 7fcbb42548c0 1 -- start start 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b69816e300 con 0x55b6973fe880 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b69816e480 con 0x55b6973fcd80 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b69816e600 con 0x55b6973fc900 2024-05-14T11:23:35.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49202/0 (socket says 172.21.15.116:49202) 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59262/0 (socket says 172.21.15.116:59262) 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 -- 172.21.15.116:0/3025563004 learned_addr learned my addr 172.21.15.116:0/3025563004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb1800640 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 -- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fc900 msgr2=0x55b698167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 -- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 msgr2=0x55b698167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 -- 172.21.15.116:0/3025563004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6933801e0 con 0x55b6973fe880 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb2200640 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x55b6982c5140 tx=0x55b6982cd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb62a392447246b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbabe00640 1 -- 172.21.15.116:0/3025563004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c09c0 con 0x55b6973fe880 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbabe00640 1 -- 172.21.15.116:0/3025563004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b6973d4c40 con 0x55b6973fe880 2024-05-14T11:23:35.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbabe00640 1 -- 172.21.15.116:0/3025563004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c04e0 con 0x55b6973fe880 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3025563004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 msgr2=0x55b698167b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x55b6982c5140 tx=0x55b6982cd800 comp rx=0 tx=0).stop 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3025563004 shutdown_connections 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3025563004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3025563004 >> 172.21.15.116:0/3025563004 conn(0x55b6973fd200 msgr2=0x55b6976aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3025563004 shutdown_connections 2024-05-14T11:23:35.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.296+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3025563004 wait complete. 2024-05-14T11:23:35.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 Processor -- start 2024-05-14T11:23:35.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- start start 2024-05-14T11:23:35.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b69816f500 con 0x55b6973fe880 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b69816e900 con 0x55b6973fc900 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55b69816e780 con 0x55b6973fcd80 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49208/0 (socket says 172.21.15.116:49208) 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 learned_addr learned my addr 172.21.15.116:0/4182400414 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fcd80 msgr2=0x55b698167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fc900 msgr2=0x55b698167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fc900 0x55b698167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6933812c0 con 0x55b6973fe880 2024-05-14T11:23:35.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fcd80 0x55b698167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x55b6982c5e00 tx=0x55b6974a4960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbaa000640 1 -- 172.21.15.116:0/4182400414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c0000 con 0x55b6973fe880 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbaa000640 1 -- 172.21.15.116:0/4182400414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b6973d4540 con 0x55b6973fe880 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbaa000640 1 -- 172.21.15.116:0/4182400414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6973c09c0 con 0x55b6973fe880 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b693381a40 con 0x55b6973fe880 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b693381c20 con 0x55b6973fe880 2024-05-14T11:23:35.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.300+0000 7fcbb42548c0 4 datascan.init: resolving metadata pool 13 2024-05-14T11:23:35.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbaa000640 1 -- 172.21.15.116:0/4182400414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55b6976d0c00 con 0x55b6973fe880 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbaa000640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55b6973ffa80 0x55b698168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbaa000640 1 -- 172.21.15.116:0/4182400414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x55b6976b3400 con 0x55b6973fe880 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb42548c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb42548c0 4 datascan.main: resolving metadata pool 13 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55b6973ffa80 0x55b698168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55b6973fed00 0x55b698168680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55b6973fed00 0x55b698168680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55b6973ffa80 0x55b698168100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55b6974a70e0 tx=0x55b697758e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55b6973fed00 0x55b698168680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55b6977d0000 0x55b6977d4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0000 2024-05-14T11:23:35.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55b6977d0000 0x55b6977d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.304+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55b6977d0000 0x55b6977d4000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=177b] v88'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55b6977d0480 0x55b6977d4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0480 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55b6977d0480 0x55b6977d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55b6977d0480 0x55b6977d4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55b6977d0900 0x55b6977d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0900 2024-05-14T11:23:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55b6977d0900 0x55b6977d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55b6977d0900 0x55b6977d4b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55b6977d1200 0x55b6977d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1200 2024-05-14T11:23:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55b6977d1200 0x55b6977d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55b6977d1200 0x55b6977d5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(6 [pgnls start_epoch 88 out=149b] v87'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.308+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0000 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0900 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(8 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55b6977d1680 0x55b6977d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1680 2024-05-14T11:23:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55b6977d1680 0x55b6977d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55b6977d1680 0x55b6977d5b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1200 2024-05-14T11:23:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0480 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(11 [pgnls start_epoch 88 out=81b] v87'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0000 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v85'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55b6977d1f80 0x55b6977d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55b6977d1f80 0x55b6977d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.312+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55b6977d1f80 0x55b6977d6680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(13 [pgnls start_epoch 88 out=77b] v87'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55b6977d1b00 0x55b6977d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1b00 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55b6977d1b00 0x55b6977d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55b6977d1b00 0x55b6977d6100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1b00 2024-05-14T11:23:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(15 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v85'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0900 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0000 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1680 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1b00 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1b00 2024-05-14T11:23:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.316+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(22 [pgnls start_epoch 88 out=97b] v87'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(23 [pgnls start_epoch 88 out=74b] v83'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(24 [pgnls start_epoch 88 out=73b] v87'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1200 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(26 [pgnls start_epoch 88 out=98b] v87'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1680 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(27 [pgnls start_epoch 88 out=76b] v87'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1f80 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(28 [pgnls start_epoch 88 out=73b] v87'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(29 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d1680 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0900 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(31 [pgnls start_epoch 88 out=73b] v83'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6977d0900 2024-05-14T11:23:35.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.320+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 5 ==== osd_op_reply(32 [pgnls start_epoch 88 out=101b] v87'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-14T11:23:35.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca400 con 0x55b6977d0000 2024-05-14T11:23:35.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-14T11:23:35.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb400 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb800 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 20 datascan.scan_links: injected ino detected 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cbc00 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb000 con 0x55b6977d0000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca400 con 0x55b6977d1200 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-14T11:23:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb400 con 0x55b6977d0900 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb800 con 0x55b6977d0480 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8000 con 0x55b6973fed00 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.324+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8400 con 0x55b6977d1f80 2024-05-14T11:23:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8c00 con 0x55b6973fed00 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-14T11:23:35.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9000 con 0x55b6973fed00 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8000 con 0x55b6977d0900 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 11 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0000 2024-05-14T11:23:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 10 ==== osd_op_reply(48 [pgnls start_epoch 88 out=177b] v88'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0480 2024-05-14T11:23:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 12 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0900 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 8 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.328+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1200 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(52 [pgnls start_epoch 88 out=149b] v87'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0000 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 11 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0900 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 9 ==== osd_op_reply(54 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1680 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1200 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0480 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 5 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v87'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0000 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 12 ==== osd_op_reply(58 [pgnls start_epoch 88 out=49b] v85'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 8 ==== osd_op_reply(59 [pgnls start_epoch 88 out=77b] v87'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1b00 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1b00 2024-05-14T11:23:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 13 ==== osd_op_reply(61 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 9 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v85'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0900 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 10 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.332+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0000 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 13 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1680 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1b00 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1b00 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 10 ==== osd_op_reply(67 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 11 ==== osd_op_reply(68 [pgnls start_epoch 88 out=97b] v87'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 12 ==== osd_op_reply(69 [pgnls start_epoch 88 out=74b] v83'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 14 ==== osd_op_reply(70 [pgnls start_epoch 88 out=73b] v87'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1200 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(71 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 15 ==== osd_op_reply(72 [pgnls start_epoch 88 out=98b] v87'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1680 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(73 [pgnls start_epoch 88 out=76b] v87'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.336+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1f80 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 13 ==== osd_op_reply(74 [pgnls start_epoch 88 out=73b] v87'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6973fed00 2024-05-14T11:23:35.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 16 ==== osd_op_reply(75 [pgnls start_epoch 88 out=73b] v87'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d1680 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(76 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b697502000 con 0x55b6977d1680 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0900 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 11 ==== osd_op_reply(77 [pgnls start_epoch 88 out=73b] v83'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8800 con 0x55b6977d0900 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 12 ==== osd_op_reply(78 [pgnls start_epoch 88 out=101b] v87'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb800 con 0x55b6977d0000 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-14T11:23:35.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6c00 con 0x55b6977d0000 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6800 con 0x55b6977d0000 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 20 datascan.scan_links: injected ino detected 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-14T11:23:35.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6400 con 0x55b6977d0000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cbc00 con 0x55b6977d0000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb000 con 0x55b6977d1200 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502500 con 0x55b6977d1200 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.340+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca400 con 0x55b6977d0900 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb800 con 0x55b6977d0480 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.2 v2:172.21.15.116:6817/2784504541 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55b697502000 con 0x55b6977d0480 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-14T11:23:35.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977ca000 con 0x55b6973fed00 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977cb400 con 0x55b6977d1f80 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6c00 con 0x55b6973fed00 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8400 con 0x55b6973fed00 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f8000 con 0x55b6973fed00 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9400 con 0x55b6977d0900 2024-05-14T11:23:35.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.6 v2:172.21.15.149:6800/3938301688 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55b697502780 con 0x55b6977d0900 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.344+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9800 con 0x55b6977d0000 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-14T11:23:35.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9c00 con 0x55b6977d0000 2024-05-14T11:23:35.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v88'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-14T11:23:35.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9800 con 0x55b6977d0000 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.scan_links: first is now 2 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55b6977f9c00 con 0x55b6977d0000 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb2200640 1 -- 172.21.15.116:0/4182400414 <== osd.1 v2:172.21.15.116:6800/4228261718 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v88'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b697502500 con 0x55b6977d0000 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 10 datascan.scan_links: updating inotable 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6800 con 0x55b6973fed00 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-14T11:23:35.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.348+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6400 con 0x55b6973fed00 2024-05-14T11:23:35.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb1800640 1 -- 172.21.15.116:0/4182400414 <== osd.7 v2:172.21.15.149:6808/2875938804 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v88'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55b697502000 con 0x55b6973fed00 2024-05-14T11:23:35.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 10 datascan.scan_links: updating snaptable 2024-05-14T11:23:35.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55b6982a6c00 con 0x55b6977d1f80 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 <== osd.4 v2:172.21.15.149:6809/3313182122 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55b697502780 con 0x55b6977d1f80 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55b6977d1200 msgr2=0x55b6977d5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55b6977d1200 0x55b6977d5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55b6977d0000 msgr2=0x55b6977d4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55b6977d0000 0x55b6977d4000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55b6977d0480 msgr2=0x55b6977d4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55b6977d0480 0x55b6977d4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55b6977d1b00 msgr2=0x55b6977d6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55b6977d1b00 0x55b6977d6100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 reap_dead start 2024-05-14T11:23:35.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55b6977d1f80 msgr2=0x55b6977d6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55b6977d1f80 0x55b6977d6680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 reap_dead start 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55b6977d1680 msgr2=0x55b6977d5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55b6977d1680 0x55b6977d5b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55b6977d0900 msgr2=0x55b6977d4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55b6977d0900 0x55b6977d4b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55b6973fed00 msgr2=0x55b698168680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55b6973fed00 0x55b698168680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55b6973ffa80 msgr2=0x55b698168100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55b6973ffa80 0x55b698168100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55b6974a70e0 tx=0x55b697758e40 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 msgr2=0x55b698167b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/4182400414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fe880 0x55b698167b80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x55b6982c5e00 tx=0x55b6974a4960 comp rx=0 tx=0).stop 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb0e00640 1 -- 172.21.15.116:0/4182400414 reap_dead start 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 shutdown_connections 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 >> 172.21.15.116:0/4182400414 conn(0x55b6973fd200 msgr2=0x55b6976aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 shutdown_connections 2024-05-14T11:23:35.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/4182400414 wait complete. 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 msgr2=0x55b698166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55b6981f7b60 tx=0x55b6974a4a20 comp rx=0 tx=0).stop 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 shutdown_connections 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6973fdf80 0x55b698166000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55b6973fdb00 0x55b698166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 --2- 172.21.15.116:0/3755025452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6973fd680 0x55b698166b00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 >> 172.21.15.116:0/3755025452 conn(0x55b6973fe400 msgr2=0x55b693389800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 shutdown_connections 2024-05-14T11:23:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.352+0000 7fcbb42548c0 1 -- 172.21.15.116:0/3755025452 wait complete. 2024-05-14T11:23:35.359 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.121424, result: 2024-05-14T11:23:35.359 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:23:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.508+0000 7f205dc00640 1 Processor -- start 2024-05-14T11:23:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.508+0000 7f205dc00640 1 -- start start 2024-05-14T11:23:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f205dc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20580d1e40 0x7f20580cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f205dc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f205806c700 0x7f205806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f205dc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205806f0a0 con 0x7f20580cd1c0 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f205dc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f20580cbb60 con 0x7f20580d1e40 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f205dc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f205806f420 con 0x7f205806c700 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f2056a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f20580cd1c0 0x7f20580d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51498/0 (socket says 172.21.15.116:51498) 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f2056a00640 1 -- 172.21.15.116:0/2369703922 learned_addr learned my addr 172.21.15.116:0/2369703922 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f204e000640 1 --2- 172.21.15.116:0/2369703922 >> v2:172.21.15.116:3300/0 conn(0x7f20580d1e40 0x7f20580cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f2057400640 1 --2- 172.21.15.116:0/2369703922 >> v2:172.21.15.149:3300/0 conn(0x7f205806c700 0x7f205806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f2056000640 1 -- 172.21.15.116:0/2369703922 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272006549 0 0) 0x7f205806f0a0 con 0x7f20580cd1c0 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f2056000640 1 -- 172.21.15.116:0/2369703922 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f203c003530 con 0x7f20580cd1c0 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f204e000640 1 -- 172.21.15.116:0/2369703922 >> v2:172.21.15.149:3300/0 conn(0x7f205806c700 msgr2=0x7f205806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f204e000640 1 --2- 172.21.15.116:0/2369703922 >> v2:172.21.15.149:3300/0 conn(0x7f205806c700 0x7f205806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f204e000640 1 -- 172.21.15.116:0/2369703922 >> v1:172.21.15.116:6789/0 conn(0x7f20580cd1c0 legacy=0x7f20580d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.512+0000 7f204e000640 1 -- 172.21.15.116:0/2369703922 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205806f700 con 0x7f20580d1e40 2024-05-14T11:23:35.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f204e000640 1 --2- 172.21.15.116:0/2369703922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580d1e40 0x7f20580cd5d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f205806e2c0 tx=0x7f204801c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2710d72668f79000 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f2056000640 1 -- 172.21.15.116:0/2369703922 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480b0070 con 0x7f20580d1e40 2024-05-14T11:23:35.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f2056000640 1 -- 172.21.15.116:0/2369703922 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20480ab600 con 0x7f20580d1e40 2024-05-14T11:23:35.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f2056000640 1 -- 172.21.15.116:0/2369703922 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480ab9b0 con 0x7f20580d1e40 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- 172.21.15.116:0/2369703922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580d1e40 msgr2=0x7f20580cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 --2- 172.21.15.116:0/2369703922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580d1e40 0x7f20580cd5d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f205806e2c0 tx=0x7f204801c2a0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- 172.21.15.116:0/2369703922 shutdown_connections 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 --2- 172.21.15.116:0/2369703922 >> v2:172.21.15.149:3300/0 conn(0x7f205806c700 0x7f205806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 --2- 172.21.15.116:0/2369703922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580d1e40 0x7f20580cd5d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- 172.21.15.116:0/2369703922 >> 172.21.15.116:0/2369703922 conn(0x7f20580b4720 msgr2=0x7f20580b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- 172.21.15.116:0/2369703922 shutdown_connections 2024-05-14T11:23:35.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- 172.21.15.116:0/2369703922 wait complete. 2024-05-14T11:23:35.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 Processor -- start 2024-05-14T11:23:35.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.516+0000 7f205dc00640 1 -- start start 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f205806c700 0x7f20581576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20580cd1c0 0x7f20580c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f20581699b0 con 0x7f20580cae60 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f205815cf10 con 0x7f20580cd1c0 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2058169b30 con 0x7f205806c700 2024-05-14T11:23:35.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49222/0 (socket says 172.21.15.116:49222) 2024-05-14T11:23:35.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 -- 172.21.15.116:0/2529704263 learned_addr learned my addr 172.21.15.116:0/2529704263 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:35.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2056a00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f205806c700 0x7f20581576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f204e000640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20580cd1c0 0x7f20580c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 -- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f205806c700 msgr2=0x7f20581576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f205806c700 0x7f20581576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 -- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20580cd1c0 msgr2=0x7f20580c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20580cd1c0 0x7f20580c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 -- 172.21.15.116:0/2529704263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2058157c30 con 0x7f20580cae60 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f2057400640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f205000efc0 tx=0x7f205000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050018020 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2050014480 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20580744c0 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050009e60 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20580ca300 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.520+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205806a9c0 con 0x7f20580cae60 2024-05-14T11:23:35.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.524+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f205001b020 con 0x7f20580cae60 2024-05-14T11:23:35.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.524+0000 7f204f400640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 0x7f2034071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:35.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.524+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f20500902c0 con 0x7f20580cae60 2024-05-14T11:23:35.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.524+0000 7f2056a00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 0x7f2034071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:35.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.524+0000 7f2056a00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 0x7f2034071910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2040007a70 tx=0x7f20400079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:35.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.528+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2050022020 con 0x7f20580cae60 2024-05-14T11:23:35.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:35.720+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2058069de0 con 0x7f20580cae60 2024-05-14T11:23:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f204f400640 1 -- 172.21.15.116:0/2529704263 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v70) ==== 162+0+0 (secure 0 0 0) 0x7f2050010040 con 0x7f20580cae60 2024-05-14T11:23:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 msgr2=0x7f2034071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 0x7f2034071910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2040007a70 tx=0x7f20400079c0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 msgr2=0x7f20580c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f205000efc0 tx=0x7f205000c490 comp rx=0 tx=0).stop 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 shutdown_connections 2024-05-14T11:23:36.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f203406f400 0x7f2034071910 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20580cd1c0 0x7f20580c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20580cae60 0x7f20580c9770 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 --2- 172.21.15.116:0/2529704263 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f205806c700 0x7f20581576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 >> 172.21.15.116:0/2529704263 conn(0x7f20580b4720 msgr2=0x7f20580d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 shutdown_connections 2024-05-14T11:23:36.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.008+0000 7f205dc00640 1 -- 172.21.15.116:0/2529704263 wait complete. 2024-05-14T11:23:36.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:23:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 Processor -- start 2024-05-14T11:23:36.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 -- start start 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f92ac0d23b0 0x7f92ac0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0b2400 0x7f92ac0d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ac0d3d30 con 0x7f92ac0d3920 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f92ac0b4890 con 0x7f92ac0d23b0 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92b1200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f92ac0d52c0 con 0x7f92ac0b2400 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f92ac0d23b0 0x7f92ac0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92aa800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f92ac0d3920 0x7f92ac0d5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51514/0 (socket says 172.21.15.116:51514) 2024-05-14T11:23:36.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92aa800640 1 -- 172.21.15.116:0/246397107 learned_addr learned my addr 172.21.15.116:0/246397107 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9400640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0b2400 0x7f92ac0d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9e00640 1 -- 172.21.15.116:0/246397107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0b2400 msgr2=0x7f92ac0d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9e00640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0b2400 0x7f92ac0d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9e00640 1 -- 172.21.15.116:0/246397107 >> v1:172.21.15.116:6789/0 conn(0x7f92ac0d3920 legacy=0x7f92ac0d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.180+0000 7f92a9e00640 1 -- 172.21.15.116:0/246397107 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac0b80e0 con 0x7f92ac0d23b0 2024-05-14T11:23:36.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92a9e00640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0d23b0 0x7f92ac0b4350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f929c0097c0 tx=0x7f929c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d2c1a64807a9dc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92a8a00640 1 -- 172.21.15.116:0/246397107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0b0070 con 0x7f92ac0d23b0 2024-05-14T11:23:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92a8a00640 1 -- 172.21.15.116:0/246397107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929c0ab600 con 0x7f92ac0d23b0 2024-05-14T11:23:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92a8a00640 1 -- 172.21.15.116:0/246397107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0ab9d0 con 0x7f92ac0d23b0 2024-05-14T11:23:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 -- 172.21.15.116:0/246397107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0d23b0 msgr2=0x7f92ac0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0d23b0 0x7f92ac0b4350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f929c0097c0 tx=0x7f929c01c4a0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 -- 172.21.15.116:0/246397107 shutdown_connections 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0b2400 0x7f92ac0d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 --2- 172.21.15.116:0/246397107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0d23b0 0x7f92ac0b4350 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 -- 172.21.15.116:0/246397107 >> 172.21.15.116:0/246397107 conn(0x7f92ac09ea30 msgr2=0x7f92ac08cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 -- 172.21.15.116:0/246397107 shutdown_connections 2024-05-14T11:23:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.184+0000 7f92b1200640 1 -- 172.21.15.116:0/246397107 wait complete. 2024-05-14T11:23:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 Processor -- start 2024-05-14T11:23:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 -- start start 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92ac0d23b0 0x7f92ac06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0d3920 0x7f92ac158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac0d6270 con 0x7f92ac0b2400 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac0d27c0 con 0x7f92ac0d23b0 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f92ac0b2810 con 0x7f92ac0d3920 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49244/0 (socket says 172.21.15.116:49244) 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 -- 172.21.15.116:0/1452880174 learned_addr learned my addr 172.21.15.116:0/1452880174 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9400640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92ac0d23b0 0x7f92ac06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92aa800640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0d3920 0x7f92ac158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 -- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0d3920 msgr2=0x7f92ac158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0d3920 0x7f92ac158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 -- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92ac0d23b0 msgr2=0x7f92ac06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92ac0d23b0 0x7f92ac06d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 -- 172.21.15.116:0/1452880174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac0a0e80 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a9e00640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f929c01a0d0 tx=0x7f929c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0b0050 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929c0bc070 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0b5830 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.188+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac15e340 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac15fb80 con 0x7f92ac0b2400 2024-05-14T11:23:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ac06a9c0 con 0x7f92ac0b2400 2024-05-14T11:23:36.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f929c0b59d0 con 0x7f92ac0b2400 2024-05-14T11:23:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92a2000640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 0x7f928c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f929c129dd0 con 0x7f92ac0b2400 2024-05-14T11:23:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92a9400640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 0x7f928c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.192+0000 7f92a9400640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 0x7f928c071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f92ac09fad0 tx=0x7f9298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.196+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f929c016090 con 0x7f92ac0b2400 2024-05-14T11:23:36.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92ac0b56a0 con 0x7f92ac0b2400 2024-05-14T11:23:36.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92a2000640 1 -- 172.21.15.116:0/1452880174 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5186 (secure 0 0 0) 0x7f929c0f6a00 con 0x7f92ac0b2400 2024-05-14T11:23:36.389 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:36.389 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:36.014807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-14T11:23:36.389 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 71 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 msgr2=0x7f928c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 0x7f928c071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f92ac09fad0 tx=0x7f9298008040 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 msgr2=0x7f92ac0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.388+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f929c01a0d0 tx=0x7f929c01ace0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 shutdown_connections 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928c06f400 0x7f928c071910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92ac0d23b0 0x7f92ac06d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92ac0d3920 0x7f92ac158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 --2- 172.21.15.116:0/1452880174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92ac0b2400 0x7f92ac0cf810 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:36.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 >> 172.21.15.116:0/1452880174 conn(0x7f92ac09ea30 msgr2=0x7f92ac0b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:36.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 shutdown_connections 2024-05-14T11:23:36.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:36.392+0000 7f92b1200640 1 -- 172.21.15.116:0/1452880174 wait complete. 2024-05-14T11:23:36.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:23:36.014807+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7027}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7027': {'gid': 7027, 'name': 'b', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/2777717906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 2777717906}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 2777717906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:23:36.813 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7027:up:replay 2024-05-14T11:23:37.813 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:23:37.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.964+0000 7f107be00640 1 Processor -- start 2024-05-14T11:23:37.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.964+0000 7f107be00640 1 -- start start 2024-05-14T11:23:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f107be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f107406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f107be00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1074160510 con 0x7f10740af380 2024-05-14T11:23:37.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f107be00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10741607f0 con 0x7f10740d1670 2024-05-14T11:23:37.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f107be00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f10741573b0 con 0x7f1074156fa0 2024-05-14T11:23:37.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f10740af380 0x7f10740af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35506/0 (socket says 172.21.15.116:35506) 2024-05-14T11:23:37.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079e00640 1 -- 172.21.15.116:0/4159422605 learned_addr learned my addr 172.21.15.116:0/4159422605 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f107406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516351158 0 0) 0x7f1074160510 con 0x7f10740af380 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f105c003530 con 0x7f10740af380 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2117050661 0 0) 0x7f10741607f0 con 0x7f10740d1670 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1074160510 con 0x7f10740d1670 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601897653 0 0) 0x7f105c003530 con 0x7f10740af380 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 msgr2=0x7f107406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f107406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 >> v1:172.21.15.149:6789/0 conn(0x7f10740d1670 legacy=0x7f10740d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1074160ad0 con 0x7f10740af380 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f106c001a80 con 0x7f10740af380 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 >> v1:172.21.15.116:6789/0 conn(0x7f10740af380 legacy=0x7f10740af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f105c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c0041d0 0x7f105c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105c004da0 0x7f105c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f105c00de10 con 0x7f105c004da0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f105c00dfc0 con 0x7f105c0041d0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f105c00e1c0 con 0x7f1074156fa0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f105c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1078a00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c0041d0 0x7f105c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079e00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105c004da0 0x7f105c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 -- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c0041d0 msgr2=0x7f105c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c0041d0 0x7f105c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 -- 172.21.15.116:0/4159422605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105c004da0 msgr2=0x7f105c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105c004da0 0x7f105c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.968+0000 7f1079400640 1 -- 172.21.15.116:0/4159422605 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c003530 con 0x7f1074156fa0 2024-05-14T11:23:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f1079400640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f105c003c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ab0 tx=0x7f106801ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be5d44a6c14ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680b2070 con 0x7f1074156fa0 2024-05-14T11:23:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10680a5680 con 0x7f1074156fa0 2024-05-14T11:23:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f1073e00640 1 -- 172.21.15.116:0/4159422605 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680096b0 con 0x7f1074156fa0 2024-05-14T11:23:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 -- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 msgr2=0x7f105c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 --2- 172.21.15.116:0/4159422605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f105c003c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ab0 tx=0x7f106801ca90 comp rx=0 tx=0).stop 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f1079e00640 1 -- 172.21.15.116:0/4159422605 reap_dead start 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 -- 172.21.15.116:0/4159422605 shutdown_connections 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 -- 172.21.15.116:0/4159422605 >> 172.21.15.116:0/4159422605 conn(0x7f10740b4620 msgr2=0x7f10740b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 -- 172.21.15.116:0/4159422605 shutdown_connections 2024-05-14T11:23:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.972+0000 7f107be00640 1 -- 172.21.15.116:0/4159422605 wait complete. 2024-05-14T11:23:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 Processor -- start 2024-05-14T11:23:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 -- start start 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f1074155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10740cfe00 0x7f10741f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f107406ffe0 con 0x7f10740ce5c0 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1074161120 con 0x7f10740cfe00 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f107be00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1074070160 con 0x7f1074156fa0 2024-05-14T11:23:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60978/0 (socket says 172.21.15.116:60978) 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 -- 172.21.15.116:0/1232458338 learned_addr learned my addr 172.21.15.116:0/1232458338 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1079400640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f1074155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1079e00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10740cfe00 0x7f10741f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 -- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 msgr2=0x7f1074155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f1074155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 -- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10740cfe00 msgr2=0x7f10741f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10740cfe00 0x7f10741f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 -- 172.21.15.116:0/1232458338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10740ceca0 con 0x7f10740ce5c0 2024-05-14T11:23:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1078a00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f106000f850 tx=0x7f106000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.976+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600052e0 con 0x7f10740ce5c0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1060018070 con 0x7f10740ce5c0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106000d970 con 0x7f10740ce5c0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10741535c0 con 0x7f10740ce5c0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1074153b80 con 0x7f10740ce5c0 2024-05-14T11:23:37.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107406a9c0 con 0x7f10740ce5c0 2024-05-14T11:23:37.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f106000db10 con 0x7f10740ce5c0 2024-05-14T11:23:37.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1071600640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 0x7f1054071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:37.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f106008f490 con 0x7f10740ce5c0 2024-05-14T11:23:37.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.980+0000 7f1079400640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 0x7f1054071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:37.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.984+0000 7f1079400640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 0x7f1054071910 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f10680af000 tx=0x7f10680046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:37.984+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f106005c0c0 con 0x7f10740ce5c0 2024-05-14T11:23:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.180+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10740cea30 con 0x7f10740ce5c0 2024-05-14T11:23:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.180+0000 7f1071600640 1 -- 172.21.15.116:0/1232458338 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5186 (secure 0 0 0) 0x7f106005b9c0 con 0x7f10740ce5c0 2024-05-14T11:23:38.181 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:38.182 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:37.028072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-14T11:23:38.182 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 73 2024-05-14T11:23:38.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 msgr2=0x7f1054071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 0x7f1054071910 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f10680af000 tx=0x7f10680046d0 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 msgr2=0x7f10740cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f106000f850 tx=0x7f106000fd10 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 shutdown_connections 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f105406f400 0x7f1054071910 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10740cfe00 0x7f10741f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10740ce5c0 0x7f10740cf8c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 --2- 172.21.15.116:0/1232458338 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1074156fa0 0x7f1074155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 >> 172.21.15.116:0/1232458338 conn(0x7f10740b4620 msgr2=0x7f10740b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 shutdown_connections 2024-05-14T11:23:38.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:38.184+0000 7f107be00640 1 -- 172.21.15.116:0/1232458338 wait complete. 2024-05-14T11:23:38.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:23:37.028072+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7027}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7027': {'gid': 7027, 'name': 'b', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.149:6835/2777717906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 2777717906}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 2777717906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:23:38.607 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7027:up:rejoin 2024-05-14T11:23:39.607 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:23:39.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.764+0000 7f33f5800640 1 Processor -- start 2024-05-14T11:23:39.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.764+0000 7f33f5800640 1 -- start start 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33f5800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f33f006f730 0x7f33f006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33f5800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33f5800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f0070080 con 0x7f33f00b0770 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33f5800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f33f006db20 con 0x7f33f006f730 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33f5800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33f006cf10 con 0x7f33f0072a30 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33eea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f33f00b0770 0x7f33f006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35520/0 (socket says 172.21.15.116:35520) 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33eea00640 1 -- 172.21.15.116:0/880645747 learned_addr learned my addr 172.21.15.116:0/880645747 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ef400640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 --2- 172.21.15.116:0/880645747 >> v2:172.21.15.149:3300/0 conn(0x7f33f006f730 0x7f33f006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872101392 0 0) 0x7f33e0001120 con 0x7f33f00b0770 2024-05-14T11:23:39.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33d4004640 con 0x7f33f00b0770 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3506029693 0 0) 0x7f33d4004640 con 0x7f33f00b0770 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 msgr2=0x7f33f0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33f0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 >> v2:172.21.15.149:3300/0 conn(0x7f33f006f730 msgr2=0x7f33f006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 --2- 172.21.15.116:0/880645747 >> v2:172.21.15.149:3300/0 conn(0x7f33f006f730 0x7f33f006fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f006d1f0 con 0x7f33f00b0770 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f33e00018d0 con 0x7f33f00b0770 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 >> v1:172.21.15.116:6789/0 conn(0x7f33f00b0770 legacy=0x7f33f006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33d4004b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33d40050f0 0x7f33d4005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d4005e60 0x7f33d4006270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33f006db20 con 0x7f33d40050f0 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33f006cf10 con 0x7f33d4005e60 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33d400cc70 con 0x7f33f0072a30 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33d40050f0 0x7f33d4005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33eea00640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33d4004b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ef400640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d4005e60 0x7f33d4006270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 -- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 msgr2=0x7f33d4004b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33d4004b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 -- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d4005e60 msgr2=0x7f33d4006270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d4005e60 0x7f33d4006270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33e6000640 1 -- 172.21.15.116:0/880645747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d4004640 con 0x7f33d40050f0 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33ef400640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d4005e60 0x7f33d4006270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:39.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.768+0000 7f33eea00640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f0072a30 0x7f33d4004b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:39.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33e6000640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33d40050f0 0x7f33d4005920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f90 tx=0x7f33d801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b159a23f21f1e651 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:39.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d80b2070 con 0x7f33d40050f0 2024-05-14T11:23:39.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d80a5bc0 con 0x7f33d40050f0 2024-05-14T11:23:39.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33ee000640 1 -- 172.21.15.116:0/880645747 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8009c10 con 0x7f33d40050f0 2024-05-14T11:23:39.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 -- 172.21.15.116:0/880645747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33d40050f0 msgr2=0x7f33d4005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 --2- 172.21.15.116:0/880645747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33d40050f0 0x7f33d4005920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f90 tx=0x7f33d801cc50 comp rx=0 tx=0).stop 2024-05-14T11:23:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33ef400640 1 -- 172.21.15.116:0/880645747 reap_dead start 2024-05-14T11:23:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 -- 172.21.15.116:0/880645747 shutdown_connections 2024-05-14T11:23:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 -- 172.21.15.116:0/880645747 >> 172.21.15.116:0/880645747 conn(0x7f33f00b4620 msgr2=0x7f33f00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 -- 172.21.15.116:0/880645747 shutdown_connections 2024-05-14T11:23:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.772+0000 7f33f5800640 1 -- 172.21.15.116:0/880645747 wait complete. 2024-05-14T11:23:39.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 Processor -- start 2024-05-14T11:23:39.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 -- start start 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d40050f0 0x7f33f0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f00cd890 0x7f33f009bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0072fc0 con 0x7f33f009c270 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0072e40 con 0x7f33d40050f0 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33f5800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33f0073140 con 0x7f33f00cd890 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60984/0 (socket says 172.21.15.116:60984) 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 -- 172.21.15.116:0/4011475379 learned_addr learned my addr 172.21.15.116:0/4011475379 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33eea00640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d40050f0 0x7f33f0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33e6000640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f00cd890 0x7f33f009bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 -- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f00cd890 msgr2=0x7f33f009bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f00cd890 0x7f33f009bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 -- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d40050f0 msgr2=0x7f33f0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d40050f0 0x7f33f0159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 -- 172.21.15.116:0/4011475379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00cebb0 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33ef400640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f33f0070d10 tx=0x7f33e8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.776+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8004440 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e801b070 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8003860 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f00cee90 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f00cf450 con 0x7f33f009c270 2024-05-14T11:23:39.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f006a9c0 con 0x7f33f009c270 2024-05-14T11:23:39.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f33e8003a00 con 0x7f33f009c270 2024-05-14T11:23:39.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33e7400640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 0x7f33cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:39.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f33e808efe0 con 0x7f33f009c270 2024-05-14T11:23:39.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.780+0000 7f33eea00640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 0x7f33cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:39.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.784+0000 7f33eea00640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 0x7f33cc071940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000fc0 tx=0x7f33e00018f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.788+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f33e8024020 con 0x7f33f009c270 2024-05-14T11:23:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.976+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33f00b1990 con 0x7f33f009c270 2024-05-14T11:23:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.976+0000 7f33e7400640 1 -- 172.21.15.116:0/4011475379 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7f33e805b5e0 con 0x7f33f009c270 2024-05-14T11:23:39.977 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:23:39.977 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:23:39.978 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:23:39.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 msgr2=0x7f33cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 0x7f33cc071940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000fc0 tx=0x7f33e00018f0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 msgr2=0x7f33f009c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f33f0070d10 tx=0x7f33e8011920 comp rx=0 tx=0).stop 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 shutdown_connections 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f33cc06f430 0x7f33cc071940 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f009c270 0x7f33f009c680 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33f00cd890 0x7f33f009bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 --2- 172.21.15.116:0/4011475379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33d40050f0 0x7f33f0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 >> 172.21.15.116:0/4011475379 conn(0x7f33f00b4620 msgr2=0x7f33f015b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 shutdown_connections 2024-05-14T11:23:39.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:39.980+0000 7f33f5800640 1 -- 172.21.15.116:0/4011475379 wait complete. 2024-05-14T11:23:40.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:21:55.758114+0000', 'modified': '2024-05-14T11:23:39.029230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7027}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7027': {'gid': 7027, 'name': 'b', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.149:6835/2777717906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6833', 'nonce': 2777717906}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 2777717906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7027, 'qdb_cluster': [7027]} 2024-05-14T11:23:40.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:23:40.402 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-14T11:23:40.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 Processor -- start 2024-05-14T11:23:40.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 -- start start 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f92a0156fa0 0x7f92a006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a0160510 con 0x7f92a00d1670 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a01607f0 con 0x7f92a00af380 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a7800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f92a01573b0 con 0x7f92a0156fa0 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f929fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f92a00d1670 0x7f92a00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35526/0 (socket says 172.21.15.116:35526) 2024-05-14T11:23:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f929fe00640 1 -- 172.21.15.116:0/641552111 learned_addr learned my addr 172.21.15.116:0/641552111 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f92a4e00640 1 --2- 172.21.15.116:0/641552111 >> v2:172.21.15.149:3300/0 conn(0x7f92a0156fa0 0x7f92a006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723237136 0 0) 0x7f92a0160510 con 0x7f92a00d1670 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.552+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9284005740 con 0x7f92a00d1670 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 222733224 0 0) 0x7f9284005740 con 0x7f92a00d1670 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 >> v2:172.21.15.149:3300/0 conn(0x7f92a0156fa0 msgr2=0x7f92a006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 --2- 172.21.15.116:0/641552111 >> v2:172.21.15.149:3300/0 conn(0x7f92a0156fa0 0x7f92a006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 >> v1:172.21.15.149:6789/0 conn(0x7f92a00af380 legacy=0x7f92a00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0160ad0 con 0x7f92a00d1670 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f928c001700 con 0x7f92a00d1670 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a4e00640 1 --2- 172.21.15.116:0/641552111 >> v2:172.21.15.149:3300/0 conn(0x7f92a0156fa0 0x7f92a006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 >> v1:172.21.15.116:6789/0 conn(0x7f92a00d1670 legacy=0x7f92a00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9284005c20 0x7f9284006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9284006a20 0x7f9284006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 0x7f928400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f928400e350 con 0x7f92840073e0 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f928400e500 con 0x7f9284005c20 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f928400e700 con 0x7f9284006a20 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 0x7f928400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f929fe00640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9284006a20 0x7f9284006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a4e00640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9284005c20 0x7f9284006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 -- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9284006a20 msgr2=0x7f9284006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9284006a20 0x7f9284006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 -- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9284005c20 msgr2=0x7f9284006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9284005c20 0x7f9284006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 -- 172.21.15.116:0/641552111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9284005740 con 0x7f92840073e0 2024-05-14T11:23:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 -- 172.21.15.116:0/641552111 reap_dead start 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.556+0000 7f92a5800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 0x7f928400dbe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9298001f90 tx=0x7f929801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f38201cbea987a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298001b60 con 0x7f92840073e0 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9298003b80 con 0x7f92840073e0 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f929f400640 1 -- 172.21.15.116:0/641552111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980a4940 con 0x7f92840073e0 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 msgr2=0x7f928400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 0x7f928400dbe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9298001f90 tx=0x7f929801c580 comp rx=0 tx=0).stop 2024-05-14T11:23:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- 172.21.15.116:0/641552111 shutdown_connections 2024-05-14T11:23:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 --2- 172.21.15.116:0/641552111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92840073e0 0x7f928400dbe0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- 172.21.15.116:0/641552111 >> 172.21.15.116:0/641552111 conn(0x7f92a00b4620 msgr2=0x7f92a00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- 172.21.15.116:0/641552111 shutdown_connections 2024-05-14T11:23:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- 172.21.15.116:0/641552111 wait complete. 2024-05-14T11:23:40.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 Processor -- start 2024-05-14T11:23:40.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.560+0000 7f92a7800640 1 -- start start 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92840073e0 0x7f92a0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 0x7f92a00d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0070160 con 0x7f92a00ceaa0 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92a006ffe0 con 0x7f92a00d0180 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00702e0 con 0x7f92840073e0 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32772/0 (socket says 172.21.15.116:32772) 2024-05-14T11:23:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 -- 172.21.15.116:0/2202991983 learned_addr learned my addr 172.21.15.116:0/2202991983 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92840073e0 0x7f92a0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a5800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 0x7f92a00d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 -- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92840073e0 msgr2=0x7f92a0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92840073e0 0x7f92a0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 -- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 msgr2=0x7f92a00d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 0x7f92a00d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 -- 172.21.15.116:0/2202991983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a00d0ad0 con 0x7f92a00ceaa0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a5800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 0x7f92a00d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929fe00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f928c001bc0 tx=0x7f928c0129a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929cc00640 1 -- 172.21.15.116:0/2202991983 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c01c070 con 0x7f92a00ceaa0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929cc00640 1 -- 172.21.15.116:0/2202991983 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c014770 con 0x7f92a00ceaa0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a00cfd70 con 0x7f92a00ceaa0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f929cc00640 1 -- 172.21.15.116:0/2202991983 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c014b60 con 0x7f92a00ceaa0 2024-05-14T11:23:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.564+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a00cfa90 con 0x7f92a00ceaa0 2024-05-14T11:23:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f929cc00640 1 -- 172.21.15.116:0/2202991983 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f928c01f070 con 0x7f92a00ceaa0 2024-05-14T11:23:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f929cc00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 0x7f9280071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f929cc00640 1 -- 172.21.15.116:0/2202991983 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f928c091d10 con 0x7f92a00ceaa0 2024-05-14T11:23:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 0x7f9280071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 Processor -- start 2024-05-14T11:23:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 -- start start 2024-05-14T11:23:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 0x7f9280071940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f92900093f0 tx=0x7f9290009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a3246c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a324576f80 0x55a3246c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a324565620 con 0x55a3245606b0 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a324480740 con 0x55a324576f80 2024-05-14T11:23:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a3244ce090 con 0x55a32418e7c0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32786/0 (socket says 172.21.15.116:32786) 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a5800640 1 -- 172.21.15.116:0/1082486290 learned_addr learned my addr 172.21.15.116:0/1082486290 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f929fe00640 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a324576f80 0x55a3246c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.568+0000 7f92a4e00640 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a3246c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 -- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 msgr2=0x55a3246c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a3246c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 -- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a324576f80 msgr2=0x55a3246c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a324576f80 0x55a3246c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 -- 172.21.15.116:0/1082486290 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3246620a0 con 0x55a3245606b0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92a5800640 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f929801c3b0 tx=0x7f92980014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c6de9d8d68a5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f9295600640 1 -- 172.21.15.116:0/1082486290 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980a4940 con 0x55a3245606b0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f9295600640 1 -- 172.21.15.116:0/1082486290 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92980a4ae0 con 0x55a3245606b0 2024-05-14T11:23:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f9295600640 1 -- 172.21.15.116:0/1082486290 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980050d0 con 0x55a3245606b0 2024-05-14T11:23:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- 172.21.15.116:0/1082486290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 msgr2=0x55a3246c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f929801c3b0 tx=0x7f92980014a0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- 172.21.15.116:0/1082486290 shutdown_connections 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a324576f80 0x55a3246c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a3246c8da0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 --2- 172.21.15.116:0/1082486290 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a3246c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- 172.21.15.116:0/1082486290 >> 172.21.15.116:0/1082486290 conn(0x55a32464d340 msgr2=0x55a32467cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- 172.21.15.116:0/1082486290 shutdown_connections 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- 172.21.15.116:0/1082486290 wait complete. 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 -- start start 2024-05-14T11:23:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.572+0000 7f92aedd0000 1 Processor -- start 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32418e7c0 0x55a3246833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a324576f80 0x55a3246ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a3244ca050 con 0x55a3245606b0 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a324486ec0 con 0x55a32418e7c0 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a32446dc90 con 0x55a324576f80 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 10 client.0 _ll_register_callbacks cb 0x55a324669e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32794/0 (socket says 172.21.15.116:32794) 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 -- 172.21.15.116:0/796937523 learned_addr learned my addr 172.21.15.116:0/796937523 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92a4e00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32418e7c0 0x55a3246833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92a5800640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a324576f80 0x55a3246ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 -- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a324576f80 msgr2=0x55a3246ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a324576f80 0x55a3246ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 -- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32418e7c0 msgr2=0x55a3246833e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32418e7c0 0x55a3246833e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 -- 172.21.15.116:0/796937523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3245786f0 con 0x55a3245606b0 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f929fe00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f928c001870 tx=0x7f928c05dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3cc6ecf95f3249 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:40.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0622c0 con 0x55a3245606b0 2024-05-14T11:23:40.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f928c05d550 con 0x55a3245606b0 2024-05-14T11:23:40.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92980a4ae0 con 0x55a3245606b0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f928c067010 con 0x55a3245606b0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 10 client.7401 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a3245746a0 con 0x55a3245606b0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92980a4ae0 con 0x55a3245606b0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f928c059c60 con 0x55a3245606b0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 10 client.7401 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 10 client.7401 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 20 client.7401 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a3241737d0 0x55a3246d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 4 client.7401 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a324466be0 con 0x55a3241737d0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92a4e00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a3241737d0 0x55a3246d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f928a000640 10 client.7401 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:40.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.576+0000 7f92a4e00640 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a3241737d0 0x55a3246d98e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12859b69b9267f4b server_cookie=589464b88a70e84a in_seq=0 out_seq=0 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f928a000640 1 -- 172.21.15.116:0/796937523 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a324466be0 con 0x55a3241737d0 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f928a000640 10 client.7401 handle_command_reply: tid=0 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 client.7401 shutdown 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 msgr2=0x55a324683920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f928c001870 tx=0x7f928c05dec0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 shutdown_connections 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a3241737d0 0x55a3246d98e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a324576f80 0x55a3246ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3245606b0 0x55a324683920 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 --2- 172.21.15.116:0/796937523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32418e7c0 0x55a3246833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 >> 172.21.15.116:0/796937523 conn(0x55a32464d340 msgr2=0x55a32467d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f928a000640 0 client.7401 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:40.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 shutdown_connections 2024-05-14T11:23:40.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 1 -- 172.21.15.116:0/796937523 wait complete. 2024-05-14T11:23:40.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.580+0000 7f92aedd0000 20 client.7401 trim_cache size 0 max 16384 2024-05-14T11:23:40.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.596+0000 7f92aedd0000 1 Processor -- start 2024-05-14T11:23:40.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.596+0000 7f92aedd0000 1 -- start start 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32464d340 0x55a3246ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a3245606b0 0x55a32467d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0203510 con 0x55a32418e7c0 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92a02028e0 con 0x55a32464d340 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f92a0202e70 con 0x55a3245606b0 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32806/0 (socket says 172.21.15.116:32806) 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 -- 172.21.15.116:0/2319543013 learned_addr learned my addr 172.21.15.116:0/2319543013 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92a5800640 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a3245606b0 0x55a32467d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32464d340 0x55a3246ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 -- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a3245606b0 msgr2=0x55a32467d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a3245606b0 0x55a32467d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 -- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32464d340 msgr2=0x55a3246ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32464d340 0x55a3246ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 -- 172.21.15.116:0/2319543013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3244a7af0 con 0x55a32418e7c0 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f929fe00640 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f928c000c10 tx=0x7f928c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6284d495732f5f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f9294c00640 1 -- 172.21.15.116:0/2319543013 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c05d550 con 0x55a32418e7c0 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f9294c00640 1 -- 172.21.15.116:0/2319543013 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c08c060 con 0x55a32418e7c0 2024-05-14T11:23:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f9294c00640 1 -- 172.21.15.116:0/2319543013 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0620c0 con 0x55a32418e7c0 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- 172.21.15.116:0/2319543013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 msgr2=0x55a3246ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f928c000c10 tx=0x7f928c019000 comp rx=0 tx=0).stop 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- 172.21.15.116:0/2319543013 shutdown_connections 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a3245606b0 0x55a32467d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32418e7c0 0x55a3246ad690 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2319543013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a32464d340 0x55a3246ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- 172.21.15.116:0/2319543013 >> 172.21.15.116:0/2319543013 conn(0x55a3246abb40 msgr2=0x55a3246d4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- 172.21.15.116:0/2319543013 shutdown_connections 2024-05-14T11:23:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- 172.21.15.116:0/2319543013 wait complete. 2024-05-14T11:23:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.600+0000 7f92aedd0000 1 -- start start 2024-05-14T11:23:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 Processor -- start 2024-05-14T11:23:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a32473b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3245606b0 0x55a32473b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a32448b920 con 0x55a32464d340 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3244717d0 con 0x55a3245606b0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a3244db490 con 0x55a32418e7c0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 10 client.0 _ll_register_callbacks cb 0x7f928c05e120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a32473b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32814/0 (socket says 172.21.15.116:32814) 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f929fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3245606b0 0x55a32473b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 -- 172.21.15.116:0/2945154856 learned_addr learned my addr 172.21.15.116:0/2945154856 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 -- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 msgr2=0x55a32473b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a32473b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 -- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3245606b0 msgr2=0x55a32473b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3245606b0 0x55a32473b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 -- 172.21.15.116:0/2945154856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3244a6e40 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92a5800640 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f929801c3b0 tx=0x7f9298019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f851aa5f93f027 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980be070 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f92980050d0 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928c08c060 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f92980ca050 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 10 client.7407 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a32454e4e0 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c08c060 con 0x55a32464d340 2024-05-14T11:23:40.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f92980cd020 con 0x55a32464d340 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 10 client.7407 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 10 client.7407 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 20 client.7407 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a32418ce80 0x55a3246f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 4 client.7407 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.604+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "status"}) -- 0x55a3244dbca0 con 0x55a32418ce80 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a32418ce80 0x55a3246f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a4e00640 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a32418ce80 0x55a3246f7150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cde5b0fc0dff24d server_cookie=fdb69760c6423b6c in_seq=0 out_seq=0 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f928a000640 10 client.7407 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f928a000640 1 -- 172.21.15.116:0/2945154856 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a3244dbca0 con 0x55a32418ce80 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f928a000640 10 client.7407 handle_command_reply: tid=0 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 client.7407 shutdown 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 msgr2=0x55a3246f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f929801c3b0 tx=0x7f9298019640 comp rx=0 tx=0).stop 2024-05-14T11:23:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 shutdown_connections 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55a32418ce80 0x55a3246f7150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3245606b0 0x55a32473b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a32464d340 0x55a3246f67f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 --2- 172.21.15.116:0/2945154856 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a32418e7c0 0x55a32473b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f928a000640 0 client.7407 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 >> 172.21.15.116:0/2945154856 conn(0x55a3246abb40 msgr2=0x55a3246ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 shutdown_connections 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 1 -- 172.21.15.116:0/2945154856 wait complete. 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92aedd0000 20 client.7407 trim_cache size 0 max 16384 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:23:40.612 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7027, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.5915865169999996, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 74, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 88, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 5.5051370430000004 2024-05-14T11:23:40.613 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 msgr2=0x7f9280071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 0x7f9280071940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f92900093f0 tx=0x7f9290009190 comp rx=0 tx=0).stop 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 msgr2=0x7f92a00ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f928c001bc0 tx=0x7f928c0129a0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 shutdown_connections 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f928006f430 0x7f9280071940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92a00d0180 0x7f92a00d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a00ceaa0 0x7f92a00ceeb0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 --2- 172.21.15.116:0/2202991983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92840073e0 0x7f92a0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 >> 172.21.15.116:0/2202991983 conn(0x7f92a00b4620 msgr2=0x7f92a015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 shutdown_connections 2024-05-14T11:23:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.608+0000 7f92a7800640 1 -- 172.21.15.116:0/2202991983 wait complete. 2024-05-14T11:23:40.628 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:23:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.772+0000 7f7e6fe00640 1 Processor -- start 2024-05-14T11:23:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.772+0000 7f7e6fe00640 1 -- start start 2024-05-14T11:23:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e680d1670 0x7f7e680d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e680af380 0x7f7e680af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6fe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e680d1fc0 con 0x7f7e68156fa0 2024-05-14T11:23:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7e680706f0 con 0x7f7e680af380 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7e681573b0 con 0x7f7e680d1670 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e680af380 0x7f7e680af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e680af380 0x7f7e680af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32824/0 (socket says 172.21.15.116:32824) 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 -- 172.21.15.116:0/3955013720 learned_addr learned my addr 172.21.15.116:0/3955013720 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6ca00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e680d1670 0x7f7e680d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 -- 172.21.15.116:0/3955013720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e680d1670 msgr2=0x7f7e680d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e680d1670 0x7f7e680d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 -- 172.21.15.116:0/3955013720 >> v1:172.21.15.149:6789/0 conn(0x7f7e68156fa0 legacy=0x7f7e6806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.776+0000 7f7e6de00640 1 -- 172.21.15.116:0/3955013720 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e68160510 con 0x7f7e680af380 2024-05-14T11:23:40.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6de00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680af380 0x7f7e680af790 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009e50 tx=0x7f7e6001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0831b6b5cf73be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e67e00640 1 -- 172.21.15.116:0/3955013720 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600a57f0 con 0x7f7e680af380 2024-05-14T11:23:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e67e00640 1 -- 172.21.15.116:0/3955013720 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e600a5990 con 0x7f7e680af380 2024-05-14T11:23:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e67e00640 1 -- 172.21.15.116:0/3955013720 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600a5d60 con 0x7f7e680af380 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 -- 172.21.15.116:0/3955013720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680af380 msgr2=0x7f7e680af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680af380 0x7f7e680af790 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009e50 tx=0x7f7e6001ca30 comp rx=0 tx=0).stop 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 -- 172.21.15.116:0/3955013720 shutdown_connections 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680af380 0x7f7e680af790 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/3955013720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e680d1670 0x7f7e680d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 -- 172.21.15.116:0/3955013720 >> 172.21.15.116:0/3955013720 conn(0x7f7e680b4620 msgr2=0x7f7e680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 -- 172.21.15.116:0/3955013720 shutdown_connections 2024-05-14T11:23:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.780+0000 7f7e6fe00640 1 -- 172.21.15.116:0/3955013720 wait complete. 2024-05-14T11:23:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 Processor -- start 2024-05-14T11:23:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- start start 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680af380 0x7f7e680d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e68156fa0 0x7f7e68155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680b0660 con 0x7f7e680d1670 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e680b04e0 con 0x7f7e680af380 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7e68169810 con 0x7f7e68156fa0 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32838/0 (socket says 172.21.15.116:32838) 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 -- 172.21.15.116:0/4025927083 learned_addr learned my addr 172.21.15.116:0/4025927083 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6d400640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680af380 0x7f7e680d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6de00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e68156fa0 0x7f7e68155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 -- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e68156fa0 msgr2=0x7f7e68155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e68156fa0 0x7f7e68155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 -- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680af380 msgr2=0x7f7e680d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680af380 0x7f7e680d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 -- 172.21.15.116:0/4025927083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e68152840 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6ca00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400e870 tx=0x7f7e5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54017070 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5400c940 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400ccf0 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e68152b20 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.784+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e680cf7c0 con 0x7f7e680d1670 2024-05-14T11:23:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6806a9c0 con 0x7f7e680d1670 2024-05-14T11:23:40.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e5401a070 con 0x7f7e680d1670 2024-05-14T11:23:40.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e65600640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 0x7f7e48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:40.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f7e5408f7d0 con 0x7f7e680d1670 2024-05-14T11:23:40.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e6d400640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 0x7f7e48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:40.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.788+0000 7f7e6d400640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 0x7f7e48071940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007fd0 tx=0x7f7e5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:40.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.792+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7e5405bd20 con 0x7f7e680d1670 2024-05-14T11:23:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7e68153ce0 con 0x7f7e680d1670 2024-05-14T11:23:40.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e65600640 1 -- 172.21.15.116:0/4025927083 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f7e54060020 con 0x7f7e680d1670 2024-05-14T11:23:40.984 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:23:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 msgr2=0x7f7e48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 0x7f7e48071940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007fd0 tx=0x7f7e5c009040 comp rx=0 tx=0).stop 2024-05-14T11:23:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 msgr2=0x7f7e6814d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400e870 tx=0x7f7e5400ec70 comp rx=0 tx=0).stop 2024-05-14T11:23:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 shutdown_connections 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7e4806f430 0x7f7e48071940 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7e68156fa0 0x7f7e68155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e680d1670 0x7f7e6814d7a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 --2- 172.21.15.116:0/4025927083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680af380 0x7f7e680d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 >> 172.21.15.116:0/4025927083 conn(0x7f7e680b4620 msgr2=0x7f7e680686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 shutdown_connections 2024-05-14T11:23:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:40.984+0000 7f7e6fe00640 1 -- 172.21.15.116:0/4025927083 wait complete. 2024-05-14T11:23:41.005 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:23:41.006 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:23:41.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.007 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:23:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:23:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41981sec preferred_lft 41981sec 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.017 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:23:41.018 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:23:41.019 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:23:41.018+0000 7fcd04600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:23:41.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.020 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:23:41.084 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:23:41.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.087 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:23:41.145 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:23:41.145 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:23:41.145 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:23:41.145 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:23:41.145 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:23:41.146 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:23:41.146 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:23:41.146 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:23:41.146 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:23:41.146 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:23:41.146 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:23:41.147 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:23:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:23:41.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:23:41.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.280 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:23:41.334 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:23:41.335 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:23:41.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:23:41.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.336 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:23:41.397 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:23:41.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:23:41.398 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:41.450 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:23:41.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:23:41.451 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:41.506 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:23:41.570 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:23:41.623 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:23:41.624 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:23:41.625 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:23:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:23:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:23:41.626 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:23:41.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.627 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:23:41.678 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:23:41.678 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:23:41.721 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:23:41.741 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:23:41.796 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:23:41.797 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:23:41.798 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:23:41.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.799 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:23:41.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:23:41.824+0000 7f5b58d9e580 -1 init, newargv = 0x7f5b3c0054f0 newargc=15 2024-05-14T11:23:41.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[45547]: starting ceph client 2024-05-14T11:23:41.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[45547]: starting fuse 2024-05-14T11:23:41.834 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:23:41.834 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:23:41.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:23:41.834 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:23:41.834 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:23:41.835 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:23:41.836 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:23:41.957 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.45547.asok 2024-05-14T11:23:41.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:23:41.968 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45547.asok 2024-05-14T11:23:41.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:41.968 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45547.asok status 2024-05-14T11:23:42.146 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "45547", 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7419, 2024-05-14T11:23:42.147 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 7419 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2772843692 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2772843692 2024-05-14T11:23:42.148 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.7419 192.168.0.1:0/2772843692", 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/2772843692", 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 74, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 88, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:23:42.149 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:23:42.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:42.162 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:23:42.162 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:42.171 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:23:42.171 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:42.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:42.171 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:23:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:42.234 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-14T11:23:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 Processor -- start 2024-05-14T11:23:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 -- start start 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 0x7fb34c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c06fa90 con 0x7fb34c0cbf70 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34c06fd70 con 0x7fb34c0d14b0 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb353200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c0d00d0 con 0x7fb34c15cd00 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb352800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb34c0cbf70 0x7fb34c0cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35530/0 (socket says 172.21.15.116:35530) 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb352800640 1 -- 172.21.15.116:0/4109435743 learned_addr learned my addr 172.21.15.116:0/4109435743 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb351e00640 1 --2- 172.21.15.116:0/4109435743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 0x7fb34c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198651661 0 0) 0x7fb34c06fa90 con 0x7fb34c0cbf70 2024-05-14T11:23:42.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb334005740 con 0x7fb34c0cbf70 2024-05-14T11:23:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb351e00640 1 -- 172.21.15.116:0/4109435743 >> v1:172.21.15.149:6789/0 conn(0x7fb34c0d14b0 legacy=0x7fb34c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838288699 0 0) 0x7fb34c06fd70 con 0x7fb34c0d14b0 2024-05-14T11:23:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb351e00640 1 -- 172.21.15.116:0/4109435743 >> v1:172.21.15.116:6789/0 conn(0x7fb34c0cbf70 legacy=0x7fb34c0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.424+0000 7fb351e00640 1 -- 172.21.15.116:0/4109435743 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c070050 con 0x7fb34c15cd00 2024-05-14T11:23:42.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb351e00640 1 --2- 172.21.15.116:0/4109435743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 0x7fb34c0cfb90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097f0 tx=0x7fb34001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe8872d3e6378481 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0cd2a0 con 0x7fb34c15cd00 2024-05-14T11:23:42.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400af070 con 0x7fb34c15cd00 2024-05-14T11:23:42.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb350a00640 1 -- 172.21.15.116:0/4109435743 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3400aa830 con 0x7fb34c15cd00 2024-05-14T11:23:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 -- 172.21.15.116:0/4109435743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 msgr2=0x7fb34c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 --2- 172.21.15.116:0/4109435743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 0x7fb34c0cfb90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097f0 tx=0x7fb34001c4b0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 -- 172.21.15.116:0/4109435743 shutdown_connections 2024-05-14T11:23:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 --2- 172.21.15.116:0/4109435743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c15cd00 0x7fb34c0cfb90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 -- 172.21.15.116:0/4109435743 >> 172.21.15.116:0/4109435743 conn(0x7fb34c0b43a0 msgr2=0x7fb34c0b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 -- 172.21.15.116:0/4109435743 shutdown_connections 2024-05-14T11:23:42.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.428+0000 7fb353200640 1 -- 172.21.15.116:0/4109435743 wait complete. 2024-05-14T11:23:42.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 Processor -- start 2024-05-14T11:23:42.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 -- start start 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c0d14b0 0x7fb34c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb34c15cd00 0x7fb34c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0cde80 con 0x7fb34c0cbf70 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c0cdd00 con 0x7fb34c15cd00 2024-05-14T11:23:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb353200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c0ce000 con 0x7fb34c0d14b0 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32850/0 (socket says 172.21.15.116:32850) 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 -- 172.21.15.116:0/2877764467 learned_addr learned my addr 172.21.15.116:0/2877764467 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351400640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c0d14b0 0x7fb34c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb352800640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb34c15cd00 0x7fb34c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 -- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c0d14b0 msgr2=0x7fb34c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c0d14b0 0x7fb34c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 -- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb34c15cd00 msgr2=0x7fb34c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb34c15cd00 0x7fb34c15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.432+0000 7fb351e00640 1 -- 172.21.15.116:0/2877764467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c156000 con 0x7fb34c0cbf70 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb351e00640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097f0 tx=0x7fb34001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb346000640 1 -- 172.21.15.116:0/2877764467 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340004220 con 0x7fb34c0cbf70 2024-05-14T11:23:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb346000640 1 -- 172.21.15.116:0/2877764467 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400043c0 con 0x7fb34c0cbf70 2024-05-14T11:23:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c0c9f40 con 0x7fb34c0cbf70 2024-05-14T11:23:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb346000640 1 -- 172.21.15.116:0/2877764467 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340003860 con 0x7fb34c0cbf70 2024-05-14T11:23:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c0ca500 con 0x7fb34c0cbf70 2024-05-14T11:23:42.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.436+0000 7fb346000640 1 -- 172.21.15.116:0/2877764467 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb340003a00 con 0x7fb34c0cbf70 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 Processor -- start 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 -- start start 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb346000640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 0x7fb330071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb346000640 1 -- 172.21.15.116:0/2877764467 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fb340129190 con 0x7fb34c0cbf70 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 0x7fb330071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 0x7fb330071940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb348005c10 tx=0x7fb348009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cb6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658cb7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d658b54960 con 0x55d65873d630 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d658abc200 con 0x55d658b662c0 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d658a74da0 con 0x55d658b4f9f0 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32852/0 (socket says 172.21.15.116:32852) 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 -- 172.21.15.116:0/627273823 learned_addr learned my addr 172.21.15.116:0/627273823 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb351400640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658cb7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb351e00640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cb6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 -- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 msgr2=0x55d658cb7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658cb7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 -- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 msgr2=0x55d658cb6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cb6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 -- 172.21.15.116:0/627273823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d658a79900 con 0x55d65873d630 2024-05-14T11:23:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb351e00640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cb6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:23:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb352800640 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ecd0 tx=0x7fb348011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93e65941a0777b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb33a600640 1 -- 172.21.15.116:0/627273823 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348021450 con 0x55d65873d630 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb33a600640 1 -- 172.21.15.116:0/627273823 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3480215f0 con 0x55d65873d630 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.440+0000 7fb33a600640 1 -- 172.21.15.116:0/627273823 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480219a0 con 0x55d65873d630 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- 172.21.15.116:0/627273823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 msgr2=0x55d658cb78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ecd0 tx=0x7fb348011000 comp rx=0 tx=0).stop 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- 172.21.15.116:0/627273823 shutdown_connections 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65873d630 0x55d658cb78c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658cb7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- 172.21.15.116:0/627273823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cb6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- 172.21.15.116:0/627273823 >> 172.21.15.116:0/627273823 conn(0x55d658b73760 msgr2=0x55d658c6ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- 172.21.15.116:0/627273823 shutdown_connections 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- 172.21.15.116:0/627273823 wait complete. 2024-05-14T11:23:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- start start 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 Processor -- start 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658c72fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658c79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d658a96df0 con 0x55d658b662c0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d658a78180 con 0x55d65873d630 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d658abb510 con 0x55d658b4f9f0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb35bebb000 10 client.0 _ll_register_callbacks cb 0x55d658b6f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32866/0 (socket says 172.21.15.116:32866) 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb352800640 1 -- 172.21.15.116:0/972826512 learned_addr learned my addr 172.21.15.116:0/972826512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb351e00640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658c72fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.444+0000 7fb351400640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658c79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 -- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 msgr2=0x55d658c79270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658c79270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 -- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 msgr2=0x55d658c72fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658c72fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 -- 172.21.15.116:0/972826512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d658a66960 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb352800640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb348001a60 tx=0x7fb348013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ea16a3f542272a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348021450 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fb348010050 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3480215f0 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb348033050 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 10 client.7431 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d658a74ff0 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3480215f0 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb348036020 con 0x55d658b662c0 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 10 client.7431 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 10 client.7431 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 20 client.7431 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:42.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d658761b80 0x55d658cbde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 4 client.7431 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:23:42.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d658a54ef0 con 0x55d658761b80 2024-05-14T11:23:42.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb351e00640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d658761b80 0x55d658cbde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb351e00640 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d658761b80 0x55d658cbde00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c88397714d4bc9 server_cookie=382b54a168d5d819 in_seq=0 out_seq=0 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 10 client.7431 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 1 -- 172.21.15.116:0/972826512 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d658a54ef0 con 0x55d658761b80 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 10 client.7431 handle_command_reply: tid=0 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 client.7431 shutdown 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 msgr2=0x55d658c797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb348001a60 tx=0x7fb348013000 comp rx=0 tx=0).stop 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 shutdown_connections 2024-05-14T11:23:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d658761b80 0x55d658cbde00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658c797b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658c79270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb32ea00640 0 client.7431 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 --2- 172.21.15.116:0/972826512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658c72fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 >> 172.21.15.116:0/972826512 conn(0x55d658b73760 msgr2=0x55d658c6c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 shutdown_connections 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 1 -- 172.21.15.116:0/972826512 wait complete. 2024-05-14T11:23:42.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.448+0000 7fb35bebb000 20 client.7431 trim_cache size 0 max 16384 2024-05-14T11:23:42.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 Processor -- start 2024-05-14T11:23:42.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 -- start start 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658d24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658d25350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c204f50 con 0x55d658b662c0 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c158160 con 0x55d65873d630 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb34c204900 con 0x55d658b4f9f0 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32874/0 (socket says 172.21.15.116:32874) 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 -- 172.21.15.116:0/990324651 learned_addr learned my addr 172.21.15.116:0/990324651 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb351e00640 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658d24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb351400640 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658d25350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 -- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 msgr2=0x55d658d25350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658d25350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 -- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 msgr2=0x55d658d24e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658d24e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 -- 172.21.15.116:0/990324651 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d658b3d970 con 0x55d658b662c0 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.468+0000 7fb352800640 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb348009010 tx=0x7fb34801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822b1779ad7a46af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb339c00640 1 -- 172.21.15.116:0/990324651 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348023320 con 0x55d658b662c0 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb339c00640 1 -- 172.21.15.116:0/990324651 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3480234c0 con 0x55d658b662c0 2024-05-14T11:23:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb339c00640 1 -- 172.21.15.116:0/990324651 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348021450 con 0x55d658b662c0 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- 172.21.15.116:0/990324651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 msgr2=0x55d658d25890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb348009010 tx=0x7fb34801d000 comp rx=0 tx=0).stop 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- 172.21.15.116:0/990324651 shutdown_connections 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b662c0 0x55d658d25890 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d658b4f9f0 0x55d658d25350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 --2- 172.21.15.116:0/990324651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65873d630 0x55d658d24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- 172.21.15.116:0/990324651 >> 172.21.15.116:0/990324651 conn(0x55d658b73760 msgr2=0x55d658cbcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- 172.21.15.116:0/990324651 shutdown_connections 2024-05-14T11:23:42.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- 172.21.15.116:0/990324651 wait complete. 2024-05-14T11:23:42.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 -- start start 2024-05-14T11:23:42.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.472+0000 7fb35bebb000 1 Processor -- start 2024-05-14T11:23:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d65873d630 0x55d658d29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cc2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d658ac9600 con 0x55d658b4f9f0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d658ac0830 con 0x55d658b662c0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d658a71140 con 0x55d65873d630 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 10 client.0 _ll_register_callbacks cb 0x7fb340009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32876/0 (socket says 172.21.15.116:32876) 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 -- 172.21.15.116:0/3093419309 learned_addr learned my addr 172.21.15.116:0/3093419309 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351e00640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d65873d630 0x55d658d29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb352800640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cc2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 -- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d65873d630 msgr2=0x55d658d29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d65873d630 0x55d658d29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 -- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 msgr2=0x55d658cc2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cc2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 -- 172.21.15.116:0/3093419309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d658a589e0 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351400640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0b01e0 tx=0x7fb33c0016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c76e3fbb10950b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c017430 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fb33c0175d0 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3480234c0 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fb33c027050 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 10 client.7437 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d658775470 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3480234c0 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fb33c02a020 con 0x55d658b4f9f0 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 10 client.7437 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 10 client.7437 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 20 client.7437 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:42.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d65874fbe0 0x55d658cc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 4 client.7437 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-14T11:23:42.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55d658ac9e10 con 0x55d65874fbe0 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351e00640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d65874fbe0 0x55d658cc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 10 client.7437 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb351e00640 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d65874fbe0 0x55d658cc3760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2e2d36647ec68c server_cookie=9432850030ac1734 in_seq=0 out_seq=0 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 1 -- 172.21.15.116:0/3093419309 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d658ac9e10 con 0x55d65874fbe0 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb32ea00640 10 client.7437 handle_command_reply: tid=0 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 client.7437 shutdown 2024-05-14T11:23:42.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 msgr2=0x55d658d29990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0b01e0 tx=0x7fb33c0016c0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 shutdown_connections 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.476+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d65874fbe0 0x55d658cc3760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d658b662c0 0x55d658cc2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d658b4f9f0 0x55d658d29990 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 --2- 172.21.15.116:0/3093419309 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d65873d630 0x55d658d29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 >> 172.21.15.116:0/3093419309 conn(0x55d658b73760 msgr2=0x55d658ccd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb32ea00640 0 client.7437 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 shutdown_connections 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 1 -- 172.21.15.116:0/3093419309 wait complete. 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb35bebb000 20 client.7437 trim_cache size 0 max 16384 2024-05-14T11:23:42.481 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:23:42.482 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:23:42.482 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "5a6fa1da-f490-4b7b-bce9-98b653789e4f", 2024-05-14T11:23:42.482 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:23:42.482 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:23:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 msgr2=0x7fb330071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 0x7fb330071940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb348005c10 tx=0x7fb348009040 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 msgr2=0x7fb34c09a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb3400097f0 tx=0x7fb34001acf0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 shutdown_connections 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb33006f430 0x7fb330071940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb34c15cd00 0x7fb34c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb34c0d14b0 0x7fb34c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 --2- 172.21.15.116:0/2877764467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb34c0cbf70 0x7fb34c09a9b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 >> 172.21.15.116:0/2877764467 conn(0x7fb34c0b43a0 msgr2=0x7fb34c0aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 shutdown_connections 2024-05-14T11:23:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.480+0000 7fb353200640 1 -- 172.21.15.116:0/2877764467 wait complete. 2024-05-14T11:23:42.484 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-05-14T11:23:42.482+0000 7fccff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55cc0acdc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:23:42.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:23:42.499 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:23:42.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 Processor -- start 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 -- start start 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b800b0770 0x7f5b8006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5b80072a30 0x7f5b80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8006cc40 con 0x7f5b8006f730 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8006db20 con 0x7f5b800b0770 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b86800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8006cf10 con 0x7f5b80072a30 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b800b0770 0x7f5b8006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b800b0770 0x7f5b8006c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32892/0 (socket says 172.21.15.116:32892) 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 -- 172.21.15.116:0/3965390243 learned_addr learned my addr 172.21.15.116:0/3965390243 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b76a00640 1 --1- 172.21.15.116:0/3965390243 >> v1:172.21.15.116:6789/0 conn(0x7f5b8006f730 0x7f5b8006fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:35538/0 (socket says 172.21.15.116:35538) 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/3965390243 >> v2:172.21.15.149:3300/0 conn(0x7f5b80072a30 0x7f5b80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 -- 172.21.15.116:0/3965390243 >> v2:172.21.15.149:3300/0 conn(0x7f5b80072a30 msgr2=0x7f5b80072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 --2- 172.21.15.116:0/3965390243 >> v2:172.21.15.149:3300/0 conn(0x7f5b80072a30 0x7f5b80072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 -- 172.21.15.116:0/3965390243 >> v1:172.21.15.116:6789/0 conn(0x7f5b8006f730 legacy=0x7f5b8006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7f400640 1 -- 172.21.15.116:0/3965390243 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8006d1f0 con 0x7f5b800b0770 2024-05-14T11:23:42.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.648+0000 7f5b7ea00640 1 -- 172.21.15.116:0/3965390243 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678274509 0 0) 0x7f5b8006cc40 con 0x7f5b8006f730 2024-05-14T11:23:42.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b7f400640 1 --2- 172.21.15.116:0/3965390243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b8006c700 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5b800729c0 tx=0x7f5b7001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3795bccd30e67838 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b7ea00640 1 -- 172.21.15.116:0/3965390243 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b1070 con 0x7f5b800b0770 2024-05-14T11:23:42.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b7ea00640 1 -- 172.21.15.116:0/3965390243 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700a8050 con 0x7f5b800b0770 2024-05-14T11:23:42.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b7ea00640 1 -- 172.21.15.116:0/3965390243 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700ac860 con 0x7f5b800b0770 2024-05-14T11:23:42.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 -- 172.21.15.116:0/3965390243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 msgr2=0x7f5b8006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 --2- 172.21.15.116:0/3965390243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b8006c700 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5b800729c0 tx=0x7f5b7001c460 comp rx=0 tx=0).stop 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 -- 172.21.15.116:0/3965390243 shutdown_connections 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 --2- 172.21.15.116:0/3965390243 >> v2:172.21.15.149:3300/0 conn(0x7f5b80072a30 0x7f5b80072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 --2- 172.21.15.116:0/3965390243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b8006c700 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 -- 172.21.15.116:0/3965390243 >> 172.21.15.116:0/3965390243 conn(0x7f5b800b4620 msgr2=0x7f5b800b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 -- 172.21.15.116:0/3965390243 shutdown_connections 2024-05-14T11:23:42.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.652+0000 7f5b86800640 1 -- 172.21.15.116:0/3965390243 wait complete. 2024-05-14T11:23:42.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 Processor -- start 2024-05-14T11:23:42.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 -- start start 2024-05-14T11:23:42.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b8006f730 0x7f5b800cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b80072a30 0x7f5b8014b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80073500 con 0x7f5b800b0770 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80073380 con 0x7f5b80072a30 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b86800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5b80073680 con 0x7f5b8006f730 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32896/0 (socket says 172.21.15.116:32896) 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 -- 172.21.15.116:0/2365176113 learned_addr learned my addr 172.21.15.116:0/2365176113 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b76a00640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b80072a30 0x7f5b8014b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7f400640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b8006f730 0x7f5b800cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 -- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b8006f730 msgr2=0x7f5b800cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b8006f730 0x7f5b800cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 -- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b80072a30 msgr2=0x7f5b8014b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b80072a30 0x7f5b8014b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.656+0000 7f5b7fe00640 1 -- 172.21.15.116:0/2365176113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b800cf3b0 con 0x7f5b800b0770 2024-05-14T11:23:42.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 -- 172.21.15.116:0/2365176113 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78014420 con 0x7f5b800b0770 2024-05-14T11:23:42.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 -- 172.21.15.116:0/2365176113 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78019070 con 0x7f5b800b0770 2024-05-14T11:23:42.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b800ce310 con 0x7f5b800b0770 2024-05-14T11:23:42.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 -- 172.21.15.116:0/2365176113 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78014800 con 0x7f5b800b0770 2024-05-14T11:23:42.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b800ce8d0 con 0x7f5b800b0770 2024-05-14T11:23:42.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 -- 172.21.15.116:0/2365176113 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b780149a0 con 0x7f5b800b0770 2024-05-14T11:23:42.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 0x7f5b5c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b77e00640 1 -- 172.21.15.116:0/2365176113 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f5b7808fce0 con 0x7f5b800b0770 2024-05-14T11:23:42.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.660+0000 7f5b7f400640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 0x7f5b5c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 Processor -- start 2024-05-14T11:23:42.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- start start 2024-05-14T11:23:42.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7f400640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 0x7f5b5c071940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5b700ae000 tx=0x7f5b700a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569feca4f00 0x5569fee1e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569fec23100 con 0x5569fe8a43e0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569febdbca0 con 0x5569fecb7120 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5569fec1f0c0 con 0x5569feca4f00 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32902/0 (socket says 172.21.15.116:32902) 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 -- 172.21.15.116:0/3926701663 learned_addr learned my addr 172.21.15.116:0/3926701663 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7f400640 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b76a00640 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569feca4f00 0x5569fee1e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 -- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569feca4f00 msgr2=0x5569fee1e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569feca4f00 0x5569fee1e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 -- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 msgr2=0x5569fee1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 -- 172.21.15.116:0/3926701663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569febe0800 con 0x5569fe8a43e0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5b780582b0 tx=0x7f5b7805bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80325bc55386a4fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b6fe00640 1 -- 172.21.15.116:0/3926701663 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78075070 con 0x5569fe8a43e0 2024-05-14T11:23:42.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b6fe00640 1 -- 172.21.15.116:0/3926701663 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78060530 con 0x5569fe8a43e0 2024-05-14T11:23:42.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b6fe00640 1 -- 172.21.15.116:0/3926701663 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78064fe0 con 0x5569fe8a43e0 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- 172.21.15.116:0/3926701663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 msgr2=0x5569fee1edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5b780582b0 tx=0x7f5b7805bd40 comp rx=0 tx=0).stop 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- 172.21.15.116:0/3926701663 shutdown_connections 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee1edd0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569feca4f00 0x5569fee1e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/3926701663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- 172.21.15.116:0/3926701663 >> 172.21.15.116:0/3926701663 conn(0x5569fe8e4400 msgr2=0x5569fee01310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- 172.21.15.116:0/3926701663 shutdown_connections 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.664+0000 7f5b8dee8000 1 -- 172.21.15.116:0/3926701663 wait complete. 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- start start 2024-05-14T11:23:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 Processor -- start 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fe8a43e0 0x5569fee24dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fedd7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569fecee2a0 con 0x5569feca4f00 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569fecdef40 con 0x5569fe8a43e0 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5569febdf080 con 0x5569fecb7120 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 10 client.0 _ll_register_callbacks cb 0x5569fedbfc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:42.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32906/0 (socket says 172.21.15.116:32906) 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 -- 172.21.15.116:0/4226402059 learned_addr learned my addr 172.21.15.116:0/4226402059 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fedd7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b7f400640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fe8a43e0 0x5569fee24dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 -- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 msgr2=0x5569fedd7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fedd7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 -- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fe8a43e0 msgr2=0x5569fee24dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fe8a43e0 0x5569fee24dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 -- 172.21.15.116:0/4226402059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569febcd860 con 0x5569feca4f00 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b76a00640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5b68006620 tx=0x7f5b680015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc229102b5ff2d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68001d00 con 0x5569feca4f00 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b78060530 con 0x5569feca4f00 2024-05-14T11:23:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f5b680145e0 con 0x5569feca4f00 2024-05-14T11:23:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f5b68022050 con 0x5569feca4f00 2024-05-14T11:23:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 10 client.7452 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.668+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5569febdbef0 con 0x5569feca4f00 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78060530 con 0x5569feca4f00 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f5b68025020 con 0x5569feca4f00 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 10 client.7452 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 10 client.7452 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 20 client.7452 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8c8930 0x5569fedd8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 4 client.7452 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569febbbdf0 con 0x5569fe8c8930 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b7f400640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8c8930 0x5569fedd8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b7f400640 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8c8930 0x5569fedd8cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3065b96a0b498b95 server_cookie=b375ce7b28b965d3 in_seq=0 out_seq=0 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 10 client.7452 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 1 -- 172.21.15.116:0/4226402059 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569febbbdf0 con 0x5569fe8c8930 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 10 client.7452 handle_command_reply: tid=0 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 client.7452 shutdown 2024-05-14T11:23:42.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 msgr2=0x5569fee25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5b68006620 tx=0x7f5b680015f0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 shutdown_connections 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8c8930 0x5569fedd8cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fedd7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee25310 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/4226402059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fe8a43e0 0x5569fee24dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b6cc00640 0 client.7452 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 >> 172.21.15.116:0/4226402059 conn(0x5569fe8e4400 msgr2=0x5569fee01200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 shutdown_connections 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 1 -- 172.21.15.116:0/4226402059 wait complete. 2024-05-14T11:23:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.672+0000 7f5b8dee8000 20 client.7452 trim_cache size 0 max 16384 2024-05-14T11:23:42.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 Processor -- start 2024-05-14T11:23:42.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- start start 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569feca4f00 0x5569fee344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fee349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80205150 con 0x5569fe8a43e0 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801f9e50 con 0x5569feca4f00 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5b80204b00 con 0x5569fecb7120 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32922/0 (socket says 172.21.15.116:32922) 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 -- 172.21.15.116:0/916463733 learned_addr learned my addr 172.21.15.116:0/916463733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b76a00640 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569feca4f00 0x5569fee344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fee349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 -- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 msgr2=0x5569fee349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fee349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 -- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569feca4f00 msgr2=0x5569fee344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569feca4f00 0x5569fee344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 -- 172.21.15.116:0/916463733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569fedd6010 con 0x5569fe8a43e0 2024-05-14T11:23:42.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b7f400640 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5b70019070 tx=0x7f5b70004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4678d1a2b2340f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b6f400640 1 -- 172.21.15.116:0/916463733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b1040 con 0x5569fe8a43e0 2024-05-14T11:23:42.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b6f400640 1 -- 172.21.15.116:0/916463733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700b8ab0 con 0x5569fe8a43e0 2024-05-14T11:23:42.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b6f400640 1 -- 172.21.15.116:0/916463733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b8e60 con 0x5569fe8a43e0 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- 172.21.15.116:0/916463733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 msgr2=0x5569fee33f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5b70019070 tx=0x7f5b70004740 comp rx=0 tx=0).stop 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- 172.21.15.116:0/916463733 shutdown_connections 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569feca4f00 0x5569fee344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fecb7120 0x5569fee349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/916463733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569fe8a43e0 0x5569fee33f60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- 172.21.15.116:0/916463733 >> 172.21.15.116:0/916463733 conn(0x5569fe8e4400 msgr2=0x5569fee003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- 172.21.15.116:0/916463733 shutdown_connections 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.692+0000 7f5b8dee8000 1 -- 172.21.15.116:0/916463733 wait complete. 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 -- start start 2024-05-14T11:23:42.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 Processor -- start 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fe8a43e0 0x5569fee90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee26bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569fec27730 con 0x5569feca4f00 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569fec22410 con 0x5569fecb7120 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5569fec30500 con 0x5569fe8a43e0 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 10 client.0 _ll_register_callbacks cb 0x7f5b680048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32928/0 (socket says 172.21.15.116:32928) 2024-05-14T11:23:42.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 -- 172.21.15.116:0/628954545 learned_addr learned my addr 172.21.15.116:0/628954545 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b7f400640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fe8a43e0 0x5569fee90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b7fe00640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee26bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 -- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fe8a43e0 msgr2=0x5569fee90b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fe8a43e0 0x5569fee90b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 -- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 msgr2=0x5569fee26bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee26bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 -- 172.21.15.116:0/628954545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569feca4340 con 0x5569feca4f00 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b76a00640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5b68004fd0 tx=0x7f5b68001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b706eb1d7277007 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:23:42.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:23:42.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68015880 con 0x5569feca4f00 2024-05-14T11:23:42.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f5b68015a20 con 0x5569feca4f00 2024-05-14T11:23:42.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b700b8ab0 con 0x5569feca4f00 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f5b68001af0 con 0x5569feca4f00 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.696+0000 7f5b8dee8000 10 client.7458 fetch_fsmap learned FSMap version 74 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5569febbf8e0 con 0x5569feca4f00 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700b8ab0 con 0x5569feca4f00 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f5b68025040 con 0x5569feca4f00 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 10 client.7458 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 10 client.7458 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 20 client.7458 populate_metadata read hostname 'smithi116' 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8b6990 0x5569fee27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 4 client.7458 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569fec30d10 con 0x5569fe8b6990 2024-05-14T11:23:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b7f400640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8b6990 0x5569fee27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 10 client.7458 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b7f400640 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8b6990 0x5569fee27820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8331178c0b6700d server_cookie=9fc3d4a764722ee in_seq=0 out_seq=0 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 1 -- 172.21.15.116:0/628954545 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5569fec30d10 con 0x5569fe8b6990 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 10 client.7458 handle_command_reply: tid=0 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 client.7458 shutdown 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 msgr2=0x5569fee91060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f5b68004fd0 tx=0x7f5b68001090 comp rx=0 tx=0).stop 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 shutdown_connections 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x5569fe8b6990 0x5569fee27820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569fecb7120 0x5569fee26bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569feca4f00 0x5569fee91060 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b6cc00640 0 client.7458 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:23:42.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 --2- 172.21.15.116:0/628954545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5569fe8a43e0 0x5569fee90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 >> 172.21.15.116:0/628954545 conn(0x5569fe8e4400 msgr2=0x5569fee00100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 shutdown_connections 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 1 -- 172.21.15.116:0/628954545 wait complete. 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b8dee8000 20 client.7458 trim_cache size 0 max 16384 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "5a6fa1da-f490-4b7b-bce9-98b653789e4f": { 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/dir", 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "5a6fa1da-f490-4b7b-bce9-98b653789e4f", 2024-05-14T11:23:42.704 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,repair" 2024-05-14T11:23:42.705 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:23:42.705 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:23:42.705 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:23:42.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 msgr2=0x7f5b5c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 0x7f5b5c071940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5b700ae000 tx=0x7f5b700a7000 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 msgr2=0x7f5b800cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7800e4b0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 shutdown_connections 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5b5c06f430 0x7f5b5c071940 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b80072a30 0x7f5b8014b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b800b0770 0x7f5b800cfb90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 --2- 172.21.15.116:0/2365176113 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5b8006f730 0x7f5b800cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 >> 172.21.15.116:0/2365176113 conn(0x7f5b800b4620 msgr2=0x7f5b8015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 shutdown_connections 2024-05-14T11:23:42.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:23:42.700+0000 7f5b86800640 1 -- 172.21.15.116:0/2365176113 wait complete. 2024-05-14T11:23:42.721 INFO:tasks.cephfs.filesystem:scrub status for tag:5a6fa1da-f490-4b7b-bce9-98b653789e4f - {'path': '/dir', 'tag': '5a6fa1da-f490-4b7b-bce9-98b653789e4f', 'options': 'recursive,repair'} 2024-05-14T11:23:43.155 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:23:43.158 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:24:12.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:12.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-14T11:24:12.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 Processor -- start 2024-05-14T11:24:12.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 -- start start 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f60201673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f602006cf10 con 0x7f602006f730 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f602006d1f0 con 0x7f60200b0770 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f6026800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f602006db20 con 0x7f6020164f80 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f602006f730 0x7f602006fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56784/0 (socket says 172.21.15.116:56784) 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601ea00640 1 -- 172.21.15.116:0/1460536505 learned_addr learned my addr 172.21.15.116:0/1460536505 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:12.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601fe00640 1 --2- 172.21.15.116:0/1460536505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f60201673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2274211570 0 0) 0x7f602006cf10 con 0x7f602006f730 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6004003530 con 0x7f602006f730 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 163572064 0 0) 0x7f602006d1f0 con 0x7f60200b0770 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f602006cf10 con 0x7f60200b0770 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601fe00640 1 -- 172.21.15.116:0/1460536505 >> v1:172.21.15.149:6789/0 conn(0x7f60200b0770 legacy=0x7f602006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.883+0000 7f601fe00640 1 -- 172.21.15.116:0/1460536505 >> v1:172.21.15.116:6789/0 conn(0x7f602006f730 legacy=0x7f602006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f601fe00640 1 -- 172.21.15.116:0/1460536505 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602006d4d0 con 0x7f6020164f80 2024-05-14T11:24:12.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f601fe00640 1 --2- 172.21.15.116:0/1460536505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f60201673a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60180097f0 tx=0x7f601801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b83178d090ab04 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180b0070 con 0x7f6020164f80 2024-05-14T11:24:12.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60180ab600 con 0x7f6020164f80 2024-05-14T11:24:12.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f601e000640 1 -- 172.21.15.116:0/1460536505 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180ab9d0 con 0x7f6020164f80 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f6026800640 1 -- 172.21.15.116:0/1460536505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 msgr2=0x7f60201673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.887+0000 7f6026800640 1 --2- 172.21.15.116:0/1460536505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f60201673a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60180097f0 tx=0x7f601801c420 comp rx=0 tx=0).stop 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 -- 172.21.15.116:0/1460536505 shutdown_connections 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 --2- 172.21.15.116:0/1460536505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f60201673a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 -- 172.21.15.116:0/1460536505 >> 172.21.15.116:0/1460536505 conn(0x7f60200b4620 msgr2=0x7f60200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 -- 172.21.15.116:0/1460536505 shutdown_connections 2024-05-14T11:24:12.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 -- 172.21.15.116:0/1460536505 wait complete. 2024-05-14T11:24:12.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 Processor -- start 2024-05-14T11:24:12.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 -- start start 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.891+0000 7f6026800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60200b0770 0x7f60200cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f602014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6020169a70 con 0x7f602006f730 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60201698f0 con 0x7f60200b0770 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6020169bf0 con 0x7f6020164f80 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52702/0 (socket says 172.21.15.116:52702) 2024-05-14T11:24:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 -- 172.21.15.116:0/1894251393 learned_addr learned my addr 172.21.15.116:0/1894251393 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601fe00640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f602014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601ea00640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60200b0770 0x7f60200cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 -- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 msgr2=0x7f602014fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f602014fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 -- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60200b0770 msgr2=0x7f60200cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60200b0770 0x7f60200cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 -- 172.21.15.116:0/1894251393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602015bab0 con 0x7f602006f730 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f601f400640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f601000c730 tx=0x7f601000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6017400640 1 -- 172.21.15.116:0/1894251393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010003b80 con 0x7f602006f730 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6017400640 1 -- 172.21.15.116:0/1894251393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6010003d40 con 0x7f602006f730 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6017400640 1 -- 172.21.15.116:0/1894251393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000aa90 con 0x7f602006f730 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602015bfb0 con 0x7f602006f730 2024-05-14T11:24:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6020150f70 con 0x7f602006f730 2024-05-14T11:24:12.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.895+0000 7f6017400640 1 -- 172.21.15.116:0/1894251393 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f601000ac30 con 0x7f602006f730 2024-05-14T11:24:12.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f6017400640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 0x7f6000071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f6017400640 1 -- 172.21.15.116:0/1894251393 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f601008f350 con 0x7f602006f730 2024-05-14T11:24:12.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 0x7f6000071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 Processor -- start 2024-05-14T11:24:12.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 -- start start 2024-05-14T11:24:12.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 0x7f6000071910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f600c000c10 tx=0x7f600c007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbf2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bbf2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d43b9f7100 con 0x55d43ba78f00 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d43b9afca0 con 0x55d43b6783e0 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d43b9f30c0 con 0x55d43ba8b120 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52708/0 (socket says 172.21.15.116:52708) 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 -- 172.21.15.116:0/979450587 learned_addr learned my addr 172.21.15.116:0/979450587 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601f400640 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbf2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601fe00640 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bbf2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 -- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 msgr2=0x55d43bbf2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbf2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 -- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 msgr2=0x55d43bbf2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bbf2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 -- 172.21.15.116:0/979450587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43b9b4800 con 0x55d43ba78f00 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.899+0000 7f601ea00640 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f600c001720 tx=0x7f600c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13057e4acf3737d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f600be00640 1 -- 172.21.15.116:0/979450587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c025070 con 0x55d43ba78f00 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f600be00640 1 -- 172.21.15.116:0/979450587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c01b050 con 0x55d43ba78f00 2024-05-14T11:24:12.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f600be00640 1 -- 172.21.15.116:0/979450587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c014bd0 con 0x55d43ba78f00 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- 172.21.15.116:0/979450587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 msgr2=0x55d43bbf2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f600c001720 tx=0x7f600c012000 comp rx=0 tx=0).stop 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- 172.21.15.116:0/979450587 shutdown_connections 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bbf2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba78f00 0x55d43bbf2890 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- 172.21.15.116:0/979450587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbf2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- 172.21.15.116:0/979450587 >> 172.21.15.116:0/979450587 conn(0x55d43b6b8400 msgr2=0x55d43bbd5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- 172.21.15.116:0/979450587 shutdown_connections 2024-05-14T11:24:12.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- 172.21.15.116:0/979450587 wait complete. 2024-05-14T11:24:12.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- start start 2024-05-14T11:24:12.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 Processor -- start 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba78f00 0x55d43bbf9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d43bac22a0 con 0x55d43b6783e0 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d43bab2f40 con 0x55d43ba78f00 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d43b9b3080 con 0x55d43ba8b120 2024-05-14T11:24:12.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f602de23000 10 client.0 _ll_register_callbacks cb 0x55d43bb93c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.903+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52720/0 (socket says 172.21.15.116:52720) 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601ea00640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba78f00 0x55d43bbf9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 -- 172.21.15.116:0/1618947811 learned_addr learned my addr 172.21.15.116:0/1618947811 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601fe00640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 -- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 msgr2=0x55d43bbabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 -- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba78f00 msgr2=0x55d43bbf9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba78f00 0x55d43bbf9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 -- 172.21.15.116:0/1618947811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43b9a1860 con 0x55d43b6783e0 2024-05-14T11:24:12.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601f400640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f601000a430 tx=0x7f601005b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7f798f760e4600 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601005b9c0 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f601005ac20 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600c01b050 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f601005fcc0 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 10 client.7470 fetch_fsmap learned FSMap version 74 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d43b9afef0 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c01b050 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f601008d170 con 0x55d43b6783e0 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 10 client.7470 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 10 client.7470 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 20 client.7470 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b69c930 0x55d43bbacd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 4 client.7470 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:12.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d43b98fdf0 con 0x55d43b69c930 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601ea00640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b69c930 0x55d43bbacd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 10 client.7470 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f601ea00640 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b69c930 0x55d43bbacd00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745730903f0fbada server_cookie=4e949e83c41473b6 in_seq=0 out_seq=0 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 1 -- 172.21.15.116:0/1618947811 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d43b98fdf0 con 0x55d43b69c930 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 10 client.7470 handle_command_reply: tid=0 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 client.7470 shutdown 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 msgr2=0x55d43bbf8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f601000a430 tx=0x7f601005b130 comp rx=0 tx=0).stop 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 shutdown_connections 2024-05-14T11:24:12.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b69c930 0x55d43bbacd00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba78f00 0x55d43bbf9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba8b120 0x55d43bbabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f6008c00640 0 client.7470 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 --2- 172.21.15.116:0/1618947811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bbf8dd0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.907+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 >> 172.21.15.116:0/1618947811 conn(0x55d43b6b8400 msgr2=0x55d43bbd5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.911+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 shutdown_connections 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.911+0000 7f602de23000 1 -- 172.21.15.116:0/1618947811 wait complete. 2024-05-14T11:24:12.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.911+0000 7f602de23000 20 client.7470 trim_cache size 0 max 16384 2024-05-14T11:24:12.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 Processor -- start 2024-05-14T11:24:12.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 -- start start 2024-05-14T11:24:12.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bc606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc60c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6020205220 con 0x55d43ba8b120 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60200ceaf0 con 0x55d43b6783e0 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60200cf140 con 0x55d43ba78f00 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52732/0 (socket says 172.21.15.116:52732) 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 -- 172.21.15.116:0/77181024 learned_addr learned my addr 172.21.15.116:0/77181024 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 -- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 msgr2=0x55d43bc60c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601ea00640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc60c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601f400640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bc606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc60c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 -- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 msgr2=0x55d43bc606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bc606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601fe00640 1 -- 172.21.15.116:0/77181024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43bbaa010 con 0x55d43ba8b120 2024-05-14T11:24:12.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601ea00640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc60c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:12.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.927+0000 7f601f400640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bc606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:12.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f601fe00640 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f60180097f0 tx=0x7f6018004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa81b621a9b5ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f600b400640 1 -- 172.21.15.116:0/77181024 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180b0040 con 0x55d43ba8b120 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f600b400640 1 -- 172.21.15.116:0/77181024 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60180bc070 con 0x55d43ba8b120 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f600b400640 1 -- 172.21.15.116:0/77181024 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018003860 con 0x55d43ba8b120 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- 172.21.15.116:0/77181024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 msgr2=0x55d43bc61170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f60180097f0 tx=0x7f6018004740 comp rx=0 tx=0).stop 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- 172.21.15.116:0/77181024 shutdown_connections 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43ba8b120 0x55d43bc61170 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc60c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- 172.21.15.116:0/77181024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43b6783e0 0x55d43bc606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- 172.21.15.116:0/77181024 >> 172.21.15.116:0/77181024 conn(0x55d43b6b8400 msgr2=0x55d43bbd43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- 172.21.15.116:0/77181024 shutdown_connections 2024-05-14T11:24:12.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- 172.21.15.116:0/77181024 wait complete. 2024-05-14T11:24:12.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- start start 2024-05-14T11:24:12.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 Processor -- start 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba8b120 0x55d43bbfabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d43b9fb730 con 0x55d43b6783e0 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d43b9f6410 con 0x55d43ba8b120 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d43ba04500 con 0x55d43ba78f00 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.931+0000 7f602de23000 10 client.0 _ll_register_callbacks cb 0x7f601000c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52742/0 (socket says 172.21.15.116:52742) 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 -- 172.21.15.116:0/4285629926 learned_addr learned my addr 172.21.15.116:0/4285629926 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601fe00640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba8b120 0x55d43bbfabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601ea00640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 -- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 msgr2=0x55d43bc65060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc65060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 -- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba8b120 msgr2=0x55d43bbfabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba8b120 0x55d43bbfabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 -- 172.21.15.116:0/4285629926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43ba78340 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601f400640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6010001930 tx=0x7f601005e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4541d54eb53021d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010064730 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f601005ae20 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60180bc070 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f601000ce40 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 10 client.7476 fetch_fsmap learned FSMap version 74 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d43b9938e0 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60180bc070 con 0x55d43b6783e0 2024-05-14T11:24:12.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6010057330 con 0x55d43b6783e0 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 10 client.7476 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 10 client.7476 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 20 client.7476 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b68a990 0x55d43bbfb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 4 client.7476 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d43ba04d10 con 0x55d43b68a990 2024-05-14T11:24:12.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601ea00640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b68a990 0x55d43bbfb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:12.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f601ea00640 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b68a990 0x55d43bbfb630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b11e87f3d62701 server_cookie=134acaa4d9515823 in_seq=0 out_seq=0 2024-05-14T11:24:12.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 10 client.7476 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:12.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 1 -- 172.21.15.116:0/4285629926 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d43ba04d10 con 0x55d43b68a990 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 10 client.7476 handle_command_reply: tid=0 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 client.7476 shutdown 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 msgr2=0x55d43bc64b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6010001930 tx=0x7f601005e070 comp rx=0 tx=0).stop 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 shutdown_connections 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x55d43b68a990 0x55d43bbfb630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d43ba8b120 0x55d43bbfabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6008c00640 0 client.7476 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d43ba78f00 0x55d43bc65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 --2- 172.21.15.116:0/4285629926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d43b6783e0 0x55d43bc64b20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 >> 172.21.15.116:0/4285629926 conn(0x55d43b6b8400 msgr2=0x55d43bbd4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 shutdown_connections 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 1 -- 172.21.15.116:0/4285629926 wait complete. 2024-05-14T11:24:12.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f602de23000 20 client.7476 trim_cache size 0 max 16384 2024-05-14T11:24:12.942 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:12.942 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:24:12.942 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:24:12.942 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 msgr2=0x7f6000071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 0x7f6000071910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f600c000c10 tx=0x7f600c007ef0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 msgr2=0x7f60200d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.935+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f601000c730 tx=0x7f601000cbf0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 shutdown_connections 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f600006f400 0x7f6000071910 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60200b0770 0x7f60200cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6020164f80 0x7f602014fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 --2- 172.21.15.116:0/1894251393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f602006f730 0x7f60200d03f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:12.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 >> 172.21.15.116:0/1894251393 conn(0x7f60200b4620 msgr2=0x7f60200b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:12.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 shutdown_connections 2024-05-14T11:24:12.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:12.939+0000 7f6026800640 1 -- 172.21.15.116:0/1894251393 wait complete. 2024-05-14T11:24:12.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:24:12.958 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:13.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 Processor -- start 2024-05-14T11:24:13.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 -- start start 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33080d3000 con 0x7f330815eaa0 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33080d1480 con 0x7f33080702b0 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f330ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33080cd830 con 0x7f33080d26b0 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f3307e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f330815eaa0 0x7f3308168860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56794/0 (socket says 172.21.15.116:56794) 2024-05-14T11:24:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f3307e00640 1 -- 172.21.15.116:0/4234539006 learned_addr learned my addr 172.21.15.116:0/4234539006 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f3307400640 1 --2- 172.21.15.116:0/4234539006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f3306000640 1 -- 172.21.15.116:0/4234539006 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576482346 0 0) 0x7f33000014a0 con 0x7f330815eaa0 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.103+0000 7f3306000640 1 -- 172.21.15.116:0/4234539006 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32ec003530 con 0x7f330815eaa0 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3307400640 1 -- 172.21.15.116:0/4234539006 >> v1:172.21.15.149:6789/0 conn(0x7f33080702b0 legacy=0x7f33080706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3307400640 1 -- 172.21.15.116:0/4234539006 >> v1:172.21.15.116:6789/0 conn(0x7f330815eaa0 legacy=0x7f3308168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3307400640 1 -- 172.21.15.116:0/4234539006 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33080d1760 con 0x7f33080d26b0 2024-05-14T11:24:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3307400640 1 --2- 172.21.15.116:0/4234539006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080d2ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32f80097f0 tx=0x7f32f801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f990145cded8f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3306000640 1 -- 172.21.15.116:0/4234539006 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80ab620 con 0x7f33080d26b0 2024-05-14T11:24:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3306000640 1 -- 172.21.15.116:0/4234539006 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f80b0070 con 0x7f33080d26b0 2024-05-14T11:24:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.107+0000 7f3306000640 1 -- 172.21.15.116:0/4234539006 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80ab9e0 con 0x7f33080d26b0 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- 172.21.15.116:0/4234539006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 msgr2=0x7f33080d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 --2- 172.21.15.116:0/4234539006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080d2ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32f80097f0 tx=0x7f32f801c420 comp rx=0 tx=0).stop 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- 172.21.15.116:0/4234539006 shutdown_connections 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 --2- 172.21.15.116:0/4234539006 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080d2ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- 172.21.15.116:0/4234539006 >> 172.21.15.116:0/4234539006 conn(0x7f33080b56e0 msgr2=0x7f33080d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- 172.21.15.116:0/4234539006 shutdown_connections 2024-05-14T11:24:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- 172.21.15.116:0/4234539006 wait complete. 2024-05-14T11:24:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 Processor -- start 2024-05-14T11:24:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.111+0000 7f330ca00640 1 -- start start 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33080702b0 0x7f3308151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33080711e0 con 0x7f330815eaa0 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33081622a0 con 0x7f33080702b0 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f33080ca160 con 0x7f33080d26b0 2024-05-14T11:24:13.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52752/0 (socket says 172.21.15.116:52752) 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 -- 172.21.15.116:0/1378654240 learned_addr learned my addr 172.21.15.116:0/1378654240 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3306a00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307400640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33080702b0 0x7f3308151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 -- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 msgr2=0x7f33080b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 -- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33080702b0 msgr2=0x7f3308151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33080702b0 0x7f3308151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 -- 172.21.15.116:0/1378654240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33080b8a70 con 0x7f330815eaa0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f3307e00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f330000cae0 tx=0x7f330000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000ac40 con 0x7f330815eaa0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330000ade0 con 0x7f330815eaa0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300012620 con 0x7f330815eaa0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33080b6f40 con 0x7f330815eaa0 2024-05-14T11:24:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33080b7500 con 0x7f330815eaa0 2024-05-14T11:24:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3308050880 con 0x7f330815eaa0 2024-05-14T11:24:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.115+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f33000127c0 con 0x7f330815eaa0 2024-05-14T11:24:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.119+0000 7f32ff400640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 0x7f32e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.119+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f330008fb50 con 0x7f330815eaa0 2024-05-14T11:24:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.119+0000 7f3307400640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 0x7f32e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.119+0000 7f3307400640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 0x7f32e4071910 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f32f80af000 tx=0x7f32f80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.123+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f33080d3000 con 0x7f330815eaa0 2024-05-14T11:24:13.294 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:24:13.299 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:24:13.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33080b2be0 con 0x7f330815eaa0 2024-05-14T11:24:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f32ff400640 1 -- 172.21.15.116:0/1378654240 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7f330005c0a0 con 0x7f330815eaa0 2024-05-14T11:24:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:13.316 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:24:13.317 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 msgr2=0x7f32e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 0x7f32e4071910 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f32f80af000 tx=0x7f32f80a7000 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 msgr2=0x7f33080ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f330000cae0 tx=0x7f330000a060 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 shutdown_connections 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f32e406f400 0x7f32e4071910 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f330815eaa0 0x7f33080ca400 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f33080d26b0 0x7f33080b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 --2- 172.21.15.116:0/1378654240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33080702b0 0x7f3308151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 >> 172.21.15.116:0/1378654240 conn(0x7f33080b56e0 msgr2=0x7f33080cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 shutdown_connections 2024-05-14T11:24:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.315+0000 7f330ca00640 1 -- 172.21.15.116:0/1378654240 wait complete. 2024-05-14T11:24:13.339 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-14T11:24:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.531+0000 7f75df200640 1 Processor -- start 2024-05-14T11:24:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.531+0000 7f75df200640 1 -- start start 2024-05-14T11:24:13.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75df200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f75d80d1670 0x7f75d80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75df200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d806ff20 con 0x7f75d815db30 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75df200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d80d1fc0 con 0x7f75d806c700 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75df200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80ae100 con 0x7f75d80d1670 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f75d80d1670 0x7f75d80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f75d80d1670 0x7f75d80d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52754/0 (socket says 172.21.15.116:52754) 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 -- 172.21.15.116:0/3914246287 learned_addr learned my addr 172.21.15.116:0/3914246287 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75d7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f75d815db30 0x7f75d80aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56808/0 (socket says 172.21.15.116:56808) 2024-05-14T11:24:13.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 -- 172.21.15.116:0/3914246287 >> v1:172.21.15.149:6789/0 conn(0x7f75d806c700 legacy=0x7f75d806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 -- 172.21.15.116:0/3914246287 >> v1:172.21.15.116:6789/0 conn(0x7f75d815db30 legacy=0x7f75d80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75cf400640 1 -- 172.21.15.116:0/3914246287 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d8070470 con 0x7f75d80d1670 2024-05-14T11:24:13.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.535+0000 7f75d7400640 1 -- 172.21.15.116:0/3914246287 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466758345 0 0) 0x7f75d806ff20 con 0x7f75d815db30 2024-05-14T11:24:13.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75cf400640 1 --2- 172.21.15.116:0/3914246287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d80d1670 0x7f75d80d1a80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f75d80ae280 tx=0x7f75c001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b124e1b3942adb48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75d7400640 1 -- 172.21.15.116:0/3914246287 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00b0070 con 0x7f75d80d1670 2024-05-14T11:24:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75d7400640 1 -- 172.21.15.116:0/3914246287 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c00ab600 con 0x7f75d80d1670 2024-05-14T11:24:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75d7400640 1 -- 172.21.15.116:0/3914246287 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00ab9d0 con 0x7f75d80d1670 2024-05-14T11:24:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 -- 172.21.15.116:0/3914246287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d80d1670 msgr2=0x7f75d80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 --2- 172.21.15.116:0/3914246287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d80d1670 0x7f75d80d1a80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f75d80ae280 tx=0x7f75c001c240 comp rx=0 tx=0).stop 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 -- 172.21.15.116:0/3914246287 shutdown_connections 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 --2- 172.21.15.116:0/3914246287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d80d1670 0x7f75d80d1a80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 -- 172.21.15.116:0/3914246287 >> 172.21.15.116:0/3914246287 conn(0x7f75d80b4620 msgr2=0x7f75d80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 -- 172.21.15.116:0/3914246287 shutdown_connections 2024-05-14T11:24:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.539+0000 7f75df200640 1 -- 172.21.15.116:0/3914246287 wait complete. 2024-05-14T11:24:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 Processor -- start 2024-05-14T11:24:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- start start 2024-05-14T11:24:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 0x7f75d814a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 0x7f75d814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f75d816bf90 con 0x7f75d806c700 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75d815df40 con 0x7f75d815db30 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f75d816c110 con 0x7f75d80d1670 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52766/0 (socket says 172.21.15.116:52766) 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 -- 172.21.15.116:0/1181352759 learned_addr learned my addr 172.21.15.116:0/1181352759 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75dd200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 0x7f75d814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75cf400640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 0x7f75d814a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 -- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 msgr2=0x7f75d814a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 0x7f75d814a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 -- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 msgr2=0x7f75d814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 0x7f75d814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 -- 172.21.15.116:0/1181352759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d8152810 con 0x7f75d806c700 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75dd200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 0x7f75d814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75cf400640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 0x7f75d814a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c430 tx=0x7f75c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d4c00640 1 -- 172.21.15.116:0/1181352759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80074e0 con 0x7f75d806c700 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d4c00640 1 -- 172.21.15.116:0/1181352759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c80047f0 con 0x7f75d806c700 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d809d860 con 0x7f75d806c700 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75d4c00640 1 -- 172.21.15.116:0/1181352759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004be0 con 0x7f75d806c700 2024-05-14T11:24:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.543+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d809de20 con 0x7f75d806c700 2024-05-14T11:24:13.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75d4c00640 1 -- 172.21.15.116:0/1181352759 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75c8011070 con 0x7f75d806c700 2024-05-14T11:24:13.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75d4c00640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 0x7f75b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75d4c00640 1 -- 172.21.15.116:0/1181352759 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f75c808f170 con 0x7f75d806c700 2024-05-14T11:24:13.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75cf400640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 0x7f75b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75e681d000 1 Processor -- start 2024-05-14T11:24:13.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75e681d000 1 -- start start 2024-05-14T11:24:13.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.547+0000 7f75cf400640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 0x7f75b4071940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75c00ad000 tx=0x7f75c00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace8aebd0 0x557acea11e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ace8171a0 con 0x557ace498580 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ace7cfb80 con 0x557ace89e2d0 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557ace813160 con 0x557ace8aebd0 2024-05-14T11:24:13.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52776/0 (socket says 172.21.15.116:52776) 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 -- 172.21.15.116:0/3199466607 learned_addr learned my addr 172.21.15.116:0/3199466607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75d7e00640 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace8aebd0 0x557acea11e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75cf400640 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 -- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace8aebd0 msgr2=0x557acea11e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace8aebd0 0x557acea11e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 -- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 msgr2=0x557acea123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea123b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 -- 172.21.15.116:0/3199466607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace823630 con 0x557ace498580 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75dd200640 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009fa0 tx=0x7f75d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dde894b42b3d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75ccc00640 1 -- 172.21.15.116:0/3199466607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0003980 con 0x557ace498580 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75ccc00640 1 -- 172.21.15.116:0/3199466607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0003b20 con 0x557ace498580 2024-05-14T11:24:13.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75ccc00640 1 -- 172.21.15.116:0/3199466607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0004020 con 0x557ace498580 2024-05-14T11:24:13.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- 172.21.15.116:0/3199466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 msgr2=0x557acea128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009fa0 tx=0x7f75d000c890 comp rx=0 tx=0).stop 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- 172.21.15.116:0/3199466607 shutdown_connections 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace498580 0x557acea128f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 --2- 172.21.15.116:0/3199466607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace8aebd0 0x557acea11e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- 172.21.15.116:0/3199466607 >> 172.21.15.116:0/3199466607 conn(0x557ace4d7760 msgr2=0x557ace9f4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- 172.21.15.116:0/3199466607 shutdown_connections 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- 172.21.15.116:0/3199466607 wait complete. 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 -- start start 2024-05-14T11:24:13.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.551+0000 7f75e681d000 1 Processor -- start 2024-05-14T11:24:13.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ace7b6ab0 con 0x557ace8aebd0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ace7cbf20 con 0x557ace498580 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557ace876da0 con 0x557ace89e2d0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 10 client.0 _ll_register_callbacks cb 0x557ace9b2f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75cf400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52780/0 (socket says 172.21.15.116:52780) 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 -- 172.21.15.116:0/215617926 learned_addr learned my addr 172.21.15.116:0/215617926 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea194e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36076/0 (socket says 172.21.15.116:36076) 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 -- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 msgr2=0x557acea19a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea19a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 -- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 msgr2=0x557acea194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 -- 172.21.15.116:0/215617926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace7d5ce0 con 0x557ace8aebd0 2024-05-14T11:24:13.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75dd200640 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75d0017040 tx=0x7f75d0004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3aaf83bac80a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0025020 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f75d000ace0 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d0003b20 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f75d002f050 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 10 client.7497 fetch_fsmap learned FSMap version 74 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x557ace7c1740 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0003b20 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f75d0032020 con 0x557ace8aebd0 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 10 client.7497 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 10 client.7497 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 20 client.7497 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4bcad0 0x557ace9cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 4 client.7497 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:13.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.555+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ace7afe60 con 0x557ace4bcad0 2024-05-14T11:24:13.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75d7e00640 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4bcad0 0x557ace9cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75c5600640 10 client.7497 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:13.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75d7e00640 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4bcad0 0x557ace9cc040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a683997379af219 server_cookie=956f3f3c1f86602b in_seq=0 out_seq=0 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75c5600640 1 -- 172.21.15.116:0/215617926 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557ace7afe60 con 0x557ace4bcad0 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75c5600640 10 client.7497 handle_command_reply: tid=0 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 client.7497 shutdown 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 msgr2=0x557ace9cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f75d0017040 tx=0x7f75d0004a10 comp rx=0 tx=0).stop 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 shutdown_connections 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4bcad0 0x557ace9cc040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557ace9cb2d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 --2- 172.21.15.116:0/215617926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75c5600640 0 client.7497 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:13.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 >> 172.21.15.116:0/215617926 conn(0x557ace4d7760 msgr2=0x557ace9f3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 shutdown_connections 2024-05-14T11:24:13.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 1 -- 172.21.15.116:0/215617926 wait complete. 2024-05-14T11:24:13.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.559+0000 7f75e681d000 20 client.7497 trim_cache size 0 max 16384 2024-05-14T11:24:13.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.571+0000 7f75e681d000 1 Processor -- start 2024-05-14T11:24:13.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.571+0000 7f75e681d000 1 -- start start 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea29680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea29bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8203690 con 0x557ace8aebd0 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75d809cec0 con 0x557ace498580 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f75d8203040 con 0x557ace89e2d0 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52792/0 (socket says 172.21.15.116:52792) 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 -- 172.21.15.116:0/1435009187 learned_addr learned my addr 172.21.15.116:0/1435009187 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 -- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 msgr2=0x557acea29bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea29bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 -- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 msgr2=0x557acea29680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea29680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 -- 172.21.15.116:0/1435009187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace7cfdd0 con 0x557ace8aebd0 2024-05-14T11:24:13.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75dd200640 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f75d000a100 tx=0x7f75d0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d67fb2735a8f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75c7e00640 1 -- 172.21.15.116:0/1435009187 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0025030 con 0x557ace8aebd0 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75c7e00640 1 -- 172.21.15.116:0/1435009187 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0003b80 con 0x557ace8aebd0 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75c7e00640 1 -- 172.21.15.116:0/1435009187 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d000a900 con 0x557ace8aebd0 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- 172.21.15.116:0/1435009187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 msgr2=0x557acea1b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f75d000a100 tx=0x7f75d0020000 comp rx=0 tx=0).stop 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- 172.21.15.116:0/1435009187 shutdown_connections 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea1b310 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace89e2d0 0x557acea29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 --2- 172.21.15.116:0/1435009187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace498580 0x557acea29680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- 172.21.15.116:0/1435009187 >> 172.21.15.116:0/1435009187 conn(0x557ace4d7760 msgr2=0x557ace9cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- 172.21.15.116:0/1435009187 shutdown_connections 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- 172.21.15.116:0/1435009187 wait complete. 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.575+0000 7f75e681d000 1 -- start start 2024-05-14T11:24:13.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 Processor -- start 2024-05-14T11:24:13.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace498580 0x557acea1e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ace7c05f0 con 0x557ace8aebd0 2024-05-14T11:24:13.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ace811e80 con 0x557ace89e2d0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x557ace7d2f60 con 0x557ace498580 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 10 client.0 _ll_register_callbacks cb 0x7f75c00ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52808/0 (socket says 172.21.15.116:52808) 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 -- 172.21.15.116:0/1159457490 learned_addr learned my addr 172.21.15.116:0/1159457490 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace498580 0x557acea1e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75cf400640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 -- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace498580 msgr2=0x557acea1e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace498580 0x557acea1e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 -- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 msgr2=0x557acea443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 -- 172.21.15.116:0/1159457490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace897c40 con 0x557ace8aebd0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75cf400640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75dd200640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f75d000c8b0 tx=0x7f75d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d9235db9de5db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0040060 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f75d0038040 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d0003b80 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f75d0046050 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 10 client.7503 fetch_fsmap learned FSMap version 74 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x557ace4d03c0 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0003b80 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f75d0049020 con 0x557ace8aebd0 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 10 client.7503 fetch_fsmap finished waiting for FSMap version 74 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 10 client.7503 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 20 client.7503 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4aab30 0x557acea88680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 4 client.7503 mds_command: new command op to 7027 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 --> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x557ace824db0 con 0x557ace4aab30 2024-05-14T11:24:13.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.579+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4aab30 0x557acea88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75d7e00640 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4aab30 0x557acea88680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee37efff7c27695 server_cookie=2119a51a30e6acc4 in_seq=0 out_seq=0 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75c5600640 10 client.7503 ms_handle_connect on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75c5600640 1 -- 172.21.15.116:0/1159457490 <== mds.0 v2:172.21.15.149:6833/2777717906 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ace824db0 con 0x557ace4aab30 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75c5600640 10 client.7503 handle_command_reply: tid=0 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 client.7503 shutdown 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 msgr2=0x557acea44920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f75d000c8b0 tx=0x7f75d0013000 comp rx=0 tx=0).stop 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 shutdown_connections 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:6833/2777717906,v1:172.21.15.149:6835/2777717906] conn(0x557ace4aab30 0x557acea88680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ace89e2d0 0x557acea443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ace8aebd0 0x557acea44920 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75c5600640 0 client.7503 ms_handle_reset on v2:172.21.15.149:6833/2777717906 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 --2- 172.21.15.116:0/1159457490 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x557ace498580 0x557acea1e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 >> 172.21.15.116:0/1159457490 conn(0x557ace4d7760 msgr2=0x557acea28670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 shutdown_connections 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 1 -- 172.21.15.116:0/1159457490 wait complete. 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75e681d000 20 client.7503 trim_cache size 0 max 16384 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:13.587 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-05-14T11:24:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 msgr2=0x7f75b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 0x7f75b4071940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75c00ad000 tx=0x7f75c00a6000 comp rx=0 tx=0).stop 2024-05-14T11:24:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 msgr2=0x7f75d8151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c430 tx=0x7f75c800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 shutdown_connections 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f75b406f430 0x7f75b4071940 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75d815db30 0x7f75d814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f75d80d1670 0x7f75d814a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 --2- 172.21.15.116:0/1181352759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f75d806c700 0x7f75d8151550 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 >> 172.21.15.116:0/1181352759 conn(0x7f75d80b4620 msgr2=0x7f75d80686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 shutdown_connections 2024-05-14T11:24:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.583+0000 7f75df200640 1 -- 172.21.15.116:0/1181352759 wait complete. 2024-05-14T11:24:13.604 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-14T11:24:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 Processor -- start 2024-05-14T11:24:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 -- start start 2024-05-14T11:24:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc80b2f40 con 0x7f0bc80d1c20 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc80b3220 con 0x7f0bc80ce4c0 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bcd400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc80b3500 con 0x7f0bc8160080 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc7200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0bc80d1c20 0x7f0bc80b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56810/0 (socket says 172.21.15.116:56810) 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc7200640 1 -- 172.21.15.116:0/454890815 learned_addr learned my addr 172.21.15.116:0/454890815 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc6800640 1 --2- 172.21.15.116:0/454890815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781653385 0 0) 0x7f0bc80b2f40 con 0x7f0bc80d1c20 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bac003530 con 0x7f0bc80d1c20 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc6800640 1 -- 172.21.15.116:0/454890815 >> v1:172.21.15.149:6789/0 conn(0x7f0bc80ce4c0 legacy=0x7f0bc80d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc6800640 1 -- 172.21.15.116:0/454890815 >> v1:172.21.15.116:6789/0 conn(0x7f0bc80d1c20 legacy=0x7f0bc80b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442136595 0 0) 0x7f0bc80b3220 con 0x7f0bc80ce4c0 2024-05-14T11:24:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.751+0000 7f0bc6800640 1 -- 172.21.15.116:0/454890815 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc80b37e0 con 0x7f0bc8160080 2024-05-14T11:24:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bc6800640 1 --2- 172.21.15.116:0/454890815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc80cdf80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80097f0 tx=0x7f0bb801c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2915c0c081ece696 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80b1070 con 0x7f0bc8160080 2024-05-14T11:24:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb80ac410 con 0x7f0bc8160080 2024-05-14T11:24:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bc5400640 1 -- 172.21.15.116:0/454890815 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80ac7c0 con 0x7f0bc8160080 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 -- 172.21.15.116:0/454890815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 msgr2=0x7f0bc80cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 --2- 172.21.15.116:0/454890815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc80cdf80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80097f0 tx=0x7f0bb801c390 comp rx=0 tx=0).stop 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 -- 172.21.15.116:0/454890815 shutdown_connections 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 --2- 172.21.15.116:0/454890815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc80cdf80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 -- 172.21.15.116:0/454890815 >> 172.21.15.116:0/454890815 conn(0x7f0bc80d0770 msgr2=0x7f0bc80d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 -- 172.21.15.116:0/454890815 shutdown_connections 2024-05-14T11:24:13.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.755+0000 7f0bcd400640 1 -- 172.21.15.116:0/454890815 wait complete. 2024-05-14T11:24:13.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 Processor -- start 2024-05-14T11:24:13.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 -- start start 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc80ce4c0 0x7f0bc8157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc816dd50 con 0x7f0bc815b060 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc806ffe0 con 0x7f0bc80ce4c0 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bcd400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc816ded0 con 0x7f0bc8160080 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bc7200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bc7200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52818/0 (socket says 172.21.15.116:52818) 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bc7200640 1 -- 172.21.15.116:0/4092558506 learned_addr learned my addr 172.21.15.116:0/4092558506 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bc6800640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc80ce4c0 0x7f0bc8157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.759+0000 7f0bc5e00640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 -- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 msgr2=0x7f0bc8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 -- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc80ce4c0 msgr2=0x7f0bc8157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc80ce4c0 0x7f0bc8157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 -- 172.21.15.116:0/4092558506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc815bee0 con 0x7f0bc815b060 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc5e00640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bc7200640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000cad0 tx=0x7f0bc000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000ac30 con 0x7f0bc815b060 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc000add0 con 0x7f0bc815b060 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc815c1c0 con 0x7f0bc815b060 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0012600 con 0x7f0bc815b060 2024-05-14T11:24:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc8158450 con 0x7f0bc815b060 2024-05-14T11:24:13.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc815c6e0 con 0x7f0bc815b060 2024-05-14T11:24:13.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.763+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bc00127a0 con 0x7f0bc815b060 2024-05-14T11:24:13.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.767+0000 7f0bbea00640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 0x7f0ba8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:13.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.767+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f0bc008fc00 con 0x7f0bc815b060 2024-05-14T11:24:13.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.767+0000 7f0bc6800640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 0x7f0ba8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:13.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.767+0000 7f0bc6800640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 0x7f0ba8071940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80ae000 tx=0x7f0bb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.771+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0bc815c6e0 con 0x7f0bc815b060 2024-05-14T11:24:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.959+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0bc806a9c0 con 0x7f0bc815b060 2024-05-14T11:24:13.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.959+0000 7f0bbea00640 1 -- 172.21.15.116:0/4092558506 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0bc005c720 con 0x7f0bc815b060 2024-05-14T11:24:13.962 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:13.962 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 msgr2=0x7f0ba8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 0x7f0ba8071940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80ae000 tx=0x7f0bb8008040 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 msgr2=0x7f0bc815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000cad0 tx=0x7f0bc000cf90 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 shutdown_connections 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ba806f430 0x7f0ba8071940 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bc815b060 0x7f0bc815b470 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0bc8160080 0x7f0bc8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 --2- 172.21.15.116:0/4092558506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc80ce4c0 0x7f0bc8157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 >> 172.21.15.116:0/4092558506 conn(0x7f0bc80d0770 msgr2=0x7f0bc815c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 shutdown_connections 2024-05-14T11:24:13.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:13.963+0000 7f0bcd400640 1 -- 172.21.15.116:0/4092558506 wait complete. 2024-05-14T11:24:13.984 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-14T11:24:13.984 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:24:13.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:24:14.062 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:24:14.143 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:24:14.143 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:24:14.143 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:14.143 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:14.152 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:14.152 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:14.202 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:24:14.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 Processor -- start 2024-05-14T11:24:14.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 -- start start 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae6c164220 0x7fae6c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fae6c06ce10 0x7fae6c16aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6c0d1fc0 con 0x7fae6c0d1bb0 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c1600a0 con 0x7fae6c164220 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae73600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c06f5d0 con 0x7fae6c06ce10 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae70c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae6c164220 0x7fae6c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae70c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae6c164220 0x7fae6c0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52826/0 (socket says 172.21.15.116:52826) 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae70c00640 1 -- 172.21.15.116:0/3676522321 learned_addr learned my addr 172.21.15.116:0/3676522321 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.391+0000 7fae71600640 1 --2- 172.21.15.116:0/3676522321 >> v2:172.21.15.149:3300/0 conn(0x7fae6c06ce10 0x7fae6c16aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae70c00640 1 -- 172.21.15.116:0/3676522321 >> v2:172.21.15.149:3300/0 conn(0x7fae6c06ce10 msgr2=0x7fae6c16aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:14.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae70c00640 1 --2- 172.21.15.116:0/3676522321 >> v2:172.21.15.149:3300/0 conn(0x7fae6c06ce10 0x7fae6c16aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae70c00640 1 -- 172.21.15.116:0/3676522321 >> v1:172.21.15.149:6789/0 conn(0x7fae6c0d1bb0 legacy=0x7fae6c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:14.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae70c00640 1 -- 172.21.15.116:0/3676522321 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c06f8b0 con 0x7fae6c164220 2024-05-14T11:24:14.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae70c00640 1 --2- 172.21.15.116:0/3676522321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c0d1670 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fae6c070c50 tx=0x7fae5c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcdc6fe9651c9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae6be00640 1 -- 172.21.15.116:0/3676522321 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0ab660 con 0x7fae6c164220 2024-05-14T11:24:14.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae6be00640 1 -- 172.21.15.116:0/3676522321 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c0b0070 con 0x7fae6c164220 2024-05-14T11:24:14.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae6be00640 1 -- 172.21.15.116:0/3676522321 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0aba20 con 0x7fae6c164220 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae73600640 1 -- 172.21.15.116:0/3676522321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 msgr2=0x7fae6c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.395+0000 7fae73600640 1 --2- 172.21.15.116:0/3676522321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c0d1670 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fae6c070c50 tx=0x7fae5c01c460 comp rx=0 tx=0).stop 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- 172.21.15.116:0/3676522321 shutdown_connections 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 --2- 172.21.15.116:0/3676522321 >> v2:172.21.15.149:3300/0 conn(0x7fae6c06ce10 0x7fae6c16aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 --2- 172.21.15.116:0/3676522321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c0d1670 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- 172.21.15.116:0/3676522321 >> 172.21.15.116:0/3676522321 conn(0x7fae6c0b4620 msgr2=0x7fae6c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- 172.21.15.116:0/3676522321 shutdown_connections 2024-05-14T11:24:14.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- 172.21.15.116:0/3676522321 wait complete. 2024-05-14T11:24:14.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 Processor -- start 2024-05-14T11:24:14.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- start start 2024-05-14T11:24:14.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c06ce10 0x7fae6c14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 0x7fae6c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0af2f0 con 0x7fae6c164220 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c073650 con 0x7fae6c06ce10 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.399+0000 7fae73600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c164630 con 0x7fae6c0d1bb0 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52830/0 (socket says 172.21.15.116:52830) 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae70c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c06ce10 0x7fae6c14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 -- 172.21.15.116:0/3542532853 learned_addr learned my addr 172.21.15.116:0/3542532853 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae63e00640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 0x7fae6c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 -- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 msgr2=0x7fae6c152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 0x7fae6c152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 -- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c06ce10 msgr2=0x7fae6c14e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c06ce10 0x7fae6c14e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 -- 172.21.15.116:0/3542532853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c153530 con 0x7fae6c164220 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae63e00640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 0x7fae6c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:14.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae71600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fae64009fd0 tx=0x7fae6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64014420 con 0x7fae6c164220 2024-05-14T11:24:14.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64019070 con 0x7fae6c164220 2024-05-14T11:24:14.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64014800 con 0x7fae6c164220 2024-05-14T11:24:14.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c0b0f30 con 0x7fae6c164220 2024-05-14T11:24:14.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c0b11d0 con 0x7fae6c164220 2024-05-14T11:24:14.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c0af640 con 0x7fae6c164220 2024-05-14T11:24:14.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.403+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fae640149a0 con 0x7fae6c164220 2024-05-14T11:24:14.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.407+0000 7fae69600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 0x7fae48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.407+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fae6408fd90 con 0x7fae6c164220 2024-05-14T11:24:14.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.407+0000 7fae70c00640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 0x7fae48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.407+0000 7fae70c00640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 0x7fae48071940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0af000 tx=0x7fae5c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.411+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fae6405c930 con 0x7fae6c164220 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.579+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae6c06ecf0 con 0x7fae6c164220 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.579+0000 7fae69600640 1 -- 172.21.15.116:0/3542532853 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v88) ==== 80+0+246 (secure 0 0 0) 0x7fae6405c230 con 0x7fae6c164220 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6833/1421948627 2024-05-15T11:23:34.960706+0000 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6835/1421948627 2024-05-15T11:23:34.960706+0000 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6835/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:24:14.581 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.149:6833/536312712 2024-05-15T11:22:16.806991+0000 2024-05-14T11:24:14.582 INFO:teuthology.orchestra.run.smithi116.stderr:listed 4 entries 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 msgr2=0x7fae48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 0x7fae48071940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0af000 tx=0x7fae5c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 msgr2=0x7fae6c15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fae64009fd0 tx=0x7fae6400e4b0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 shutdown_connections 2024-05-14T11:24:14.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fae4806f430 0x7fae48071940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6c164220 0x7fae6c15d3d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fae6c0d1bb0 0x7fae6c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 --2- 172.21.15.116:0/3542532853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c06ce10 0x7fae6c14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 >> 172.21.15.116:0/3542532853 conn(0x7fae6c0b4620 msgr2=0x7fae6c15e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 shutdown_connections 2024-05-14T11:24:14.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.583+0000 7fae73600640 1 -- 172.21.15.116:0/3542532853 wait complete. 2024-05-14T11:24:14.603 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:24:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:14.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:14.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[45547]: fuse finished with error 0 and tester_r 0 2024-05-14T11:24:14.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:24:14.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:14.624 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:24:14.694 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:24:14.694 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:24:14.694 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:14.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:24:14.704 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:24:14.705 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:14.705 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:14.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:24:14.742 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:24:14.743 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:24:14.743 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:14.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:24:14.756 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:14.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 Processor -- start 2024-05-14T11:24:14.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 -- start start 2024-05-14T11:24:14.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9fc0d1e60 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa9fc0b2f40 con 0x7fa9fc160080 2024-05-14T11:24:14.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7faa02600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc0b3220 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9fb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9fb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:36134/0 (socket says 172.21.15.116:36134) 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9fb400640 1 -- 172.21.15.116:0/1153656333 learned_addr learned my addr 172.21.15.116:0/1153656333 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:60732/0 (socket says 172.21.15.116:60732) 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 -- 172.21.15.116:0/1153656333 >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 msgr2=0x7fa9fc0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 --2- 172.21.15.116:0/1153656333 >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 -- 172.21.15.116:0/1153656333 >> v1:172.21.15.149:6789/0 conn(0x7fa9fc0d1a50 legacy=0x7fa9fc0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9faa00640 1 -- 172.21.15.116:0/1153656333 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc0b3500 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.935+0000 7fa9fb400640 1 --2- 172.21.15.116:0/1153656333 >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:14.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7fa9faa00640 1 --2- 172.21.15.116:0/1153656333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876ab82a41191096 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7fa9fa000640 1 -- 172.21.15.116:0/1153656333 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80b0070 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7fa9fa000640 1 -- 172.21.15.116:0/1153656333 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e8004230 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7fa9fa000640 1 -- 172.21.15.116:0/1153656333 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8003620 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 -- 172.21.15.116:0/1153656333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 msgr2=0x7fa9fc0d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 --2- 172.21.15.116:0/1153656333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e801c350 comp rx=0 tx=0).stop 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 -- 172.21.15.116:0/1153656333 shutdown_connections 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 --2- 172.21.15.116:0/1153656333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc0d1510 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 --2- 172.21.15.116:0/1153656333 >> v2:172.21.15.149:3300/0 conn(0x7fa9fc160080 0x7fa9fc0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.939+0000 7faa02600640 1 -- 172.21.15.116:0/1153656333 >> 172.21.15.116:0/1153656333 conn(0x7fa9fc0d0770 msgr2=0x7fa9fc0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- 172.21.15.116:0/1153656333 shutdown_connections 2024-05-14T11:24:14.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- 172.21.15.116:0/1153656333 wait complete. 2024-05-14T11:24:14.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 Processor -- start 2024-05-14T11:24:14.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- start start 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc1508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 0x7fa9fc071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc16cd40 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc06ffe0 con 0x7fa9fc160080 2024-05-14T11:24:14.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7faa02600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9fc16cec0 con 0x7fa9fc0ce4c0 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7fa9faa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7fa9faa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52838/0 (socket says 172.21.15.116:52838) 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.943+0000 7fa9faa00640 1 -- 172.21.15.116:0/3051099527 learned_addr learned my addr 172.21.15.116:0/3051099527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9fb400640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc1508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9fbe00640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 0x7fa9fc071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 -- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 msgr2=0x7fa9fc1508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc1508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 -- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 msgr2=0x7fa9fc071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 0x7fa9fc071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 -- 172.21.15.116:0/3051099527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc162050 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9fbe00640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 0x7fa9fc071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:14.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9faa00640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e80190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80b9020 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e8003df0 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc162560 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80b0040 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc162b20 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc06a9c0 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.947+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9e80bc020 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.951+0000 7fa9f3400640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 0x7fa9dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.951+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fa9e81290e0 con 0x7fa9fc0d1a50 2024-05-14T11:24:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.951+0000 7fa9fb400640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 0x7fa9dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.951+0000 7fa9fb400640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 0x7fa9dc071940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006cd0 tx=0x7fa9ec009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:14.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:14.955+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa9e80c3070 con 0x7fa9fc0d1a50 2024-05-14T11:24:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.143+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9fc0b9590 con 0x7fa9fc0d1a50 2024-05-14T11:24:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7fa9f3400640 1 -- 172.21.15.116:0/3051099527 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7fa9e8016090 con 0x7fa9fc0d1a50 2024-05-14T11:24:15.147 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:15.147 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:24:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 msgr2=0x7fa9dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 0x7fa9dc071940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006cd0 tx=0x7fa9ec009240 comp rx=0 tx=0).stop 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 msgr2=0x7fa9fc161b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e80190f0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 shutdown_connections 2024-05-14T11:24:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa9dc06f430 0x7fa9dc071940 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc160080 0x7fa9fc071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9fc0d1a50 0x7fa9fc161b10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 --2- 172.21.15.116:0/3051099527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9fc0ce4c0 0x7fa9fc1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 >> 172.21.15.116:0/3051099527 conn(0x7fa9fc0d0770 msgr2=0x7fa9fc0cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 shutdown_connections 2024-05-14T11:24:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.147+0000 7faa02600640 1 -- 172.21.15.116:0/3051099527 wait complete. 2024-05-14T11:24:15.170 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:15.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.315+0000 7f52c6a00640 1 Processor -- start 2024-05-14T11:24:15.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.315+0000 7f52c6a00640 1 -- start start 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52c006f730 0x7f52c006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c6a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c6a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c0070080 con 0x7f52c00b0770 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c6a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f52c006db20 con 0x7f52c006f730 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c6a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52c006cf10 con 0x7f52c0072a30 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bf400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52c006f730 0x7f52c006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bfe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f52c00b0770 0x7f52c006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56818/0 (socket says 172.21.15.116:56818) 2024-05-14T11:24:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bfe00640 1 -- 172.21.15.116:0/3400588215 learned_addr learned my addr 172.21.15.116:0/3400588215 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c4a00640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bf400640 1 -- 172.21.15.116:0/3400588215 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 msgr2=0x7f52c0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bf400640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bf400640 1 -- 172.21.15.116:0/3400588215 >> v1:172.21.15.116:6789/0 conn(0x7f52c00b0770 legacy=0x7f52c006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bea00640 1 -- 172.21.15.116:0/3400588215 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553429615 0 0) 0x7f52b0001120 con 0x7f52c00b0770 2024-05-14T11:24:15.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52bf400640 1 -- 172.21.15.116:0/3400588215 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c006d1f0 con 0x7f52c006f730 2024-05-14T11:24:15.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.319+0000 7f52c4a00640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:15.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52bf400640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c006fb40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0097c0 tx=0x7f52ac01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f0220950d87d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52bea00640 1 -- 172.21.15.116:0/3400588215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0b2070 con 0x7f52c006f730 2024-05-14T11:24:15.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52bea00640 1 -- 172.21.15.116:0/3400588215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac0a9070 con 0x7f52c006f730 2024-05-14T11:24:15.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52bea00640 1 -- 172.21.15.116:0/3400588215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0ad880 con 0x7f52c006f730 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- 172.21.15.116:0/3400588215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 msgr2=0x7f52c006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c006fb40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0097c0 tx=0x7f52ac01c3e0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- 172.21.15.116:0/3400588215 shutdown_connections 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 --2- 172.21.15.116:0/3400588215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c006fb40 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- 172.21.15.116:0/3400588215 >> 172.21.15.116:0/3400588215 conn(0x7f52c00b4620 msgr2=0x7f52c00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- 172.21.15.116:0/3400588215 shutdown_connections 2024-05-14T11:24:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- 172.21.15.116:0/3400588215 wait complete. 2024-05-14T11:24:15.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 Processor -- start 2024-05-14T11:24:15.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.323+0000 7f52c6a00640 1 -- start start 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 0x7f52c015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0073500 con 0x7f52c006f730 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0073380 con 0x7f52c00b0770 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0073680 con 0x7f52c0072a30 2024-05-14T11:24:15.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52854/0 (socket says 172.21.15.116:52854) 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 -- 172.21.15.116:0/1724140204 learned_addr learned my addr 172.21.15.116:0/1724140204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bf400640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c4a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 0x7f52c015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 -- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 msgr2=0x7f52c015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 -- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 msgr2=0x7f52c015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 0x7f52c015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 -- 172.21.15.116:0/1724140204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c00cd310 con 0x7f52c006f730 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c4a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 0x7f52c015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52bfe00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c430 tx=0x7f52b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00074e0 con 0x7f52c006f730 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b00047f0 con 0x7f52c006f730 2024-05-14T11:24:15.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c00ca4a0 con 0x7f52c006f730 2024-05-14T11:24:15.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0007680 con 0x7f52c006f730 2024-05-14T11:24:15.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c00caa60 con 0x7f52c006f730 2024-05-14T11:24:15.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.327+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c015e5a0 con 0x7f52c006f730 2024-05-14T11:24:15.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.331+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52b0011070 con 0x7f52c006f730 2024-05-14T11:24:15.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.331+0000 7f52b7e00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 0x7f52a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.331+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f52b008f1f0 con 0x7f52c006f730 2024-05-14T11:24:15.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.331+0000 7f52bf400640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 0x7f52a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.331+0000 7f52bf400640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 0x7f52a0071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0af000 tx=0x7f52ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.335+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f52c0070080 con 0x7f52c006f730 2024-05-14T11:24:15.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.527+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c006a9c0 con 0x7f52c006f730 2024-05-14T11:24:15.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.527+0000 7f52b7e00640 1 -- 172.21.15.116:0/1724140204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7f52b005bd90 con 0x7f52c006f730 2024-05-14T11:24:15.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:15.530 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:24:15.531 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 msgr2=0x7f52a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 0x7f52a0071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0af000 tx=0x7f52ac008040 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 msgr2=0x7f52c01504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c430 tx=0x7f52b000c8f0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 shutdown_connections 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f52a006f400 0x7f52a0071910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52c00b0770 0x7f52c015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52c0072a30 0x7f52c015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 --2- 172.21.15.116:0/1724140204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52c006f730 0x7f52c01504a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 >> 172.21.15.116:0/1724140204 conn(0x7f52c00b4620 msgr2=0x7f52c015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 shutdown_connections 2024-05-14T11:24:15.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.531+0000 7f52c6a00640 1 -- 172.21.15.116:0/1724140204 wait complete. 2024-05-14T11:24:15.551 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:15.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.695+0000 7f04bbe00640 1 Processor -- start 2024-05-14T11:24:15.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.695+0000 7f04bbe00640 1 -- start start 2024-05-14T11:24:15.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04bbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f04b40af830 0x7f04b40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40d1bb0 0x7f04b406edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04bbe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b40d1fc0 con 0x7f04b40af420 2024-05-14T11:24:15.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04bbe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f04b415d340 con 0x7f04b40af830 2024-05-14T11:24:15.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04bbe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f04b415d620 con 0x7f04b40d1bb0 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f04b40af830 0x7f04b40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b9400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f04b40af420 0x7f04b4074630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56824/0 (socket says 172.21.15.116:56824) 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b9400640 1 -- 172.21.15.116:0/3583890849 learned_addr learned my addr 172.21.15.116:0/3583890849 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b9e00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40d1bb0 0x7f04b406edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b8a00640 1 -- 172.21.15.116:0/3583890849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40d1bb0 msgr2=0x7f04b406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:15.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b8a00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40d1bb0 0x7f04b406edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b8a00640 1 -- 172.21.15.116:0/3583890849 >> v1:172.21.15.116:6789/0 conn(0x7f04b40af420 legacy=0x7f04b4074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b3e00640 1 -- 172.21.15.116:0/3583890849 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276861260 0 0) 0x7f04a8001120 con 0x7f04b40af420 2024-05-14T11:24:15.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.699+0000 7f04b8a00640 1 -- 172.21.15.116:0/3583890849 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b415d900 con 0x7f04b40af830 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04b8a00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b40d1670 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1da82e0da2e842 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04b3e00640 1 -- 172.21.15.116:0/3583890849 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00b1070 con 0x7f04b40af830 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04b3e00640 1 -- 172.21.15.116:0/3583890849 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a00ac620 con 0x7f04b40af830 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04b3e00640 1 -- 172.21.15.116:0/3583890849 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00ac9f0 con 0x7f04b40af830 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- 172.21.15.116:0/3583890849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 msgr2=0x7f04b40d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b40d1670 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a001c440 comp rx=0 tx=0).stop 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- 172.21.15.116:0/3583890849 shutdown_connections 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40d1bb0 0x7f04b406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 --2- 172.21.15.116:0/3583890849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b40d1670 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- 172.21.15.116:0/3583890849 >> 172.21.15.116:0/3583890849 conn(0x7f04b40b4620 msgr2=0x7f04b40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:15.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- 172.21.15.116:0/3583890849 shutdown_connections 2024-05-14T11:24:15.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- 172.21.15.116:0/3583890849 wait complete. 2024-05-14T11:24:15.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 Processor -- start 2024-05-14T11:24:15.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.703+0000 7f04bbe00640 1 -- start start 2024-05-14T11:24:15.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40af420 0x7f04b414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04b40d1bb0 0x7f04b406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4169880 con 0x7f04b40af830 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4169a00 con 0x7f04b40d1bb0 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f04b4169b80 con 0x7f04b40af420 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52878/0 (socket says 172.21.15.116:52878) 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 -- 172.21.15.116:0/2345055730 learned_addr learned my addr 172.21.15.116:0/2345055730 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b9400640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40af420 0x7f04b414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b9e00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04b40d1bb0 0x7f04b406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 -- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40af420 msgr2=0x7f04b414e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40af420 0x7f04b414e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 -- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04b40d1bb0 msgr2=0x7f04b406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04b40d1bb0 0x7f04b406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 -- 172.21.15.116:0/2345055730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b414fe70 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b8a00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a00191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00b1040 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a00bb070 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b40b0d20 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00b5650 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b40b12e0 con 0x7f04b40af830 2024-05-14T11:24:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.707+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b406a9c0 con 0x7f04b40af830 2024-05-14T11:24:15.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.711+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04a00b57f0 con 0x7f04b40af830 2024-05-14T11:24:15.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.711+0000 7f04b1600640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 0x7f0494071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:15.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.711+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f04a0128e00 con 0x7f04b40af830 2024-05-14T11:24:15.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.711+0000 7f04b9400640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 0x7f0494071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:15.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.711+0000 7f04b9400640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 0x7f0494071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007fd0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:15.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.715+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f04a0016090 con 0x7f04b40af830 2024-05-14T11:24:15.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04b415fd20 con 0x7f04b40af830 2024-05-14T11:24:15.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04b1600640 1 -- 172.21.15.116:0/2345055730 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+17387 (secure 0 0 0) 0x7f04a00f59a0 con 0x7f04b40af830 2024-05-14T11:24:15.877 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:15.877 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":88,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:23:34.960736+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:21:53.462471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-14T11:21:54.724572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6833/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/536312712":"2024-05-15T11:22:16.806991+0000","172.21.15.149:6833/536312712":"2024-05-15T11:22:16.806991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 msgr2=0x7f0494071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 0x7f0494071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007fd0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 msgr2=0x7f04b406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a00191e0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 shutdown_connections 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f049406f430 0x7f0494071940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04b40d1bb0 0x7f04b406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04b40af830 0x7f04b406c700 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 --2- 172.21.15.116:0/2345055730 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04b40af420 0x7f04b414e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 >> 172.21.15.116:0/2345055730 conn(0x7f04b40b4620 msgr2=0x7f04b40b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 shutdown_connections 2024-05-14T11:24:15.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:15.875+0000 7f04bbe00640 1 -- 172.21.15.116:0/2345055730 wait complete. 2024-05-14T11:24:15.899 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:24:15.899 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.043+0000 7f3b50000640 1 Processor -- start 2024-05-14T11:24:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.043+0000 7f3b50000640 1 -- start start 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b50000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3b480d1670 0x7f3b480d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b50000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b480d1fc0 con 0x7f3b48156fa0 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b50000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b48160510 con 0x7f3b480af380 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b50000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3b480706f0 con 0x7f3b480d1670 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4d600640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3b48156fa0 0x7f3b4806ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56836/0 (socket says 172.21.15.116:56836) 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4d600640 1 -- 172.21.15.116:0/3719012365 learned_addr learned my addr 172.21.15.116:0/3719012365 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4cc00640 1 --2- 172.21.15.116:0/3719012365 >> v2:172.21.15.116:3300/0 conn(0x7f3b480d1670 0x7f3b480d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b47e00640 1 -- 172.21.15.116:0/3719012365 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376094370 0 0) 0x7f3b3c0014a0 con 0x7f3b48156fa0 2024-05-14T11:24:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b47e00640 1 -- 172.21.15.116:0/3719012365 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b30005740 con 0x7f3b48156fa0 2024-05-14T11:24:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4cc00640 1 -- 172.21.15.116:0/3719012365 >> v1:172.21.15.149:6789/0 conn(0x7f3b480af380 legacy=0x7f3b480af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4cc00640 1 -- 172.21.15.116:0/3719012365 >> v1:172.21.15.116:6789/0 conn(0x7f3b48156fa0 legacy=0x7f3b4806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4cc00640 1 -- 172.21.15.116:0/3719012365 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481607f0 con 0x7f3b480d1670 2024-05-14T11:24:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.047+0000 7f3b4cc00640 1 --2- 172.21.15.116:0/3719012365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b480d1a80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097c0 tx=0x7f3b3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db62371eac9c3d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b47e00640 1 -- 172.21.15.116:0/3719012365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340b0070 con 0x7f3b480d1670 2024-05-14T11:24:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b47e00640 1 -- 172.21.15.116:0/3719012365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b340ab600 con 0x7f3b480d1670 2024-05-14T11:24:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b47e00640 1 -- 172.21.15.116:0/3719012365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340ab9d0 con 0x7f3b480d1670 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- 172.21.15.116:0/3719012365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 msgr2=0x7f3b480d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 --2- 172.21.15.116:0/3719012365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b480d1a80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097c0 tx=0x7f3b3401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- 172.21.15.116:0/3719012365 shutdown_connections 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 --2- 172.21.15.116:0/3719012365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b480d1a80 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- 172.21.15.116:0/3719012365 >> 172.21.15.116:0/3719012365 conn(0x7f3b480b4620 msgr2=0x7f3b480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- 172.21.15.116:0/3719012365 shutdown_connections 2024-05-14T11:24:16.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- 172.21.15.116:0/3719012365 wait complete. 2024-05-14T11:24:16.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 Processor -- start 2024-05-14T11:24:16.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.051+0000 7f3b50000640 1 -- start start 2024-05-14T11:24:16.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 0x7f3b48150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 0x7f3b481501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806ca90 con 0x7f3b480d1670 2024-05-14T11:24:16.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806c910 con 0x7f3b480af380 2024-05-14T11:24:16.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4806cc10 con 0x7f3b48156fa0 2024-05-14T11:24:16.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52898/0 (socket says 172.21.15.116:52898) 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 -- 172.21.15.116:0/2081035536 learned_addr learned my addr 172.21.15.116:0/2081035536 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4d600640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 0x7f3b48150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4e000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 0x7f3b481501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 -- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 msgr2=0x7f3b481501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 0x7f3b481501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 -- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 msgr2=0x7f3b48150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 0x7f3b48150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 -- 172.21.15.116:0/2081035536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b48157a60 con 0x7f3b480d1670 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4e000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 0x7f3b481501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4d600640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 0x7f3b48150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b4cc00640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097c0 tx=0x7f3b34019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340ba070 con 0x7f3b480d1670 2024-05-14T11:24:16.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b340b43f0 con 0x7f3b480d1670 2024-05-14T11:24:16.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340b0040 con 0x7f3b480d1670 2024-05-14T11:24:16.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481506e0 con 0x7f3b480d1670 2024-05-14T11:24:16.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b48074580 con 0x7f3b480d1670 2024-05-14T11:24:16.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.055+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4806a9c0 con 0x7f3b480d1670 2024-05-14T11:24:16.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.059+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b340b47c0 con 0x7f3b480d1670 2024-05-14T11:24:16.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.059+0000 7f3b45600640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 0x7f3b28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.059+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f3b34128870 con 0x7f3b480d1670 2024-05-14T11:24:16.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.059+0000 7f3b4d600640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 0x7f3b28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.059+0000 7f3b4d600640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 0x7f3b28071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c001780 tx=0x7f3b3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.063+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3b340c3080 con 0x7f3b480d1670 2024-05-14T11:24:16.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b4806fdc0 con 0x7f3b480d1670 2024-05-14T11:24:16.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b45600640 1 -- 172.21.15.116:0/2081035536 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7f3b34016090 con 0x7f3b480d1670 2024-05-14T11:24:16.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:16.256 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:24:16.256 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 msgr2=0x7f3b28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 0x7f3b28071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c001780 tx=0x7f3b3c009040 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 msgr2=0x7f3b48157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097c0 tx=0x7f3b34019240 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 shutdown_connections 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3b2806f400 0x7f3b28071910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3b48156fa0 0x7f3b481501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b480d1670 0x7f3b48157520 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 --2- 172.21.15.116:0/2081035536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b480af380 0x7f3b48150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 >> 172.21.15.116:0/2081035536 conn(0x7f3b480b4620 msgr2=0x7f3b4806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 shutdown_connections 2024-05-14T11:24:16.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.255+0000 7f3b50000640 1 -- 172.21.15.116:0/2081035536 wait complete. 2024-05-14T11:24:16.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:16.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 Processor -- start 2024-05-14T11:24:16.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 -- start start 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9720166750 0x7f9720168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f972006ff20 con 0x7f972015db30 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9720070470 con 0x7f97200d1670 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f9724e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f972006d710 con 0x7f9720166750 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971de00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f972015db30 0x7f97200aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56852/0 (socket says 172.21.15.116:56852) 2024-05-14T11:24:16.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971de00640 1 -- 172.21.15.116:0/1986279541 learned_addr learned my addr 172.21.15.116:0/1986279541 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971e800640 1 --2- 172.21.15.116:0/1986279541 >> v2:172.21.15.149:3300/0 conn(0x7f9720166750 0x7f9720168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461526533 0 0) 0x7f972006ff20 con 0x7f972015db30 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9704003530 con 0x7f972015db30 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2151016153 0 0) 0x7f9704003530 con 0x7f972015db30 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 >> v2:172.21.15.149:3300/0 conn(0x7f9720166750 msgr2=0x7f9720168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:16.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 --2- 172.21.15.116:0/1986279541 >> v2:172.21.15.149:3300/0 conn(0x7f9720166750 0x7f9720168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 >> v1:172.21.15.149:6789/0 conn(0x7f97200d1670 legacy=0x7f97200d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9720070750 con 0x7f972015db30 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9708001700 con 0x7f972015db30 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 >> v1:172.21.15.116:6789/0 conn(0x7f972015db30 legacy=0x7f97200aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9704003a60 0x7f9704003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f970400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f970400c310 con 0x7f97040053b0 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f970400c4c0 con 0x7f9704004810 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.423+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f970400c6c0 con 0x7f9704003a60 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f970400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f9715400640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971de00640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9704003a60 0x7f9704003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 -- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9704003a60 msgr2=0x7f9704003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:16.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9704003a60 0x7f9704003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 -- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9704004810 msgr2=0x7f9704004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:16.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 -- 172.21.15.116:0/1986279541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704003530 con 0x7f97040053b0 2024-05-14T11:24:16.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 -- 172.21.15.116:0/1986279541 reap_dead start 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971e800640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f970400ba60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9718009f90 tx=0x7f971801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457f383d308ec684 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97180b2070 con 0x7f97040053b0 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97180a5a70 con 0x7f97040053b0 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.427+0000 7f971d400640 1 -- 172.21.15.116:0/1986279541 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718009c10 con 0x7f97040053b0 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 msgr2=0x7f970400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f970400ba60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9718009f90 tx=0x7f971801cc80 comp rx=0 tx=0).stop 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- 172.21.15.116:0/1986279541 shutdown_connections 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 --2- 172.21.15.116:0/1986279541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f970400ba60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- 172.21.15.116:0/1986279541 >> 172.21.15.116:0/1986279541 conn(0x7f97200b4620 msgr2=0x7f97200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- 172.21.15.116:0/1986279541 shutdown_connections 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- 172.21.15.116:0/1986279541 wait complete. 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 Processor -- start 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.431+0000 7f9724e00640 1 -- start start 2024-05-14T11:24:16.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 0x7f9720138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 0x7f97201fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f97200aee30 con 0x7f97040053b0 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97200aecb0 con 0x7f97201391e0 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f97200aefb0 con 0x7f9720150c20 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52924/0 (socket says 172.21.15.116:52924) 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 -- 172.21.15.116:0/1708050406 learned_addr learned my addr 172.21.15.116:0/1708050406 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9715400640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 0x7f9720138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971e800640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 0x7f97201fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 -- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 msgr2=0x7f9720138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 0x7f9720138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 -- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 msgr2=0x7f97201fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 0x7f97201fa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 -- 172.21.15.116:0/1708050406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97201395f0 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9715400640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 0x7f9720138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971e800640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 0x7f97201fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f971de00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f97080072c0 tx=0x7f9708007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970801a070 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97080016b0 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9720153480 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800fc00 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9720153a40 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972006a9c0 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.435+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f970801d070 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.439+0000 7f9716a00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.439+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f970808ed70 con 0x7f97040053b0 2024-05-14T11:24:16.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.439+0000 7f9715400640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.439+0000 7f9715400640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 0x7f96fc071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c10 tx=0x7f9710009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.443+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f97201395f0 con 0x7f97040053b0 2024-05-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f972014f480 con 0x7f97040053b0 2024-05-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9716a00640 1 -- 172.21.15.116:0/1708050406 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5193 (secure 0 0 0) 0x7f970805b9a0 con 0x7f97040053b0 2024-05-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:21:55.758114+0000","modified":"2024-05-14T11:23:39.029230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7027":{"gid":7027,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6835/2777717906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":2777717906},{"type":"v1","addr":"172.21.15.149:6835","nonce":2777717906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7027,"qdb_cluster":[7027]},"id":6}]} 2024-05-14T11:24:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 74 2024-05-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 msgr2=0x7f96fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 0x7f96fc071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9710000c10 tx=0x7f9710009040 comp rx=0 tx=0).stop 2024-05-14T11:24:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 msgr2=0x7f9720168300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f97080072c0 tx=0x7f9708007780 comp rx=0 tx=0).stop 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 shutdown_connections 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f96fc06f400 0x7f96fc071910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97201391e0 0x7f97201fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9720150c20 0x7f9720138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 --2- 172.21.15.116:0/1708050406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97040053b0 0x7f9720168300 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 >> 172.21.15.116:0/1708050406 conn(0x7f97200b4620 msgr2=0x7f97200b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 shutdown_connections 2024-05-14T11:24:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.635+0000 7f9724e00640 1 -- 172.21.15.116:0/1708050406 wait complete. 2024-05-14T11:24:16.658 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:16.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.803+0000 7fe022200640 1 Processor -- start 2024-05-14T11:24:16.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.803+0000 7fe022200640 1 -- start start 2024-05-14T11:24:16.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.803+0000 7fe022200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe01c0b0770 0x7fe01c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe022200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe01c072a30 0x7fe01c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe022200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c06cc40 con 0x7fe01c06f730 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe022200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c06db20 con 0x7fe01c072a30 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe022200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c06cf10 con 0x7fe01c0b0770 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe01c072a30 0x7fe01c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe01c072a30 0x7fe01c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52930/0 (socket says 172.21.15.116:52930) 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 -- 172.21.15.116:0/1042034186 learned_addr learned my addr 172.21.15.116:0/1042034186 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01b400640 1 --2- 172.21.15.116:0/1042034186 >> v2:172.21.15.149:3300/0 conn(0x7fe01c0b0770 0x7fe01c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 -- 172.21.15.116:0/1042034186 >> v2:172.21.15.149:3300/0 conn(0x7fe01c0b0770 msgr2=0x7fe01c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 --2- 172.21.15.116:0/1042034186 >> v2:172.21.15.149:3300/0 conn(0x7fe01c0b0770 0x7fe01c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 -- 172.21.15.116:0/1042034186 >> v1:172.21.15.149:6789/0 conn(0x7fe01c06f730 legacy=0x7fe01c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:16.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 -- 172.21.15.116:0/1042034186 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c06d1f0 con 0x7fe01c072a30 2024-05-14T11:24:16.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01be00640 1 --2- 172.21.15.116:0/1042034186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c072e40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0729c0 tx=0x7fe01401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfce2b3871b939c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01aa00640 1 -- 172.21.15.116:0/1042034186 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140a5790 con 0x7fe01c072a30 2024-05-14T11:24:16.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01aa00640 1 -- 172.21.15.116:0/1042034186 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0140a5930 con 0x7fe01c072a30 2024-05-14T11:24:16.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.807+0000 7fe01aa00640 1 -- 172.21.15.116:0/1042034186 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140a5d00 con 0x7fe01c072a30 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- 172.21.15.116:0/1042034186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 msgr2=0x7fe01c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 --2- 172.21.15.116:0/1042034186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c072e40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0729c0 tx=0x7fe01401c910 comp rx=0 tx=0).stop 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- 172.21.15.116:0/1042034186 shutdown_connections 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 --2- 172.21.15.116:0/1042034186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c072e40 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 --2- 172.21.15.116:0/1042034186 >> v2:172.21.15.149:3300/0 conn(0x7fe01c0b0770 0x7fe01c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- 172.21.15.116:0/1042034186 >> 172.21.15.116:0/1042034186 conn(0x7fe01c0b4620 msgr2=0x7fe01c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- 172.21.15.116:0/1042034186 shutdown_connections 2024-05-14T11:24:16.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- 172.21.15.116:0/1042034186 wait complete. 2024-05-14T11:24:16.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 Processor -- start 2024-05-14T11:24:16.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.811+0000 7fe022200640 1 -- start start 2024-05-14T11:24:16.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe01c06f730 0x7fe01c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe01c0b0770 0x7fe01c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c169120 con 0x7fe01c072a30 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c168fa0 con 0x7fe01c0b0770 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c1692a0 con 0x7fe01c06f730 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52942/0 (socket says 172.21.15.116:52942) 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 -- 172.21.15.116:0/2303697140 learned_addr learned my addr 172.21.15.116:0/2303697140 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe01b400640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe01c06f730 0x7fe01c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe01be00640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe01c0b0770 0x7fe01c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 -- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe01c06f730 msgr2=0x7fe01c0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe01c06f730 0x7fe01c0cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 -- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe01c0b0770 msgr2=0x7fe01c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe01c0b0770 0x7fe01c154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 -- 172.21.15.116:0/2303697140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c09ae60 con 0x7fe01c072a30 2024-05-14T11:24:16.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe012a00640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b440 tx=0x7fe00400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004015020 con 0x7fe01c072a30 2024-05-14T11:24:16.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe004011890 con 0x7fe01c072a30 2024-05-14T11:24:16.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004011c80 con 0x7fe01c072a30 2024-05-14T11:24:16.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c0cf750 con 0x7fe01c072a30 2024-05-14T11:24:16.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0cfd10 con 0x7fe01c072a30 2024-05-14T11:24:16.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.815+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c06a9c0 con 0x7fe01c072a30 2024-05-14T11:24:16.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.819+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe004018020 con 0x7fe01c072a30 2024-05-14T11:24:16.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.819+0000 7fe013e00640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 0x7fdff8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:16.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.819+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7fe00408f290 con 0x7fe01c072a30 2024-05-14T11:24:16.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.819+0000 7fe01b400640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 0x7fdff8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:16.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.819+0000 7fe01b400640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 0x7fdff8071910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0d1240 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:16.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.823+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe00401f030 con 0x7fe01c072a30 2024-05-14T11:24:16.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.983+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe01c06f510 con 0x7fe01c072a30 2024-05-14T11:24:16.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.983+0000 7fe013e00640 1 -- 172.21.15.116:0/2303697140 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+17387 (secure 0 0 0) 0x7fe00400d040 con 0x7fe01c072a30 2024-05-14T11:24:16.985 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:16.985 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":88,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:23:34.960736+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:21:53.462471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-14T11:21:54.724572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6833/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/536312712":"2024-05-15T11:22:16.806991+0000","172.21.15.149:6833/536312712":"2024-05-15T11:22:16.806991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:16.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 msgr2=0x7fdff8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 0x7fdff8071910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0d1240 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 msgr2=0x7fe01c09a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b440 tx=0x7fe00400b900 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 shutdown_connections 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fdff806f400 0x7fdff8071910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe01c0b0770 0x7fe01c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe01c072a30 0x7fe01c09a920 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 --2- 172.21.15.116:0/2303697140 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe01c06f730 0x7fe01c0cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 >> 172.21.15.116:0/2303697140 conn(0x7fe01c0b4620 msgr2=0x7fe01c0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 shutdown_connections 2024-05-14T11:24:16.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:16.987+0000 7fe022200640 1 -- 172.21.15.116:0/2303697140 wait complete. 2024-05-14T11:24:17.011 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:24:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.155+0000 7f4bc4e00640 1 Processor -- start 2024-05-14T11:24:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.155+0000 7f4bc4e00640 1 -- start start 2024-05-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bc4e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4bc00b0770 0x7f4bc006c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bc4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bc4e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc0070080 con 0x7f4bc015fc00 2024-05-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bc4e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc006db20 con 0x7f4bc00b0770 2024-05-14T11:24:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bc4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc006cc40 con 0x7f4bc006f730 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bbde00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4bc00b0770 0x7f4bc006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc006fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:60830/0 (socket says 172.21.15.116:60830) 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bb5400640 1 -- 172.21.15.116:0/35915752 learned_addr learned my addr 172.21.15.116:0/35915752 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bbde00640 1 -- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 msgr2=0x7f4bc006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bbde00640 1 --2- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc006fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bbde00640 1 -- 172.21.15.116:0/35915752 >> v1:172.21.15.149:6789/0 conn(0x7f4bc015fc00 legacy=0x7f4bc0160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.159+0000 7f4bbde00640 1 -- 172.21.15.116:0/35915752 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc006cf10 con 0x7f4bc00b0770 2024-05-14T11:24:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bbde00640 1 --2- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc00b0770 0x7f4bc006c700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097c0 tx=0x7f4bb001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbac5c07f307668 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bbd400640 1 -- 172.21.15.116:0/35915752 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00b2070 con 0x7f4bc00b0770 2024-05-14T11:24:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bbd400640 1 -- 172.21.15.116:0/35915752 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb00a9070 con 0x7f4bc00b0770 2024-05-14T11:24:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bbd400640 1 -- 172.21.15.116:0/35915752 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00ad880 con 0x7f4bc00b0770 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc00b0770 msgr2=0x7f4bc006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc00b0770 0x7f4bc006c700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097c0 tx=0x7f4bb001c3e0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- 172.21.15.116:0/35915752 shutdown_connections 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/35915752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc00b0770 0x7f4bc006c700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- 172.21.15.116:0/35915752 >> 172.21.15.116:0/35915752 conn(0x7f4bc00b4620 msgr2=0x7f4bc00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- 172.21.15.116:0/35915752 shutdown_connections 2024-05-14T11:24:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- 172.21.15.116:0/35915752 wait complete. 2024-05-14T11:24:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 Processor -- start 2024-05-14T11:24:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.163+0000 7f4bc4e00640 1 -- start start 2024-05-14T11:24:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc015fc00 0x7f4bc01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0073a40 con 0x7f4bc00b0770 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc006dfb0 con 0x7f4bc015fc00 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc0168fa0 con 0x7f4bc006f730 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52944/0 (socket says 172.21.15.116:52944) 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 -- 172.21.15.116:0/1826705995 learned_addr learned my addr 172.21.15.116:0/1826705995 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bbde00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bbe800640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc015fc00 0x7f4bc01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 -- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 msgr2=0x7f4bc0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc0159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 -- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc015fc00 msgr2=0x7f4bc01525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc015fc00 0x7f4bc01525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 -- 172.21.15.116:0/1826705995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc0158950 con 0x7f4bc00b0770 2024-05-14T11:24:17.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bbde00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:17.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb5400640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e910 tx=0x7f4ba800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8016070 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba800cc40 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc0158c30 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800b040 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc014e6e0 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.167+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc015a290 con 0x7f4bc00b0770 2024-05-14T11:24:17.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.171+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ba8019070 con 0x7f4bc00b0770 2024-05-14T11:24:17.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.171+0000 7f4bb6a00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 0x7f4b9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.171+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6389+0+0 (secure 0 0 0) 0x7f4ba808f060 con 0x7f4bc00b0770 2024-05-14T11:24:17.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.171+0000 7f4bbde00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 0x7f4b9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.171+0000 7f4bbde00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 0x7f4b9c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00af000 tx=0x7f4bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.175+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4ba805bcb0 con 0x7f4bc00b0770 2024-05-14T11:24:17.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.367+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4bc014e4d0 con 0x7f4bc00b0770 2024-05-14T11:24:17.371 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:24:17.371+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:24:17.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.371+0000 7f4bb6a00640 1 -- 172.21.15.116:0/1826705995 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f4ba805b5b0 con 0x7f4bc00b0770 2024-05-14T11:24:17.373 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 msgr2=0x7f4b9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 0x7f4b9c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00af000 tx=0x7f4bb0008040 comp rx=0 tx=0).stop 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 msgr2=0x7f4bc014af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e910 tx=0x7f4ba800edd0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 shutdown_connections 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b9c06f400 0x7f4b9c071910 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bc015fc00 0x7f4bc01525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc00b0770 0x7f4bc014af90 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 --2- 172.21.15.116:0/1826705995 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4bc006f730 0x7f4bc0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 >> 172.21.15.116:0/1826705995 conn(0x7f4bc00b4620 msgr2=0x7f4bc00b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:17.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 shutdown_connections 2024-05-14T11:24:17.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.375+0000 7f4bc4e00640 1 -- 172.21.15.116:0/1826705995 wait complete. 2024-05-14T11:24:17.383 INFO:tasks.ceph.mds.b.smithi149.stderr: -2232> 2024-05-14T11:23:41.018+0000 7fcd04600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:24:17.384 INFO:tasks.ceph.mds.b.smithi149.stderr: -1845> 2024-05-14T11:23:42.482+0000 7fccff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55cc0acdc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-14T11:24:17.396 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:24:17.507 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:24:17.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.547+0000 7f5e2cc00640 1 Processor -- start 2024-05-14T11:24:17.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.547+0000 7f5e2cc00640 1 -- start start 2024-05-14T11:24:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.547+0000 7f5e2cc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e2cc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5e2806f730 0x7f5e2806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e2cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2815fc00 0x7f5e28160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e2cc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2806db20 con 0x7f5e280b0770 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e2cc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5e28070080 con 0x7f5e2806f730 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e2cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2806cc40 con 0x7f5e2815fc00 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e25e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5e2806f730 0x7f5e2806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e26800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2815fc00 0x7f5e28160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5e2806f730 0x7f5e2806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:58978/0 (socket says 172.21.15.116:58978) 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e25e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38228/0 (socket says 172.21.15.116:38228) 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 -- 172.21.15.116:0/1895735854 learned_addr learned my addr 172.21.15.116:0/1895735854 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 -- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2815fc00 msgr2=0x7f5e28160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 --2- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2815fc00 0x7f5e28160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 -- 172.21.15.116:0/1895735854 >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 msgr2=0x7f5e2806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 --2- 172.21.15.116:0/1895735854 >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 -- 172.21.15.116:0/1895735854 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2806cf10 con 0x7f5e2806f730 2024-05-14T11:24:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e25e00640 1 --2- 172.21.15.116:0/1895735854 >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:17.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.551+0000 7f5e1d400640 1 --2- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e2806f730 0x7f5e2806fb40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e2806f4e0 tx=0x7f5e1001d830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10da686d22924972 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e25400640 1 -- 172.21.15.116:0/1895735854 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100b3070 con 0x7f5e2806f730 2024-05-14T11:24:17.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e25400640 1 -- 172.21.15.116:0/1895735854 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e100a6820 con 0x7f5e2806f730 2024-05-14T11:24:17.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e25400640 1 -- 172.21.15.116:0/1895735854 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000b040 con 0x7f5e2806f730 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e2806f730 msgr2=0x7f5e2806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e2806f730 0x7f5e2806fb40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e2806f4e0 tx=0x7f5e1001d830 comp rx=0 tx=0).stop 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- 172.21.15.116:0/1895735854 shutdown_connections 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2815fc00 0x7f5e28160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/1895735854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e2806f730 0x7f5e2806fb40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/1895735854 >> v2:172.21.15.116:3300/0 conn(0x7f5e280b0770 0x7f5e2806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- 172.21.15.116:0/1895735854 >> 172.21.15.116:0/1895735854 conn(0x7f5e280b4620 msgr2=0x7f5e280b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- 172.21.15.116:0/1895735854 shutdown_connections 2024-05-14T11:24:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- 172.21.15.116:0/1895735854 wait complete. 2024-05-14T11:24:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 Processor -- start 2024-05-14T11:24:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.555+0000 7f5e2cc00640 1 -- start start 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2806f730 0x7f5e2814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e280b0770 0x7f5e28154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28160960 con 0x7f5e2815fc00 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2806dfb0 con 0x7f5e280b0770 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5e28072a30 con 0x7f5e2806f730 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38230/0 (socket says 172.21.15.116:38230) 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 -- 172.21.15.116:0/2678203797 learned_addr learned my addr 172.21.15.116:0/2678203797 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e1d400640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e280b0770 0x7f5e28154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e25e00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2806f730 0x7f5e2814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 -- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2806f730 msgr2=0x7f5e2814b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2806f730 0x7f5e2814b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 -- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e280b0770 msgr2=0x7f5e28154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e280b0770 0x7f5e28154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 -- 172.21.15.116:0/2678203797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e281542f0 con 0x7f5e2815fc00 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e26800640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000efc0 tx=0x7f5e2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20018020 con 0x7f5e2815fc00 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20014480 con 0x7f5e2815fc00 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20009e60 con 0x7f5e2815fc00 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e280cd9f0 con 0x7f5e2815fc00 2024-05-14T11:24:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2814bd70 con 0x7f5e2815fc00 2024-05-14T11:24:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.559+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2806a9c0 con 0x7f5e2815fc00 2024-05-14T11:24:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.563+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5e2001b020 con 0x7f5e2815fc00 2024-05-14T11:24:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.563+0000 7f5e1ea00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 0x7f5e04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.563+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6475+0+0 (secure 0 0 0) 0x7f5e20090280 con 0x7f5e2815fc00 2024-05-14T11:24:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.563+0000 7f5e25e00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 0x7f5e04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:17.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.563+0000 7f5e25e00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 0x7f5e04071910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e280d1240 tx=0x7f5e18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:17.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.567+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5e20022030 con 0x7f5e2815fc00 2024-05-14T11:24:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:17.767+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5e2806f510 con 0x7f5e2815fc00 2024-05-14T11:24:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e1ea00640 1 -- 172.21.15.116:0/2678203797 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f5e20010040 con 0x7f5e2815fc00 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 msgr2=0x7f5e04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 0x7f5e04071910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e280d1240 tx=0x7f5e18008040 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 msgr2=0x7f5e281571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000efc0 tx=0x7f5e2000c490 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 shutdown_connections 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5e0406f400 0x7f5e04071910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e280b0770 0x7f5e28154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e2815fc00 0x7f5e281571f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 --2- 172.21.15.116:0/2678203797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e2806f730 0x7f5e2814b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 >> 172.21.15.116:0/2678203797 conn(0x7f5e280b4620 msgr2=0x7f5e280b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 shutdown_connections 2024-05-14T11:24:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.375+0000 7f5e2cc00640 1 -- 172.21.15.116:0/2678203797 wait complete. 2024-05-14T11:24:18.403 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:24:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.555+0000 7f64a0200640 1 Processor -- start 2024-05-14T11:24:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.555+0000 7f64a0200640 1 -- start start 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f64a0200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6498070640 0x7f649806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f64a0200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6498143010 con 0x7f6498070a50 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f64a0200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649815d360 con 0x7f64980d0290 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f64a0200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f649815dc90 con 0x7f6498070640 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649ce00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6498070a50 0x7f64981421f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59332/0 (socket says 172.21.15.116:59332) 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649ce00640 1 -- 172.21.15.116:0/1800732143 learned_addr learned my addr 172.21.15.116:0/1800732143 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649d800640 1 --2- 172.21.15.116:0/1800732143 >> v2:172.21.15.149:3300/0 conn(0x7f6498070640 0x7f649806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f6497e00640 1 -- 172.21.15.116:0/1800732143 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68500956 0 0) 0x7f6498143010 con 0x7f6498070a50 2024-05-14T11:24:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f6497e00640 1 -- 172.21.15.116:0/1800732143 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6480003530 con 0x7f6498070a50 2024-05-14T11:24:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649d800640 1 -- 172.21.15.116:0/1800732143 >> v1:172.21.15.149:6789/0 conn(0x7f64980d0290 legacy=0x7f6498142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649d800640 1 -- 172.21.15.116:0/1800732143 >> v1:172.21.15.116:6789/0 conn(0x7f6498070a50 legacy=0x7f64981421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.559+0000 7f649d800640 1 -- 172.21.15.116:0/1800732143 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649815d640 con 0x7f6498070640 2024-05-14T11:24:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f649d800640 1 --2- 172.21.15.116:0/1800732143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498070640 0x7f649806e870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f649815de10 tx=0x7f648c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8649932477928b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f6497e00640 1 -- 172.21.15.116:0/1800732143 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0b0070 con 0x7f6498070640 2024-05-14T11:24:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f6497e00640 1 -- 172.21.15.116:0/1800732143 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c0ab600 con 0x7f6498070640 2024-05-14T11:24:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f6497e00640 1 -- 172.21.15.116:0/1800732143 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0ab9d0 con 0x7f6498070640 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 -- 172.21.15.116:0/1800732143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498070640 msgr2=0x7f649806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 --2- 172.21.15.116:0/1800732143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498070640 0x7f649806e870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f649815de10 tx=0x7f648c01c410 comp rx=0 tx=0).stop 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 -- 172.21.15.116:0/1800732143 shutdown_connections 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 --2- 172.21.15.116:0/1800732143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498070640 0x7f649806e870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 -- 172.21.15.116:0/1800732143 >> 172.21.15.116:0/1800732143 conn(0x7f64980cde80 msgr2=0x7f64980ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 -- 172.21.15.116:0/1800732143 shutdown_connections 2024-05-14T11:24:18.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.563+0000 7f64a0200640 1 -- 172.21.15.116:0/1800732143 wait complete. 2024-05-14T11:24:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 Processor -- start 2024-05-14T11:24:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- start start 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6498070640 0x7f64981f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64980d0290 0x7f649813e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64980cd980 con 0x7f6498070a50 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64980cd800 con 0x7f64980d0290 2024-05-14T11:24:18.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f649816b5d0 con 0x7f6498070640 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38242/0 (socket says 172.21.15.116:38242) 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 -- 172.21.15.116:0/1212975466 learned_addr learned my addr 172.21.15.116:0/1212975466 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649e200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64980d0290 0x7f649813e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649d800640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6498070640 0x7f64981f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 -- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6498070640 msgr2=0x7f64981f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6498070640 0x7f64981f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 -- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64980d0290 msgr2=0x7f649813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64980d0290 0x7f649813e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 -- 172.21.15.116:0/1212975466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64981441e0 con 0x7f6498070a50 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f649ce00640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f648400c410 tx=0x7f648400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484017070 con 0x7f6498070a50 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6484005d10 con 0x7f6498070a50 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484007310 con 0x7f6498070a50 2024-05-14T11:24:18.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64981fdbc0 con 0x7f6498070a50 2024-05-14T11:24:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.567+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981fe180 con 0x7f6498070a50 2024-05-14T11:24:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649806aa30 con 0x7f6498070a50 2024-05-14T11:24:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f648401a070 con 0x7f6498070a50 2024-05-14T11:24:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f6495600640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 0x7f6478071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6475+0+0 (secure 0 0 0) 0x7f648408e8f0 con 0x7f6498070a50 2024-05-14T11:24:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f649d800640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 0x7f6478071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.571+0000 7f649d800640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 0x7f6478071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f648c0ad000 tx=0x7f648c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.575+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f648405b4d0 con 0x7f6498070a50 2024-05-14T11:24:18.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:18.735+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f649806de80 con 0x7f6498070a50 2024-05-14T11:24:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.379+0000 7f6495600640 1 -- 172.21.15.116:0/1212975466 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f648405add0 con 0x7f6498070a50 2024-05-14T11:24:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 msgr2=0x7f6478071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 0x7f6478071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f648c0ad000 tx=0x7f648c0a6000 comp rx=0 tx=0).stop 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 msgr2=0x7f649813de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f648400c410 tx=0x7f648400c8d0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 shutdown_connections 2024-05-14T11:24:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f647806f400 0x7f6478071910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64980d0290 0x7f649813e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6498070a50 0x7f649813de70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 --2- 172.21.15.116:0/1212975466 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6498070640 0x7f64981f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 >> 172.21.15.116:0/1212975466 conn(0x7f64980cde80 msgr2=0x7f649815bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 shutdown_connections 2024-05-14T11:24:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.383+0000 7f64a0200640 1 -- 172.21.15.116:0/1212975466 wait complete. 2024-05-14T11:24:19.408 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:24:19.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.555+0000 7ffa7e400640 1 Processor -- start 2024-05-14T11:24:19.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.555+0000 7ffa7e400640 1 -- start start 2024-05-14T11:24:19.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa7e400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ffa78165060 0x7ffa78167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa7e400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7815c800 con 0x7ffa780b5f30 2024-05-14T11:24:19.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa7e400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7815caa0 con 0x7ffa7806cf90 2024-05-14T11:24:19.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa7e400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7815d0f0 con 0x7ffa78165060 2024-05-14T11:24:19.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffa780b5f30 0x7ffa780b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59344/0 (socket says 172.21.15.116:59344) 2024-05-14T11:24:19.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 -- 172.21.15.116:0/719826825 learned_addr learned my addr 172.21.15.116:0/719826825 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77e00640 1 --2- 172.21.15.116:0/719826825 >> v2:172.21.15.149:3300/0 conn(0x7ffa78165060 0x7ffa78167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713003223 0 0) 0x7ffa7815c800 con 0x7ffa780b5f30 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa5c003450 con 0x7ffa780b5f30 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 385112573 0 0) 0x7ffa5c003450 con 0x7ffa780b5f30 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 >> v2:172.21.15.149:3300/0 conn(0x7ffa78165060 msgr2=0x7ffa78167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 --2- 172.21.15.116:0/719826825 >> v2:172.21.15.149:3300/0 conn(0x7ffa78165060 0x7ffa78167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 >> v1:172.21.15.149:6789/0 conn(0x7ffa7806cf90 legacy=0x7ffa7806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7815cd80 con 0x7ffa780b5f30 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7ffa68001720 con 0x7ffa780b5f30 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 >> v1:172.21.15.116:6789/0 conn(0x7ffa780b5f30 legacy=0x7ffa780b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 0x7ffa5c003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c008970 0x7ffa5c00ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c00b650 0x7ffa5c00db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7815d0f0 con 0x7ffa5c003b80 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c00e440 con 0x7ffa5c008970 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c00e640 con 0x7ffa5c00b650 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 0x7ffa5c003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa6fe00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c008970 0x7ffa5c00ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77e00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c00b650 0x7ffa5c00db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 -- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c00b650 msgr2=0x7ffa5c00db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c00b650 0x7ffa5c00db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 -- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c008970 msgr2=0x7ffa5c00ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c008970 0x7ffa5c00ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77400640 1 -- 172.21.15.116:0/719826825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c003450 con 0x7ffa5c003b80 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77e00640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c00b650 0x7ffa5c00db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:19.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.559+0000 7ffa77e00640 1 -- 172.21.15.116:0/719826825 reap_dead start 2024-05-14T11:24:19.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa77400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 0x7ffa5c003ff0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c210 tx=0x7ffa6801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789870765d9ee9b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:19.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68007d70 con 0x7ffa5c003b80 2024-05-14T11:24:19.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa680a9020 con 0x7ffa5c003b80 2024-05-14T11:24:19.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa76a00640 1 -- 172.21.15.116:0/719826825 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68005ae0 con 0x7ffa5c003b80 2024-05-14T11:24:19.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 -- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 msgr2=0x7ffa5c003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 0x7ffa5c003ff0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c210 tx=0x7ffa6801cb80 comp rx=0 tx=0).stop 2024-05-14T11:24:19.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 -- 172.21.15.116:0/719826825 shutdown_connections 2024-05-14T11:24:19.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 --2- 172.21.15.116:0/719826825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa5c003b80 0x7ffa5c003ff0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 -- 172.21.15.116:0/719826825 >> 172.21.15.116:0/719826825 conn(0x7ffa780cde80 msgr2=0x7ffa780ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:19.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 -- 172.21.15.116:0/719826825 shutdown_connections 2024-05-14T11:24:19.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.563+0000 7ffa7e400640 1 -- 172.21.15.116:0/719826825 wait complete. 2024-05-14T11:24:19.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 Processor -- start 2024-05-14T11:24:19.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 -- start start 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c003b80 0x7ffa781591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 0x7ffa7814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780b6750 con 0x7ffa7814e770 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780b5200 con 0x7ffa78143250 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.567+0000 7ffa7e400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7814f490 con 0x7ffa5c003b80 2024-05-14T11:24:19.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38266/0 (socket says 172.21.15.116:38266) 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 -- 172.21.15.116:0/3114448923 learned_addr learned my addr 172.21.15.116:0/3114448923 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c003b80 0x7ffa781591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6fe00640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 0x7ffa7814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 -- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c003b80 msgr2=0x7ffa781591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c003b80 0x7ffa781591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 -- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 msgr2=0x7ffa7814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 0x7ffa7814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 -- 172.21.15.116:0/3114448923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78158120 con 0x7ffa7814e770 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6fe00640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 0x7ffa7814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:19.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa77e00640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffa780d04a0 tx=0x7ffa70010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa700039a0 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa70003b60 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa78158400 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004020 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781589c0 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7806aa30 con 0x7ffa7814e770 2024-05-14T11:24:19.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.571+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffa7001e020 con 0x7ffa7814e770 2024-05-14T11:24:19.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.575+0000 7ffa6f400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 0x7ffa54071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:19.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.575+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6037+0+0 (secure 0 0 0) 0x7ffa7008fb20 con 0x7ffa7814e770 2024-05-14T11:24:19.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.575+0000 7ffa77400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 0x7ffa54071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:19.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.575+0000 7ffa77400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 0x7ffa54071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa680aa000 tx=0x7ffa7815cc70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:19.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.579+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffa7005c820 con 0x7ffa7814e770 2024-05-14T11:24:19.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:19.743+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffa78157e70 con 0x7ffa7814e770 2024-05-14T11:24:20.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.391+0000 7ffa6f400640 1 -- 172.21.15.116:0/3114448923 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7ffa7005c120 con 0x7ffa7814e770 2024-05-14T11:24:20.395 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 msgr2=0x7ffa54071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 0x7ffa54071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa680aa000 tx=0x7ffa7815cc70 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 msgr2=0x7ffa7814eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ffa780d04a0 tx=0x7ffa70010590 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 shutdown_connections 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffa5406f400 0x7ffa54071910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78143250 0x7ffa7814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa7814e770 0x7ffa7814eb80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 --2- 172.21.15.116:0/3114448923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa5c003b80 0x7ffa781591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 >> 172.21.15.116:0/3114448923 conn(0x7ffa780cde80 msgr2=0x7ffa780cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 shutdown_connections 2024-05-14T11:24:20.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.395+0000 7ffa7e400640 1 -- 172.21.15.116:0/3114448923 wait complete. 2024-05-14T11:24:20.425 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:24:20.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.575+0000 7f48ba400640 1 Processor -- start 2024-05-14T11:24:20.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.575+0000 7f48ba400640 1 -- start start 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48ba400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f48b406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48ba400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b406cc40 con 0x7f48b406f730 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48ba400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b406cf10 con 0x7f48b4072a30 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48ba400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48b406db20 con 0x7f48b40b0770 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48abe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f48b406f730 0x7f48b406fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59346/0 (socket says 172.21.15.116:59346) 2024-05-14T11:24:20.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48abe00640 1 -- 172.21.15.116:0/496559042 learned_addr learned my addr 172.21.15.116:0/496559042 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:20.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3400640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f48b406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882797608 0 0) 0x7f48b406cc40 con 0x7f48b406f730 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4898003530 con 0x7f48b406f730 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 35134151 0 0) 0x7f4898003530 con 0x7f48b406f730 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 msgr2=0x7f48b406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f48b406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 >> v1:172.21.15.149:6789/0 conn(0x7f48b4072a30 legacy=0x7f48b4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b406d1f0 con 0x7f48b406f730 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f489c001700 con 0x7f48b406f730 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 >> v1:172.21.15.116:6789/0 conn(0x7f48b406f730 legacy=0x7f48b406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f4898003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004010 0x7f4898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4898004d50 0x7f4898005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48b406db20 con 0x7f4898004d50 2024-05-14T11:24:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f489800bc90 con 0x7f4898004010 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f489800be90 con 0x7f48b40b0770 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4898004d50 0x7f4898005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3400640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f4898003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48abe00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004010 0x7f4898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 -- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 msgr2=0x7f4898003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b40b0770 0x7f4898003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 -- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004010 msgr2=0x7f4898004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004010 0x7f4898004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.579+0000 7f48b3e00640 1 -- 172.21.15.116:0/496559042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4898003530 con 0x7f4898004d50 2024-05-14T11:24:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48b3e00640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4898004d50 0x7f4898005200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f48b40d22e0 tx=0x7f48ac01c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e5d9097281f7f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac001a80 con 0x7f4898004d50 2024-05-14T11:24:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac003b80 con 0x7f4898004d50 2024-05-14T11:24:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48b2a00640 1 -- 172.21.15.116:0/496559042 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac005080 con 0x7f4898004d50 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 -- 172.21.15.116:0/496559042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4898004d50 msgr2=0x7f4898005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 --2- 172.21.15.116:0/496559042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4898004d50 0x7f4898005200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f48b40d22e0 tx=0x7f48ac01c6f0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48b3e00640 1 -- 172.21.15.116:0/496559042 reap_dead start 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 -- 172.21.15.116:0/496559042 shutdown_connections 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 -- 172.21.15.116:0/496559042 >> 172.21.15.116:0/496559042 conn(0x7f48b40b4620 msgr2=0x7f48b40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 -- 172.21.15.116:0/496559042 shutdown_connections 2024-05-14T11:24:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.583+0000 7f48ba400640 1 -- 172.21.15.116:0/496559042 wait complete. 2024-05-14T11:24:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 Processor -- start 2024-05-14T11:24:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- start start 2024-05-14T11:24:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004d50 0x7f48b414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b41510c0 0x7f48b415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48b406dfb0 con 0x7f48b4147e40 2024-05-14T11:24:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48b406d7b0 con 0x7f4898004d50 2024-05-14T11:24:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48b406c700 con 0x7f48b41510c0 2024-05-14T11:24:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38278/0 (socket says 172.21.15.116:38278) 2024-05-14T11:24:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 -- 172.21.15.116:0/1042831104 learned_addr learned my addr 172.21.15.116:0/1042831104 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48b3e00640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b41510c0 0x7f48b415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48b3400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004d50 0x7f48b414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 -- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b41510c0 msgr2=0x7f48b415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b41510c0 0x7f48b415abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 -- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004d50 msgr2=0x7f48b414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004d50 0x7f48b414bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 -- 172.21.15.116:0/1042831104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b41514d0 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48abe00640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f489c012510 tx=0x7f489c0129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c01c070 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c014770 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b414eeb0 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c014b60 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.587+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b414f470 con 0x7f48b4147e40 2024-05-14T11:24:20.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b406a9c0 con 0x7f48b4147e40 2024-05-14T11:24:20.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f489c01f070 con 0x7f48b4147e40 2024-05-14T11:24:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48ab400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7f489c091820 con 0x7f48b4147e40 2024-05-14T11:24:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48b3400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.591+0000 7f48b3400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f48b40d1240 tx=0x7f48a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.595+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f48b41514d0 con 0x7f48b4147e40 2024-05-14T11:24:20.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.755+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f48b414c920 con 0x7f48b4147e40 2024-05-14T11:24:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.759+0000 7f48ab400640 1 -- 172.21.15.116:0/1042831104 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f489c05e8d0 con 0x7f48b4147e40 2024-05-14T11:24:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 msgr2=0x7f4890071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f48b40d1240 tx=0x7f48a4008040 comp rx=0 tx=0).stop 2024-05-14T11:24:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 msgr2=0x7f48b4150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f489c012510 tx=0x7f489c0129d0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 shutdown_connections 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f489006f400 0x7f4890071910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48b41510c0 0x7f48b415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48b4147e40 0x7f48b4150b80 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 --2- 172.21.15.116:0/1042831104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4898004d50 0x7f48b414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 >> 172.21.15.116:0/1042831104 conn(0x7f48b40b4620 msgr2=0x7f48b4152820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 shutdown_connections 2024-05-14T11:24:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.763+0000 7f48ba400640 1 -- 172.21.15.116:0/1042831104 wait complete. 2024-05-14T11:24:20.785 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-14T11:24:20.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 Processor -- start 2024-05-14T11:24:20.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 -- start start 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7c80d1670 0x7ff7c80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c80d1fc0 con 0x7ff7c8156fa0 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c8160510 con 0x7ff7c80af380 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7cd400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c80706f0 con 0x7ff7c80d1670 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c6800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff7c8156fa0 0x7ff7c806ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59362/0 (socket says 172.21.15.116:59362) 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c6800640 1 -- 172.21.15.116:0/1994817854 learned_addr learned my addr 172.21.15.116:0/1994817854 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:20.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5e00640 1 --2- 172.21.15.116:0/1994817854 >> v2:172.21.15.116:3300/0 conn(0x7ff7c80d1670 0x7ff7c80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5400640 1 -- 172.21.15.116:0/1994817854 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420580463 0 0) 0x7ff7b8001120 con 0x7ff7c8156fa0 2024-05-14T11:24:20.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5400640 1 -- 172.21.15.116:0/1994817854 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7ac003530 con 0x7ff7c8156fa0 2024-05-14T11:24:20.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5e00640 1 -- 172.21.15.116:0/1994817854 >> v1:172.21.15.149:6789/0 conn(0x7ff7c80af380 legacy=0x7ff7c80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5e00640 1 -- 172.21.15.116:0/1994817854 >> v1:172.21.15.116:6789/0 conn(0x7ff7c8156fa0 legacy=0x7ff7c806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.931+0000 7ff7c5e00640 1 -- 172.21.15.116:0/1994817854 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81607f0 con 0x7ff7c80d1670 2024-05-14T11:24:20.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7c5e00640 1 --2- 172.21.15.116:0/1994817854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80d1670 0x7ff7c80d1a80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40097c0 tx=0x7ff7b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33083144f1954497 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7c5400640 1 -- 172.21.15.116:0/1994817854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b0070 con 0x7ff7c80d1670 2024-05-14T11:24:20.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7c5400640 1 -- 172.21.15.116:0/1994817854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b40ab600 con 0x7ff7c80d1670 2024-05-14T11:24:20.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7c5400640 1 -- 172.21.15.116:0/1994817854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40ab9d0 con 0x7ff7c80d1670 2024-05-14T11:24:20.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 -- 172.21.15.116:0/1994817854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80d1670 msgr2=0x7ff7c80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1994817854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80d1670 0x7ff7c80d1a80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40097c0 tx=0x7ff7b401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 -- 172.21.15.116:0/1994817854 shutdown_connections 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1994817854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80d1670 0x7ff7c80d1a80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 -- 172.21.15.116:0/1994817854 >> 172.21.15.116:0/1994817854 conn(0x7ff7c80b4620 msgr2=0x7ff7c80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 -- 172.21.15.116:0/1994817854 shutdown_connections 2024-05-14T11:24:20.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.935+0000 7ff7cd400640 1 -- 172.21.15.116:0/1994817854 wait complete. 2024-05-14T11:24:20.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 Processor -- start 2024-05-14T11:24:20.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 -- start start 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c80d1670 0x7ff7c814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7c8156fa0 0x7ff7c81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c816abb0 con 0x7ff7c80af380 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c816aa30 con 0x7ff7c80d1670 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7cd400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c816ad30 con 0x7ff7c8156fa0 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38298/0 (socket says 172.21.15.116:38298) 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 -- 172.21.15.116:0/1532444863 learned_addr learned my addr 172.21.15.116:0/1532444863 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c5e00640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c80d1670 0x7ff7c814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c7200640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7c8156fa0 0x7ff7c81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 -- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7c8156fa0 msgr2=0x7ff7c81599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7c8156fa0 0x7ff7c81599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 -- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c80d1670 msgr2=0x7ff7c814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c80d1670 0x7ff7c814fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 -- 172.21.15.116:0/1532444863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c8150a30 con 0x7ff7c80af380 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.939+0000 7ff7c6800640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c730 tx=0x7ff7b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8017070 con 0x7ff7c80af380 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b8005ac0 con 0x7ff7c80af380 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c8150d10 con 0x7ff7c80af380 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8001d20 con 0x7ff7c80af380 2024-05-14T11:24:20.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c8157a50 con 0x7ff7c80af380 2024-05-14T11:24:20.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c806a9c0 con 0x7ff7c80af380 2024-05-14T11:24:20.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff7b801a070 con 0x7ff7c80af380 2024-05-14T11:24:20.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 0x7ff7a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:20.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7ff7b808e4a0 con 0x7ff7c80af380 2024-05-14T11:24:20.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.943+0000 7ff7c5e00640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 0x7ff7a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:20.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.947+0000 7ff7c5e00640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 0x7ff7a8071940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40ad000 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:20.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:20.947+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff7c80d1fc0 con 0x7ff7c80af380 2024-05-14T11:24:21.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.159+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7ff7c806fdc0 con 0x7ff7c80af380 2024-05-14T11:24:21.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7bea00640 1 -- 172.21.15.116:0/1532444863 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v384) ==== 167+0+0 (secure 0 0 0) 0x7ff7b8063430 con 0x7ff7c80af380 2024-05-14T11:24:21.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 msgr2=0x7ff7a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 0x7ff7a8071940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40ad000 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2024-05-14T11:24:21.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 msgr2=0x7ff7c815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c730 tx=0x7ff7b800cbf0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 shutdown_connections 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff7a806f430 0x7ff7a8071940 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7c80d1670 0x7ff7c814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7c8156fa0 0x7ff7c81599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 --2- 172.21.15.116:0/1532444863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7c80af380 0x7ff7c815c090 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 >> 172.21.15.116:0/1532444863 conn(0x7ff7c80b4620 msgr2=0x7ff7c806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 shutdown_connections 2024-05-14T11:24:21.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.403+0000 7ff7cd400640 1 -- 172.21.15.116:0/1532444863 wait complete. 2024-05-14T11:24:21.427 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-14T11:24:21.427 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:24:21.427 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.571+0000 7fe536e00640 1 Processor -- start 2024-05-14T11:24:21.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.571+0000 7fe536e00640 1 -- start start 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe536e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe5300d1670 0x7fe5300d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe536e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53006ff20 con 0x7fe53006c700 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe536e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5300d1fc0 con 0x7fe53015db30 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe536e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe5300ae100 con 0x7fe5300d1670 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe534e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe53006c700 0x7fe53006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59376/0 (socket says 172.21.15.116:59376) 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe534e00640 1 -- 172.21.15.116:0/2468177838 learned_addr learned my addr 172.21.15.116:0/2468177838 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:21.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> v2:172.21.15.149:3300/0 conn(0x7fe5300d1670 0x7fe5300d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893783228 0 0) 0x7fe53006ff20 con 0x7fe53006c700 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe514003530 con 0x7fe53006c700 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423350647 0 0) 0x7fe520001120 con 0x7fe53015db30 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe53006ff20 con 0x7fe53015db30 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 741736405 0 0) 0x7fe514003530 con 0x7fe53006c700 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 >> v2:172.21.15.149:3300/0 conn(0x7fe5300d1670 msgr2=0x7fe5300d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 --2- 172.21.15.116:0/2468177838 >> v2:172.21.15.149:3300/0 conn(0x7fe5300d1670 0x7fe5300d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 >> v1:172.21.15.149:6789/0 conn(0x7fe53015db30 legacy=0x7fe5300aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe530070470 con 0x7fe53006c700 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fe528001a80 con 0x7fe53006c700 2024-05-14T11:24:21.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> v2:172.21.15.149:3300/0 conn(0x7fe5300d1670 0x7fe5300d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 >> v1:172.21.15.116:6789/0 conn(0x7fe53006c700 legacy=0x7fe53006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514003c40 0x7fe5140040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 0x7fe514004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5140053f0 0x7fe51400baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5300ae100 con 0x7fe514004810 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe51400c380 con 0x7fe514003c40 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe51400c580 con 0x7fe5140053f0 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 0x7fe514004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52fe00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514003c40 0x7fe5140040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe534e00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5140053f0 0x7fe51400baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 -- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5140053f0 msgr2=0x7fe51400baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5140053f0 0x7fe51400baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 -- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514003c40 msgr2=0x7fe5140040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514003c40 0x7fe5140040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe52f400640 1 -- 172.21.15.116:0/2468177838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe514003530 con 0x7fe514004810 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe534e00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5140053f0 0x7fe51400baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.575+0000 7fe534e00640 1 -- 172.21.15.116:0/2468177838 reap_dead start 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe52f400640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 0x7fe514004ce0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009cb0 tx=0x7fe51c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa51af510888a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0b2070 con 0x7fe514004810 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c0a5680 con 0x7fe514004810 2024-05-14T11:24:21.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe52ea00640 1 -- 172.21.15.116:0/2468177838 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009880 con 0x7fe514004810 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 -- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 msgr2=0x7fe514004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 0x7fe514004ce0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009cb0 tx=0x7fe51c01cc10 comp rx=0 tx=0).stop 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 -- 172.21.15.116:0/2468177838 shutdown_connections 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 --2- 172.21.15.116:0/2468177838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe514004810 0x7fe514004ce0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 -- 172.21.15.116:0/2468177838 >> 172.21.15.116:0/2468177838 conn(0x7fe5300b4620 msgr2=0x7fe5300b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:21.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 -- 172.21.15.116:0/2468177838 shutdown_connections 2024-05-14T11:24:21.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.579+0000 7fe536e00640 1 -- 172.21.15.116:0/2468177838 wait complete. 2024-05-14T11:24:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 Processor -- start 2024-05-14T11:24:21.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- start start 2024-05-14T11:24:21.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514004810 0x7fe5301fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 0x7fe53015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5300aeeb0 con 0x7fe53009b4b0 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5300ae870 con 0x7fe514004810 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5300af030 con 0x7fe53015eda0 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38308/0 (socket says 172.21.15.116:38308) 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 -- 172.21.15.116:0/3277094333 learned_addr learned my addr 172.21.15.116:0/3277094333 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe534e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 0x7fe53015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52fe00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514004810 0x7fe5301fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 -- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 msgr2=0x7fe53015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 0x7fe53015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 -- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514004810 msgr2=0x7fe5301fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514004810 0x7fe5301fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 -- 172.21.15.116:0/3277094333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5301fd480 con 0x7fe53009b4b0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe534e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 0x7fe53015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe52f400640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe51c01ca40 tx=0x7fe51c0199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0bb070 con 0x7fe53009b4b0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c0ad440 con 0x7fe53009b4b0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0b2040 con 0x7fe53009b4b0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53009c220 con 0x7fe53009b4b0 2024-05-14T11:24:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.583+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53009c7e0 con 0x7fe53009b4b0 2024-05-14T11:24:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53006a9c0 con 0x7fe53009b4b0 2024-05-14T11:24:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe51c0be070 con 0x7fe53009b4b0 2024-05-14T11:24:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe527e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 0x7fe510071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7fe51c128980 con 0x7fe53009b4b0 2024-05-14T11:24:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe52fe00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 0x7fe510071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.587+0000 7fe52fe00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 0x7fe510071940 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe520000c10 tx=0x7fe520009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.591+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe51c0c5080 con 0x7fe53009b4b0 2024-05-14T11:24:21.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe53009d600 con 0x7fe53009b4b0 2024-05-14T11:24:21.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe527e00640 1 -- 172.21.15.116:0/3277094333 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+13606 (secure 0 0 0) 0x7fe51c016920 con 0x7fe53009b4b0 2024-05-14T11:24:21.752 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:21.752 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":91,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:24:20.387513+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6833/2777717906":"2024-05-15T11:24:17.375126+0000","172.21.15.149:6833/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/1421948627":"2024-05-15T11:23:34.960706+0000","172.21.15.149:6835/536312712":"2024-05-15T11:22:16.806991+0000","172.21.15.149:6835/2777717906":"2024-05-15T11:24:17.375126+0000","172.21.15.149:6833/536312712":"2024-05-15T11:22:16.806991+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:21.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 msgr2=0x7fe510071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 0x7fe510071940 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe520000c10 tx=0x7fe520009040 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 msgr2=0x7fe53009b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe51c01ca40 tx=0x7fe51c0199b0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 shutdown_connections 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe51006f430 0x7fe510071940 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53015eda0 0x7fe53015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe53009b4b0 0x7fe53009b8c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 --2- 172.21.15.116:0/3277094333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe514004810 0x7fe5301fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 >> 172.21.15.116:0/3277094333 conn(0x7fe5300b4620 msgr2=0x7fe53015a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 shutdown_connections 2024-05-14T11:24:21.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.751+0000 7fe536e00640 1 -- 172.21.15.116:0/3277094333 wait complete. 2024-05-14T11:24:21.774 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:24:21.775 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:24:21.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 Processor -- start 2024-05-14T11:24:21.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 -- start start 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f012c15c1d0 0x7f012c0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f012c15fc50 0x7f012c160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f012c06d790 con 0x7f012c15fc50 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f012c0ce690 con 0x7f012c15c1d0 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0130c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f012c06cc10 con 0x7f012c0cdd40 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f012c15fc50 0x7f012c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f012c15fc50 0x7f012c160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38314/0 (socket says 172.21.15.116:38314) 2024-05-14T11:24:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 -- 172.21.15.116:0/930709783 learned_addr learned my addr 172.21.15.116:0/930709783 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0129e00640 1 --2- 172.21.15.116:0/930709783 >> v2:172.21.15.149:3300/0 conn(0x7f012c15c1d0 0x7f012c0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f0129400640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 -- 172.21.15.116:0/930709783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 msgr2=0x7f012c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 -- 172.21.15.116:0/930709783 >> v2:172.21.15.149:3300/0 conn(0x7f012c15c1d0 msgr2=0x7f012c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 --2- 172.21.15.116:0/930709783 >> v2:172.21.15.149:3300/0 conn(0x7f012c15c1d0 0x7f012c0d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.923+0000 7f012a800640 1 -- 172.21.15.116:0/930709783 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c06cef0 con 0x7f012c15fc50 2024-05-14T11:24:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f012a800640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15fc50 0x7f012c160060 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b52d0 tx=0x7f012401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88329af3c3edbc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0128a00640 1 -- 172.21.15.116:0/930709783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240a5560 con 0x7f012c15fc50 2024-05-14T11:24:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0128a00640 1 -- 172.21.15.116:0/930709783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240a5700 con 0x7f012c15fc50 2024-05-14T11:24:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0128a00640 1 -- 172.21.15.116:0/930709783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240a5ad0 con 0x7f012c15fc50 2024-05-14T11:24:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 -- 172.21.15.116:0/930709783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15fc50 msgr2=0x7f012c160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15fc50 0x7f012c160060 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b52d0 tx=0x7f012401c960 comp rx=0 tx=0).stop 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 -- 172.21.15.116:0/930709783 shutdown_connections 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15fc50 0x7f012c160060 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 --2- 172.21.15.116:0/930709783 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 --2- 172.21.15.116:0/930709783 >> v2:172.21.15.149:3300/0 conn(0x7f012c15c1d0 0x7f012c0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 -- 172.21.15.116:0/930709783 >> 172.21.15.116:0/930709783 conn(0x7f012c0b4720 msgr2=0x7f012c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 -- 172.21.15.116:0/930709783 shutdown_connections 2024-05-14T11:24:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.927+0000 7f0130c00640 1 -- 172.21.15.116:0/930709783 wait complete. 2024-05-14T11:24:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 Processor -- start 2024-05-14T11:24:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 -- start start 2024-05-14T11:24:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c074470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f012c15fc50 0x7f012c0cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f012c0b0b70 con 0x7f012c15c1d0 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f012c0b09f0 con 0x7f012c15fc50 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0130c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f012c0cbd60 con 0x7f012c0cdd40 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0129400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0129400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38326/0 (socket says 172.21.15.116:38326) 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0129400640 1 -- 172.21.15.116:0/3985418323 learned_addr learned my addr 172.21.15.116:0/3985418323 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f012a800640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f012c15fc50 0x7f012c0cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.931+0000 7f0129e00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c074470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 -- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 msgr2=0x7f012c074470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c074470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 -- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f012c15fc50 msgr2=0x7f012c0cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f012c15fc50 0x7f012c0cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 -- 172.21.15.116:0/3985418323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c0c6310 con 0x7f012c15c1d0 2024-05-14T11:24:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0129400640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f011800e8a0 tx=0x7f011800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118017070 con 0x7f012c15c1d0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f011800ee90 con 0x7f012c15c1d0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800cb80 con 0x7f012c15c1d0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c0c6030 con 0x7f012c15c1d0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c1656a0 con 0x7f012c15c1d0 2024-05-14T11:24:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c06a9c0 con 0x7f012c15c1d0 2024-05-14T11:24:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.935+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f011801a070 con 0x7f012c15c1d0 2024-05-14T11:24:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.939+0000 7f0122000640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 0x7f010c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.939+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7f011808f3a0 con 0x7f012c15c1d0 2024-05-14T11:24:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.939+0000 7f0129e00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 0x7f010c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.939+0000 7f0129e00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 0x7f010c071910 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f011c005c10 tx=0x7f011c00b750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:21.943+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f011805bd00 con 0x7f012c15c1d0 2024-05-14T11:24:22.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f012c069de0 con 0x7f012c15c1d0 2024-05-14T11:24:22.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0122000640 1 -- 172.21.15.116:0/3985418323 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f0118060000 con 0x7f012c15c1d0 2024-05-14T11:24:22.100 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:24:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 msgr2=0x7f010c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 0x7f010c071910 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f011c005c10 tx=0x7f011c00b750 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 msgr2=0x7f012c1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f011800e8a0 tx=0x7f011800ed60 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 shutdown_connections 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f010c06f400 0x7f010c071910 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f012c15fc50 0x7f012c0cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f012c15c1d0 0x7f012c1571a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 --2- 172.21.15.116:0/3985418323 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f012c0cdd40 0x7f012c074470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 >> 172.21.15.116:0/3985418323 conn(0x7f012c0b4720 msgr2=0x7f012c0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 shutdown_connections 2024-05-14T11:24:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.099+0000 7f0130c00640 1 -- 172.21.15.116:0/3985418323 wait complete. 2024-05-14T11:24:22.122 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-14T11:24:22.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 Processor -- start 2024-05-14T11:24:22.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 -- start start 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 0x7fb0000d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0000d1fc0 con 0x7fb0000af380 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb000160510 con 0x7fb000156fa0 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7fb004a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb0000706f0 con 0x7fb0000d1670 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffe800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb0000af380 0x7fb0000af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59386/0 (socket says 172.21.15.116:59386) 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffe800640 1 -- 172.21.15.116:0/1135939574 learned_addr learned my addr 172.21.15.116:0/1135939574 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:22.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffd400640 1 --2- 172.21.15.116:0/1135939574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 0x7fb0000d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012656442 0 0) 0x7faff80014a0 con 0x7fb0000af380 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe4003530 con 0x7fb0000af380 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024497267 0 0) 0x7fb000160510 con 0x7fb000156fa0 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.267+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faff80014a0 con 0x7fb000156fa0 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffd400640 1 -- 172.21.15.116:0/1135939574 >> v1:172.21.15.149:6789/0 conn(0x7fb000156fa0 legacy=0x7fb00006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffd400640 1 -- 172.21.15.116:0/1135939574 >> v1:172.21.15.116:6789/0 conn(0x7fb0000af380 legacy=0x7fb0000af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffd400640 1 -- 172.21.15.116:0/1135939574 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0001607f0 con 0x7fb0000d1670 2024-05-14T11:24:22.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffd400640 1 --2- 172.21.15.116:0/1135939574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 0x7fb0000d1a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafec0097f0 tx=0x7fafec01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4b18cf1b67b928 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:22.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0b0070 con 0x7fb0000d1670 2024-05-14T11:24:22.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0ab600 con 0x7fb0000d1670 2024-05-14T11:24:22.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.271+0000 7faffca00640 1 -- 172.21.15.116:0/1135939574 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0ab9d0 con 0x7fb0000d1670 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- 172.21.15.116:0/1135939574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 msgr2=0x7fb0000d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 --2- 172.21.15.116:0/1135939574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 0x7fb0000d1a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafec0097f0 tx=0x7fafec01c4e0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- 172.21.15.116:0/1135939574 shutdown_connections 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 --2- 172.21.15.116:0/1135939574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0000d1670 0x7fb0000d1a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- 172.21.15.116:0/1135939574 >> 172.21.15.116:0/1135939574 conn(0x7fb0000b4620 msgr2=0x7fb0000b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- 172.21.15.116:0/1135939574 shutdown_connections 2024-05-14T11:24:22.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- 172.21.15.116:0/1135939574 wait complete. 2024-05-14T11:24:22.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 Processor -- start 2024-05-14T11:24:22.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.275+0000 7fb004a00640 1 -- start start 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0000d1670 0x7fb000073440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb000156fa0 0x7fb000165950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb00016abb0 con 0x7fb0000af380 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb00016aa30 con 0x7fb0000d1670 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb00016ad30 con 0x7fb000156fa0 2024-05-14T11:24:22.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38332/0 (socket says 172.21.15.116:38332) 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 -- 172.21.15.116:0/633825819 learned_addr learned my addr 172.21.15.116:0/633825819 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffd400640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0000d1670 0x7fb000073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffe800640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb000156fa0 0x7fb000165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 -- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb000156fa0 msgr2=0x7fb000165950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb000156fa0 0x7fb000165950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 -- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0000d1670 msgr2=0x7fb000073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0000d1670 0x7fb000073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 -- 172.21.15.116:0/633825819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb000154890 con 0x7fb0000af380 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faffde00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7faff000c460 tx=0x7faff000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0017070 con 0x7fb0000af380 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff00052f0 con 0x7fb0000af380 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0003840 con 0x7fb0000af380 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb000154b70 con 0x7fb0000af380 2024-05-14T11:24:22.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb000155130 con 0x7fb0000af380 2024-05-14T11:24:22.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00006a9c0 con 0x7fb0000af380 2024-05-14T11:24:22.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.279+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faff001a070 con 0x7fb0000af380 2024-05-14T11:24:22.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.283+0000 7faff6000640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 0x7fafe0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:22.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.283+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7faff008e540 con 0x7fb0000af380 2024-05-14T11:24:22.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.283+0000 7faffd400640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 0x7fafe0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:22.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.283+0000 7faffd400640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 0x7fafe0071940 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:22.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.287+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7faff005ae90 con 0x7fb0000af380 2024-05-14T11:24:22.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:22.499+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fb00006fdc0 con 0x7fb0000af380 2024-05-14T11:24:23.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.411+0000 7faff6000640 1 -- 172.21.15.116:0/633825819 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v386) ==== 149+0+0 (secure 0 0 0) 0x7faff005f190 con 0x7fb0000af380 2024-05-14T11:24:23.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 msgr2=0x7fafe0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 0x7fafe0071940 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).stop 2024-05-14T11:24:23.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 msgr2=0x7fb0001fdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7faff000c460 tx=0x7faff000c920 comp rx=0 tx=0).stop 2024-05-14T11:24:23.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 shutdown_connections 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fafe006f430 0x7fafe0071940 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0000d1670 0x7fb000073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb000156fa0 0x7fb000165950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 --2- 172.21.15.116:0/633825819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0000af380 0x7fb0001fdd30 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 >> 172.21.15.116:0/633825819 conn(0x7fb0000b4620 msgr2=0x7fb0000b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 shutdown_connections 2024-05-14T11:24:23.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.415+0000 7fb004a00640 1 -- 172.21.15.116:0/633825819 wait complete. 2024-05-14T11:24:23.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:24:23.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.591+0000 7f1141000640 1 Processor -- start 2024-05-14T11:24:23.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 -- start start 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f113c06f450 0x7f113c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f113c0d1730 0x7f113c0d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c06f860 con 0x7f113c0cce60 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f113c06db80 con 0x7f113c06f450 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1141000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f113c0d2080 con 0x7f113c0d1730 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f113c06f450 0x7f113c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f113c06f450 0x7f113c06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38340/0 (socket says 172.21.15.116:38340) 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 -- 172.21.15.116:0/504340721 learned_addr learned my addr 172.21.15.116:0/504340721 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1133e00640 1 --2- 172.21.15.116:0/504340721 >> v2:172.21.15.149:3300/0 conn(0x7f113c0d1730 0x7f113c0d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 -- 172.21.15.116:0/504340721 >> v2:172.21.15.149:3300/0 conn(0x7f113c0d1730 msgr2=0x7f113c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:23.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 --2- 172.21.15.116:0/504340721 >> v2:172.21.15.149:3300/0 conn(0x7f113c0d1730 0x7f113c0d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 -- 172.21.15.116:0/504340721 >> v1:172.21.15.149:6789/0 conn(0x7f113c0cce60 legacy=0x7f113c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:23.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.595+0000 7f1139e00640 1 -- 172.21.15.116:0/504340721 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c06fae0 con 0x7f113c06f450 2024-05-14T11:24:23.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1139e00640 1 --2- 172.21.15.116:0/504340721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c06d640 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f113c0d2430 tx=0x7f112c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e5fbb8044c507d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:23.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1139400640 1 -- 172.21.15.116:0/504340721 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b0070 con 0x7f113c06f450 2024-05-14T11:24:23.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1139400640 1 -- 172.21.15.116:0/504340721 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c0ab600 con 0x7f113c06f450 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1139400640 1 -- 172.21.15.116:0/504340721 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0ce190 con 0x7f113c06f450 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 -- 172.21.15.116:0/504340721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 msgr2=0x7f113c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 --2- 172.21.15.116:0/504340721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c06d640 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f113c0d2430 tx=0x7f112c01c340 comp rx=0 tx=0).stop 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 -- 172.21.15.116:0/504340721 shutdown_connections 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 --2- 172.21.15.116:0/504340721 >> v2:172.21.15.149:3300/0 conn(0x7f113c0d1730 0x7f113c0d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 --2- 172.21.15.116:0/504340721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c06d640 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 -- 172.21.15.116:0/504340721 >> 172.21.15.116:0/504340721 conn(0x7f113c0b4720 msgr2=0x7f113c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 -- 172.21.15.116:0/504340721 shutdown_connections 2024-05-14T11:24:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.599+0000 7f1141000640 1 -- 172.21.15.116:0/504340721 wait complete. 2024-05-14T11:24:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 Processor -- start 2024-05-14T11:24:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- start start 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0cce60 0x7f113c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c0d1730 0x7f113c1f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f113c16a610 con 0x7f113c06f450 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f113c070660 con 0x7f113c0d1730 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f113c16a790 con 0x7f113c0cce60 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38354/0 (socket says 172.21.15.116:38354) 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 -- 172.21.15.116:0/2381268085 learned_addr learned my addr 172.21.15.116:0/2381268085 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f113a800640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c0d1730 0x7f113c1f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1133e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0cce60 0x7f113c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 -- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0cce60 msgr2=0x7f113c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0cce60 0x7f113c072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 -- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c0d1730 msgr2=0x7f113c1f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c0d1730 0x7f113c1f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 -- 172.21.15.116:0/2381268085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c15c400 con 0x7f113c06f450 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1139e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f112c0ae040 tx=0x7f112c01ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b0040 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c0bc070 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b6620 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c15c6e0 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.603+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c073930 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113c0c8e90 con 0x7f113c06f450 2024-05-14T11:24:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f112c0b67c0 con 0x7f113c06f450 2024-05-14T11:24:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1132000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 0x7f1118071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7f112c1294d0 con 0x7f113c06f450 2024-05-14T11:24:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1133e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 0x7f1118071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.607+0000 7f1133e00640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 0x7f1118071910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f113c0c7ff0 tx=0x7f1124008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.611+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f112c0c6020 con 0x7f113c06f450 2024-05-14T11:24:23.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.771+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f113c06a9c0 con 0x7f113c06f450 2024-05-14T11:24:23.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1132000640 1 -- 172.21.15.116:0/2381268085 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f113c06f860 con 0x7f113c06f450 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 msgr2=0x7f1118071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 0x7f1118071910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f113c0c7ff0 tx=0x7f1124008040 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 msgr2=0x7f113c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f112c0ae040 tx=0x7f112c01ab80 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 shutdown_connections 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f111806f400 0x7f1118071910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c0d1730 0x7f113c1f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0cce60 0x7f113c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 --2- 172.21.15.116:0/2381268085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f113c06f450 0x7f113c0729c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 >> 172.21.15.116:0/2381268085 conn(0x7f113c0b4720 msgr2=0x7f113c0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:23.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 shutdown_connections 2024-05-14T11:24:23.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:23.775+0000 7f1141000640 1 -- 172.21.15.116:0/2381268085 wait complete. 2024-05-14T11:24:23.797 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:24:23.798 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:24:23.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:24:23.979 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:24:23.979 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:23.979 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:23.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:24:23.988 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:24:23.988 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:23.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:24:23.999 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:23.999 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:24.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:24:24.039 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:24:24.039 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:24.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:24:24.048 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:24:24.048 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:24.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 Processor -- start 2024-05-14T11:24:24.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 -- start start 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3610156fa0 0x7f361006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36100af380 0x7f36100af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3610160510 con 0x7f36100d1670 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f36101573b0 con 0x7f36100af380 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.227+0000 7f3616200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3610161120 con 0x7f3610156fa0 2024-05-14T11:24:24.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36100af380 0x7f36100af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36100af380 0x7f36100af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38364/0 (socket says 172.21.15.116:38364) 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 -- 172.21.15.116:0/3552834970 learned_addr learned my addr 172.21.15.116:0/3552834970 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360f400640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3610156fa0 0x7f361006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 -- 172.21.15.116:0/3552834970 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3610156fa0 msgr2=0x7f361006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3610156fa0 0x7f361006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 -- 172.21.15.116:0/3552834970 >> v1:172.21.15.149:6789/0 conn(0x7f36100d1670 legacy=0x7f36100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 -- 172.21.15.116:0/3552834970 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36101607f0 con 0x7f36100af380 2024-05-14T11:24:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360fe00640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36100af790 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3608009e50 tx=0x7f360801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0cb35fcf76ac49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360ea00640 1 -- 172.21.15.116:0/3552834970 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36080a58d0 con 0x7f36100af380 2024-05-14T11:24:24.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360ea00640 1 -- 172.21.15.116:0/3552834970 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36080a5a70 con 0x7f36100af380 2024-05-14T11:24:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.231+0000 7f360ea00640 1 -- 172.21.15.116:0/3552834970 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36080a5e40 con 0x7f36100af380 2024-05-14T11:24:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- 172.21.15.116:0/3552834970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 msgr2=0x7f36100af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36100af790 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3608009e50 tx=0x7f360801ca00 comp rx=0 tx=0).stop 2024-05-14T11:24:24.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- 172.21.15.116:0/3552834970 shutdown_connections 2024-05-14T11:24:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36100af790 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 --2- 172.21.15.116:0/3552834970 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3610156fa0 0x7f361006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- 172.21.15.116:0/3552834970 >> 172.21.15.116:0/3552834970 conn(0x7f36100b4620 msgr2=0x7f36100b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- 172.21.15.116:0/3552834970 shutdown_connections 2024-05-14T11:24:24.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- 172.21.15.116:0/3552834970 wait complete. 2024-05-14T11:24:24.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 Processor -- start 2024-05-14T11:24:24.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.235+0000 7f3616200640 1 -- start start 2024-05-14T11:24:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36100d1670 0x7f36100cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3610156fa0 0x7f3610152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36100b0660 con 0x7f36100af380 2024-05-14T11:24:24.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36100b04e0 con 0x7f3610156fa0 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3610169810 con 0x7f36100d1670 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38378/0 (socket says 172.21.15.116:38378) 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 -- 172.21.15.116:0/1361841147 learned_addr learned my addr 172.21.15.116:0/1361841147 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3606a00640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36100d1670 0x7f36100cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360fe00640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3610156fa0 0x7f3610152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 -- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36100d1670 msgr2=0x7f36100cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36100d1670 0x7f36100cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 -- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3610156fa0 msgr2=0x7f3610152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3610156fa0 0x7f3610152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 -- 172.21.15.116:0/1361841147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36101561f0 con 0x7f36100af380 2024-05-14T11:24:24.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f360f400640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f360000d600 tx=0x7f360000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600004240 con 0x7f36100af380 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36000043e0 con 0x7f36100af380 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600003840 con 0x7f36100af380 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36101564d0 con 0x7f36100af380 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36100d01d0 con 0x7f36100af380 2024-05-14T11:24:24.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.239+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361006a9c0 con 0x7f36100af380 2024-05-14T11:24:24.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.243+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f36000039e0 con 0x7f36100af380 2024-05-14T11:24:24.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.243+0000 7f3607e00640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 0x7f35ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.243+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7f360008f270 con 0x7f36100af380 2024-05-14T11:24:24.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.243+0000 7f3606a00640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 0x7f35ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.243+0000 7f3606a00640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 0x7f35ec071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f36100b3910 tx=0x7f35f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.247+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f360005bcf0 con 0x7f36100af380 2024-05-14T11:24:24.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.439+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36100b3940 con 0x7f36100af380 2024-05-14T11:24:24.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.439+0000 7f3607e00640 1 -- 172.21.15.116:0/1361841147 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3878 (secure 0 0 0) 0x7f360005fff0 con 0x7f36100af380 2024-05-14T11:24:24.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:24.441 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3378214020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3378214020},{"type":"v1","addr":"172.21.15.149:6835","nonce":3378214020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/613702563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":613702563},{"type":"v1","addr":"172.21.15.149:6834","nonce":613702563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6610,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/3948023420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3948023420},{"type":"v1","addr":"172.21.15.116:6836","nonce":3948023420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3834370102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":3834370102},{"type":"v1","addr":"172.21.15.116:6837","nonce":3834370102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-05-14T11:24:24.442 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 76 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 msgr2=0x7f35ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 0x7f35ec071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f36100b3910 tx=0x7f35f8009040 comp rx=0 tx=0).stop 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 msgr2=0x7f36101fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f360000d600 tx=0x7f360000dac0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 shutdown_connections 2024-05-14T11:24:24.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f35ec06f400 0x7f35ec071910 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3610156fa0 0x7f3610152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36100d1670 0x7f36100cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 --2- 172.21.15.116:0/1361841147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36100af380 0x7f36101fb040 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 >> 172.21.15.116:0/1361841147 conn(0x7f36100b4620 msgr2=0x7f36100b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 shutdown_connections 2024-05-14T11:24:24.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.443+0000 7f3616200640 1 -- 172.21.15.116:0/1361841147 wait complete. 2024-05-14T11:24:24.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:24:24.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:24:24.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:24:24.464 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:24.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:24:24.465 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:24:24.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:24:24.465 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:24.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:24:24.465 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:24:24.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:24:24.465 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:24.465 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:24:24.466 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:24:24.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:24:24.466 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:24.478 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:24:24.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:24:24.480 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:24:24.480 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:24:24.482 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:24:24.482 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:24:24.491 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:24:24.491 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:24:24.524 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:24:24.525 DEBUG:teuthology.parallel:result is None 2024-05-14T11:24:24.527 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:24:24.527 DEBUG:teuthology.parallel:result is None 2024-05-14T11:24:24.534 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:24:24.534 DEBUG:teuthology.parallel:result is None 2024-05-14T11:24:24.536 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:24:24.536 DEBUG:teuthology.parallel:result is None 2024-05-14T11:24:24.537 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:24:24.625 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:24:24.628 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:24:24.639 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:24:24.642 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:24:24.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.691+0000 7f9cd0000640 1 Processor -- start 2024-05-14T11:24:24.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.691+0000 7f9cd0000640 1 -- start start 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cd0000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 0x7f9cc806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cd0000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9cc80af380 0x7f9cc80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cd0000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc8160510 con 0x7f9cc80d1670 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cd0000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc81573b0 con 0x7f9cc80af380 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cd0000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc8161120 con 0x7f9cc8156fa0 2024-05-14T11:24:24.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cccc00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9cc80d1670 0x7f9cc80d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59408/0 (socket says 172.21.15.116:59408) 2024-05-14T11:24:24.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cccc00640 1 -- 172.21.15.116:0/1940246640 learned_addr learned my addr 172.21.15.116:0/1940246640 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cce000640 1 --2- 172.21.15.116:0/1940246640 >> v2:172.21.15.149:3300/0 conn(0x7f9cc80af380 0x7f9cc80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9ccd600640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 0x7f9cc806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384823775 0 0) 0x7f9cc8160510 con 0x7f9cc80d1670 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb0003530 con 0x7f9cc80d1670 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42538358 0 0) 0x7f9cb0003530 con 0x7f9cc80d1670 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cce000640 1 -- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 msgr2=0x7f9cc806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cce000640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 0x7f9cc806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cce000640 1 -- 172.21.15.116:0/1940246640 >> v1:172.21.15.116:6789/0 conn(0x7f9cc80d1670 legacy=0x7f9cc80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9cce000640 1 -- 172.21.15.116:0/1940246640 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc81607f0 con 0x7f9cc80af380 2024-05-14T11:24:24.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.695+0000 7f9ccd600640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 0x7f9cc806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cce000640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc80af380 0x7f9cc80af790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009d90 tx=0x7f9cc001c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a8f313d939d493 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b2070 con 0x7f9cc80af380 2024-05-14T11:24:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc00a5660 con 0x7f9cc80af380 2024-05-14T11:24:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cc7e00640 1 -- 172.21.15.116:0/1940246640 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4001700 con 0x7f9cc80af380 2024-05-14T11:24:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 -- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc80af380 msgr2=0x7f9cc80af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc80af380 0x7f9cc80af790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009d90 tx=0x7f9cc001c860 comp rx=0 tx=0).stop 2024-05-14T11:24:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 -- 172.21.15.116:0/1940246640 shutdown_connections 2024-05-14T11:24:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc80af380 0x7f9cc80af790 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 --2- 172.21.15.116:0/1940246640 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc8156fa0 0x7f9cc806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 -- 172.21.15.116:0/1940246640 >> 172.21.15.116:0/1940246640 conn(0x7f9cc80b4620 msgr2=0x7f9cc80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 -- 172.21.15.116:0/1940246640 shutdown_connections 2024-05-14T11:24:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 -- 172.21.15.116:0/1940246640 wait complete. 2024-05-14T11:24:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.699+0000 7f9cd0000640 1 Processor -- start 2024-05-14T11:24:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- start start 2024-05-14T11:24:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc80d1670 0x7f9cc80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc8156fa0 0x7f9cc80b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80b0660 con 0x7f9cc80af380 2024-05-14T11:24:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80b04e0 con 0x7f9cc8156fa0 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc8169810 con 0x7f9cc80d1670 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38414/0 (socket says 172.21.15.116:38414) 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 -- 172.21.15.116:0/570786965 learned_addr learned my addr 172.21.15.116:0/570786965 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cce000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc8156fa0 0x7f9cc80b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cccc00640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc80d1670 0x7f9cc80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 -- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc80d1670 msgr2=0x7f9cc80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc80d1670 0x7f9cc80cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 -- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc8156fa0 msgr2=0x7f9cc80b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc8156fa0 0x7f9cc80b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 -- 172.21.15.116:0/570786965 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc809ae80 con 0x7f9cc80af380 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9ccd600640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e960 tx=0x7f9cbc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc017070 con 0x7f9cc80af380 2024-05-14T11:24:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00ca00 con 0x7f9cc80af380 2024-05-14T11:24:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00cdd0 con 0x7f9cc80af380 2024-05-14T11:24:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc80d0250 con 0x7f9cc80af380 2024-05-14T11:24:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc80d0530 con 0x7f9cc80af380 2024-05-14T11:24:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.703+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc806a9c0 con 0x7f9cc80af380 2024-05-14T11:24:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.707+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cbc01a070 con 0x7f9cc80af380 2024-05-14T11:24:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.707+0000 7f9cc5600640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 0x7f9ca8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.707+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5507+0+0 (secure 0 0 0) 0x7f9cbc08f100 con 0x7f9cc80af380 2024-05-14T11:24:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.707+0000 7f9cccc00640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 0x7f9ca8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.707+0000 7f9cccc00640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 0x7f9ca8071910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4001020 tx=0x7f9cb4000fb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.711+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9cbc05b9c0 con 0x7f9cc80af380 2024-05-14T11:24:24.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:24.879+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9cc81531f0 con 0x7f9cc80af380 2024-05-14T11:24:25.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cc5600640 1 -- 172.21.15.116:0/570786965 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) ==== 97+0+0 (secure 0 0 0) 0x7f9cbc05fcc0 con 0x7f9cc80af380 2024-05-14T11:24:25.428 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 msgr2=0x7f9ca8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 0x7f9ca8071910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4001020 tx=0x7f9cb4000fb0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 msgr2=0x7f9cc809a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e960 tx=0x7f9cbc00ee20 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 shutdown_connections 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9ca806f400 0x7f9ca8071910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc8156fa0 0x7f9cc80b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9cc80d1670 0x7f9cc80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 --2- 172.21.15.116:0/570786965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cc80af380 0x7f9cc809a940 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 >> 172.21.15.116:0/570786965 conn(0x7f9cc80b4620 msgr2=0x7f9cc80b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:25.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 shutdown_connections 2024-05-14T11:24:25.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.427+0000 7f9cd0000640 1 -- 172.21.15.116:0/570786965 wait complete. 2024-05-14T11:24:25.456 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:24:25.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 Processor -- start 2024-05-14T11:24:25.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 -- start start 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63ec0af830 0x7f63ec0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f63ec0d1bb0 0x7f63ec06edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ec0d1fc0 con 0x7f63ec0af420 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec15d340 con 0x7f63ec0af830 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f3600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec15d620 con 0x7f63ec0d1bb0 2024-05-14T11:24:25.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63ec0af830 0x7f63ec0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f0c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63ec0af420 0x7f63ec074630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59414/0 (socket says 172.21.15.116:59414) 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63ec0af830 0x7f63ec0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38422/0 (socket says 172.21.15.116:38422) 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63f0c00640 1 -- 172.21.15.116:0/2628906937 learned_addr learned my addr 172.21.15.116:0/2628906937 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 -- 172.21.15.116:0/2628906937 >> v2:172.21.15.149:3300/0 conn(0x7f63ec0d1bb0 msgr2=0x7f63ec06edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 --2- 172.21.15.116:0/2628906937 >> v2:172.21.15.149:3300/0 conn(0x7f63ec0d1bb0 0x7f63ec06edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 -- 172.21.15.116:0/2628906937 >> v1:172.21.15.116:6789/0 conn(0x7f63ec0af420 legacy=0x7f63ec074630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63ebe00640 1 -- 172.21.15.116:0/2628906937 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec15d900 con 0x7f63ec0af830 2024-05-14T11:24:25.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.603+0000 7f63eb400640 1 -- 172.21.15.116:0/2628906937 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336096042 0 0) 0x7f63e0001120 con 0x7f63ec0af420 2024-05-14T11:24:25.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63ebe00640 1 --2- 172.21.15.116:0/2628906937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec0d1670 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f63ec160420 tx=0x7f63d801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30490bd1566590b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:25.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63eb400640 1 -- 172.21.15.116:0/2628906937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b1070 con 0x7f63ec0af830 2024-05-14T11:24:25.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63eb400640 1 -- 172.21.15.116:0/2628906937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80ac5f0 con 0x7f63ec0af830 2024-05-14T11:24:25.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63eb400640 1 -- 172.21.15.116:0/2628906937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ac9c0 con 0x7f63ec0af830 2024-05-14T11:24:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 -- 172.21.15.116:0/2628906937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 msgr2=0x7f63ec0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 --2- 172.21.15.116:0/2628906937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec0d1670 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f63ec160420 tx=0x7f63d801c420 comp rx=0 tx=0).stop 2024-05-14T11:24:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 -- 172.21.15.116:0/2628906937 shutdown_connections 2024-05-14T11:24:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 --2- 172.21.15.116:0/2628906937 >> v2:172.21.15.149:3300/0 conn(0x7f63ec0d1bb0 0x7f63ec06edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 --2- 172.21.15.116:0/2628906937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec0d1670 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 -- 172.21.15.116:0/2628906937 >> 172.21.15.116:0/2628906937 conn(0x7f63ec0b4620 msgr2=0x7f63ec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:25.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 -- 172.21.15.116:0/2628906937 shutdown_connections 2024-05-14T11:24:25.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.607+0000 7f63f3600640 1 -- 172.21.15.116:0/2628906937 wait complete. 2024-05-14T11:24:25.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 Processor -- start 2024-05-14T11:24:25.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 -- start start 2024-05-14T11:24:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 0x7f63ec150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63ec0d1bb0 0x7f63ec155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec169880 con 0x7f63ec0af830 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec169a00 con 0x7f63ec0af420 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f3600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec169b80 con 0x7f63ec0d1bb0 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38438/0 (socket says 172.21.15.116:38438) 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63ebe00640 1 -- 172.21.15.116:0/1806000192 learned_addr learned my addr 172.21.15.116:0/1806000192 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.611+0000 7f63f0c00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 0x7f63ec150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63f1600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63ec0d1bb0 0x7f63ec155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 -- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63ec0d1bb0 msgr2=0x7f63ec155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63ec0d1bb0 0x7f63ec155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 -- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 msgr2=0x7f63ec150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 0x7f63ec150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 -- 172.21.15.116:0/1806000192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec153e60 con 0x7f63ec0af830 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63f0c00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 0x7f63ec150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63ebe00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f63d801c900 tx=0x7f63d80191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b1040 con 0x7f63ec0af830 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80bb070 con 0x7f63ec0af830 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b5650 con 0x7f63ec0af830 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec154140 con 0x7f63ec0af830 2024-05-14T11:24:25.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec0b1e10 con 0x7f63ec0af830 2024-05-14T11:24:25.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec06a9c0 con 0x7f63ec0af830 2024-05-14T11:24:25.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63d80b57f0 con 0x7f63ec0af830 2024-05-14T11:24:25.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.615+0000 7f63e8c00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 0x7f63cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:25.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.619+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7f63d8128bd0 con 0x7f63ec0af830 2024-05-14T11:24:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.619+0000 7f63f0c00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 0x7f63cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:25.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.619+0000 7f63f0c00640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 0x7f63cc071940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f70 tx=0x7f63e0007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:25.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.623+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f63d80c5020 con 0x7f63ec0af830 2024-05-14T11:24:25.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.835+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f63ec06fe00 con 0x7f63ec0af830 2024-05-14T11:24:25.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.835+0000 7f63e8c00640 1 -- 172.21.15.116:0/1806000192 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f63d80a9070 con 0x7f63ec0af830 2024-05-14T11:24:25.837 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:25.837 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.838 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.840 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.842 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.843 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.844 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:24:25.845 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:24:25.846 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.847 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:24:25.848 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.849 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.850 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:24:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:24:25.852 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 msgr2=0x7f63cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 0x7f63cc071940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f70 tx=0x7f63e0007cb0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 msgr2=0x7f63ec154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f63d801c900 tx=0x7f63d80191c0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 shutdown_connections 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f63cc06f430 0x7f63cc071940 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63ec0d1bb0 0x7f63ec155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ec0af830 0x7f63ec154bc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 --2- 172.21.15.116:0/1806000192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63ec0af420 0x7f63ec150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:25.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 >> 172.21.15.116:0/1806000192 conn(0x7f63ec0b4620 msgr2=0x7f63ec15f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:25.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 shutdown_connections 2024-05-14T11:24:25.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:25.839+0000 7f63f3600640 1 -- 172.21.15.116:0/1806000192 wait complete. 2024-05-14T11:24:25.859 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:24:26.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.003+0000 7f6f6ee00640 1 Processor -- start 2024-05-14T11:24:26.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.003+0000 7f6f6ee00640 1 -- start start 2024-05-14T11:24:26.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ee00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6f68166750 0x7f6f68168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ee00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6806ff20 con 0x7f6f6815db30 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ee00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f68070470 con 0x7f6f680d1670 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ee00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6806d710 con 0x7f6f68166750 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f67e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6f6815db30 0x7f6f680aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59418/0 (socket says 172.21.15.116:59418) 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f67e00640 1 -- 172.21.15.116:0/3856350582 learned_addr learned my addr 172.21.15.116:0/3856350582 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ce00640 1 --2- 172.21.15.116:0/3856350582 >> v2:172.21.15.116:3300/0 conn(0x7f6f68166750 0x7f6f68168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f67400640 1 -- 172.21.15.116:0/3856350582 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076795388 0 0) 0x7f6f6806ff20 con 0x7f6f6815db30 2024-05-14T11:24:26.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f67400640 1 -- 172.21.15.116:0/3856350582 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f4c003530 con 0x7f6f6815db30 2024-05-14T11:24:26.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ce00640 1 -- 172.21.15.116:0/3856350582 >> v1:172.21.15.149:6789/0 conn(0x7f6f680d1670 legacy=0x7f6f680d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ce00640 1 -- 172.21.15.116:0/3856350582 >> v1:172.21.15.116:6789/0 conn(0x7f6f6815db30 legacy=0x7f6f680aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.007+0000 7f6f6ce00640 1 -- 172.21.15.116:0/3856350582 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f68070750 con 0x7f6f68166750 2024-05-14T11:24:26.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ce00640 1 --2- 172.21.15.116:0/3856350582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f68166750 0x7f6f68168b70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009d90 tx=0x7f6f6001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b0924cf7ea123e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f67400640 1 -- 172.21.15.116:0/3856350582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600b2070 con 0x7f6f68166750 2024-05-14T11:24:26.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f67400640 1 -- 172.21.15.116:0/3856350582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f600a5710 con 0x7f6f68166750 2024-05-14T11:24:26.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f67400640 1 -- 172.21.15.116:0/3856350582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600a5ae0 con 0x7f6f68166750 2024-05-14T11:24:26.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3856350582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f68166750 msgr2=0x7f6f68168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3856350582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f68166750 0x7f6f68168b70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009d90 tx=0x7f6f6001c850 comp rx=0 tx=0).stop 2024-05-14T11:24:26.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3856350582 shutdown_connections 2024-05-14T11:24:26.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3856350582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f68166750 0x7f6f68168b70 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3856350582 >> 172.21.15.116:0/3856350582 conn(0x7f6f680b4620 msgr2=0x7f6f680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:26.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3856350582 shutdown_connections 2024-05-14T11:24:26.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3856350582 wait complete. 2024-05-14T11:24:26.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 Processor -- start 2024-05-14T11:24:26.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.011+0000 7f6f6ee00640 1 -- start start 2024-05-14T11:24:26.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6f680d1670 0x7f6f68165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f68166750 0x7f6f680cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6816bae0 con 0x7f6f6815db30 2024-05-14T11:24:26.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680ae870 con 0x7f6f68166750 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6816bc60 con 0x7f6f680d1670 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38446/0 (socket says 172.21.15.116:38446) 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 -- 172.21.15.116:0/3645257631 learned_addr learned my addr 172.21.15.116:0/3645257631 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ce00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f68166750 0x7f6f680cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f67e00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6f680d1670 0x7f6f68165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 -- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6f680d1670 msgr2=0x7f6f68165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6f680d1670 0x7f6f68165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 -- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f68166750 msgr2=0x7f6f680cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f68166750 0x7f6f680cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 -- 172.21.15.116:0/3645257631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f680b3bd0 con 0x7f6f6815db30 2024-05-14T11:24:26.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f5f400640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800c3f0 tx=0x7f6f5800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58017070 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f580052d0 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f680b3050 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58003820 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f680b3610 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.015+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6806a9c0 con 0x7f6f6815db30 2024-05-14T11:24:26.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.019+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6f5801a070 con 0x7f6f6815db30 2024-05-14T11:24:26.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.019+0000 7f6f64c00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 0x7f6f44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.019+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7f6f5808e360 con 0x7f6f6815db30 2024-05-14T11:24:26.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.019+0000 7f6f67e00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 0x7f6f44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.019+0000 7f6f67e00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 0x7f6f44071940 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f50007a70 tx=0x7f6f500079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.023+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6f58005480 con 0x7f6f6815db30 2024-05-14T11:24:26.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.183+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6f680aec40 con 0x7f6f6815db30 2024-05-14T11:24:26.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.183+0000 7f6f64c00640 1 -- 172.21.15.116:0/3645257631 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) ==== 84+0+65 (secure 0 0 0) 0x7f6f580633f0 con 0x7f6f6815db30 2024-05-14T11:24:26.185 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:26.186 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:24:26.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 msgr2=0x7f6f44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 0x7f6f44071940 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f50007a70 tx=0x7f6f500079c0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 msgr2=0x7f6f68073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800c3f0 tx=0x7f6f5800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 shutdown_connections 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f4406f430 0x7f6f44071940 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f68166750 0x7f6f680cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f6815db30 0x7f6f68073000 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 --2- 172.21.15.116:0/3645257631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6f680d1670 0x7f6f68165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 >> 172.21.15.116:0/3645257631 conn(0x7f6f680b4620 msgr2=0x7f6f6815ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 shutdown_connections 2024-05-14T11:24:26.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.187+0000 7f6f6ee00640 1 -- 172.21.15.116:0/3645257631 wait complete. 2024-05-14T11:24:26.207 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:24:26.207 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:24:26.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.355+0000 7fc6d5400640 1 Processor -- start 2024-05-14T11:24:26.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.355+0000 7fc6d5400640 1 -- start start 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6d5400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 0x7fc6d006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6d5400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6d5400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6d00d04c0 con 0x7fc6d00cda30 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6d5400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fc6d00d0270 con 0x7fc6d00cd620 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6d5400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d00b0c30 con 0x7fc6d006d710 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6ce800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 0x7fc6d006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:60384/0 (socket says 172.21.15.116:60384) 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6c5e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fc6d00cda30 0x7fc6d0161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:46084/0 (socket says 172.21.15.116:46084) 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 -- 172.21.15.116:0/1644266531 learned_addr learned my addr 172.21.15.116:0/1644266531 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6ce800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 0x7fc6d006f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:59242/0 (socket says 172.21.15.116:59242) 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 -- 172.21.15.116:0/1644266531 >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 msgr2=0x7fc6d006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 --2- 172.21.15.116:0/1644266531 >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 0x7fc6d006f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 -- 172.21.15.116:0/1644266531 >> v1:172.21.15.149:6789/0 conn(0x7fc6d00cda30 legacy=0x7fc6d0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.359+0000 7fc6cf200640 1 -- 172.21.15.116:0/1644266531 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d00d07a0 con 0x7fc6d006d710 2024-05-14T11:24:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6cf200640 1 --2- 172.21.15.116:0/1644266531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009e50 tx=0x7fc6c801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687b0bcd2a9d456 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6cde00640 1 -- 172.21.15.116:0/1644266531 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80a5610 con 0x7fc6d006d710 2024-05-14T11:24:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6cde00640 1 -- 172.21.15.116:0/1644266531 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c80a57b0 con 0x7fc6d006d710 2024-05-14T11:24:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6cde00640 1 -- 172.21.15.116:0/1644266531 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80a5b80 con 0x7fc6d006d710 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- 172.21.15.116:0/1644266531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 msgr2=0x7fc6d006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 --2- 172.21.15.116:0/1644266531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009e50 tx=0x7fc6c801ca30 comp rx=0 tx=0).stop 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- 172.21.15.116:0/1644266531 shutdown_connections 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 --2- 172.21.15.116:0/1644266531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d006d710 0x7fc6d006fbf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 --2- 172.21.15.116:0/1644266531 >> v2:172.21.15.149:3300/0 conn(0x7fc6d00cd620 0x7fc6d006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- 172.21.15.116:0/1644266531 >> 172.21.15.116:0/1644266531 conn(0x7fc6d00cde80 msgr2=0x7fc6d00ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- 172.21.15.116:0/1644266531 shutdown_connections 2024-05-14T11:24:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- 172.21.15.116:0/1644266531 wait complete. 2024-05-14T11:24:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 Processor -- start 2024-05-14T11:24:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.363+0000 7fc6d5400640 1 -- start start 2024-05-14T11:24:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d00cd620 0x7fc6d015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6d015e780 0x7fc6d00b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d016ae40 con 0x7fc6d006d710 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d016acc0 con 0x7fc6d015e780 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d016afc0 con 0x7fc6d00cd620 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38452/0 (socket says 172.21.15.116:38452) 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 -- 172.21.15.116:0/756537187 learned_addr learned my addr 172.21.15.116:0/756537187 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6cf200640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6d015e780 0x7fc6d00b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6c5e00640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d00cd620 0x7fc6d015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 -- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d00cd620 msgr2=0x7fc6d015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d00cd620 0x7fc6d015e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 -- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6d015e780 msgr2=0x7fc6d00b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6d015e780 0x7fc6d00b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 -- 172.21.15.116:0/756537187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d015efb0 con 0x7fc6d006d710 2024-05-14T11:24:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6ce800640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e8a0 tx=0x7fc6c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0017070 con 0x7fc6d006d710 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c000c940 con 0x7fc6d006d710 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d015f290 con 0x7fc6d006d710 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000ccf0 con 0x7fc6d006d710 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d015f850 con 0x7fc6d006d710 2024-05-14T11:24:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.367+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d006aa30 con 0x7fc6d006d710 2024-05-14T11:24:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.371+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc6c001a070 con 0x7fc6d006d710 2024-05-14T11:24:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.371+0000 7fc6c7400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 0x7fc6ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.371+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7fc6c008f2d0 con 0x7fc6d006d710 2024-05-14T11:24:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.371+0000 7fc6c5e00640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 0x7fc6ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.371+0000 7fc6c5e00640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 0x7fc6ac071940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01fc8f0 tx=0x7fc6b8007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.375+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc6c005bd20 con 0x7fc6d006d710 2024-05-14T11:24:26.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:26.539+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc6d0156f50 con 0x7fc6d006d710 2024-05-14T11:24:27.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6c7400640 1 -- 172.21.15.116:0/756537187 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) ==== 138+0+0 (secure 0 0 0) 0x7fc6c00620b0 con 0x7fc6d006d710 2024-05-14T11:24:27.443 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 msgr2=0x7fc6ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 0x7fc6ac071940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01fc8f0 tx=0x7fc6b8007cb0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 msgr2=0x7fc6d013ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e8a0 tx=0x7fc6c000eca0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 shutdown_connections 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fc6ac06f430 0x7fc6ac071940 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6d015e780 0x7fc6d00b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc6d00cd620 0x7fc6d015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 --2- 172.21.15.116:0/756537187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6d006d710 0x7fc6d013ed30 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 >> 172.21.15.116:0/756537187 conn(0x7fc6d00cde80 msgr2=0x7fc6d014ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 shutdown_connections 2024-05-14T11:24:27.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.443+0000 7fc6d5400640 1 -- 172.21.15.116:0/756537187 wait complete. 2024-05-14T11:24:27.473 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:24:27.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 Processor -- start 2024-05-14T11:24:27.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 -- start start 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e400b2f80 0x7f1e400d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1e400b3390 0x7f1e400d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e400cdfb0 con 0x7f1e4015fc70 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1e400ce250 con 0x7f1e400b2f80 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e48600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1e400ce530 con 0x7f1e400b3390 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e45200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e400b2f80 0x7f1e400d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e45200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e400b2f80 0x7f1e400d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53832/0 (socket says 172.21.15.116:53832) 2024-05-14T11:24:27.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e45200640 1 -- 172.21.15.116:0/83999391 learned_addr learned my addr 172.21.15.116:0/83999391 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.627+0000 7f1e46600640 1 --2- 172.21.15.116:0/83999391 >> v2:172.21.15.149:3300/0 conn(0x7f1e400b3390 0x7f1e400d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e45200640 1 -- 172.21.15.116:0/83999391 >> v2:172.21.15.149:3300/0 conn(0x7f1e400b3390 msgr2=0x7f1e400d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e45200640 1 --2- 172.21.15.116:0/83999391 >> v2:172.21.15.149:3300/0 conn(0x7f1e400b3390 0x7f1e400d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e45200640 1 -- 172.21.15.116:0/83999391 >> v1:172.21.15.149:6789/0 conn(0x7f1e4015fc70 legacy=0x7f1e4006f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:27.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e45200640 1 -- 172.21.15.116:0/83999391 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e400ce810 con 0x7f1e400b2f80 2024-05-14T11:24:27.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e45200640 1 --2- 172.21.15.116:0/83999391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400d1420 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1e400d2fb0 tx=0x7f1e2c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d19644e27acf2c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:27.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e3fe00640 1 -- 172.21.15.116:0/83999391 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0b5070 con 0x7f1e400b2f80 2024-05-14T11:24:27.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e3fe00640 1 -- 172.21.15.116:0/83999391 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c0a89d0 con 0x7f1e400b2f80 2024-05-14T11:24:27.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.631+0000 7f1e3fe00640 1 -- 172.21.15.116:0/83999391 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0a8da0 con 0x7f1e400b2f80 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- 172.21.15.116:0/83999391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 msgr2=0x7f1e400d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 --2- 172.21.15.116:0/83999391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400d1420 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1e400d2fb0 tx=0x7f1e2c01ed10 comp rx=0 tx=0).stop 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- 172.21.15.116:0/83999391 shutdown_connections 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 --2- 172.21.15.116:0/83999391 >> v2:172.21.15.149:3300/0 conn(0x7f1e400b3390 0x7f1e400d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 --2- 172.21.15.116:0/83999391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400d1420 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- 172.21.15.116:0/83999391 >> 172.21.15.116:0/83999391 conn(0x7f1e400b5710 msgr2=0x7f1e400d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- 172.21.15.116:0/83999391 shutdown_connections 2024-05-14T11:24:27.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- 172.21.15.116:0/83999391 wait complete. 2024-05-14T11:24:27.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 Processor -- start 2024-05-14T11:24:27.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 -- start start 2024-05-14T11:24:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.635+0000 7f1e48600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 0x7f1e40156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 0x7f1e401f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40160200 con 0x7f1e400b2f80 2024-05-14T11:24:27.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1e40160080 con 0x7f1e4015fc70 2024-05-14T11:24:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1e400ced30 con 0x7f1e400b3390 2024-05-14T11:24:27.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53848/0 (socket says 172.21.15.116:53848) 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 -- 172.21.15.116:0/2362862680 learned_addr learned my addr 172.21.15.116:0/2362862680 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e46600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 0x7f1e401f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45200640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 0x7f1e40156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 -- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 msgr2=0x7f1e40156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 0x7f1e40156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 -- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 msgr2=0x7f1e401f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 0x7f1e401f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 -- 172.21.15.116:0/2362862680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40070b10 con 0x7f1e400b2f80 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e46600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 0x7f1e401f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45200640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 0x7f1e40156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e45c00640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400b9c0 tx=0x7f1e3400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400cba0 con 0x7f1e400b2f80 2024-05-14T11:24:27.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3400cd40 con 0x7f1e400b2f80 2024-05-14T11:24:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e40070df0 con 0x7f1e400b2f80 2024-05-14T11:24:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34003bc0 con 0x7f1e400b2f80 2024-05-14T11:24:27.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e400713b0 con 0x7f1e400b2f80 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4006a9c0 con 0x7f1e400b2f80 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.639+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e34006d10 con 0x7f1e400b2f80 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.643+0000 7f1e3d600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 0x7f1e20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.643+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5619+0+0 (secure 0 0 0) 0x7f1e3408ee60 con 0x7f1e400b2f80 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.643+0000 7f1e45200640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 0x7f1e20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:27.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.643+0000 7f1e45200640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 0x7f1e20071940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b2000 tx=0x7f1e2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:27.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.647+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1e3405bd90 con 0x7f1e400b2f80 2024-05-14T11:24:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:27.803+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e400d3df0 con 0x7f1e400b2f80 2024-05-14T11:24:28.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.451+0000 7f1e3d600640 1 -- 172.21.15.116:0/2362862680 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) ==== 170+0+0 (secure 0 0 0) 0x7f1e3405b690 con 0x7f1e400b2f80 2024-05-14T11:24:28.455 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:24:28.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 msgr2=0x7f1e20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 0x7f1e20071940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0b2000 tx=0x7f1e2c008040 comp rx=0 tx=0).stop 2024-05-14T11:24:28.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 msgr2=0x7f1e400b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400b9c0 tx=0x7f1e3400be80 comp rx=0 tx=0).stop 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 shutdown_connections 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1e2006f430 0x7f1e20071940 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e4015fc70 0x7f1e401f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1e400b3390 0x7f1e40156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 --2- 172.21.15.116:0/2362862680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e400b2f80 0x7f1e400b6d70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 >> 172.21.15.116:0/2362862680 conn(0x7f1e400b5710 msgr2=0x7f1e40157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 shutdown_connections 2024-05-14T11:24:28.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.455+0000 7f1e48600640 1 -- 172.21.15.116:0/2362862680 wait complete. 2024-05-14T11:24:28.495 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:24:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 Processor -- start 2024-05-14T11:24:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 -- start start 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd3240cd1c0 0x7fd3240cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32406f0a0 con 0x7fd3240d1730 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3240cbb60 con 0x7fd32406c700 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd32b800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd32406f420 con 0x7fd3240cd1c0 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd31be00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd3240d1730 0x7fd3240d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52332/0 (socket says 172.21.15.116:52332) 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd31be00640 1 -- 172.21.15.116:0/1343878786 learned_addr learned my addr 172.21.15.116:0/1343878786 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd328e00640 1 --2- 172.21.15.116:0/1343878786 >> v2:172.21.15.149:3300/0 conn(0x7fd3240cd1c0 0x7fd3240cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452493528 0 0) 0x7fd32406f0a0 con 0x7fd3240d1730 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.643+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd308003530 con 0x7fd3240d1730 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd328e00640 1 -- 172.21.15.116:0/1343878786 >> v1:172.21.15.149:6789/0 conn(0x7fd32406c700 legacy=0x7fd32406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649908787 0 0) 0x7fd31c0014a0 con 0x7fd32406c700 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd328e00640 1 -- 172.21.15.116:0/1343878786 >> v1:172.21.15.116:6789/0 conn(0x7fd3240d1730 legacy=0x7fd3240d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd328e00640 1 -- 172.21.15.116:0/1343878786 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32406f700 con 0x7fd3240cd1c0 2024-05-14T11:24:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd328e00640 1 --2- 172.21.15.116:0/1343878786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd3240cd5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097c0 tx=0x7fd31401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46f6e826dc6655 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140aa600 con 0x7fd3240cd1c0 2024-05-14T11:24:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3140af070 con 0x7fd3240cd1c0 2024-05-14T11:24:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.647+0000 7fd323e00640 1 -- 172.21.15.116:0/1343878786 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140aa9c0 con 0x7fd3240cd1c0 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- 172.21.15.116:0/1343878786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 msgr2=0x7fd3240cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 --2- 172.21.15.116:0/1343878786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd3240cd5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097c0 tx=0x7fd31401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- 172.21.15.116:0/1343878786 shutdown_connections 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 --2- 172.21.15.116:0/1343878786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd3240cd5d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- 172.21.15.116:0/1343878786 >> 172.21.15.116:0/1343878786 conn(0x7fd3240b4720 msgr2=0x7fd3240b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- 172.21.15.116:0/1343878786 shutdown_connections 2024-05-14T11:24:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- 172.21.15.116:0/1343878786 wait complete. 2024-05-14T11:24:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 Processor -- start 2024-05-14T11:24:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- start start 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd324168150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3240d1730 0x7fd32409a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd32416a9c0 con 0x7fd32406c700 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.651+0000 7fd32b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd32415cf10 con 0x7fd3240cd1c0 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd32b800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd32416ab40 con 0x7fd3240d1730 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53850/0 (socket says 172.21.15.116:53850) 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 -- 172.21.15.116:0/3383123119 learned_addr learned my addr 172.21.15.116:0/3383123119 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd31be00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd324168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd329800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3240d1730 0x7fd32409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 -- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3240d1730 msgr2=0x7fd32409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3240d1730 0x7fd32409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 -- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 msgr2=0x7fd324168150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd324168150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 -- 172.21.15.116:0/3383123119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd324168690 con 0x7fd32406c700 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd31be00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd324168150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd328e00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097c0 tx=0x7fd31401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140be600 con 0x7fd32406c700 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3140be7a0 con 0x7fd32406c700 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140beb90 con 0x7fd32406c700 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3241670b0 con 0x7fd32406c700 2024-05-14T11:24:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd324167670 con 0x7fd32406c700 2024-05-14T11:24:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32406a9c0 con 0x7fd32406c700 2024-05-14T11:24:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.655+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3140a9070 con 0x7fd32406c700 2024-05-14T11:24:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.659+0000 7fd321600640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 0x7fd300071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.659+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6001+0+0 (secure 0 0 0) 0x7fd314129ff0 con 0x7fd32406c700 2024-05-14T11:24:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.659+0000 7fd31be00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 0x7fd300071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.659+0000 7fd31be00640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 0x7fd300071910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005890 tx=0x7fd30c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:28.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.663+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd314016090 con 0x7fd32406c700 2024-05-14T11:24:28.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:28.851+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd324069de0 con 0x7fd32406c700 2024-05-14T11:24:29.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.471+0000 7fd321600640 1 -- 172.21.15.116:0/3383123119 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v79) ==== 174+0+0 (secure 0 0 0) 0x7fd3140f66c0 con 0x7fd32406c700 2024-05-14T11:24:29.475 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 15 and data pool 16 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 msgr2=0x7fd300071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 0x7fd300071910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005890 tx=0x7fd30c009040 comp rx=0 tx=0).stop 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 msgr2=0x7fd32415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097c0 tx=0x7fd31401ace0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 shutdown_connections 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd30006f400 0x7fd300071910 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005890 tx=0x7fd30c009040 comp rx=0 tx=0).stop 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3240cd1c0 0x7fd324168150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd3240d1730 0x7fd32409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 --2- 172.21.15.116:0/3383123119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd32406c700 0x7fd32415bfd0 secure :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097c0 tx=0x7fd31401ace0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.483+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 >> 172.21.15.116:0/3383123119 conn(0x7fd3240b4720 msgr2=0x7fd3240c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.487+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 shutdown_connections 2024-05-14T11:24:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.487+0000 7fd32b800640 1 -- 172.21.15.116:0/3383123119 wait complete. 2024-05-14T11:24:29.521 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:29.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 Processor -- start 2024-05-14T11:24:29.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 -- start start 2024-05-14T11:24:29.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f51b41432c0 0x7f51b41436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:29.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b4143c10 con 0x7f51b415eb60 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b40b0da0 con 0x7f51b40cdac0 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b9800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f51b4140f90 con 0x7f51b41432c0 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b2a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f51b415eb60 0x7f51b40b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52336/0 (socket says 172.21.15.116:52336) 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b2a00640 1 -- 172.21.15.116:0/2707977063 learned_addr learned my addr 172.21.15.116:0/2707977063 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51aa000640 1 --2- 172.21.15.116:0/2707977063 >> v2:172.21.15.116:3300/0 conn(0x7f51b41432c0 0x7f51b41436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b2000640 1 -- 172.21.15.116:0/2707977063 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28543585 0 0) 0x7f51a4001120 con 0x7f51b415eb60 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51b2000640 1 -- 172.21.15.116:0/2707977063 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5198003530 con 0x7f51b415eb60 2024-05-14T11:24:29.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51aa000640 1 -- 172.21.15.116:0/2707977063 >> v1:172.21.15.149:6789/0 conn(0x7f51b40cdac0 legacy=0x7f51b40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51aa000640 1 -- 172.21.15.116:0/2707977063 >> v1:172.21.15.116:6789/0 conn(0x7f51b415eb60 legacy=0x7f51b40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.667+0000 7f51aa000640 1 -- 172.21.15.116:0/2707977063 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b406c7a0 con 0x7f51b41432c0 2024-05-14T11:24:29.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51aa000640 1 --2- 172.21.15.116:0/2707977063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b41432c0 0x7f51b41436d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f519c0097c0 tx=0x7f519c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8237edca4313114 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:29.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b2000640 1 -- 172.21.15.116:0/2707977063 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0b0070 con 0x7f51b41432c0 2024-05-14T11:24:29.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b2000640 1 -- 172.21.15.116:0/2707977063 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519c0ab600 con 0x7f51b41432c0 2024-05-14T11:24:29.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b2000640 1 -- 172.21.15.116:0/2707977063 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0ab9b0 con 0x7f51b41432c0 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 -- 172.21.15.116:0/2707977063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b41432c0 msgr2=0x7f51b41436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 --2- 172.21.15.116:0/2707977063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b41432c0 0x7f51b41436d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f519c0097c0 tx=0x7f519c01c410 comp rx=0 tx=0).stop 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 -- 172.21.15.116:0/2707977063 shutdown_connections 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 --2- 172.21.15.116:0/2707977063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b41432c0 0x7f51b41436d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 -- 172.21.15.116:0/2707977063 >> 172.21.15.116:0/2707977063 conn(0x7f51b40ce6b0 msgr2=0x7f51b40ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 -- 172.21.15.116:0/2707977063 shutdown_connections 2024-05-14T11:24:29.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.671+0000 7f51b9800640 1 -- 172.21.15.116:0/2707977063 wait complete. 2024-05-14T11:24:29.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 Processor -- start 2024-05-14T11:24:29.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 -- start start 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b41432c0 0x7f51b413e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 0x7f51b41422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406e7c0 con 0x7f51b40cdac0 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406cd60 con 0x7f51b415eb60 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b9800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f51b40702a0 con 0x7f51b41432c0 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53870/0 (socket says 172.21.15.116:53870) 2024-05-14T11:24:29.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 -- 172.21.15.116:0/1350384049 learned_addr learned my addr 172.21.15.116:0/1350384049 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b3400640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 0x7f51b41422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 -- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b41432c0 msgr2=0x7f51b413e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b41432c0 0x7f51b413e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 -- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 msgr2=0x7f51b41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 0x7f51b41422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b2a00640 1 -- 172.21.15.116:0/1350384049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b4158fa0 con 0x7f51b40cdac0 2024-05-14T11:24:29.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.675+0000 7f51b3400640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 0x7f51b41422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51b2a00640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c760 tx=0x7f51a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4017070 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a40058c0 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b4158ce0 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4005cb0 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b413f3f0 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b406a9c0 con 0x7f51b40cdac0 2024-05-14T11:24:29.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.679+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51a401a070 con 0x7f51b40cdac0 2024-05-14T11:24:29.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.683+0000 7f51ab400640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 0x7f5190071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:29.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.683+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7f51a408e710 con 0x7f51b40cdac0 2024-05-14T11:24:29.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.683+0000 7f51aa000640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 0x7f5190071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:29.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.683+0000 7f51aa000640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 0x7f5190071940 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f519c0ad000 tx=0x7f519c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:29.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.687+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f51b4143c10 con 0x7f51b40cdac0 2024-05-14T11:24:29.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.847+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b40b3450 con 0x7f51b40cdac0 2024-05-14T11:24:29.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.847+0000 7f51ab400640 1 -- 172.21.15.116:0/1350384049 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16934 (secure 0 0 0) 0x7f51a405b3d0 con 0x7f51b40cdac0 2024-05-14T11:24:29.849 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:29.849 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":98,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:24:28.859782+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:24:26.553545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-14T11:24:27.820765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:29.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 msgr2=0x7f5190071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 0x7f5190071940 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f519c0ad000 tx=0x7f519c004740 comp rx=0 tx=0).stop 2024-05-14T11:24:29.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 msgr2=0x7f51b41587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:29.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c760 tx=0x7f51a400cc20 comp rx=0 tx=0).stop 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 shutdown_connections 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f519006f430 0x7f5190071940 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b415eb60 0x7f51b41422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b41432c0 0x7f51b413e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 --2- 172.21.15.116:0/1350384049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51b40cdac0 0x7f51b41587a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 >> 172.21.15.116:0/1350384049 conn(0x7f51b40ce6b0 msgr2=0x7f51b40cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 shutdown_connections 2024-05-14T11:24:29.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:29.851+0000 7f51b9800640 1 -- 172.21.15.116:0/1350384049 wait complete. 2024-05-14T11:24:29.871 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.015+0000 7f2654400640 1 Processor -- start 2024-05-14T11:24:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.015+0000 7f2654400640 1 -- start start 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2654400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f264c156fa0 0x7f264c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2654400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c160510 con 0x7f264c0af380 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2654400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c1607f0 con 0x7f264c0d1670 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2654400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f264c1573b0 con 0x7f264c156fa0 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2651a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f264c156fa0 0x7f264c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2652400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f264c0af380 0x7f264c0af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52344/0 (socket says 172.21.15.116:52344) 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2652400640 1 -- 172.21.15.116:0/4236324200 learned_addr learned my addr 172.21.15.116:0/4236324200 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:30.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2651a00640 1 -- 172.21.15.116:0/4236324200 >> v1:172.21.15.149:6789/0 conn(0x7f264c0d1670 legacy=0x7f264c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:30.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2651a00640 1 -- 172.21.15.116:0/4236324200 >> v1:172.21.15.116:6789/0 conn(0x7f264c0af380 legacy=0x7f264c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f264be00640 1 -- 172.21.15.116:0/4236324200 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094835581 0 0) 0x7f264c160510 con 0x7f264c0af380 2024-05-14T11:24:30.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.019+0000 7f2651a00640 1 -- 172.21.15.116:0/4236324200 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c160ad0 con 0x7f264c156fa0 2024-05-14T11:24:30.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2651a00640 1 --2- 172.21.15.116:0/4236324200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c06ffe0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f26400097c0 tx=0x7f264001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3563c8167e3c1411 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f264be00640 1 -- 172.21.15.116:0/4236324200 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400b1070 con 0x7f264c156fa0 2024-05-14T11:24:30.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f264be00640 1 -- 172.21.15.116:0/4236324200 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26400ac410 con 0x7f264c156fa0 2024-05-14T11:24:30.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f264be00640 1 -- 172.21.15.116:0/4236324200 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400ac7e0 con 0x7f264c156fa0 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 -- 172.21.15.116:0/4236324200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 msgr2=0x7f264c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 --2- 172.21.15.116:0/4236324200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c06ffe0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f26400097c0 tx=0x7f264001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 -- 172.21.15.116:0/4236324200 shutdown_connections 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 --2- 172.21.15.116:0/4236324200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c06ffe0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 -- 172.21.15.116:0/4236324200 >> 172.21.15.116:0/4236324200 conn(0x7f264c0b4620 msgr2=0x7f264c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:30.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 -- 172.21.15.116:0/4236324200 shutdown_connections 2024-05-14T11:24:30.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 -- 172.21.15.116:0/4236324200 wait complete. 2024-05-14T11:24:30.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.023+0000 7f2654400640 1 Processor -- start 2024-05-14T11:24:30.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- start start 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f264c0af380 0x7f264c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c0d1670 0x7f264c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f264c16abb0 con 0x7f264c156fa0 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f264c16aa30 con 0x7f264c0d1670 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f264c16ad30 con 0x7f264c0af380 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53884/0 (socket says 172.21.15.116:53884) 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 -- 172.21.15.116:0/2139650733 learned_addr learned my addr 172.21.15.116:0/2139650733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2651000640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c0d1670 0x7f264c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2651a00640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f264c0af380 0x7f264c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 -- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f264c0af380 msgr2=0x7f264c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f264c0af380 0x7f264c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 -- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c0d1670 msgr2=0x7f264c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c0d1670 0x7f264c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 -- 172.21.15.116:0/2139650733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c151f10 con 0x7f264c156fa0 2024-05-14T11:24:30.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2652400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f264400c6b0 tx=0x7f264400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644018070 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2644004db0 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264400a880 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c1521f0 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1527b0 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.027+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c151c80 con 0x7f264c156fa0 2024-05-14T11:24:30.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.031+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f264401b070 con 0x7f264c156fa0 2024-05-14T11:24:30.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.031+0000 7f2649600640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 0x7f262c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.031+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7f264408ea70 con 0x7f264c156fa0 2024-05-14T11:24:30.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.031+0000 7f2651a00640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 0x7f262c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.031+0000 7f2651a00640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 0x7f262c071910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26400ae000 tx=0x7f2640008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.035+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f264405b0e0 con 0x7f264c156fa0 2024-05-14T11:24:30.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.191+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f264c06a9c0 con 0x7f264c156fa0 2024-05-14T11:24:30.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.191+0000 7f2649600640 1 -- 172.21.15.116:0/2139650733 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16934 (secure 0 0 0) 0x7f264405f3e0 con 0x7f264c156fa0 2024-05-14T11:24:30.194 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:30.194 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":98,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:24:28.859782+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:24:26.553545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-14T11:24:27.820765+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:30.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 msgr2=0x7f262c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 0x7f262c071910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26400ae000 tx=0x7f2640008040 comp rx=0 tx=0).stop 2024-05-14T11:24:30.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 msgr2=0x7f264c0b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f264400c6b0 tx=0x7f264400cb70 comp rx=0 tx=0).stop 2024-05-14T11:24:30.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 shutdown_connections 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f262c06f400 0x7f262c071910 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c0d1670 0x7f264c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264c156fa0 0x7f264c0b2010 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 --2- 172.21.15.116:0/2139650733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f264c0af380 0x7f264c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 >> 172.21.15.116:0/2139650733 conn(0x7f264c0b4620 msgr2=0x7f264c15c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 shutdown_connections 2024-05-14T11:24:30.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.195+0000 7f2654400640 1 -- 172.21.15.116:0/2139650733 wait complete. 2024-05-14T11:24:30.216 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:24:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 Processor -- start 2024-05-14T11:24:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 -- start start 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9f80cd5d0 0x7fe9f806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9f806f0a0 con 0x7fe9f80cd1c0 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9f80cbb60 con 0x7fe9f80d1e40 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9ffc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe9f806f420 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9fdc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9f80cd5d0 0x7fe9f806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9fd200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe9f80cd1c0 0x7fe9f80d1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52348/0 (socket says 172.21.15.116:52348) 2024-05-14T11:24:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.363+0000 7fe9fd200640 1 -- 172.21.15.116:0/71772117 learned_addr learned my addr 172.21.15.116:0/71772117 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9fdc00640 1 -- 172.21.15.116:0/71772117 >> v1:172.21.15.149:6789/0 conn(0x7fe9f80d1e40 legacy=0x7fe9f806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9fdc00640 1 -- 172.21.15.116:0/71772117 >> v1:172.21.15.116:6789/0 conn(0x7fe9f80cd1c0 legacy=0x7fe9f80d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9fdc00640 1 -- 172.21.15.116:0/71772117 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f806f700 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9fdc00640 1 --2- 172.21.15.116:0/71772117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80cd5d0 0x7fe9f806ce10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00097c0 tx=0x7fe9f001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40903da3f93dc495 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9f7400640 1 -- 172.21.15.116:0/71772117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00a7070 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9f7400640 1 -- 172.21.15.116:0/71772117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f00b0070 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9f7400640 1 -- 172.21.15.116:0/71772117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00ab890 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9ffc00640 1 -- 172.21.15.116:0/71772117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80cd5d0 msgr2=0x7fe9f806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.367+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/71772117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80cd5d0 0x7fe9f806ce10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00097c0 tx=0x7fe9f001c470 comp rx=0 tx=0).stop 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- 172.21.15.116:0/71772117 shutdown_connections 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/71772117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80cd5d0 0x7fe9f806ce10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- 172.21.15.116:0/71772117 >> 172.21.15.116:0/71772117 conn(0x7fe9f80b4720 msgr2=0x7fe9f80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- 172.21.15.116:0/71772117 shutdown_connections 2024-05-14T11:24:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- 172.21.15.116:0/71772117 wait complete. 2024-05-14T11:24:30.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 Processor -- start 2024-05-14T11:24:30.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- start start 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80cd1c0 0x7fe9f80c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9f80cd5d0 0x7fe9f80c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f815cf10 con 0x7fe9f80b2850 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f80cd9e0 con 0x7fe9f80cd1c0 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.371+0000 7fe9ffc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f806d760 con 0x7fe9f80cd5d0 2024-05-14T11:24:30.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53914/0 (socket says 172.21.15.116:53914) 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 -- 172.21.15.116:0/170067777 learned_addr learned my addr 172.21.15.116:0/170067777 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9f7e00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9f80cd5d0 0x7fe9f80c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fd200640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80cd1c0 0x7fe9f80c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 -- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9f80cd5d0 msgr2=0x7fe9f80c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9f80cd5d0 0x7fe9f80c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 -- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80cd1c0 msgr2=0x7fe9f80c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80cd1c0 0x7fe9f80c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:30.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 -- 172.21.15.116:0/170067777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f80c7cc0 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9fdc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe9f001a0a0 tx=0x7fe9f001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00bf610 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f00bf7b0 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00bfb80 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f80c70b0 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f80c7670 con 0x7fe9f80b2850 2024-05-14T11:24:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.375+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f806a9c0 con 0x7fe9f80b2850 2024-05-14T11:24:30.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.379+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9f00bd020 con 0x7fe9f80b2850 2024-05-14T11:24:30.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.379+0000 7fe9f4c00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 0x7fe9d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:30.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.379+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7fe9f01297f0 con 0x7fe9f80b2850 2024-05-14T11:24:30.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.379+0000 7fe9fd200640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 0x7fe9d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:30.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.379+0000 7fe9fd200640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 0x7fe9d8071940 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80012f0 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:30.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.383+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe9f00c6070 con 0x7fe9f80b2850 2024-05-14T11:24:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:30.583+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe9f80c82d0 con 0x7fe9f80b2850 2024-05-14T11:24:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.487+0000 7fe9f4c00640 1 -- 172.21.15.116:0/170067777 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7fe9f0016090 con 0x7fe9f80b2850 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 msgr2=0x7fe9d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 0x7fe9d8071940 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80012f0 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 msgr2=0x7fe9f80c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe9f001a0a0 tx=0x7fe9f001acb0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 shutdown_connections 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe9d806f430 0x7fe9d8071940 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9f80b2850 0x7fe9f80c8ab0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9f80cd5d0 0x7fe9f80c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 --2- 172.21.15.116:0/170067777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80cd1c0 0x7fe9f80c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 >> 172.21.15.116:0/170067777 conn(0x7fe9f80b4720 msgr2=0x7fe9f80cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 shutdown_connections 2024-05-14T11:24:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.491+0000 7fe9ffc00640 1 -- 172.21.15.116:0/170067777 wait complete. 2024-05-14T11:24:31.515 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:24:31.515 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:24:31.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.667+0000 7f0cf4600640 1 Processor -- start 2024-05-14T11:24:31.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.667+0000 7f0cf4600640 1 -- start start 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.667+0000 7f0cf4600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f0cec0ce450 0x7f0cec0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf4600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0cec0cbd60 0x7f0cec0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf4600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec06cc10 con 0x7f0cec15c1d0 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf4600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec06d790 con 0x7f0cec0cbd60 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf4600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec06cef0 con 0x7f0cec0ce450 2024-05-14T11:24:31.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0cec0cbd60 0x7f0cec0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0cec0cbd60 0x7f0cec0cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53918/0 (socket says 172.21.15.116:53918) 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 -- 172.21.15.116:0/3407563704 learned_addr learned my addr 172.21.15.116:0/3407563704 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf1200640 1 --2- 172.21.15.116:0/3407563704 >> v2:172.21.15.149:3300/0 conn(0x7f0cec0ce450 0x7f0cec0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 -- 172.21.15.116:0/3407563704 >> v2:172.21.15.149:3300/0 conn(0x7f0cec0ce450 msgr2=0x7f0cec0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 --2- 172.21.15.116:0/3407563704 >> v2:172.21.15.149:3300/0 conn(0x7f0cec0ce450 0x7f0cec0d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 -- 172.21.15.116:0/3407563704 >> v1:172.21.15.149:6789/0 conn(0x7f0cec15c1d0 legacy=0x7f0cec0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:31.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 -- 172.21.15.116:0/3407563704 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec06d1d0 con 0x7f0cec0cbd60 2024-05-14T11:24:31.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cf2600640 1 --2- 172.21.15.116:0/3407563704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0cc170 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b0e60 tx=0x7f0ce401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fffcb8c051ad320 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:31.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cebe00640 1 -- 172.21.15.116:0/3407563704 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40a64e0 con 0x7f0cec0cbd60 2024-05-14T11:24:31.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cebe00640 1 -- 172.21.15.116:0/3407563704 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40a6680 con 0x7f0cec0cbd60 2024-05-14T11:24:31.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.671+0000 7f0cebe00640 1 -- 172.21.15.116:0/3407563704 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40a6a50 con 0x7f0cec0cbd60 2024-05-14T11:24:31.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- 172.21.15.116:0/3407563704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec0cbd60 msgr2=0x7f0cec0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:31.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 --2- 172.21.15.116:0/3407563704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0cc170 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b0e60 tx=0x7f0ce401c8d0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- 172.21.15.116:0/3407563704 shutdown_connections 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 --2- 172.21.15.116:0/3407563704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0cc170 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 --2- 172.21.15.116:0/3407563704 >> v2:172.21.15.149:3300/0 conn(0x7f0cec0ce450 0x7f0cec0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- 172.21.15.116:0/3407563704 >> 172.21.15.116:0/3407563704 conn(0x7f0cec0b4720 msgr2=0x7f0cec0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- 172.21.15.116:0/3407563704 shutdown_connections 2024-05-14T11:24:31.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- 172.21.15.116:0/3407563704 wait complete. 2024-05-14T11:24:31.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 Processor -- start 2024-05-14T11:24:31.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.675+0000 7f0cf4600640 1 -- start start 2024-05-14T11:24:31.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0cec15c1d0 0x7f0cec092290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec0cca40 con 0x7f0cec072880 2024-05-14T11:24:31.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec0cc8c0 con 0x7f0cec0cbd60 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0ccbc0 con 0x7f0cec15c1d0 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53920/0 (socket says 172.21.15.116:53920) 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 -- 172.21.15.116:0/845586660 learned_addr learned my addr 172.21.15.116:0/845586660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf1200640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0cec15c1d0 0x7f0cec092290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf1c00640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 -- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0cec15c1d0 msgr2=0x7f0cec092290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0cec15c1d0 0x7f0cec092290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 -- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cec0cbd60 msgr2=0x7f0cec0c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 -- 172.21.15.116:0/845586660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec092d10 con 0x7f0cec072880 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf2600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401af80 tx=0x7f0ce4019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40a6c40 con 0x7f0cec072880 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40a6de0 con 0x7f0cec072880 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec097070 con 0x7f0cec072880 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40af6c0 con 0x7f0cec072880 2024-05-14T11:24:31.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec0917b0 con 0x7f0cec072880 2024-05-14T11:24:31.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.679+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec0c43e0 con 0x7f0cec072880 2024-05-14T11:24:31.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.683+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ce40af860 con 0x7f0cec072880 2024-05-14T11:24:31.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.683+0000 7f0ce9600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 0x7f0ccc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:31.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.683+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f0ce41298a0 con 0x7f0cec072880 2024-05-14T11:24:31.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.683+0000 7f0cf1c00640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 0x7f0ccc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:31.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.683+0000 7f0cf1c00640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 0x7f0ccc071910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00098e0 tx=0x7f0ce0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:31.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.687+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0cec0c43e0 con 0x7f0cec072880 2024-05-14T11:24:31.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:31.879+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0cec069de0 con 0x7f0cec072880 2024-05-14T11:24:32.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.491+0000 7f0ce9600640 1 -- 172.21.15.116:0/845586660 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f0ce40165e0 con 0x7f0cec072880 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 msgr2=0x7f0ccc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 0x7f0ccc071910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00098e0 tx=0x7f0ce0009060 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 msgr2=0x7f0cec0927d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401af80 tx=0x7f0ce4019670 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 shutdown_connections 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0ccc06f400 0x7f0ccc071910 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cec072880 0x7f0cec0927d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0cec15c1d0 0x7f0cec092290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 --2- 172.21.15.116:0/845586660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cec0cbd60 0x7f0cec0c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 >> 172.21.15.116:0/845586660 conn(0x7f0cec0b4720 msgr2=0x7f0cec098b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 shutdown_connections 2024-05-14T11:24:32.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.495+0000 7f0cf4600640 1 -- 172.21.15.116:0/845586660 wait complete. 2024-05-14T11:24:32.516 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:24:32.516 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:32.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 Processor -- start 2024-05-14T11:24:32.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 -- start start 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fed700d1c20 0x7fed700ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed700b3060 con 0x7fed700b40c0 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed700b3300 con 0x7fed700cdf80 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed74e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fed700b35e0 con 0x7fed700d1c20 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed65400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fed700d1c20 0x7fed700ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed6e800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fed700b40c0 0x7fed700b44d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52352/0 (socket says 172.21.15.116:52352) 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.663+0000 7fed6e800640 1 -- 172.21.15.116:0/3236169308 learned_addr learned my addr 172.21.15.116:0/3236169308 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed65400640 1 -- 172.21.15.116:0/3236169308 >> v1:172.21.15.149:6789/0 conn(0x7fed700cdf80 legacy=0x7fed700d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed65400640 1 -- 172.21.15.116:0/3236169308 >> v1:172.21.15.116:6789/0 conn(0x7fed700b40c0 legacy=0x7fed700b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed65400640 1 -- 172.21.15.116:0/3236169308 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed700b38c0 con 0x7fed700d1c20 2024-05-14T11:24:32.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed6d400640 1 -- 172.21.15.116:0/3236169308 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314903722 0 0) 0x7fed700b3060 con 0x7fed700b40c0 2024-05-14T11:24:32.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed65400640 1 --2- 172.21.15.116:0/3236169308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed700d1c20 0x7fed700ce390 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed600097c0 tx=0x7fed6001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39332bb7382a4784 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:32.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed6d400640 1 -- 172.21.15.116:0/3236169308 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600aa5f0 con 0x7fed700d1c20 2024-05-14T11:24:32.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed6d400640 1 -- 172.21.15.116:0/3236169308 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed600af070 con 0x7fed700d1c20 2024-05-14T11:24:32.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.667+0000 7fed6d400640 1 -- 172.21.15.116:0/3236169308 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600aa9b0 con 0x7fed700d1c20 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- 172.21.15.116:0/3236169308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed700d1c20 msgr2=0x7fed700ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 --2- 172.21.15.116:0/3236169308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed700d1c20 0x7fed700ce390 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed600097c0 tx=0x7fed6001c470 comp rx=0 tx=0).stop 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- 172.21.15.116:0/3236169308 shutdown_connections 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 --2- 172.21.15.116:0/3236169308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed700d1c20 0x7fed700ce390 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- 172.21.15.116:0/3236169308 >> 172.21.15.116:0/3236169308 conn(0x7fed700d0770 msgr2=0x7fed700d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- 172.21.15.116:0/3236169308 shutdown_connections 2024-05-14T11:24:32.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- 172.21.15.116:0/3236169308 wait complete. 2024-05-14T11:24:32.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 Processor -- start 2024-05-14T11:24:32.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- start start 2024-05-14T11:24:32.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed700b40c0 0x7fed70154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:32.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fed700cdf80 0x7fed7015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fed7016dd70 con 0x7fed70154e00 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed7006ffd0 con 0x7fed700b40c0 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.671+0000 7fed74e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fed7016def0 con 0x7fed700cdf80 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53938/0 (socket says 172.21.15.116:53938) 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 -- 172.21.15.116:0/724473315 learned_addr learned my addr 172.21.15.116:0/724473315 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6de00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed700b40c0 0x7fed70154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed65400640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fed700cdf80 0x7fed7015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 -- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fed700cdf80 msgr2=0x7fed7015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fed700cdf80 0x7fed7015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 -- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed700b40c0 msgr2=0x7fed70154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed700b40c0 0x7fed70154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 -- 172.21.15.116:0/724473315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed70159960 con 0x7fed70154e00 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed6e800640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c6b0 tx=0x7fed6800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:32.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68018070 con 0x7fed70154e00 2024-05-14T11:24:32.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed68004db0 con 0x7fed70154e00 2024-05-14T11:24:32.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6800a820 con 0x7fed70154e00 2024-05-14T11:24:32.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed70159c40 con 0x7fed70154e00 2024-05-14T11:24:32.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7015a200 con 0x7fed70154e00 2024-05-14T11:24:32.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7006a9c0 con 0x7fed70154e00 2024-05-14T11:24:32.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.675+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fed6801b070 con 0x7fed70154e00 2024-05-14T11:24:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.679+0000 7fed66a00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 0x7fed4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.679+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fed6808eac0 con 0x7fed70154e00 2024-05-14T11:24:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.679+0000 7fed6de00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 0x7fed4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:32.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.679+0000 7fed6de00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 0x7fed4c071940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed58000f70 tx=0x7fed58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:32.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.683+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fed6805b780 con 0x7fed70154e00 2024-05-14T11:24:32.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.871+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed7015b770 con 0x7fed70154e00 2024-05-14T11:24:32.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.871+0000 7fed66a00640 1 -- 172.21.15.116:0/724473315 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5183 (secure 0 0 0) 0x7fed6805b080 con 0x7fed70154e00 2024-05-14T11:24:32.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:32.874 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:31.883949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7638":{"gid":7638,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3421632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3421632},{"type":"v1","addr":"172.21.15.116:6835","nonce":3421632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7638,"qdb_cluster":[7638]},"id":7}]} 2024-05-14T11:24:32.874 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 83 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 msgr2=0x7fed4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 0x7fed4c071940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed58000f70 tx=0x7fed58009040 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 msgr2=0x7fed70155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c6b0 tx=0x7fed6800cb70 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 shutdown_connections 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fed4c06f430 0x7fed4c071940 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed70154e00 0x7fed70155210 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fed700cdf80 0x7fed7015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 --2- 172.21.15.116:0/724473315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed700b40c0 0x7fed70154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 >> 172.21.15.116:0/724473315 conn(0x7fed700d0770 msgr2=0x7fed7015bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 shutdown_connections 2024-05-14T11:24:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:32.875+0000 7fed74e00640 1 -- 172.21.15.116:0/724473315 wait complete. 2024-05-14T11:24:32.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:24:33.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.039+0000 7f00ca200640 1 Processor -- start 2024-05-14T11:24:33.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.039+0000 7f00ca200640 1 -- start start 2024-05-14T11:24:33.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00ca200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 0x7f00c406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00ca200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00c40d1670 0x7f00c40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00ca200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c40d1fc0 con 0x7f00c406e9e0 2024-05-14T11:24:33.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00ca200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f00c41573b0 con 0x7f00c40d1670 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00ca200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00c4161120 con 0x7f00c4156fa0 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00c40d1670 0x7f00c40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00c40d1670 0x7f00c40d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53944/0 (socket says 172.21.15.116:53944) 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 -- 172.21.15.116:0/4056883805 learned_addr learned my addr 172.21.15.116:0/4056883805 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00c3e00640 1 --1- 172.21.15.116:0/4056883805 >> v1:172.21.15.116:6789/0 conn(0x7f00c406e9e0 0x7f00c406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52360/0 (socket says 172.21.15.116:52360) 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00c3400640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 0x7f00c406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 -- 172.21.15.116:0/4056883805 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 msgr2=0x7f00c406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 0x7f00c406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 -- 172.21.15.116:0/4056883805 >> v1:172.21.15.116:6789/0 conn(0x7f00c406e9e0 legacy=0x7f00c406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00baa00640 1 -- 172.21.15.116:0/4056883805 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c4160510 con 0x7f00c40d1670 2024-05-14T11:24:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.043+0000 7f00c3400640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 0x7f00c406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:33.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00baa00640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c40d1670 0x7f00c40d1a80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009870 tx=0x7f00ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e0801ae29cc74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00c2a00640 1 -- 172.21.15.116:0/4056883805 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0aa430 con 0x7f00c40d1670 2024-05-14T11:24:33.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00c2a00640 1 -- 172.21.15.116:0/4056883805 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac0af070 con 0x7f00c40d1670 2024-05-14T11:24:33.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00c2a00640 1 -- 172.21.15.116:0/4056883805 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0aa7f0 con 0x7f00c40d1670 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- 172.21.15.116:0/4056883805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c40d1670 msgr2=0x7f00c40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c40d1670 0x7f00c40d1a80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009870 tx=0x7f00ac01c4d0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- 172.21.15.116:0/4056883805 shutdown_connections 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c40d1670 0x7f00c40d1a80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 --2- 172.21.15.116:0/4056883805 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c4156fa0 0x7f00c406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- 172.21.15.116:0/4056883805 >> 172.21.15.116:0/4056883805 conn(0x7f00c40b4620 msgr2=0x7f00c40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- 172.21.15.116:0/4056883805 shutdown_connections 2024-05-14T11:24:33.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- 172.21.15.116:0/4056883805 wait complete. 2024-05-14T11:24:33.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 Processor -- start 2024-05-14T11:24:33.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.047+0000 7f00ca200640 1 -- start start 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 0x7f00c4149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 0x7f00c40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4070520 con 0x7f00c406e9e0 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f00c40b07a0 con 0x7f00c4156fa0 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00c40706a0 con 0x7f00c40d1670 2024-05-14T11:24:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53958/0 (socket says 172.21.15.116:53958) 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 -- 172.21.15.116:0/2685981331 learned_addr learned my addr 172.21.15.116:0/2685981331 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00baa00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 0x7f00c4149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3e00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 0x7f00c40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 -- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 msgr2=0x7f00c4149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 0x7f00c4149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 -- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 msgr2=0x7f00c40b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 0x7f00c40b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 -- 172.21.15.116:0/2685981331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c4169bb0 con 0x7f00c406e9e0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00baa00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 0x7f00c4149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3e00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 0x7f00c40b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00c3400640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f00b400d800 tx=0x7f00b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b400b6c0 con 0x7f00c406e9e0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4019070 con 0x7f00c406e9e0 2024-05-14T11:24:33.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c4169e90 con 0x7f00c406e9e0 2024-05-14T11:24:33.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b400baa0 con 0x7f00c406e9e0 2024-05-14T11:24:33.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c416a170 con 0x7f00c406e9e0 2024-05-14T11:24:33.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.051+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c406a9c0 con 0x7f00c406e9e0 2024-05-14T11:24:33.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.055+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00b400bc40 con 0x7f00c406e9e0 2024-05-14T11:24:33.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.055+0000 7f00bbe00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 0x7f00a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.055+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f00b408f980 con 0x7f00c406e9e0 2024-05-14T11:24:33.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.055+0000 7f00baa00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 0x7f00a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.055+0000 7f00baa00640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 0x7f00a0071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0ae000 tx=0x7f00ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.059+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f00b405c640 con 0x7f00c406e9e0 2024-05-14T11:24:33.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.215+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00c4156410 con 0x7f00c406e9e0 2024-05-14T11:24:33.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00bbe00640 1 -- 172.21.15.116:0/2685981331 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16921 (secure 0 0 0) 0x7f00b4086e20 con 0x7f00c406e9e0 2024-05-14T11:24:33.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:33.220 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":99,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:24:30.461202+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:24:26.553545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-14T11:24:27.820765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 msgr2=0x7f00a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 0x7f00a0071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0ae000 tx=0x7f00ac008040 comp rx=0 tx=0).stop 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 msgr2=0x7f00c414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f00b400d800 tx=0x7f00b400dcc0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 shutdown_connections 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f00a006f400 0x7f00a0071910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00c4156fa0 0x7f00c40b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00c40d1670 0x7f00c4149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 --2- 172.21.15.116:0/2685981331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c406e9e0 0x7f00c414f1a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 >> 172.21.15.116:0/2685981331 conn(0x7f00c40b4620 msgr2=0x7f00c40b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 shutdown_connections 2024-05-14T11:24:33.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.219+0000 7f00ca200640 1 -- 172.21.15.116:0/2685981331 wait complete. 2024-05-14T11:24:33.242 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:24:33.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:24:33.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.387+0000 7f70d7000640 1 Processor -- start 2024-05-14T11:24:33.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.387+0000 7f70d7000640 1 -- start start 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f70d015db30 0x7f70d00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f70d0161600 0x7f70d0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f70d00ae100 con 0x7f70d015db30 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f70d00ae870 con 0x7f70d0161600 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f70d006ff20 con 0x7f70d00d1670 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f70d015db30 0x7f70d00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f70d015db30 0x7f70d00aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53968/0 (socket says 172.21.15.116:53968) 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 -- 172.21.15.116:0/1521020368 learned_addr learned my addr 172.21.15.116:0/1521020368 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70c7400640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70d5000640 1 --2- 172.21.15.116:0/1521020368 >> v2:172.21.15.149:3300/0 conn(0x7f70d0161600 0x7f70d0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 -- 172.21.15.116:0/1521020368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 msgr2=0x7f70d00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d00d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 -- 172.21.15.116:0/1521020368 >> v2:172.21.15.149:3300/0 conn(0x7f70d0161600 msgr2=0x7f70d0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 --2- 172.21.15.116:0/1521020368 >> v2:172.21.15.149:3300/0 conn(0x7f70d0161600 0x7f70d0161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.391+0000 7f70cfe00640 1 -- 172.21.15.116:0/1521020368 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d00d1fc0 con 0x7f70d015db30 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70cfe00640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d015db30 0x7f70d00aeaa0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b41d0 tx=0x7f70c001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a41d0c864f2a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70cf400640 1 -- 172.21.15.116:0/1521020368 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a56d0 con 0x7f70d015db30 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70cf400640 1 -- 172.21.15.116:0/1521020368 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c00a5870 con 0x7f70d015db30 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70cf400640 1 -- 172.21.15.116:0/1521020368 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a5c40 con 0x7f70d015db30 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- 172.21.15.116:0/1521020368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d015db30 msgr2=0x7f70d00aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d015db30 0x7f70d00aeaa0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b41d0 tx=0x7f70c001c960 comp rx=0 tx=0).stop 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- 172.21.15.116:0/1521020368 shutdown_connections 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 --2- 172.21.15.116:0/1521020368 >> v2:172.21.15.149:3300/0 conn(0x7f70d0161600 0x7f70d0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 --2- 172.21.15.116:0/1521020368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d015db30 0x7f70d00aeaa0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- 172.21.15.116:0/1521020368 >> 172.21.15.116:0/1521020368 conn(0x7f70d00b4620 msgr2=0x7f70d00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- 172.21.15.116:0/1521020368 shutdown_connections 2024-05-14T11:24:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- 172.21.15.116:0/1521020368 wait complete. 2024-05-14T11:24:33.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 Processor -- start 2024-05-14T11:24:33.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.395+0000 7f70d7000640 1 -- start start 2024-05-14T11:24:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d0160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70d015db30 0x7f70d0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0162360 con 0x7f70d0161600 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f70d00aefe0 con 0x7f70d015db30 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f70d015df40 con 0x7f70d00d1670 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53980/0 (socket says 172.21.15.116:53980) 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 -- 172.21.15.116:0/3958166296 learned_addr learned my addr 172.21.15.116:0/3958166296 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70cfe00640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d0160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70c7400640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70d015db30 0x7f70d0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 -- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 msgr2=0x7f70d0160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d0160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 -- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70d015db30 msgr2=0x7f70d0152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70d015db30 0x7f70d0152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 -- 172.21.15.116:0/3958166296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d014f6a0 con 0x7f70d0161600 2024-05-14T11:24:33.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d5000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f70c800e7f0 tx=0x7f70c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8017070 con 0x7f70d0161600 2024-05-14T11:24:33.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c800c940 con 0x7f70d0161600 2024-05-14T11:24:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c800eea0 con 0x7f70d0161600 2024-05-14T11:24:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d014f3c0 con 0x7f70d0161600 2024-05-14T11:24:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d0151ab0 con 0x7f70d0161600 2024-05-14T11:24:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.399+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d006a9c0 con 0x7f70d0161600 2024-05-14T11:24:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.403+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f70c801a070 con 0x7f70d0161600 2024-05-14T11:24:33.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.403+0000 7f70ccc00640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 0x7f70ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.403+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f70c808f1f0 con 0x7f70d0161600 2024-05-14T11:24:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.403+0000 7f70cfe00640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 0x7f70ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.403+0000 7f70cfe00640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 0x7f70ac071940 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ae000 tx=0x7f70c00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.407+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f70c805b860 con 0x7f70d0161600 2024-05-14T11:24:33.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.567+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70d00b3b80 con 0x7f70ac06f430 2024-05-14T11:24:33.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.575+0000 7f70ccc00640 1 -- 172.21.15.116:0/3958166296 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252003 (secure 0 0 0) 0x7f70d00b3b80 con 0x7f70ac06f430 2024-05-14T11:24:33.578 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:33.579 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:24:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 msgr2=0x7f70ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 0x7f70ac071940 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ae000 tx=0x7f70c00a7070 comp rx=0 tx=0).stop 2024-05-14T11:24:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 msgr2=0x7f70d0152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f70c800e7f0 tx=0x7f70c800ecb0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 shutdown_connections 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f70ac06f430 0x7f70ac071940 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70d015db30 0x7f70d0152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d0161600 0x7f70d0152b50 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 --2- 172.21.15.116:0/3958166296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70d00d1670 0x7f70d0160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 >> 172.21.15.116:0/3958166296 conn(0x7f70d00b4620 msgr2=0x7f70d015b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 shutdown_connections 2024-05-14T11:24:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.579+0000 7f70d7000640 1 -- 172.21.15.116:0/3958166296 wait complete. 2024-05-14T11:24:33.598 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-05-14T11:24:32.487953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260188,"kb_used_data":10728,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714532,"statfs":{"total":773094113280,"available":772827680768,"internally_reserved":0,"allocated":10985472,"data_stored":5682604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.038023"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492239+0000","last_change":"2024-05-14T11:24:29.484520+0000","last_active":"2024-05-14T11:24:29.492239+0000","last_peered":"2024-05-14T11:24:29.492239+0000","last_clean":"2024-05-14T11:24:29.492239+0000","last_became_active":"2024-05-14T11:24:29.480423+0000","last_became_peered":"2024-05-14T11:24:29.480423+0000","last_unstale":"2024-05-14T11:24:29.492239+0000","last_undegraded":"2024-05-14T11:24:29.492239+0000","last_fullsized":"2024-05-14T11:24:29.492239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:16.350044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487958+0000","last_change":"2024-05-14T11:24:29.485294+0000","last_active":"2024-05-14T11:24:29.487958+0000","last_peered":"2024-05-14T11:24:29.487958+0000","last_clean":"2024-05-14T11:24:29.487958+0000","last_became_active":"2024-05-14T11:24:29.484551+0000","last_became_peered":"2024-05-14T11:24:29.484551+0000","last_unstale":"2024-05-14T11:24:29.487958+0000","last_undegraded":"2024-05-14T11:24:29.487958+0000","last_fullsized":"2024-05-14T11:24:29.487958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497485+0000","last_change":"2024-05-14T11:24:29.494224+0000","last_active":"2024-05-14T11:24:29.497485+0000","last_peered":"2024-05-14T11:24:29.497485+0000","last_clean":"2024-05-14T11:24:29.497485+0000","last_became_active":"2024-05-14T11:24:29.493907+0000","last_became_peered":"2024-05-14T11:24:29.493907+0000","last_unstale":"2024-05-14T11:24:29.497485+0000","last_undegraded":"2024-05-14T11:24:29.497485+0000","last_fullsized":"2024-05-14T11:24:29.497485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:22.019019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474989+0000","last_change":"2024-05-14T11:24:29.472454+0000","last_active":"2024-05-14T11:24:29.474989+0000","last_peered":"2024-05-14T11:24:29.474989+0000","last_clean":"2024-05-14T11:24:29.474989+0000","last_became_active":"2024-05-14T11:24:29.472135+0000","last_became_peered":"2024-05-14T11:24:29.472135+0000","last_unstale":"2024-05-14T11:24:29.474989+0000","last_undegraded":"2024-05-14T11:24:29.474989+0000","last_fullsized":"2024-05-14T11:24:29.474989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493574+0000","last_change":"2024-05-14T11:24:29.484540+0000","last_active":"2024-05-14T11:24:29.493574+0000","last_peered":"2024-05-14T11:24:29.493574+0000","last_clean":"2024-05-14T11:24:29.493574+0000","last_became_active":"2024-05-14T11:24:29.484248+0000","last_became_peered":"2024-05-14T11:24:29.484248+0000","last_unstale":"2024-05-14T11:24:29.493574+0000","last_undegraded":"2024-05-14T11:24:29.493574+0000","last_fullsized":"2024-05-14T11:24:29.493574+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:31.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487673+0000","last_change":"2024-05-14T11:24:29.485211+0000","last_active":"2024-05-14T11:24:29.487673+0000","last_peered":"2024-05-14T11:24:29.487673+0000","last_clean":"2024-05-14T11:24:29.487673+0000","last_became_active":"2024-05-14T11:24:29.484892+0000","last_became_peered":"2024-05-14T11:24:29.484892+0000","last_unstale":"2024-05-14T11:24:29.487673+0000","last_undegraded":"2024-05-14T11:24:29.487673+0000","last_fullsized":"2024-05-14T11:24:29.487673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.490849+0000","last_change":"2024-05-14T11:24:29.489213+0000","last_active":"2024-05-14T11:24:29.490849+0000","last_peered":"2024-05-14T11:24:29.490849+0000","last_clean":"2024-05-14T11:24:29.490849+0000","last_became_active":"2024-05-14T11:24:29.488499+0000","last_became_peered":"2024-05-14T11:24:29.488499+0000","last_unstale":"2024-05-14T11:24:29.490849+0000","last_undegraded":"2024-05-14T11:24:29.490849+0000","last_fullsized":"2024-05-14T11:24:29.490849+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499432+0000","last_change":"2024-05-14T11:24:29.494430+0000","last_active":"2024-05-14T11:24:29.499432+0000","last_peered":"2024-05-14T11:24:29.499432+0000","last_clean":"2024-05-14T11:24:29.499432+0000","last_became_active":"2024-05-14T11:24:29.494032+0000","last_became_peered":"2024-05-14T11:24:29.494032+0000","last_unstale":"2024-05-14T11:24:29.499432+0000","last_undegraded":"2024-05-14T11:24:29.499432+0000","last_fullsized":"2024-05-14T11:24:29.499432+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:43.455297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480816+0000","last_change":"2024-05-14T11:24:29.476804+0000","last_active":"2024-05-14T11:24:29.480816+0000","last_peered":"2024-05-14T11:24:29.480816+0000","last_clean":"2024-05-14T11:24:29.480816+0000","last_became_active":"2024-05-14T11:24:29.476443+0000","last_became_peered":"2024-05-14T11:24:29.476443+0000","last_unstale":"2024-05-14T11:24:29.480816+0000","last_undegraded":"2024-05-14T11:24:29.480816+0000","last_fullsized":"2024-05-14T11:24:29.480816+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:03.775167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491646+0000","last_change":"2024-05-14T11:24:29.482646+0000","last_active":"2024-05-14T11:24:29.491646+0000","last_peered":"2024-05-14T11:24:29.491646+0000","last_clean":"2024-05-14T11:24:29.491646+0000","last_became_active":"2024-05-14T11:24:29.482337+0000","last_became_peered":"2024-05-14T11:24:29.482337+0000","last_unstale":"2024-05-14T11:24:29.491646+0000","last_undegraded":"2024-05-14T11:24:29.491646+0000","last_fullsized":"2024-05-14T11:24:29.491646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:03.775167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495216+0000","last_change":"2024-05-14T11:24:29.491433+0000","last_active":"2024-05-14T11:24:29.495216+0000","last_peered":"2024-05-14T11:24:29.495216+0000","last_clean":"2024-05-14T11:24:29.495216+0000","last_became_active":"2024-05-14T11:24:29.491114+0000","last_became_peered":"2024-05-14T11:24:29.491114+0000","last_unstale":"2024-05-14T11:24:29.495216+0000","last_undegraded":"2024-05-14T11:24:29.495216+0000","last_fullsized":"2024-05-14T11:24:29.495216+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477763+0000","last_change":"2024-05-14T11:24:29.473072+0000","last_active":"2024-05-14T11:24:29.477763+0000","last_peered":"2024-05-14T11:24:29.477763+0000","last_clean":"2024-05-14T11:24:29.477763+0000","last_became_active":"2024-05-14T11:24:29.472731+0000","last_became_peered":"2024-05-14T11:24:29.472731+0000","last_unstale":"2024-05-14T11:24:29.477763+0000","last_undegraded":"2024-05-14T11:24:29.477763+0000","last_fullsized":"2024-05-14T11:24:29.477763+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496358+0000","last_change":"2024-05-14T11:24:29.493866+0000","last_active":"2024-05-14T11:24:29.496358+0000","last_peered":"2024-05-14T11:24:29.496358+0000","last_clean":"2024-05-14T11:24:29.496358+0000","last_became_active":"2024-05-14T11:24:29.493366+0000","last_became_peered":"2024-05-14T11:24:29.493366+0000","last_unstale":"2024-05-14T11:24:29.496358+0000","last_undegraded":"2024-05-14T11:24:29.496358+0000","last_fullsized":"2024-05-14T11:24:29.496358+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:50.275542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494854+0000","last_change":"2024-05-14T11:24:29.492972+0000","last_active":"2024-05-14T11:24:29.494854+0000","last_peered":"2024-05-14T11:24:29.494854+0000","last_clean":"2024-05-14T11:24:29.494854+0000","last_became_active":"2024-05-14T11:24:29.492641+0000","last_became_peered":"2024-05-14T11:24:29.492641+0000","last_unstale":"2024-05-14T11:24:29.494854+0000","last_undegraded":"2024-05-14T11:24:29.494854+0000","last_fullsized":"2024-05-14T11:24:29.494854+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:15:52.028291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496030+0000","last_change":"2024-05-14T11:24:29.494501+0000","last_active":"2024-05-14T11:24:29.496030+0000","last_peered":"2024-05-14T11:24:29.496030+0000","last_clean":"2024-05-14T11:24:29.496030+0000","last_became_active":"2024-05-14T11:24:29.494147+0000","last_became_peered":"2024-05-14T11:24:29.494147+0000","last_unstale":"2024-05-14T11:24:29.496030+0000","last_undegraded":"2024-05-14T11:24:29.496030+0000","last_fullsized":"2024-05-14T11:24:29.496030+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:31.571239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496134+0000","last_change":"2024-05-14T11:24:29.494899+0000","last_active":"2024-05-14T11:24:29.496134+0000","last_peered":"2024-05-14T11:24:29.496134+0000","last_clean":"2024-05-14T11:24:29.496134+0000","last_became_active":"2024-05-14T11:24:29.494585+0000","last_became_peered":"2024-05-14T11:24:29.494585+0000","last_unstale":"2024-05-14T11:24:29.496134+0000","last_undegraded":"2024-05-14T11:24:29.496134+0000","last_fullsized":"2024-05-14T11:24:29.496134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:57.884605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497390+0000","last_change":"2024-05-14T11:24:29.492284+0000","last_active":"2024-05-14T11:24:29.497390+0000","last_peered":"2024-05-14T11:24:29.497390+0000","last_clean":"2024-05-14T11:24:29.497390+0000","last_became_active":"2024-05-14T11:24:29.488193+0000","last_became_peered":"2024-05-14T11:24:29.488193+0000","last_unstale":"2024-05-14T11:24:29.497390+0000","last_undegraded":"2024-05-14T11:24:29.497390+0000","last_fullsized":"2024-05-14T11:24:29.497390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:55.159038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487995+0000","last_change":"2024-05-14T11:24:29.485799+0000","last_active":"2024-05-14T11:24:29.487995+0000","last_peered":"2024-05-14T11:24:29.487995+0000","last_clean":"2024-05-14T11:24:29.487995+0000","last_became_active":"2024-05-14T11:24:29.482903+0000","last_became_peered":"2024-05-14T11:24:29.482903+0000","last_unstale":"2024-05-14T11:24:29.487995+0000","last_undegraded":"2024-05-14T11:24:29.487995+0000","last_fullsized":"2024-05-14T11:24:29.487995+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491548+0000","last_change":"2024-05-14T11:24:29.486345+0000","last_active":"2024-05-14T11:24:29.491548+0000","last_peered":"2024-05-14T11:24:29.491548+0000","last_clean":"2024-05-14T11:24:29.491548+0000","last_became_active":"2024-05-14T11:24:29.486038+0000","last_became_peered":"2024-05-14T11:24:29.486038+0000","last_unstale":"2024-05-14T11:24:29.491548+0000","last_undegraded":"2024-05-14T11:24:29.491548+0000","last_fullsized":"2024-05-14T11:24:29.491548+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:59:13.050744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493107+0000","last_change":"2024-05-14T11:24:29.487286+0000","last_active":"2024-05-14T11:24:29.493107+0000","last_peered":"2024-05-14T11:24:29.493107+0000","last_clean":"2024-05-14T11:24:29.493107+0000","last_became_active":"2024-05-14T11:24:29.486974+0000","last_became_peered":"2024-05-14T11:24:29.486974+0000","last_unstale":"2024-05-14T11:24:29.493107+0000","last_undegraded":"2024-05-14T11:24:29.493107+0000","last_fullsized":"2024-05-14T11:24:29.493107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494298+0000","last_change":"2024-05-14T11:24:29.492520+0000","last_active":"2024-05-14T11:24:29.494298+0000","last_peered":"2024-05-14T11:24:29.494298+0000","last_clean":"2024-05-14T11:24:29.494298+0000","last_became_active":"2024-05-14T11:24:29.492005+0000","last_became_peered":"2024-05-14T11:24:29.492005+0000","last_unstale":"2024-05-14T11:24:29.494298+0000","last_undegraded":"2024-05-14T11:24:29.494298+0000","last_fullsized":"2024-05-14T11:24:29.494298+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:49.188374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494616+0000","last_change":"2024-05-14T11:24:29.491260+0000","last_active":"2024-05-14T11:24:29.494616+0000","last_peered":"2024-05-14T11:24:29.494616+0000","last_clean":"2024-05-14T11:24:29.494616+0000","last_became_active":"2024-05-14T11:24:29.487960+0000","last_became_peered":"2024-05-14T11:24:29.487960+0000","last_unstale":"2024-05-14T11:24:29.494616+0000","last_undegraded":"2024-05-14T11:24:29.494616+0000","last_fullsized":"2024-05-14T11:24:29.494616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491606+0000","last_change":"2024-05-14T11:24:29.488023+0000","last_active":"2024-05-14T11:24:29.491606+0000","last_peered":"2024-05-14T11:24:29.491606+0000","last_clean":"2024-05-14T11:24:29.491606+0000","last_became_active":"2024-05-14T11:24:29.487514+0000","last_became_peered":"2024-05-14T11:24:29.487514+0000","last_unstale":"2024-05-14T11:24:29.491606+0000","last_undegraded":"2024-05-14T11:24:29.491606+0000","last_fullsized":"2024-05-14T11:24:29.491606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:35:45.048217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493215+0000","last_change":"2024-05-14T11:24:29.485288+0000","last_active":"2024-05-14T11:24:29.493215+0000","last_peered":"2024-05-14T11:24:29.493215+0000","last_clean":"2024-05-14T11:24:29.493215+0000","last_became_active":"2024-05-14T11:24:29.484999+0000","last_became_peered":"2024-05-14T11:24:29.484999+0000","last_unstale":"2024-05-14T11:24:29.493215+0000","last_undegraded":"2024-05-14T11:24:29.493215+0000","last_fullsized":"2024-05-14T11:24:29.493215+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491696+0000","last_change":"2024-05-14T11:24:29.484177+0000","last_active":"2024-05-14T11:24:29.491696+0000","last_peered":"2024-05-14T11:24:29.491696+0000","last_clean":"2024-05-14T11:24:29.491696+0000","last_became_active":"2024-05-14T11:24:29.483842+0000","last_became_peered":"2024-05-14T11:24:29.483842+0000","last_unstale":"2024-05-14T11:24:29.491696+0000","last_undegraded":"2024-05-14T11:24:29.491696+0000","last_fullsized":"2024-05-14T11:24:29.491696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487864+0000","last_change":"2024-05-14T11:24:29.481868+0000","last_active":"2024-05-14T11:24:29.487864+0000","last_peered":"2024-05-14T11:24:29.487864+0000","last_clean":"2024-05-14T11:24:29.487864+0000","last_became_active":"2024-05-14T11:24:29.481551+0000","last_became_peered":"2024-05-14T11:24:29.481551+0000","last_unstale":"2024-05-14T11:24:29.487864+0000","last_undegraded":"2024-05-14T11:24:29.487864+0000","last_fullsized":"2024-05-14T11:24:29.487864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495238+0000","last_change":"2024-05-14T11:24:29.492229+0000","last_active":"2024-05-14T11:24:29.495238+0000","last_peered":"2024-05-14T11:24:29.495238+0000","last_clean":"2024-05-14T11:24:29.495238+0000","last_became_active":"2024-05-14T11:24:29.491940+0000","last_became_peered":"2024-05-14T11:24:29.491940+0000","last_unstale":"2024-05-14T11:24:29.495238+0000","last_undegraded":"2024-05-14T11:24:29.495238+0000","last_fullsized":"2024-05-14T11:24:29.495238+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487511+0000","last_change":"2024-05-14T11:24:29.484409+0000","last_active":"2024-05-14T11:24:29.487511+0000","last_peered":"2024-05-14T11:24:29.487511+0000","last_clean":"2024-05-14T11:24:29.487511+0000","last_became_active":"2024-05-14T11:24:29.484115+0000","last_became_peered":"2024-05-14T11:24:29.484115+0000","last_unstale":"2024-05-14T11:24:29.487511+0000","last_undegraded":"2024-05-14T11:24:29.487511+0000","last_fullsized":"2024-05-14T11:24:29.487511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.490270+0000","last_change":"2024-05-14T11:24:29.488584+0000","last_active":"2024-05-14T11:24:29.490270+0000","last_peered":"2024-05-14T11:24:29.490270+0000","last_clean":"2024-05-14T11:24:29.490270+0000","last_became_active":"2024-05-14T11:24:29.488230+0000","last_became_peered":"2024-05-14T11:24:29.488230+0000","last_unstale":"2024-05-14T11:24:29.490270+0000","last_undegraded":"2024-05-14T11:24:29.490270+0000","last_fullsized":"2024-05-14T11:24:29.490270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:02:00.028776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491691+0000","last_change":"2024-05-14T11:24:29.484420+0000","last_active":"2024-05-14T11:24:29.491691+0000","last_peered":"2024-05-14T11:24:29.491691+0000","last_clean":"2024-05-14T11:24:29.491691+0000","last_became_active":"2024-05-14T11:24:29.484112+0000","last_became_peered":"2024-05-14T11:24:29.484112+0000","last_unstale":"2024-05-14T11:24:29.491691+0000","last_undegraded":"2024-05-14T11:24:29.491691+0000","last_fullsized":"2024-05-14T11:24:29.491691+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.488798+0000","last_change":"2024-05-14T11:24:29.483219+0000","last_active":"2024-05-14T11:24:29.488798+0000","last_peered":"2024-05-14T11:24:29.488798+0000","last_clean":"2024-05-14T11:24:29.488798+0000","last_became_active":"2024-05-14T11:24:29.482753+0000","last_became_peered":"2024-05-14T11:24:29.482753+0000","last_unstale":"2024-05-14T11:24:29.488798+0000","last_undegraded":"2024-05-14T11:24:29.488798+0000","last_fullsized":"2024-05-14T11:24:29.488798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.489717+0000","last_change":"2024-05-14T11:24:29.482521+0000","last_active":"2024-05-14T11:24:29.489717+0000","last_peered":"2024-05-14T11:24:29.489717+0000","last_clean":"2024-05-14T11:24:29.489717+0000","last_became_active":"2024-05-14T11:24:29.482237+0000","last_became_peered":"2024-05-14T11:24:29.482237+0000","last_unstale":"2024-05-14T11:24:29.489717+0000","last_undegraded":"2024-05-14T11:24:29.489717+0000","last_fullsized":"2024-05-14T11:24:29.489717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:18.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480510+0000","last_change":"2024-05-14T11:24:29.476521+0000","last_active":"2024-05-14T11:24:29.480510+0000","last_peered":"2024-05-14T11:24:29.480510+0000","last_clean":"2024-05-14T11:24:29.480510+0000","last_became_active":"2024-05-14T11:24:29.475357+0000","last_became_peered":"2024-05-14T11:24:29.475357+0000","last_unstale":"2024-05-14T11:24:29.480510+0000","last_undegraded":"2024-05-14T11:24:29.480510+0000","last_fullsized":"2024-05-14T11:24:29.480510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:12.175619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506251+0000","last_change":"2024-05-14T11:24:28.487696+0000","last_active":"2024-05-14T11:24:29.506251+0000","last_peered":"2024-05-14T11:24:29.506251+0000","last_clean":"2024-05-14T11:24:29.506251+0000","last_became_active":"2024-05-14T11:24:28.487372+0000","last_became_peered":"2024-05-14T11:24:28.487372+0000","last_unstale":"2024-05-14T11:24:29.506251+0000","last_undegraded":"2024-05-14T11:24:29.506251+0000","last_fullsized":"2024-05-14T11:24:29.506251+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:29.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499291+0000","last_change":"2024-05-14T11:24:29.495374+0000","last_active":"2024-05-14T11:24:29.499291+0000","last_peered":"2024-05-14T11:24:29.499291+0000","last_clean":"2024-05-14T11:24:29.499291+0000","last_became_active":"2024-05-14T11:24:29.493020+0000","last_became_peered":"2024-05-14T11:24:29.493020+0000","last_unstale":"2024-05-14T11:24:29.499291+0000","last_undegraded":"2024-05-14T11:24:29.499291+0000","last_fullsized":"2024-05-14T11:24:29.499291+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:00.787515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506505+0000","last_change":"2024-05-14T11:24:28.485400+0000","last_active":"2024-05-14T11:24:29.506505+0000","last_peered":"2024-05-14T11:24:29.506505+0000","last_clean":"2024-05-14T11:24:29.506505+0000","last_became_active":"2024-05-14T11:24:28.484706+0000","last_became_peered":"2024-05-14T11:24:28.484706+0000","last_unstale":"2024-05-14T11:24:29.506505+0000","last_undegraded":"2024-05-14T11:24:29.506505+0000","last_fullsized":"2024-05-14T11:24:29.506505+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:52.595560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493386+0000","last_change":"2024-05-14T11:24:29.487675+0000","last_active":"2024-05-14T11:24:29.493386+0000","last_peered":"2024-05-14T11:24:29.493386+0000","last_clean":"2024-05-14T11:24:29.493386+0000","last_became_active":"2024-05-14T11:24:29.487292+0000","last_became_peered":"2024-05-14T11:24:29.487292+0000","last_unstale":"2024-05-14T11:24:29.493386+0000","last_undegraded":"2024-05-14T11:24:29.493386+0000","last_fullsized":"2024-05-14T11:24:29.493386+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506071+0000","last_change":"2024-05-14T11:24:28.483038+0000","last_active":"2024-05-14T11:24:29.506071+0000","last_peered":"2024-05-14T11:24:29.506071+0000","last_clean":"2024-05-14T11:24:29.506071+0000","last_became_active":"2024-05-14T11:24:28.482438+0000","last_became_peered":"2024-05-14T11:24:28.482438+0000","last_unstale":"2024-05-14T11:24:29.506071+0000","last_undegraded":"2024-05-14T11:24:29.506071+0000","last_fullsized":"2024-05-14T11:24:29.506071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:58.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493138+0000","last_change":"2024-05-14T11:24:29.487686+0000","last_active":"2024-05-14T11:24:29.493138+0000","last_peered":"2024-05-14T11:24:29.493138+0000","last_clean":"2024-05-14T11:24:29.493138+0000","last_became_active":"2024-05-14T11:24:29.485570+0000","last_became_peered":"2024-05-14T11:24:29.485570+0000","last_unstale":"2024-05-14T11:24:29.493138+0000","last_undegraded":"2024-05-14T11:24:29.493138+0000","last_fullsized":"2024-05-14T11:24:29.493138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505928+0000","last_change":"2024-05-14T11:24:28.481787+0000","last_active":"2024-05-14T11:24:29.505928+0000","last_peered":"2024-05-14T11:24:29.505928+0000","last_clean":"2024-05-14T11:24:29.505928+0000","last_became_active":"2024-05-14T11:24:28.481471+0000","last_became_peered":"2024-05-14T11:24:28.481471+0000","last_unstale":"2024-05-14T11:24:29.505928+0000","last_undegraded":"2024-05-14T11:24:29.505928+0000","last_fullsized":"2024-05-14T11:24:29.505928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:19:25.815981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495390+0000","last_change":"2024-05-14T11:24:29.483524+0000","last_active":"2024-05-14T11:24:29.495390+0000","last_peered":"2024-05-14T11:24:29.495390+0000","last_clean":"2024-05-14T11:24:29.495390+0000","last_became_active":"2024-05-14T11:24:29.483193+0000","last_became_peered":"2024-05-14T11:24:29.483193+0000","last_unstale":"2024-05-14T11:24:29.495390+0000","last_undegraded":"2024-05-14T11:24:29.495390+0000","last_fullsized":"2024-05-14T11:24:29.495390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:08:10.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.504022+0000","last_change":"2024-05-14T11:24:28.489640+0000","last_active":"2024-05-14T11:24:29.504022+0000","last_peered":"2024-05-14T11:24:29.504022+0000","last_clean":"2024-05-14T11:24:29.504022+0000","last_became_active":"2024-05-14T11:24:28.489213+0000","last_became_peered":"2024-05-14T11:24:28.489213+0000","last_unstale":"2024-05-14T11:24:29.504022+0000","last_undegraded":"2024-05-14T11:24:29.504022+0000","last_fullsized":"2024-05-14T11:24:29.504022+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:45.116165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491508+0000","last_change":"2024-05-14T11:24:29.485807+0000","last_active":"2024-05-14T11:24:29.491508+0000","last_peered":"2024-05-14T11:24:29.491508+0000","last_clean":"2024-05-14T11:24:29.491508+0000","last_became_active":"2024-05-14T11:24:29.485516+0000","last_became_peered":"2024-05-14T11:24:29.485516+0000","last_unstale":"2024-05-14T11:24:29.491508+0000","last_undegraded":"2024-05-14T11:24:29.491508+0000","last_fullsized":"2024-05-14T11:24:29.491508+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:12:15.420193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505424+0000","last_change":"2024-05-14T11:24:28.478497+0000","last_active":"2024-05-14T11:24:29.505424+0000","last_peered":"2024-05-14T11:24:29.505424+0000","last_clean":"2024-05-14T11:24:29.505424+0000","last_became_active":"2024-05-14T11:24:28.478156+0000","last_became_peered":"2024-05-14T11:24:28.478156+0000","last_unstale":"2024-05-14T11:24:29.505424+0000","last_undegraded":"2024-05-14T11:24:29.505424+0000","last_fullsized":"2024-05-14T11:24:29.505424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:57.865081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487935+0000","last_change":"2024-05-14T11:24:29.485000+0000","last_active":"2024-05-14T11:24:29.487935+0000","last_peered":"2024-05-14T11:24:29.487935+0000","last_clean":"2024-05-14T11:24:29.487935+0000","last_became_active":"2024-05-14T11:24:29.483862+0000","last_became_peered":"2024-05-14T11:24:29.483862+0000","last_unstale":"2024-05-14T11:24:29.487935+0000","last_undegraded":"2024-05-14T11:24:29.487935+0000","last_fullsized":"2024-05-14T11:24:29.487935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.469007+0000","last_change":"2024-05-14T11:24:28.483085+0000","last_active":"2024-05-14T11:24:29.469007+0000","last_peered":"2024-05-14T11:24:29.469007+0000","last_clean":"2024-05-14T11:24:29.469007+0000","last_became_active":"2024-05-14T11:24:28.482701+0000","last_became_peered":"2024-05-14T11:24:28.482701+0000","last_unstale":"2024-05-14T11:24:29.469007+0000","last_undegraded":"2024-05-14T11:24:29.469007+0000","last_fullsized":"2024-05-14T11:24:29.469007+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496711+0000","last_change":"2024-05-14T11:24:29.483810+0000","last_active":"2024-05-14T11:24:29.496711+0000","last_peered":"2024-05-14T11:24:29.496711+0000","last_clean":"2024-05-14T11:24:29.496711+0000","last_became_active":"2024-05-14T11:24:29.483258+0000","last_became_peered":"2024-05-14T11:24:29.483258+0000","last_unstale":"2024-05-14T11:24:29.496711+0000","last_undegraded":"2024-05-14T11:24:29.496711+0000","last_fullsized":"2024-05-14T11:24:29.496711+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:24.258572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.463579+0000","last_change":"2024-05-14T11:24:28.481747+0000","last_active":"2024-05-14T11:24:29.463579+0000","last_peered":"2024-05-14T11:24:29.463579+0000","last_clean":"2024-05-14T11:24:29.463579+0000","last_became_active":"2024-05-14T11:24:28.481455+0000","last_became_peered":"2024-05-14T11:24:28.481455+0000","last_unstale":"2024-05-14T11:24:29.463579+0000","last_undegraded":"2024-05-14T11:24:29.463579+0000","last_fullsized":"2024-05-14T11:24:29.463579+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:57.865081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491543+0000","last_change":"2024-05-14T11:24:29.482659+0000","last_active":"2024-05-14T11:24:29.491543+0000","last_peered":"2024-05-14T11:24:29.491543+0000","last_clean":"2024-05-14T11:24:29.491543+0000","last_became_active":"2024-05-14T11:24:29.482355+0000","last_became_peered":"2024-05-14T11:24:29.482355+0000","last_unstale":"2024-05-14T11:24:29.491543+0000","last_undegraded":"2024-05-14T11:24:29.491543+0000","last_fullsized":"2024-05-14T11:24:29.491543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:42.618784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470981+0000","last_change":"2024-05-14T11:24:28.482077+0000","last_active":"2024-05-14T11:24:29.470981+0000","last_peered":"2024-05-14T11:24:29.470981+0000","last_clean":"2024-05-14T11:24:29.470981+0000","last_became_active":"2024-05-14T11:24:28.481549+0000","last_became_peered":"2024-05-14T11:24:28.481549+0000","last_unstale":"2024-05-14T11:24:29.470981+0000","last_undegraded":"2024-05-14T11:24:29.470981+0000","last_fullsized":"2024-05-14T11:24:29.470981+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:30.633067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.500978+0000","last_change":"2024-05-14T11:24:29.499419+0000","last_active":"2024-05-14T11:24:29.500978+0000","last_peered":"2024-05-14T11:24:29.500978+0000","last_clean":"2024-05-14T11:24:29.500978+0000","last_became_active":"2024-05-14T11:24:29.499090+0000","last_became_peered":"2024-05-14T11:24:29.499090+0000","last_unstale":"2024-05-14T11:24:29.500978+0000","last_undegraded":"2024-05-14T11:24:29.500978+0000","last_fullsized":"2024-05-14T11:24:29.500978+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470375+0000","last_change":"2024-05-14T11:24:28.469226+0000","last_active":"2024-05-14T11:24:29.470375+0000","last_peered":"2024-05-14T11:24:29.470375+0000","last_clean":"2024-05-14T11:24:29.470375+0000","last_became_active":"2024-05-14T11:24:28.468777+0000","last_became_peered":"2024-05-14T11:24:28.468777+0000","last_unstale":"2024-05-14T11:24:29.470375+0000","last_undegraded":"2024-05-14T11:24:29.470375+0000","last_fullsized":"2024-05-14T11:24:29.470375+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:21:23.617550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499336+0000","last_change":"2024-05-14T11:24:29.497609+0000","last_active":"2024-05-14T11:24:29.499336+0000","last_peered":"2024-05-14T11:24:29.499336+0000","last_clean":"2024-05-14T11:24:29.499336+0000","last_became_active":"2024-05-14T11:24:29.497209+0000","last_became_peered":"2024-05-14T11:24:29.497209+0000","last_unstale":"2024-05-14T11:24:29.499336+0000","last_undegraded":"2024-05-14T11:24:29.499336+0000","last_fullsized":"2024-05-14T11:24:29.499336+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:51.987797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506076+0000","last_change":"2024-05-14T11:24:28.487409+0000","last_active":"2024-05-14T11:24:29.506076+0000","last_peered":"2024-05-14T11:24:29.506076+0000","last_clean":"2024-05-14T11:24:29.506076+0000","last_became_active":"2024-05-14T11:24:28.486959+0000","last_became_peered":"2024-05-14T11:24:28.486959+0000","last_unstale":"2024-05-14T11:24:29.506076+0000","last_undegraded":"2024-05-14T11:24:29.506076+0000","last_fullsized":"2024-05-14T11:24:29.506076+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:16.738352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499973+0000","last_change":"2024-05-14T11:24:29.498150+0000","last_active":"2024-05-14T11:24:29.499973+0000","last_peered":"2024-05-14T11:24:29.499973+0000","last_clean":"2024-05-14T11:24:29.499973+0000","last_became_active":"2024-05-14T11:24:29.497129+0000","last_became_peered":"2024-05-14T11:24:29.497129+0000","last_unstale":"2024-05-14T11:24:29.499973+0000","last_undegraded":"2024-05-14T11:24:29.499973+0000","last_fullsized":"2024-05-14T11:24:29.499973+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.509295+0000","last_change":"2024-05-14T11:24:28.481570+0000","last_active":"2024-05-14T11:24:29.509295+0000","last_peered":"2024-05-14T11:24:29.509295+0000","last_clean":"2024-05-14T11:24:29.509295+0000","last_became_active":"2024-05-14T11:24:28.481242+0000","last_became_peered":"2024-05-14T11:24:28.481242+0000","last_unstale":"2024-05-14T11:24:29.509295+0000","last_undegraded":"2024-05-14T11:24:29.509295+0000","last_fullsized":"2024-05-14T11:24:29.509295+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:19:25.815981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496370+0000","last_change":"2024-05-14T11:24:29.494265+0000","last_active":"2024-05-14T11:24:29.496370+0000","last_peered":"2024-05-14T11:24:29.496370+0000","last_clean":"2024-05-14T11:24:29.496370+0000","last_became_active":"2024-05-14T11:24:29.493922+0000","last_became_peered":"2024-05-14T11:24:29.493922+0000","last_unstale":"2024-05-14T11:24:29.496370+0000","last_undegraded":"2024-05-14T11:24:29.496370+0000","last_fullsized":"2024-05-14T11:24:29.496370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.582719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507344+0000","last_change":"2024-05-14T11:24:28.481094+0000","last_active":"2024-05-14T11:24:29.507344+0000","last_peered":"2024-05-14T11:24:29.507344+0000","last_clean":"2024-05-14T11:24:29.507344+0000","last_became_active":"2024-05-14T11:24:28.480694+0000","last_became_peered":"2024-05-14T11:24:28.480694+0000","last_unstale":"2024-05-14T11:24:29.507344+0000","last_undegraded":"2024-05-14T11:24:29.507344+0000","last_fullsized":"2024-05-14T11:24:29.507344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.481814+0000","last_change":"2024-05-14T11:24:29.475922+0000","last_active":"2024-05-14T11:24:29.481814+0000","last_peered":"2024-05-14T11:24:29.481814+0000","last_clean":"2024-05-14T11:24:29.481814+0000","last_became_active":"2024-05-14T11:24:29.475535+0000","last_became_peered":"2024-05-14T11:24:29.475535+0000","last_unstale":"2024-05-14T11:24:29.481814+0000","last_undegraded":"2024-05-14T11:24:29.481814+0000","last_fullsized":"2024-05-14T11:24:29.481814+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:59.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.463728+0000","last_change":"2024-05-14T11:24:28.482038+0000","last_active":"2024-05-14T11:24:29.463728+0000","last_peered":"2024-05-14T11:24:29.463728+0000","last_clean":"2024-05-14T11:24:29.463728+0000","last_became_active":"2024-05-14T11:24:28.481245+0000","last_became_peered":"2024-05-14T11:24:28.481245+0000","last_unstale":"2024-05-14T11:24:29.463728+0000","last_undegraded":"2024-05-14T11:24:29.463728+0000","last_fullsized":"2024-05-14T11:24:29.463728+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:56.131294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492232+0000","last_change":"2024-05-14T11:24:29.490254+0000","last_active":"2024-05-14T11:24:29.492232+0000","last_peered":"2024-05-14T11:24:29.492232+0000","last_clean":"2024-05-14T11:24:29.492232+0000","last_became_active":"2024-05-14T11:24:29.489784+0000","last_became_peered":"2024-05-14T11:24:29.489784+0000","last_unstale":"2024-05-14T11:24:29.492232+0000","last_undegraded":"2024-05-14T11:24:29.492232+0000","last_fullsized":"2024-05-14T11:24:29.492232+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:02.842633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.473640+0000","last_change":"2024-05-14T11:24:28.478187+0000","last_active":"2024-05-14T11:24:29.473640+0000","last_peered":"2024-05-14T11:24:29.473640+0000","last_clean":"2024-05-14T11:24:29.473640+0000","last_became_active":"2024-05-14T11:24:28.477875+0000","last_became_peered":"2024-05-14T11:24:28.477875+0000","last_unstale":"2024-05-14T11:24:29.473640+0000","last_undegraded":"2024-05-14T11:24:29.473640+0000","last_fullsized":"2024-05-14T11:24:29.473640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:38.000500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497313+0000","last_change":"2024-05-14T11:24:29.492094+0000","last_active":"2024-05-14T11:24:29.497313+0000","last_peered":"2024-05-14T11:24:29.497313+0000","last_clean":"2024-05-14T11:24:29.497313+0000","last_became_active":"2024-05-14T11:24:29.491105+0000","last_became_peered":"2024-05-14T11:24:29.491105+0000","last_unstale":"2024-05-14T11:24:29.497313+0000","last_undegraded":"2024-05-14T11:24:29.497313+0000","last_fullsized":"2024-05-14T11:24:29.497313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:17.333394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507575+0000","last_change":"2024-05-14T11:24:28.478525+0000","last_active":"2024-05-14T11:24:29.507575+0000","last_peered":"2024-05-14T11:24:29.507575+0000","last_clean":"2024-05-14T11:24:29.507575+0000","last_became_active":"2024-05-14T11:24:28.478026+0000","last_became_peered":"2024-05-14T11:24:28.478026+0000","last_unstale":"2024-05-14T11:24:29.507575+0000","last_undegraded":"2024-05-14T11:24:29.507575+0000","last_fullsized":"2024-05-14T11:24:29.507575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.489693+0000","last_change":"2024-05-14T11:24:29.480098+0000","last_active":"2024-05-14T11:24:29.489693+0000","last_peered":"2024-05-14T11:24:29.489693+0000","last_clean":"2024-05-14T11:24:29.489693+0000","last_became_active":"2024-05-14T11:24:29.479761+0000","last_became_peered":"2024-05-14T11:24:29.479761+0000","last_unstale":"2024-05-14T11:24:29.489693+0000","last_undegraded":"2024-05-14T11:24:29.489693+0000","last_fullsized":"2024-05-14T11:24:29.489693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:58:49.486260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505153+0000","last_change":"2024-05-14T11:24:28.482087+0000","last_active":"2024-05-14T11:24:29.505153+0000","last_peered":"2024-05-14T11:24:29.505153+0000","last_clean":"2024-05-14T11:24:29.505153+0000","last_became_active":"2024-05-14T11:24:28.481684+0000","last_became_peered":"2024-05-14T11:24:28.481684+0000","last_unstale":"2024-05-14T11:24:29.505153+0000","last_undegraded":"2024-05-14T11:24:29.505153+0000","last_fullsized":"2024-05-14T11:24:29.505153+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487493+0000","last_change":"2024-05-14T11:24:29.482840+0000","last_active":"2024-05-14T11:24:29.487493+0000","last_peered":"2024-05-14T11:24:29.487493+0000","last_clean":"2024-05-14T11:24:29.487493+0000","last_became_active":"2024-05-14T11:24:29.481947+0000","last_became_peered":"2024-05-14T11:24:29.481947+0000","last_unstale":"2024-05-14T11:24:29.487493+0000","last_undegraded":"2024-05-14T11:24:29.487493+0000","last_fullsized":"2024-05-14T11:24:29.487493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:31.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505124+0000","last_change":"2024-05-14T11:24:28.485226+0000","last_active":"2024-05-14T11:24:29.505124+0000","last_peered":"2024-05-14T11:24:29.505124+0000","last_clean":"2024-05-14T11:24:29.505124+0000","last_became_active":"2024-05-14T11:24:28.484885+0000","last_became_peered":"2024-05-14T11:24:28.484885+0000","last_unstale":"2024-05-14T11:24:29.505124+0000","last_undegraded":"2024-05-14T11:24:29.505124+0000","last_fullsized":"2024-05-14T11:24:29.505124+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:36:51.156570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496597+0000","last_change":"2024-05-14T11:24:29.493388+0000","last_active":"2024-05-14T11:24:29.496597+0000","last_peered":"2024-05-14T11:24:29.496597+0000","last_clean":"2024-05-14T11:24:29.496597+0000","last_became_active":"2024-05-14T11:24:29.490804+0000","last_became_peered":"2024-05-14T11:24:29.490804+0000","last_unstale":"2024-05-14T11:24:29.496597+0000","last_undegraded":"2024-05-14T11:24:29.496597+0000","last_fullsized":"2024-05-14T11:24:29.496597+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.478303+0000","last_change":"2024-05-14T11:24:28.488547+0000","last_active":"2024-05-14T11:24:29.478303+0000","last_peered":"2024-05-14T11:24:29.478303+0000","last_clean":"2024-05-14T11:24:29.478303+0000","last_became_active":"2024-05-14T11:24:28.487905+0000","last_became_peered":"2024-05-14T11:24:28.487905+0000","last_unstale":"2024-05-14T11:24:29.478303+0000","last_undegraded":"2024-05-14T11:24:29.478303+0000","last_fullsized":"2024-05-14T11:24:29.478303+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:30.677695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480572+0000","last_change":"2024-05-14T11:24:29.476262+0000","last_active":"2024-05-14T11:24:29.480572+0000","last_peered":"2024-05-14T11:24:29.480572+0000","last_clean":"2024-05-14T11:24:29.480572+0000","last_became_active":"2024-05-14T11:24:29.475972+0000","last_became_peered":"2024-05-14T11:24:29.475972+0000","last_unstale":"2024-05-14T11:24:29.480572+0000","last_undegraded":"2024-05-14T11:24:29.480572+0000","last_fullsized":"2024-05-14T11:24:29.480572+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:55.847557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.504794+0000","last_change":"2024-05-14T11:24:28.477500+0000","last_active":"2024-05-14T11:24:29.504794+0000","last_peered":"2024-05-14T11:24:29.504794+0000","last_clean":"2024-05-14T11:24:29.504794+0000","last_became_active":"2024-05-14T11:24:28.477202+0000","last_became_peered":"2024-05-14T11:24:28.477202+0000","last_unstale":"2024-05-14T11:24:29.504794+0000","last_undegraded":"2024-05-14T11:24:29.504794+0000","last_fullsized":"2024-05-14T11:24:29.504794+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:15.207463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497815+0000","last_change":"2024-05-14T11:24:29.494761+0000","last_active":"2024-05-14T11:24:29.497815+0000","last_peered":"2024-05-14T11:24:29.497815+0000","last_clean":"2024-05-14T11:24:29.497815+0000","last_became_active":"2024-05-14T11:24:29.491233+0000","last_became_peered":"2024-05-14T11:24:29.491233+0000","last_unstale":"2024-05-14T11:24:29.497815+0000","last_undegraded":"2024-05-14T11:24:29.497815+0000","last_fullsized":"2024-05-14T11:24:29.497815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:08.468352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474266+0000","last_change":"2024-05-14T11:24:28.485068+0000","last_active":"2024-05-14T11:24:29.474266+0000","last_peered":"2024-05-14T11:24:29.474266+0000","last_clean":"2024-05-14T11:24:29.474266+0000","last_became_active":"2024-05-14T11:24:28.484738+0000","last_became_peered":"2024-05-14T11:24:28.484738+0000","last_unstale":"2024-05-14T11:24:29.474266+0000","last_undegraded":"2024-05-14T11:24:29.474266+0000","last_fullsized":"2024-05-14T11:24:29.474266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:51:35.728704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494864+0000","last_change":"2024-05-14T11:24:29.482296+0000","last_active":"2024-05-14T11:24:29.494864+0000","last_peered":"2024-05-14T11:24:29.494864+0000","last_clean":"2024-05-14T11:24:29.494864+0000","last_became_active":"2024-05-14T11:24:29.481986+0000","last_became_peered":"2024-05-14T11:24:29.481986+0000","last_unstale":"2024-05-14T11:24:29.494864+0000","last_undegraded":"2024-05-14T11:24:29.494864+0000","last_fullsized":"2024-05-14T11:24:29.494864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.582719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.472322+0000","last_change":"2024-05-14T11:24:28.479856+0000","last_active":"2024-05-14T11:24:29.472322+0000","last_peered":"2024-05-14T11:24:29.472322+0000","last_clean":"2024-05-14T11:24:29.472322+0000","last_became_active":"2024-05-14T11:24:28.479562+0000","last_became_peered":"2024-05-14T11:24:28.479562+0000","last_unstale":"2024-05-14T11:24:29.472322+0000","last_undegraded":"2024-05-14T11:24:29.472322+0000","last_fullsized":"2024-05-14T11:24:29.472322+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494815+0000","last_change":"2024-05-14T11:24:29.493476+0000","last_active":"2024-05-14T11:24:29.494815+0000","last_peered":"2024-05-14T11:24:29.494815+0000","last_clean":"2024-05-14T11:24:29.494815+0000","last_became_active":"2024-05-14T11:24:29.493203+0000","last_became_peered":"2024-05-14T11:24:29.493203+0000","last_unstale":"2024-05-14T11:24:29.494815+0000","last_undegraded":"2024-05-14T11:24:29.494815+0000","last_fullsized":"2024-05-14T11:24:29.494815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:06:19.926453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474169+0000","last_change":"2024-05-14T11:24:28.482611+0000","last_active":"2024-05-14T11:24:29.474169+0000","last_peered":"2024-05-14T11:24:29.474169+0000","last_clean":"2024-05-14T11:24:29.474169+0000","last_became_active":"2024-05-14T11:24:28.482263+0000","last_became_peered":"2024-05-14T11:24:28.482263+0000","last_unstale":"2024-05-14T11:24:29.474169+0000","last_undegraded":"2024-05-14T11:24:29.474169+0000","last_fullsized":"2024-05-14T11:24:29.474169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:08.904227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480485+0000","last_change":"2024-05-14T11:24:29.476081+0000","last_active":"2024-05-14T11:24:29.480485+0000","last_peered":"2024-05-14T11:24:29.480485+0000","last_clean":"2024-05-14T11:24:29.480485+0000","last_became_active":"2024-05-14T11:24:29.475784+0000","last_became_peered":"2024-05-14T11:24:29.475784+0000","last_unstale":"2024-05-14T11:24:29.480485+0000","last_undegraded":"2024-05-14T11:24:29.480485+0000","last_fullsized":"2024-05-14T11:24:29.480485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:00:39.418858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507221+0000","last_change":"2024-05-14T11:24:28.478145+0000","last_active":"2024-05-14T11:24:29.507221+0000","last_peered":"2024-05-14T11:24:29.507221+0000","last_clean":"2024-05-14T11:24:29.507221+0000","last_became_active":"2024-05-14T11:24:28.477830+0000","last_became_peered":"2024-05-14T11:24:28.477830+0000","last_unstale":"2024-05-14T11:24:29.507221+0000","last_undegraded":"2024-05-14T11:24:29.507221+0000","last_fullsized":"2024-05-14T11:24:29.507221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:58.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477798+0000","last_change":"2024-05-14T11:24:29.473825+0000","last_active":"2024-05-14T11:24:29.477798+0000","last_peered":"2024-05-14T11:24:29.477798+0000","last_clean":"2024-05-14T11:24:29.477798+0000","last_became_active":"2024-05-14T11:24:29.473487+0000","last_became_peered":"2024-05-14T11:24:29.473487+0000","last_unstale":"2024-05-14T11:24:29.477798+0000","last_undegraded":"2024-05-14T11:24:29.477798+0000","last_fullsized":"2024-05-14T11:24:29.477798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470698+0000","last_change":"2024-05-14T11:24:28.483422+0000","last_active":"2024-05-14T11:24:29.470698+0000","last_peered":"2024-05-14T11:24:29.470698+0000","last_clean":"2024-05-14T11:24:29.470698+0000","last_became_active":"2024-05-14T11:24:28.482778+0000","last_became_peered":"2024-05-14T11:24:28.482778+0000","last_unstale":"2024-05-14T11:24:29.470698+0000","last_undegraded":"2024-05-14T11:24:29.470698+0000","last_fullsized":"2024-05-14T11:24:29.470698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:03:35.833592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.478318+0000","last_change":"2024-05-14T11:24:29.475928+0000","last_active":"2024-05-14T11:24:29.478318+0000","last_peered":"2024-05-14T11:24:29.478318+0000","last_clean":"2024-05-14T11:24:29.478318+0000","last_became_active":"2024-05-14T11:24:29.475602+0000","last_became_peered":"2024-05-14T11:24:29.475602+0000","last_unstale":"2024-05-14T11:24:29.478318+0000","last_undegraded":"2024-05-14T11:24:29.478318+0000","last_fullsized":"2024-05-14T11:24:29.478318+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471810+0000","last_change":"2024-05-14T11:24:28.484667+0000","last_active":"2024-05-14T11:24:29.471810+0000","last_peered":"2024-05-14T11:24:29.471810+0000","last_clean":"2024-05-14T11:24:29.471810+0000","last_became_active":"2024-05-14T11:24:28.484352+0000","last_became_peered":"2024-05-14T11:24:28.484352+0000","last_unstale":"2024-05-14T11:24:29.471810+0000","last_undegraded":"2024-05-14T11:24:29.471810+0000","last_fullsized":"2024-05-14T11:24:29.471810+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:48:47.820347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499524+0000","last_change":"2024-05-14T11:24:29.498563+0000","last_active":"2024-05-14T11:24:29.499524+0000","last_peered":"2024-05-14T11:24:29.499524+0000","last_clean":"2024-05-14T11:24:29.499524+0000","last_became_active":"2024-05-14T11:24:29.498197+0000","last_became_peered":"2024-05-14T11:24:29.498197+0000","last_unstale":"2024-05-14T11:24:29.499524+0000","last_undegraded":"2024-05-14T11:24:29.499524+0000","last_fullsized":"2024-05-14T11:24:29.499524+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:24.258572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.469848+0000","last_change":"2024-05-14T11:24:28.486653+0000","last_active":"2024-05-14T11:24:29.469848+0000","last_peered":"2024-05-14T11:24:29.469848+0000","last_clean":"2024-05-14T11:24:29.469848+0000","last_became_active":"2024-05-14T11:24:28.486363+0000","last_became_peered":"2024-05-14T11:24:28.486363+0000","last_unstale":"2024-05-14T11:24:29.469848+0000","last_undegraded":"2024-05-14T11:24:29.469848+0000","last_fullsized":"2024-05-14T11:24:29.469848+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492199+0000","last_change":"2024-05-14T11:24:29.490253+0000","last_active":"2024-05-14T11:24:29.492199+0000","last_peered":"2024-05-14T11:24:29.492199+0000","last_clean":"2024-05-14T11:24:29.492199+0000","last_became_active":"2024-05-14T11:24:29.489881+0000","last_became_peered":"2024-05-14T11:24:29.489881+0000","last_unstale":"2024-05-14T11:24:29.492199+0000","last_undegraded":"2024-05-14T11:24:29.492199+0000","last_fullsized":"2024-05-14T11:24:29.492199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:36.065423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470320+0000","last_change":"2024-05-14T11:24:28.486137+0000","last_active":"2024-05-14T11:24:29.470320+0000","last_peered":"2024-05-14T11:24:29.470320+0000","last_clean":"2024-05-14T11:24:29.470320+0000","last_became_active":"2024-05-14T11:24:28.485810+0000","last_became_peered":"2024-05-14T11:24:28.485810+0000","last_unstale":"2024-05-14T11:24:29.470320+0000","last_undegraded":"2024-05-14T11:24:29.470320+0000","last_fullsized":"2024-05-14T11:24:29.470320+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:02.760528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499504+0000","last_change":"2024-05-14T11:24:29.495003+0000","last_active":"2024-05-14T11:24:29.499504+0000","last_peered":"2024-05-14T11:24:29.499504+0000","last_clean":"2024-05-14T11:24:29.499504+0000","last_became_active":"2024-05-14T11:24:29.493494+0000","last_became_peered":"2024-05-14T11:24:29.493494+0000","last_unstale":"2024-05-14T11:24:29.499504+0000","last_undegraded":"2024-05-14T11:24:29.499504+0000","last_fullsized":"2024-05-14T11:24:29.499504+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:02:26.179275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471877+0000","last_change":"2024-05-14T11:24:28.477847+0000","last_active":"2024-05-14T11:24:29.471877+0000","last_peered":"2024-05-14T11:24:29.471877+0000","last_clean":"2024-05-14T11:24:29.471877+0000","last_became_active":"2024-05-14T11:24:28.477392+0000","last_became_peered":"2024-05-14T11:24:28.477392+0000","last_unstale":"2024-05-14T11:24:29.471877+0000","last_undegraded":"2024-05-14T11:24:29.471877+0000","last_fullsized":"2024-05-14T11:24:29.471877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:17.986233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495884+0000","last_change":"2024-05-14T11:24:29.494436+0000","last_active":"2024-05-14T11:24:29.495884+0000","last_peered":"2024-05-14T11:24:29.495884+0000","last_clean":"2024-05-14T11:24:29.495884+0000","last_became_active":"2024-05-14T11:24:29.493953+0000","last_became_peered":"2024-05-14T11:24:29.493953+0000","last_unstale":"2024-05-14T11:24:29.495884+0000","last_undegraded":"2024-05-14T11:24:29.495884+0000","last_fullsized":"2024-05-14T11:24:29.495884+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:14:41.122327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471699+0000","last_change":"2024-05-14T11:24:28.484318+0000","last_active":"2024-05-14T11:24:29.471699+0000","last_peered":"2024-05-14T11:24:29.471699+0000","last_clean":"2024-05-14T11:24:29.471699+0000","last_became_active":"2024-05-14T11:24:28.484005+0000","last_became_peered":"2024-05-14T11:24:28.484005+0000","last_unstale":"2024-05-14T11:24:29.471699+0000","last_undegraded":"2024-05-14T11:24:29.471699+0000","last_fullsized":"2024-05-14T11:24:29.471699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:51.761563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495345+0000","last_change":"2024-05-14T11:24:29.490790+0000","last_active":"2024-05-14T11:24:29.495345+0000","last_peered":"2024-05-14T11:24:29.495345+0000","last_clean":"2024-05-14T11:24:29.495345+0000","last_became_active":"2024-05-14T11:24:29.490490+0000","last_became_peered":"2024-05-14T11:24:29.490490+0000","last_unstale":"2024-05-14T11:24:29.495345+0000","last_undegraded":"2024-05-14T11:24:29.495345+0000","last_fullsized":"2024-05-14T11:24:29.495345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474277+0000","last_change":"2024-05-14T11:24:28.479358+0000","last_active":"2024-05-14T11:24:29.474277+0000","last_peered":"2024-05-14T11:24:29.474277+0000","last_clean":"2024-05-14T11:24:29.474277+0000","last_became_active":"2024-05-14T11:24:28.479052+0000","last_became_peered":"2024-05-14T11:24:28.479052+0000","last_unstale":"2024-05-14T11:24:29.474277+0000","last_undegraded":"2024-05-14T11:24:29.474277+0000","last_fullsized":"2024-05-14T11:24:29.474277+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:04:36.505749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496424+0000","last_change":"2024-05-14T11:24:29.494580+0000","last_active":"2024-05-14T11:24:29.496424+0000","last_peered":"2024-05-14T11:24:29.496424+0000","last_clean":"2024-05-14T11:24:29.496424+0000","last_became_active":"2024-05-14T11:24:29.494207+0000","last_became_peered":"2024-05-14T11:24:29.494207+0000","last_unstale":"2024-05-14T11:24:29.496424+0000","last_undegraded":"2024-05-14T11:24:29.496424+0000","last_fullsized":"2024-05-14T11:24:29.496424+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:08:10.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507618+0000","last_change":"2024-05-14T11:24:28.485089+0000","last_active":"2024-05-14T11:24:29.507618+0000","last_peered":"2024-05-14T11:24:29.507618+0000","last_clean":"2024-05-14T11:24:29.507618+0000","last_became_active":"2024-05-14T11:24:28.484228+0000","last_became_peered":"2024-05-14T11:24:28.484228+0000","last_unstale":"2024-05-14T11:24:29.507618+0000","last_undegraded":"2024-05-14T11:24:29.507618+0000","last_fullsized":"2024-05-14T11:24:29.507618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:53.962180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":253,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477789+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:24:29.477789+0000","last_peered":"2024-05-14T11:24:29.477789+0000","last_clean":"2024-05-14T11:24:29.477789+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:24:29.477789+0000","last_undegraded":"2024-05-14T11:24:29.477789+0000","last_fullsized":"2024-05-14T11:24:29.477789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1818624,"data_stored":1152659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1236992,"data_stored":564347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1216512,"data_stored":562131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":562223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":562713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":562165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":562165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1826816,"data_stored":1154201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:24:33.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:24:33.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 Processor -- start 2024-05-14T11:24:33.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 -- start start 2024-05-14T11:24:33.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f863c1602c0 0x7f863c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863c160d50 con 0x7f863c0cc910 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f863c161310 con 0x7f863c1602c0 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f863c161030 con 0x7f863c0c7b00 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f863c1602c0 0x7f863c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f863c1602c0 0x7f863c1606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53996/0 (socket says 172.21.15.116:53996) 2024-05-14T11:24:33.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 -- 172.21.15.116:0/4167961235 learned_addr learned my addr 172.21.15.116:0/4167961235 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642400640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 -- 172.21.15.116:0/4167961235 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 msgr2=0x7f863c0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 -- 172.21.15.116:0/4167961235 >> v1:172.21.15.149:6789/0 conn(0x7f863c0cc910 legacy=0x7f863c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8641a00640 1 -- 172.21.15.116:0/4167961235 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0c7300 con 0x7f863c1602c0 2024-05-14T11:24:33.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.787+0000 7f8642400640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:33.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8641a00640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c1606d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41966622ebbe7758 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f863be00640 1 -- 172.21.15.116:0/4167961235 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300a5760 con 0x7f863c1602c0 2024-05-14T11:24:33.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f863be00640 1 -- 172.21.15.116:0/4167961235 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86300a5900 con 0x7f863c1602c0 2024-05-14T11:24:33.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f863be00640 1 -- 172.21.15.116:0/4167961235 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300a5cd0 con 0x7f863c1602c0 2024-05-14T11:24:33.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 -- 172.21.15.116:0/4167961235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 msgr2=0x7f863c1606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c1606d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863001c9c0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 -- 172.21.15.116:0/4167961235 shutdown_connections 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 --2- 172.21.15.116:0/4167961235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c1606d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 -- 172.21.15.116:0/4167961235 >> 172.21.15.116:0/4167961235 conn(0x7f863c0d28c0 msgr2=0x7f863c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 -- 172.21.15.116:0/4167961235 shutdown_connections 2024-05-14T11:24:33.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.791+0000 7f8642e00640 1 -- 172.21.15.116:0/4167961235 wait complete. 2024-05-14T11:24:33.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 Processor -- start 2024-05-14T11:24:33.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 -- start start 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 0x7f863c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f863c163c30 con 0x7f863c1602c0 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f863c163ab0 con 0x7f863c0cc910 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f863c16ca80 con 0x7f863c0c7b00 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54000/0 (socket says 172.21.15.116:54000) 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642400640 1 -- 172.21.15.116:0/3868304675 learned_addr learned my addr 172.21.15.116:0/3868304675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8641a00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8641000640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 0x7f863c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642400640 1 -- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 msgr2=0x7f863c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.795+0000 7f8642400640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642400640 1 -- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 msgr2=0x7f863c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642400640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 0x7f863c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642400640 1 -- 172.21.15.116:0/3868304675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0aeed0 con 0x7f863c1602c0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8641000640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 0x7f863c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642400640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f863400c7f0 tx=0x7f863400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634015020 con 0x7f863c1602c0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8634004250 con 0x7f863c1602c0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634003620 con 0x7f863c1602c0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0b1300 con 0x7f863c1602c0 2024-05-14T11:24:33.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c0cb900 con 0x7f863c1602c0 2024-05-14T11:24:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c0cbb60 con 0x7f863c1602c0 2024-05-14T11:24:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8634018020 con 0x7f863c1602c0 2024-05-14T11:24:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.799+0000 7f8639600640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 0x7f861c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:33.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.803+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f863408f140 con 0x7f863c1602c0 2024-05-14T11:24:33.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.803+0000 7f8641a00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 0x7f861c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:33.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.803+0000 7f8641a00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 0x7f861c071910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f86300ae000 tx=0x7f86300a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:33.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.807+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f863401f030 con 0x7f863c1602c0 2024-05-14T11:24:33.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.959+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f863c050880 con 0x7f861c06f400 2024-05-14T11:24:33.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.967+0000 7f8639600640 1 -- 172.21.15.116:0/3868304675 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252003 (secure 0 0 0) 0x7f863c050880 con 0x7f861c06f400 2024-05-14T11:24:33.971 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:33.972 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 msgr2=0x7f861c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 0x7f861c071910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f86300ae000 tx=0x7f86300a7070 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 msgr2=0x7f863c0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f863400c7f0 tx=0x7f863400ccb0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 shutdown_connections 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f861c06f400 0x7f861c071910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c0cc910 0x7f863c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f863c1602c0 0x7f863c0b29e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 --2- 172.21.15.116:0/3868304675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f863c0c7b00 0x7f863c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 >> 172.21.15.116:0/3868304675 conn(0x7f863c0d28c0 msgr2=0x7f863c13e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 shutdown_connections 2024-05-14T11:24:33.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:33.971+0000 7f8642e00640 1 -- 172.21.15.116:0/3868304675 wait complete. 2024-05-14T11:24:33.992 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-05-14T11:24:32.487953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260188,"kb_used_data":10728,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714532,"statfs":{"total":773094113280,"available":772827680768,"internally_reserved":0,"allocated":10985472,"data_stored":5682604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.038023"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492239+0000","last_change":"2024-05-14T11:24:29.484520+0000","last_active":"2024-05-14T11:24:29.492239+0000","last_peered":"2024-05-14T11:24:29.492239+0000","last_clean":"2024-05-14T11:24:29.492239+0000","last_became_active":"2024-05-14T11:24:29.480423+0000","last_became_peered":"2024-05-14T11:24:29.480423+0000","last_unstale":"2024-05-14T11:24:29.492239+0000","last_undegraded":"2024-05-14T11:24:29.492239+0000","last_fullsized":"2024-05-14T11:24:29.492239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:50:16.350044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487958+0000","last_change":"2024-05-14T11:24:29.485294+0000","last_active":"2024-05-14T11:24:29.487958+0000","last_peered":"2024-05-14T11:24:29.487958+0000","last_clean":"2024-05-14T11:24:29.487958+0000","last_became_active":"2024-05-14T11:24:29.484551+0000","last_became_peered":"2024-05-14T11:24:29.484551+0000","last_unstale":"2024-05-14T11:24:29.487958+0000","last_undegraded":"2024-05-14T11:24:29.487958+0000","last_fullsized":"2024-05-14T11:24:29.487958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497485+0000","last_change":"2024-05-14T11:24:29.494224+0000","last_active":"2024-05-14T11:24:29.497485+0000","last_peered":"2024-05-14T11:24:29.497485+0000","last_clean":"2024-05-14T11:24:29.497485+0000","last_became_active":"2024-05-14T11:24:29.493907+0000","last_became_peered":"2024-05-14T11:24:29.493907+0000","last_unstale":"2024-05-14T11:24:29.497485+0000","last_undegraded":"2024-05-14T11:24:29.497485+0000","last_fullsized":"2024-05-14T11:24:29.497485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:22.019019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474989+0000","last_change":"2024-05-14T11:24:29.472454+0000","last_active":"2024-05-14T11:24:29.474989+0000","last_peered":"2024-05-14T11:24:29.474989+0000","last_clean":"2024-05-14T11:24:29.474989+0000","last_became_active":"2024-05-14T11:24:29.472135+0000","last_became_peered":"2024-05-14T11:24:29.472135+0000","last_unstale":"2024-05-14T11:24:29.474989+0000","last_undegraded":"2024-05-14T11:24:29.474989+0000","last_fullsized":"2024-05-14T11:24:29.474989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493574+0000","last_change":"2024-05-14T11:24:29.484540+0000","last_active":"2024-05-14T11:24:29.493574+0000","last_peered":"2024-05-14T11:24:29.493574+0000","last_clean":"2024-05-14T11:24:29.493574+0000","last_became_active":"2024-05-14T11:24:29.484248+0000","last_became_peered":"2024-05-14T11:24:29.484248+0000","last_unstale":"2024-05-14T11:24:29.493574+0000","last_undegraded":"2024-05-14T11:24:29.493574+0000","last_fullsized":"2024-05-14T11:24:29.493574+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:31.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487673+0000","last_change":"2024-05-14T11:24:29.485211+0000","last_active":"2024-05-14T11:24:29.487673+0000","last_peered":"2024-05-14T11:24:29.487673+0000","last_clean":"2024-05-14T11:24:29.487673+0000","last_became_active":"2024-05-14T11:24:29.484892+0000","last_became_peered":"2024-05-14T11:24:29.484892+0000","last_unstale":"2024-05-14T11:24:29.487673+0000","last_undegraded":"2024-05-14T11:24:29.487673+0000","last_fullsized":"2024-05-14T11:24:29.487673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.490849+0000","last_change":"2024-05-14T11:24:29.489213+0000","last_active":"2024-05-14T11:24:29.490849+0000","last_peered":"2024-05-14T11:24:29.490849+0000","last_clean":"2024-05-14T11:24:29.490849+0000","last_became_active":"2024-05-14T11:24:29.488499+0000","last_became_peered":"2024-05-14T11:24:29.488499+0000","last_unstale":"2024-05-14T11:24:29.490849+0000","last_undegraded":"2024-05-14T11:24:29.490849+0000","last_fullsized":"2024-05-14T11:24:29.490849+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499432+0000","last_change":"2024-05-14T11:24:29.494430+0000","last_active":"2024-05-14T11:24:29.499432+0000","last_peered":"2024-05-14T11:24:29.499432+0000","last_clean":"2024-05-14T11:24:29.499432+0000","last_became_active":"2024-05-14T11:24:29.494032+0000","last_became_peered":"2024-05-14T11:24:29.494032+0000","last_unstale":"2024-05-14T11:24:29.499432+0000","last_undegraded":"2024-05-14T11:24:29.499432+0000","last_fullsized":"2024-05-14T11:24:29.499432+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:43.455297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480816+0000","last_change":"2024-05-14T11:24:29.476804+0000","last_active":"2024-05-14T11:24:29.480816+0000","last_peered":"2024-05-14T11:24:29.480816+0000","last_clean":"2024-05-14T11:24:29.480816+0000","last_became_active":"2024-05-14T11:24:29.476443+0000","last_became_peered":"2024-05-14T11:24:29.476443+0000","last_unstale":"2024-05-14T11:24:29.480816+0000","last_undegraded":"2024-05-14T11:24:29.480816+0000","last_fullsized":"2024-05-14T11:24:29.480816+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:03.775167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491646+0000","last_change":"2024-05-14T11:24:29.482646+0000","last_active":"2024-05-14T11:24:29.491646+0000","last_peered":"2024-05-14T11:24:29.491646+0000","last_clean":"2024-05-14T11:24:29.491646+0000","last_became_active":"2024-05-14T11:24:29.482337+0000","last_became_peered":"2024-05-14T11:24:29.482337+0000","last_unstale":"2024-05-14T11:24:29.491646+0000","last_undegraded":"2024-05-14T11:24:29.491646+0000","last_fullsized":"2024-05-14T11:24:29.491646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:03.775167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495216+0000","last_change":"2024-05-14T11:24:29.491433+0000","last_active":"2024-05-14T11:24:29.495216+0000","last_peered":"2024-05-14T11:24:29.495216+0000","last_clean":"2024-05-14T11:24:29.495216+0000","last_became_active":"2024-05-14T11:24:29.491114+0000","last_became_peered":"2024-05-14T11:24:29.491114+0000","last_unstale":"2024-05-14T11:24:29.495216+0000","last_undegraded":"2024-05-14T11:24:29.495216+0000","last_fullsized":"2024-05-14T11:24:29.495216+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477763+0000","last_change":"2024-05-14T11:24:29.473072+0000","last_active":"2024-05-14T11:24:29.477763+0000","last_peered":"2024-05-14T11:24:29.477763+0000","last_clean":"2024-05-14T11:24:29.477763+0000","last_became_active":"2024-05-14T11:24:29.472731+0000","last_became_peered":"2024-05-14T11:24:29.472731+0000","last_unstale":"2024-05-14T11:24:29.477763+0000","last_undegraded":"2024-05-14T11:24:29.477763+0000","last_fullsized":"2024-05-14T11:24:29.477763+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496358+0000","last_change":"2024-05-14T11:24:29.493866+0000","last_active":"2024-05-14T11:24:29.496358+0000","last_peered":"2024-05-14T11:24:29.496358+0000","last_clean":"2024-05-14T11:24:29.496358+0000","last_became_active":"2024-05-14T11:24:29.493366+0000","last_became_peered":"2024-05-14T11:24:29.493366+0000","last_unstale":"2024-05-14T11:24:29.496358+0000","last_undegraded":"2024-05-14T11:24:29.496358+0000","last_fullsized":"2024-05-14T11:24:29.496358+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:16:50.275542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494854+0000","last_change":"2024-05-14T11:24:29.492972+0000","last_active":"2024-05-14T11:24:29.494854+0000","last_peered":"2024-05-14T11:24:29.494854+0000","last_clean":"2024-05-14T11:24:29.494854+0000","last_became_active":"2024-05-14T11:24:29.492641+0000","last_became_peered":"2024-05-14T11:24:29.492641+0000","last_unstale":"2024-05-14T11:24:29.494854+0000","last_undegraded":"2024-05-14T11:24:29.494854+0000","last_fullsized":"2024-05-14T11:24:29.494854+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:15:52.028291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496030+0000","last_change":"2024-05-14T11:24:29.494501+0000","last_active":"2024-05-14T11:24:29.496030+0000","last_peered":"2024-05-14T11:24:29.496030+0000","last_clean":"2024-05-14T11:24:29.496030+0000","last_became_active":"2024-05-14T11:24:29.494147+0000","last_became_peered":"2024-05-14T11:24:29.494147+0000","last_unstale":"2024-05-14T11:24:29.496030+0000","last_undegraded":"2024-05-14T11:24:29.496030+0000","last_fullsized":"2024-05-14T11:24:29.496030+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:31.571239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496134+0000","last_change":"2024-05-14T11:24:29.494899+0000","last_active":"2024-05-14T11:24:29.496134+0000","last_peered":"2024-05-14T11:24:29.496134+0000","last_clean":"2024-05-14T11:24:29.496134+0000","last_became_active":"2024-05-14T11:24:29.494585+0000","last_became_peered":"2024-05-14T11:24:29.494585+0000","last_unstale":"2024-05-14T11:24:29.496134+0000","last_undegraded":"2024-05-14T11:24:29.496134+0000","last_fullsized":"2024-05-14T11:24:29.496134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:59:57.884605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497390+0000","last_change":"2024-05-14T11:24:29.492284+0000","last_active":"2024-05-14T11:24:29.497390+0000","last_peered":"2024-05-14T11:24:29.497390+0000","last_clean":"2024-05-14T11:24:29.497390+0000","last_became_active":"2024-05-14T11:24:29.488193+0000","last_became_peered":"2024-05-14T11:24:29.488193+0000","last_unstale":"2024-05-14T11:24:29.497390+0000","last_undegraded":"2024-05-14T11:24:29.497390+0000","last_fullsized":"2024-05-14T11:24:29.497390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:08:55.159038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487995+0000","last_change":"2024-05-14T11:24:29.485799+0000","last_active":"2024-05-14T11:24:29.487995+0000","last_peered":"2024-05-14T11:24:29.487995+0000","last_clean":"2024-05-14T11:24:29.487995+0000","last_became_active":"2024-05-14T11:24:29.482903+0000","last_became_peered":"2024-05-14T11:24:29.482903+0000","last_unstale":"2024-05-14T11:24:29.487995+0000","last_undegraded":"2024-05-14T11:24:29.487995+0000","last_fullsized":"2024-05-14T11:24:29.487995+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491548+0000","last_change":"2024-05-14T11:24:29.486345+0000","last_active":"2024-05-14T11:24:29.491548+0000","last_peered":"2024-05-14T11:24:29.491548+0000","last_clean":"2024-05-14T11:24:29.491548+0000","last_became_active":"2024-05-14T11:24:29.486038+0000","last_became_peered":"2024-05-14T11:24:29.486038+0000","last_unstale":"2024-05-14T11:24:29.491548+0000","last_undegraded":"2024-05-14T11:24:29.491548+0000","last_fullsized":"2024-05-14T11:24:29.491548+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:59:13.050744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493107+0000","last_change":"2024-05-14T11:24:29.487286+0000","last_active":"2024-05-14T11:24:29.493107+0000","last_peered":"2024-05-14T11:24:29.493107+0000","last_clean":"2024-05-14T11:24:29.493107+0000","last_became_active":"2024-05-14T11:24:29.486974+0000","last_became_peered":"2024-05-14T11:24:29.486974+0000","last_unstale":"2024-05-14T11:24:29.493107+0000","last_undegraded":"2024-05-14T11:24:29.493107+0000","last_fullsized":"2024-05-14T11:24:29.493107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:34:10.556055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494298+0000","last_change":"2024-05-14T11:24:29.492520+0000","last_active":"2024-05-14T11:24:29.494298+0000","last_peered":"2024-05-14T11:24:29.494298+0000","last_clean":"2024-05-14T11:24:29.494298+0000","last_became_active":"2024-05-14T11:24:29.492005+0000","last_became_peered":"2024-05-14T11:24:29.492005+0000","last_unstale":"2024-05-14T11:24:29.494298+0000","last_undegraded":"2024-05-14T11:24:29.494298+0000","last_fullsized":"2024-05-14T11:24:29.494298+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:49.188374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494616+0000","last_change":"2024-05-14T11:24:29.491260+0000","last_active":"2024-05-14T11:24:29.494616+0000","last_peered":"2024-05-14T11:24:29.494616+0000","last_clean":"2024-05-14T11:24:29.494616+0000","last_became_active":"2024-05-14T11:24:29.487960+0000","last_became_peered":"2024-05-14T11:24:29.487960+0000","last_unstale":"2024-05-14T11:24:29.494616+0000","last_undegraded":"2024-05-14T11:24:29.494616+0000","last_fullsized":"2024-05-14T11:24:29.494616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491606+0000","last_change":"2024-05-14T11:24:29.488023+0000","last_active":"2024-05-14T11:24:29.491606+0000","last_peered":"2024-05-14T11:24:29.491606+0000","last_clean":"2024-05-14T11:24:29.491606+0000","last_became_active":"2024-05-14T11:24:29.487514+0000","last_became_peered":"2024-05-14T11:24:29.487514+0000","last_unstale":"2024-05-14T11:24:29.491606+0000","last_undegraded":"2024-05-14T11:24:29.491606+0000","last_fullsized":"2024-05-14T11:24:29.491606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:35:45.048217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493215+0000","last_change":"2024-05-14T11:24:29.485288+0000","last_active":"2024-05-14T11:24:29.493215+0000","last_peered":"2024-05-14T11:24:29.493215+0000","last_clean":"2024-05-14T11:24:29.493215+0000","last_became_active":"2024-05-14T11:24:29.484999+0000","last_became_peered":"2024-05-14T11:24:29.484999+0000","last_unstale":"2024-05-14T11:24:29.493215+0000","last_undegraded":"2024-05-14T11:24:29.493215+0000","last_fullsized":"2024-05-14T11:24:29.493215+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491696+0000","last_change":"2024-05-14T11:24:29.484177+0000","last_active":"2024-05-14T11:24:29.491696+0000","last_peered":"2024-05-14T11:24:29.491696+0000","last_clean":"2024-05-14T11:24:29.491696+0000","last_became_active":"2024-05-14T11:24:29.483842+0000","last_became_peered":"2024-05-14T11:24:29.483842+0000","last_unstale":"2024-05-14T11:24:29.491696+0000","last_undegraded":"2024-05-14T11:24:29.491696+0000","last_fullsized":"2024-05-14T11:24:29.491696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487864+0000","last_change":"2024-05-14T11:24:29.481868+0000","last_active":"2024-05-14T11:24:29.487864+0000","last_peered":"2024-05-14T11:24:29.487864+0000","last_clean":"2024-05-14T11:24:29.487864+0000","last_became_active":"2024-05-14T11:24:29.481551+0000","last_became_peered":"2024-05-14T11:24:29.481551+0000","last_unstale":"2024-05-14T11:24:29.487864+0000","last_undegraded":"2024-05-14T11:24:29.487864+0000","last_fullsized":"2024-05-14T11:24:29.487864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495238+0000","last_change":"2024-05-14T11:24:29.492229+0000","last_active":"2024-05-14T11:24:29.495238+0000","last_peered":"2024-05-14T11:24:29.495238+0000","last_clean":"2024-05-14T11:24:29.495238+0000","last_became_active":"2024-05-14T11:24:29.491940+0000","last_became_peered":"2024-05-14T11:24:29.491940+0000","last_unstale":"2024-05-14T11:24:29.495238+0000","last_undegraded":"2024-05-14T11:24:29.495238+0000","last_fullsized":"2024-05-14T11:24:29.495238+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487511+0000","last_change":"2024-05-14T11:24:29.484409+0000","last_active":"2024-05-14T11:24:29.487511+0000","last_peered":"2024-05-14T11:24:29.487511+0000","last_clean":"2024-05-14T11:24:29.487511+0000","last_became_active":"2024-05-14T11:24:29.484115+0000","last_became_peered":"2024-05-14T11:24:29.484115+0000","last_unstale":"2024-05-14T11:24:29.487511+0000","last_undegraded":"2024-05-14T11:24:29.487511+0000","last_fullsized":"2024-05-14T11:24:29.487511+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.490270+0000","last_change":"2024-05-14T11:24:29.488584+0000","last_active":"2024-05-14T11:24:29.490270+0000","last_peered":"2024-05-14T11:24:29.490270+0000","last_clean":"2024-05-14T11:24:29.490270+0000","last_became_active":"2024-05-14T11:24:29.488230+0000","last_became_peered":"2024-05-14T11:24:29.488230+0000","last_unstale":"2024-05-14T11:24:29.490270+0000","last_undegraded":"2024-05-14T11:24:29.490270+0000","last_fullsized":"2024-05-14T11:24:29.490270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:02:00.028776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491691+0000","last_change":"2024-05-14T11:24:29.484420+0000","last_active":"2024-05-14T11:24:29.491691+0000","last_peered":"2024-05-14T11:24:29.491691+0000","last_clean":"2024-05-14T11:24:29.491691+0000","last_became_active":"2024-05-14T11:24:29.484112+0000","last_became_peered":"2024-05-14T11:24:29.484112+0000","last_unstale":"2024-05-14T11:24:29.491691+0000","last_undegraded":"2024-05-14T11:24:29.491691+0000","last_fullsized":"2024-05-14T11:24:29.491691+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.488798+0000","last_change":"2024-05-14T11:24:29.483219+0000","last_active":"2024-05-14T11:24:29.488798+0000","last_peered":"2024-05-14T11:24:29.488798+0000","last_clean":"2024-05-14T11:24:29.488798+0000","last_became_active":"2024-05-14T11:24:29.482753+0000","last_became_peered":"2024-05-14T11:24:29.482753+0000","last_unstale":"2024-05-14T11:24:29.488798+0000","last_undegraded":"2024-05-14T11:24:29.488798+0000","last_fullsized":"2024-05-14T11:24:29.488798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:48.950268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.489717+0000","last_change":"2024-05-14T11:24:29.482521+0000","last_active":"2024-05-14T11:24:29.489717+0000","last_peered":"2024-05-14T11:24:29.489717+0000","last_clean":"2024-05-14T11:24:29.489717+0000","last_became_active":"2024-05-14T11:24:29.482237+0000","last_became_peered":"2024-05-14T11:24:29.482237+0000","last_unstale":"2024-05-14T11:24:29.489717+0000","last_undegraded":"2024-05-14T11:24:29.489717+0000","last_fullsized":"2024-05-14T11:24:29.489717+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:18.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480510+0000","last_change":"2024-05-14T11:24:29.476521+0000","last_active":"2024-05-14T11:24:29.480510+0000","last_peered":"2024-05-14T11:24:29.480510+0000","last_clean":"2024-05-14T11:24:29.480510+0000","last_became_active":"2024-05-14T11:24:29.475357+0000","last_became_peered":"2024-05-14T11:24:29.475357+0000","last_unstale":"2024-05-14T11:24:29.480510+0000","last_undegraded":"2024-05-14T11:24:29.480510+0000","last_fullsized":"2024-05-14T11:24:29.480510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:30:12.175619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506251+0000","last_change":"2024-05-14T11:24:28.487696+0000","last_active":"2024-05-14T11:24:29.506251+0000","last_peered":"2024-05-14T11:24:29.506251+0000","last_clean":"2024-05-14T11:24:29.506251+0000","last_became_active":"2024-05-14T11:24:28.487372+0000","last_became_peered":"2024-05-14T11:24:28.487372+0000","last_unstale":"2024-05-14T11:24:29.506251+0000","last_undegraded":"2024-05-14T11:24:29.506251+0000","last_fullsized":"2024-05-14T11:24:29.506251+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:29.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499291+0000","last_change":"2024-05-14T11:24:29.495374+0000","last_active":"2024-05-14T11:24:29.499291+0000","last_peered":"2024-05-14T11:24:29.499291+0000","last_clean":"2024-05-14T11:24:29.499291+0000","last_became_active":"2024-05-14T11:24:29.493020+0000","last_became_peered":"2024-05-14T11:24:29.493020+0000","last_unstale":"2024-05-14T11:24:29.499291+0000","last_undegraded":"2024-05-14T11:24:29.499291+0000","last_fullsized":"2024-05-14T11:24:29.499291+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:24:00.787515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506505+0000","last_change":"2024-05-14T11:24:28.485400+0000","last_active":"2024-05-14T11:24:29.506505+0000","last_peered":"2024-05-14T11:24:29.506505+0000","last_clean":"2024-05-14T11:24:29.506505+0000","last_became_active":"2024-05-14T11:24:28.484706+0000","last_became_peered":"2024-05-14T11:24:28.484706+0000","last_unstale":"2024-05-14T11:24:29.506505+0000","last_undegraded":"2024-05-14T11:24:29.506505+0000","last_fullsized":"2024-05-14T11:24:29.506505+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:03:52.595560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493386+0000","last_change":"2024-05-14T11:24:29.487675+0000","last_active":"2024-05-14T11:24:29.493386+0000","last_peered":"2024-05-14T11:24:29.493386+0000","last_clean":"2024-05-14T11:24:29.493386+0000","last_became_active":"2024-05-14T11:24:29.487292+0000","last_became_peered":"2024-05-14T11:24:29.487292+0000","last_unstale":"2024-05-14T11:24:29.493386+0000","last_undegraded":"2024-05-14T11:24:29.493386+0000","last_fullsized":"2024-05-14T11:24:29.493386+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506071+0000","last_change":"2024-05-14T11:24:28.483038+0000","last_active":"2024-05-14T11:24:29.506071+0000","last_peered":"2024-05-14T11:24:29.506071+0000","last_clean":"2024-05-14T11:24:29.506071+0000","last_became_active":"2024-05-14T11:24:28.482438+0000","last_became_peered":"2024-05-14T11:24:28.482438+0000","last_unstale":"2024-05-14T11:24:29.506071+0000","last_undegraded":"2024-05-14T11:24:29.506071+0000","last_fullsized":"2024-05-14T11:24:29.506071+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:58.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.493138+0000","last_change":"2024-05-14T11:24:29.487686+0000","last_active":"2024-05-14T11:24:29.493138+0000","last_peered":"2024-05-14T11:24:29.493138+0000","last_clean":"2024-05-14T11:24:29.493138+0000","last_became_active":"2024-05-14T11:24:29.485570+0000","last_became_peered":"2024-05-14T11:24:29.485570+0000","last_unstale":"2024-05-14T11:24:29.493138+0000","last_undegraded":"2024-05-14T11:24:29.493138+0000","last_fullsized":"2024-05-14T11:24:29.493138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505928+0000","last_change":"2024-05-14T11:24:28.481787+0000","last_active":"2024-05-14T11:24:29.505928+0000","last_peered":"2024-05-14T11:24:29.505928+0000","last_clean":"2024-05-14T11:24:29.505928+0000","last_became_active":"2024-05-14T11:24:28.481471+0000","last_became_peered":"2024-05-14T11:24:28.481471+0000","last_unstale":"2024-05-14T11:24:29.505928+0000","last_undegraded":"2024-05-14T11:24:29.505928+0000","last_fullsized":"2024-05-14T11:24:29.505928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:19:25.815981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495390+0000","last_change":"2024-05-14T11:24:29.483524+0000","last_active":"2024-05-14T11:24:29.495390+0000","last_peered":"2024-05-14T11:24:29.495390+0000","last_clean":"2024-05-14T11:24:29.495390+0000","last_became_active":"2024-05-14T11:24:29.483193+0000","last_became_peered":"2024-05-14T11:24:29.483193+0000","last_unstale":"2024-05-14T11:24:29.495390+0000","last_undegraded":"2024-05-14T11:24:29.495390+0000","last_fullsized":"2024-05-14T11:24:29.495390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:08:10.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.504022+0000","last_change":"2024-05-14T11:24:28.489640+0000","last_active":"2024-05-14T11:24:29.504022+0000","last_peered":"2024-05-14T11:24:29.504022+0000","last_clean":"2024-05-14T11:24:29.504022+0000","last_became_active":"2024-05-14T11:24:28.489213+0000","last_became_peered":"2024-05-14T11:24:28.489213+0000","last_unstale":"2024-05-14T11:24:29.504022+0000","last_undegraded":"2024-05-14T11:24:29.504022+0000","last_fullsized":"2024-05-14T11:24:29.504022+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:01:45.116165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491508+0000","last_change":"2024-05-14T11:24:29.485807+0000","last_active":"2024-05-14T11:24:29.491508+0000","last_peered":"2024-05-14T11:24:29.491508+0000","last_clean":"2024-05-14T11:24:29.491508+0000","last_became_active":"2024-05-14T11:24:29.485516+0000","last_became_peered":"2024-05-14T11:24:29.485516+0000","last_unstale":"2024-05-14T11:24:29.491508+0000","last_undegraded":"2024-05-14T11:24:29.491508+0000","last_fullsized":"2024-05-14T11:24:29.491508+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:12:15.420193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505424+0000","last_change":"2024-05-14T11:24:28.478497+0000","last_active":"2024-05-14T11:24:29.505424+0000","last_peered":"2024-05-14T11:24:29.505424+0000","last_clean":"2024-05-14T11:24:29.505424+0000","last_became_active":"2024-05-14T11:24:28.478156+0000","last_became_peered":"2024-05-14T11:24:28.478156+0000","last_unstale":"2024-05-14T11:24:29.505424+0000","last_undegraded":"2024-05-14T11:24:29.505424+0000","last_fullsized":"2024-05-14T11:24:29.505424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:57.865081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487935+0000","last_change":"2024-05-14T11:24:29.485000+0000","last_active":"2024-05-14T11:24:29.487935+0000","last_peered":"2024-05-14T11:24:29.487935+0000","last_clean":"2024-05-14T11:24:29.487935+0000","last_became_active":"2024-05-14T11:24:29.483862+0000","last_became_peered":"2024-05-14T11:24:29.483862+0000","last_unstale":"2024-05-14T11:24:29.487935+0000","last_undegraded":"2024-05-14T11:24:29.487935+0000","last_fullsized":"2024-05-14T11:24:29.487935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.469007+0000","last_change":"2024-05-14T11:24:28.483085+0000","last_active":"2024-05-14T11:24:29.469007+0000","last_peered":"2024-05-14T11:24:29.469007+0000","last_clean":"2024-05-14T11:24:29.469007+0000","last_became_active":"2024-05-14T11:24:28.482701+0000","last_became_peered":"2024-05-14T11:24:28.482701+0000","last_unstale":"2024-05-14T11:24:29.469007+0000","last_undegraded":"2024-05-14T11:24:29.469007+0000","last_fullsized":"2024-05-14T11:24:29.469007+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496711+0000","last_change":"2024-05-14T11:24:29.483810+0000","last_active":"2024-05-14T11:24:29.496711+0000","last_peered":"2024-05-14T11:24:29.496711+0000","last_clean":"2024-05-14T11:24:29.496711+0000","last_became_active":"2024-05-14T11:24:29.483258+0000","last_became_peered":"2024-05-14T11:24:29.483258+0000","last_unstale":"2024-05-14T11:24:29.496711+0000","last_undegraded":"2024-05-14T11:24:29.496711+0000","last_fullsized":"2024-05-14T11:24:29.496711+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:24.258572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.463579+0000","last_change":"2024-05-14T11:24:28.481747+0000","last_active":"2024-05-14T11:24:29.463579+0000","last_peered":"2024-05-14T11:24:29.463579+0000","last_clean":"2024-05-14T11:24:29.463579+0000","last_became_active":"2024-05-14T11:24:28.481455+0000","last_became_peered":"2024-05-14T11:24:28.481455+0000","last_unstale":"2024-05-14T11:24:29.463579+0000","last_undegraded":"2024-05-14T11:24:29.463579+0000","last_fullsized":"2024-05-14T11:24:29.463579+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:57.865081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.491543+0000","last_change":"2024-05-14T11:24:29.482659+0000","last_active":"2024-05-14T11:24:29.491543+0000","last_peered":"2024-05-14T11:24:29.491543+0000","last_clean":"2024-05-14T11:24:29.491543+0000","last_became_active":"2024-05-14T11:24:29.482355+0000","last_became_peered":"2024-05-14T11:24:29.482355+0000","last_unstale":"2024-05-14T11:24:29.491543+0000","last_undegraded":"2024-05-14T11:24:29.491543+0000","last_fullsized":"2024-05-14T11:24:29.491543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:42:42.618784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470981+0000","last_change":"2024-05-14T11:24:28.482077+0000","last_active":"2024-05-14T11:24:29.470981+0000","last_peered":"2024-05-14T11:24:29.470981+0000","last_clean":"2024-05-14T11:24:29.470981+0000","last_became_active":"2024-05-14T11:24:28.481549+0000","last_became_peered":"2024-05-14T11:24:28.481549+0000","last_unstale":"2024-05-14T11:24:29.470981+0000","last_undegraded":"2024-05-14T11:24:29.470981+0000","last_fullsized":"2024-05-14T11:24:29.470981+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:30.633067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.500978+0000","last_change":"2024-05-14T11:24:29.499419+0000","last_active":"2024-05-14T11:24:29.500978+0000","last_peered":"2024-05-14T11:24:29.500978+0000","last_clean":"2024-05-14T11:24:29.500978+0000","last_became_active":"2024-05-14T11:24:29.499090+0000","last_became_peered":"2024-05-14T11:24:29.499090+0000","last_unstale":"2024-05-14T11:24:29.500978+0000","last_undegraded":"2024-05-14T11:24:29.500978+0000","last_fullsized":"2024-05-14T11:24:29.500978+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470375+0000","last_change":"2024-05-14T11:24:28.469226+0000","last_active":"2024-05-14T11:24:29.470375+0000","last_peered":"2024-05-14T11:24:29.470375+0000","last_clean":"2024-05-14T11:24:29.470375+0000","last_became_active":"2024-05-14T11:24:28.468777+0000","last_became_peered":"2024-05-14T11:24:28.468777+0000","last_unstale":"2024-05-14T11:24:29.470375+0000","last_undegraded":"2024-05-14T11:24:29.470375+0000","last_fullsized":"2024-05-14T11:24:29.470375+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:21:23.617550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499336+0000","last_change":"2024-05-14T11:24:29.497609+0000","last_active":"2024-05-14T11:24:29.499336+0000","last_peered":"2024-05-14T11:24:29.499336+0000","last_clean":"2024-05-14T11:24:29.499336+0000","last_became_active":"2024-05-14T11:24:29.497209+0000","last_became_peered":"2024-05-14T11:24:29.497209+0000","last_unstale":"2024-05-14T11:24:29.499336+0000","last_undegraded":"2024-05-14T11:24:29.499336+0000","last_fullsized":"2024-05-14T11:24:29.499336+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:51.987797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.506076+0000","last_change":"2024-05-14T11:24:28.487409+0000","last_active":"2024-05-14T11:24:29.506076+0000","last_peered":"2024-05-14T11:24:29.506076+0000","last_clean":"2024-05-14T11:24:29.506076+0000","last_became_active":"2024-05-14T11:24:28.486959+0000","last_became_peered":"2024-05-14T11:24:28.486959+0000","last_unstale":"2024-05-14T11:24:29.506076+0000","last_undegraded":"2024-05-14T11:24:29.506076+0000","last_fullsized":"2024-05-14T11:24:29.506076+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:09:16.738352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499973+0000","last_change":"2024-05-14T11:24:29.498150+0000","last_active":"2024-05-14T11:24:29.499973+0000","last_peered":"2024-05-14T11:24:29.499973+0000","last_clean":"2024-05-14T11:24:29.499973+0000","last_became_active":"2024-05-14T11:24:29.497129+0000","last_became_peered":"2024-05-14T11:24:29.497129+0000","last_unstale":"2024-05-14T11:24:29.499973+0000","last_undegraded":"2024-05-14T11:24:29.499973+0000","last_fullsized":"2024-05-14T11:24:29.499973+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:57.145933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.509295+0000","last_change":"2024-05-14T11:24:28.481570+0000","last_active":"2024-05-14T11:24:29.509295+0000","last_peered":"2024-05-14T11:24:29.509295+0000","last_clean":"2024-05-14T11:24:29.509295+0000","last_became_active":"2024-05-14T11:24:28.481242+0000","last_became_peered":"2024-05-14T11:24:28.481242+0000","last_unstale":"2024-05-14T11:24:29.509295+0000","last_undegraded":"2024-05-14T11:24:29.509295+0000","last_fullsized":"2024-05-14T11:24:29.509295+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:19:25.815981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496370+0000","last_change":"2024-05-14T11:24:29.494265+0000","last_active":"2024-05-14T11:24:29.496370+0000","last_peered":"2024-05-14T11:24:29.496370+0000","last_clean":"2024-05-14T11:24:29.496370+0000","last_became_active":"2024-05-14T11:24:29.493922+0000","last_became_peered":"2024-05-14T11:24:29.493922+0000","last_unstale":"2024-05-14T11:24:29.496370+0000","last_undegraded":"2024-05-14T11:24:29.496370+0000","last_fullsized":"2024-05-14T11:24:29.496370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.582719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507344+0000","last_change":"2024-05-14T11:24:28.481094+0000","last_active":"2024-05-14T11:24:29.507344+0000","last_peered":"2024-05-14T11:24:29.507344+0000","last_clean":"2024-05-14T11:24:29.507344+0000","last_became_active":"2024-05-14T11:24:28.480694+0000","last_became_peered":"2024-05-14T11:24:28.480694+0000","last_unstale":"2024-05-14T11:24:29.507344+0000","last_undegraded":"2024-05-14T11:24:29.507344+0000","last_fullsized":"2024-05-14T11:24:29.507344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.481814+0000","last_change":"2024-05-14T11:24:29.475922+0000","last_active":"2024-05-14T11:24:29.481814+0000","last_peered":"2024-05-14T11:24:29.481814+0000","last_clean":"2024-05-14T11:24:29.481814+0000","last_became_active":"2024-05-14T11:24:29.475535+0000","last_became_peered":"2024-05-14T11:24:29.475535+0000","last_unstale":"2024-05-14T11:24:29.481814+0000","last_undegraded":"2024-05-14T11:24:29.481814+0000","last_fullsized":"2024-05-14T11:24:29.481814+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:59.663108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.463728+0000","last_change":"2024-05-14T11:24:28.482038+0000","last_active":"2024-05-14T11:24:29.463728+0000","last_peered":"2024-05-14T11:24:29.463728+0000","last_clean":"2024-05-14T11:24:29.463728+0000","last_became_active":"2024-05-14T11:24:28.481245+0000","last_became_peered":"2024-05-14T11:24:28.481245+0000","last_unstale":"2024-05-14T11:24:29.463728+0000","last_undegraded":"2024-05-14T11:24:29.463728+0000","last_fullsized":"2024-05-14T11:24:29.463728+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:09:56.131294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492232+0000","last_change":"2024-05-14T11:24:29.490254+0000","last_active":"2024-05-14T11:24:29.492232+0000","last_peered":"2024-05-14T11:24:29.492232+0000","last_clean":"2024-05-14T11:24:29.492232+0000","last_became_active":"2024-05-14T11:24:29.489784+0000","last_became_peered":"2024-05-14T11:24:29.489784+0000","last_unstale":"2024-05-14T11:24:29.492232+0000","last_undegraded":"2024-05-14T11:24:29.492232+0000","last_fullsized":"2024-05-14T11:24:29.492232+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:04:02.842633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.473640+0000","last_change":"2024-05-14T11:24:28.478187+0000","last_active":"2024-05-14T11:24:29.473640+0000","last_peered":"2024-05-14T11:24:29.473640+0000","last_clean":"2024-05-14T11:24:29.473640+0000","last_became_active":"2024-05-14T11:24:28.477875+0000","last_became_peered":"2024-05-14T11:24:28.477875+0000","last_unstale":"2024-05-14T11:24:29.473640+0000","last_undegraded":"2024-05-14T11:24:29.473640+0000","last_fullsized":"2024-05-14T11:24:29.473640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:47:38.000500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497313+0000","last_change":"2024-05-14T11:24:29.492094+0000","last_active":"2024-05-14T11:24:29.497313+0000","last_peered":"2024-05-14T11:24:29.497313+0000","last_clean":"2024-05-14T11:24:29.497313+0000","last_became_active":"2024-05-14T11:24:29.491105+0000","last_became_peered":"2024-05-14T11:24:29.491105+0000","last_unstale":"2024-05-14T11:24:29.497313+0000","last_undegraded":"2024-05-14T11:24:29.497313+0000","last_fullsized":"2024-05-14T11:24:29.497313+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:59:17.333394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507575+0000","last_change":"2024-05-14T11:24:28.478525+0000","last_active":"2024-05-14T11:24:29.507575+0000","last_peered":"2024-05-14T11:24:29.507575+0000","last_clean":"2024-05-14T11:24:29.507575+0000","last_became_active":"2024-05-14T11:24:28.478026+0000","last_became_peered":"2024-05-14T11:24:28.478026+0000","last_unstale":"2024-05-14T11:24:29.507575+0000","last_undegraded":"2024-05-14T11:24:29.507575+0000","last_fullsized":"2024-05-14T11:24:29.507575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.489693+0000","last_change":"2024-05-14T11:24:29.480098+0000","last_active":"2024-05-14T11:24:29.489693+0000","last_peered":"2024-05-14T11:24:29.489693+0000","last_clean":"2024-05-14T11:24:29.489693+0000","last_became_active":"2024-05-14T11:24:29.479761+0000","last_became_peered":"2024-05-14T11:24:29.479761+0000","last_unstale":"2024-05-14T11:24:29.489693+0000","last_undegraded":"2024-05-14T11:24:29.489693+0000","last_fullsized":"2024-05-14T11:24:29.489693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:58:49.486260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505153+0000","last_change":"2024-05-14T11:24:28.482087+0000","last_active":"2024-05-14T11:24:29.505153+0000","last_peered":"2024-05-14T11:24:29.505153+0000","last_clean":"2024-05-14T11:24:29.505153+0000","last_became_active":"2024-05-14T11:24:28.481684+0000","last_became_peered":"2024-05-14T11:24:28.481684+0000","last_unstale":"2024-05-14T11:24:29.505153+0000","last_undegraded":"2024-05-14T11:24:29.505153+0000","last_fullsized":"2024-05-14T11:24:29.505153+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:38:20.726176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.487493+0000","last_change":"2024-05-14T11:24:29.482840+0000","last_active":"2024-05-14T11:24:29.487493+0000","last_peered":"2024-05-14T11:24:29.487493+0000","last_clean":"2024-05-14T11:24:29.487493+0000","last_became_active":"2024-05-14T11:24:29.481947+0000","last_became_peered":"2024-05-14T11:24:29.481947+0000","last_unstale":"2024-05-14T11:24:29.487493+0000","last_undegraded":"2024-05-14T11:24:29.487493+0000","last_fullsized":"2024-05-14T11:24:29.487493+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:06:31.647706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.505124+0000","last_change":"2024-05-14T11:24:28.485226+0000","last_active":"2024-05-14T11:24:29.505124+0000","last_peered":"2024-05-14T11:24:29.505124+0000","last_clean":"2024-05-14T11:24:29.505124+0000","last_became_active":"2024-05-14T11:24:28.484885+0000","last_became_peered":"2024-05-14T11:24:28.484885+0000","last_unstale":"2024-05-14T11:24:29.505124+0000","last_undegraded":"2024-05-14T11:24:29.505124+0000","last_fullsized":"2024-05-14T11:24:29.505124+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:36:51.156570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496597+0000","last_change":"2024-05-14T11:24:29.493388+0000","last_active":"2024-05-14T11:24:29.496597+0000","last_peered":"2024-05-14T11:24:29.496597+0000","last_clean":"2024-05-14T11:24:29.496597+0000","last_became_active":"2024-05-14T11:24:29.490804+0000","last_became_peered":"2024-05-14T11:24:29.490804+0000","last_unstale":"2024-05-14T11:24:29.496597+0000","last_undegraded":"2024-05-14T11:24:29.496597+0000","last_fullsized":"2024-05-14T11:24:29.496597+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:40:58.879720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.478303+0000","last_change":"2024-05-14T11:24:28.488547+0000","last_active":"2024-05-14T11:24:29.478303+0000","last_peered":"2024-05-14T11:24:29.478303+0000","last_clean":"2024-05-14T11:24:29.478303+0000","last_became_active":"2024-05-14T11:24:28.487905+0000","last_became_peered":"2024-05-14T11:24:28.487905+0000","last_unstale":"2024-05-14T11:24:29.478303+0000","last_undegraded":"2024-05-14T11:24:29.478303+0000","last_fullsized":"2024-05-14T11:24:29.478303+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:30.677695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480572+0000","last_change":"2024-05-14T11:24:29.476262+0000","last_active":"2024-05-14T11:24:29.480572+0000","last_peered":"2024-05-14T11:24:29.480572+0000","last_clean":"2024-05-14T11:24:29.480572+0000","last_became_active":"2024-05-14T11:24:29.475972+0000","last_became_peered":"2024-05-14T11:24:29.475972+0000","last_unstale":"2024-05-14T11:24:29.480572+0000","last_undegraded":"2024-05-14T11:24:29.480572+0000","last_fullsized":"2024-05-14T11:24:29.480572+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:30:55.847557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.504794+0000","last_change":"2024-05-14T11:24:28.477500+0000","last_active":"2024-05-14T11:24:29.504794+0000","last_peered":"2024-05-14T11:24:29.504794+0000","last_clean":"2024-05-14T11:24:29.504794+0000","last_became_active":"2024-05-14T11:24:28.477202+0000","last_became_peered":"2024-05-14T11:24:28.477202+0000","last_unstale":"2024-05-14T11:24:29.504794+0000","last_undegraded":"2024-05-14T11:24:29.504794+0000","last_fullsized":"2024-05-14T11:24:29.504794+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:05:15.207463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.497815+0000","last_change":"2024-05-14T11:24:29.494761+0000","last_active":"2024-05-14T11:24:29.497815+0000","last_peered":"2024-05-14T11:24:29.497815+0000","last_clean":"2024-05-14T11:24:29.497815+0000","last_became_active":"2024-05-14T11:24:29.491233+0000","last_became_peered":"2024-05-14T11:24:29.491233+0000","last_unstale":"2024-05-14T11:24:29.497815+0000","last_undegraded":"2024-05-14T11:24:29.497815+0000","last_fullsized":"2024-05-14T11:24:29.497815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:08.468352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474266+0000","last_change":"2024-05-14T11:24:28.485068+0000","last_active":"2024-05-14T11:24:29.474266+0000","last_peered":"2024-05-14T11:24:29.474266+0000","last_clean":"2024-05-14T11:24:29.474266+0000","last_became_active":"2024-05-14T11:24:28.484738+0000","last_became_peered":"2024-05-14T11:24:28.484738+0000","last_unstale":"2024-05-14T11:24:29.474266+0000","last_undegraded":"2024-05-14T11:24:29.474266+0000","last_fullsized":"2024-05-14T11:24:29.474266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:51:35.728704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494864+0000","last_change":"2024-05-14T11:24:29.482296+0000","last_active":"2024-05-14T11:24:29.494864+0000","last_peered":"2024-05-14T11:24:29.494864+0000","last_clean":"2024-05-14T11:24:29.494864+0000","last_became_active":"2024-05-14T11:24:29.481986+0000","last_became_peered":"2024-05-14T11:24:29.481986+0000","last_unstale":"2024-05-14T11:24:29.494864+0000","last_undegraded":"2024-05-14T11:24:29.494864+0000","last_fullsized":"2024-05-14T11:24:29.494864+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:11:19.582719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.472322+0000","last_change":"2024-05-14T11:24:28.479856+0000","last_active":"2024-05-14T11:24:29.472322+0000","last_peered":"2024-05-14T11:24:29.472322+0000","last_clean":"2024-05-14T11:24:29.472322+0000","last_became_active":"2024-05-14T11:24:28.479562+0000","last_became_peered":"2024-05-14T11:24:28.479562+0000","last_unstale":"2024-05-14T11:24:29.472322+0000","last_undegraded":"2024-05-14T11:24:29.472322+0000","last_fullsized":"2024-05-14T11:24:29.472322+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.494815+0000","last_change":"2024-05-14T11:24:29.493476+0000","last_active":"2024-05-14T11:24:29.494815+0000","last_peered":"2024-05-14T11:24:29.494815+0000","last_clean":"2024-05-14T11:24:29.494815+0000","last_became_active":"2024-05-14T11:24:29.493203+0000","last_became_peered":"2024-05-14T11:24:29.493203+0000","last_unstale":"2024-05-14T11:24:29.494815+0000","last_undegraded":"2024-05-14T11:24:29.494815+0000","last_fullsized":"2024-05-14T11:24:29.494815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:06:19.926453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474169+0000","last_change":"2024-05-14T11:24:28.482611+0000","last_active":"2024-05-14T11:24:29.474169+0000","last_peered":"2024-05-14T11:24:29.474169+0000","last_clean":"2024-05-14T11:24:29.474169+0000","last_became_active":"2024-05-14T11:24:28.482263+0000","last_became_peered":"2024-05-14T11:24:28.482263+0000","last_unstale":"2024-05-14T11:24:29.474169+0000","last_undegraded":"2024-05-14T11:24:29.474169+0000","last_fullsized":"2024-05-14T11:24:29.474169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:30:08.904227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.480485+0000","last_change":"2024-05-14T11:24:29.476081+0000","last_active":"2024-05-14T11:24:29.480485+0000","last_peered":"2024-05-14T11:24:29.480485+0000","last_clean":"2024-05-14T11:24:29.480485+0000","last_became_active":"2024-05-14T11:24:29.475784+0000","last_became_peered":"2024-05-14T11:24:29.475784+0000","last_unstale":"2024-05-14T11:24:29.480485+0000","last_undegraded":"2024-05-14T11:24:29.480485+0000","last_fullsized":"2024-05-14T11:24:29.480485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:00:39.418858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507221+0000","last_change":"2024-05-14T11:24:28.478145+0000","last_active":"2024-05-14T11:24:29.507221+0000","last_peered":"2024-05-14T11:24:29.507221+0000","last_clean":"2024-05-14T11:24:29.507221+0000","last_became_active":"2024-05-14T11:24:28.477830+0000","last_became_peered":"2024-05-14T11:24:28.477830+0000","last_unstale":"2024-05-14T11:24:29.507221+0000","last_undegraded":"2024-05-14T11:24:29.507221+0000","last_fullsized":"2024-05-14T11:24:29.507221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:58.648468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477798+0000","last_change":"2024-05-14T11:24:29.473825+0000","last_active":"2024-05-14T11:24:29.477798+0000","last_peered":"2024-05-14T11:24:29.477798+0000","last_clean":"2024-05-14T11:24:29.477798+0000","last_became_active":"2024-05-14T11:24:29.473487+0000","last_became_peered":"2024-05-14T11:24:29.473487+0000","last_unstale":"2024-05-14T11:24:29.477798+0000","last_undegraded":"2024-05-14T11:24:29.477798+0000","last_fullsized":"2024-05-14T11:24:29.477798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:06:31.692335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470698+0000","last_change":"2024-05-14T11:24:28.483422+0000","last_active":"2024-05-14T11:24:29.470698+0000","last_peered":"2024-05-14T11:24:29.470698+0000","last_clean":"2024-05-14T11:24:29.470698+0000","last_became_active":"2024-05-14T11:24:28.482778+0000","last_became_peered":"2024-05-14T11:24:28.482778+0000","last_unstale":"2024-05-14T11:24:29.470698+0000","last_undegraded":"2024-05-14T11:24:29.470698+0000","last_fullsized":"2024-05-14T11:24:29.470698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:03:35.833592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.478318+0000","last_change":"2024-05-14T11:24:29.475928+0000","last_active":"2024-05-14T11:24:29.478318+0000","last_peered":"2024-05-14T11:24:29.478318+0000","last_clean":"2024-05-14T11:24:29.478318+0000","last_became_active":"2024-05-14T11:24:29.475602+0000","last_became_peered":"2024-05-14T11:24:29.475602+0000","last_unstale":"2024-05-14T11:24:29.478318+0000","last_undegraded":"2024-05-14T11:24:29.478318+0000","last_fullsized":"2024-05-14T11:24:29.478318+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:05.418563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471810+0000","last_change":"2024-05-14T11:24:28.484667+0000","last_active":"2024-05-14T11:24:29.471810+0000","last_peered":"2024-05-14T11:24:29.471810+0000","last_clean":"2024-05-14T11:24:29.471810+0000","last_became_active":"2024-05-14T11:24:28.484352+0000","last_became_peered":"2024-05-14T11:24:28.484352+0000","last_unstale":"2024-05-14T11:24:29.471810+0000","last_undegraded":"2024-05-14T11:24:29.471810+0000","last_fullsized":"2024-05-14T11:24:29.471810+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:48:47.820347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499524+0000","last_change":"2024-05-14T11:24:29.498563+0000","last_active":"2024-05-14T11:24:29.499524+0000","last_peered":"2024-05-14T11:24:29.499524+0000","last_clean":"2024-05-14T11:24:29.499524+0000","last_became_active":"2024-05-14T11:24:29.498197+0000","last_became_peered":"2024-05-14T11:24:29.498197+0000","last_unstale":"2024-05-14T11:24:29.499524+0000","last_undegraded":"2024-05-14T11:24:29.499524+0000","last_fullsized":"2024-05-14T11:24:29.499524+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:33:24.258572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.469848+0000","last_change":"2024-05-14T11:24:28.486653+0000","last_active":"2024-05-14T11:24:29.469848+0000","last_peered":"2024-05-14T11:24:29.469848+0000","last_clean":"2024-05-14T11:24:29.469848+0000","last_became_active":"2024-05-14T11:24:28.486363+0000","last_became_peered":"2024-05-14T11:24:28.486363+0000","last_unstale":"2024-05-14T11:24:29.469848+0000","last_undegraded":"2024-05-14T11:24:29.469848+0000","last_fullsized":"2024-05-14T11:24:29.469848+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:43:15.471377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.492199+0000","last_change":"2024-05-14T11:24:29.490253+0000","last_active":"2024-05-14T11:24:29.492199+0000","last_peered":"2024-05-14T11:24:29.492199+0000","last_clean":"2024-05-14T11:24:29.492199+0000","last_became_active":"2024-05-14T11:24:29.489881+0000","last_became_peered":"2024-05-14T11:24:29.489881+0000","last_unstale":"2024-05-14T11:24:29.492199+0000","last_undegraded":"2024-05-14T11:24:29.492199+0000","last_fullsized":"2024-05-14T11:24:29.492199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:36.065423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.470320+0000","last_change":"2024-05-14T11:24:28.486137+0000","last_active":"2024-05-14T11:24:29.470320+0000","last_peered":"2024-05-14T11:24:29.470320+0000","last_clean":"2024-05-14T11:24:29.470320+0000","last_became_active":"2024-05-14T11:24:28.485810+0000","last_became_peered":"2024-05-14T11:24:28.485810+0000","last_unstale":"2024-05-14T11:24:29.470320+0000","last_undegraded":"2024-05-14T11:24:29.470320+0000","last_fullsized":"2024-05-14T11:24:29.470320+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:51:02.760528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.499504+0000","last_change":"2024-05-14T11:24:29.495003+0000","last_active":"2024-05-14T11:24:29.499504+0000","last_peered":"2024-05-14T11:24:29.499504+0000","last_clean":"2024-05-14T11:24:29.499504+0000","last_became_active":"2024-05-14T11:24:29.493494+0000","last_became_peered":"2024-05-14T11:24:29.493494+0000","last_unstale":"2024-05-14T11:24:29.499504+0000","last_undegraded":"2024-05-14T11:24:29.499504+0000","last_fullsized":"2024-05-14T11:24:29.499504+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:02:26.179275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471877+0000","last_change":"2024-05-14T11:24:28.477847+0000","last_active":"2024-05-14T11:24:29.471877+0000","last_peered":"2024-05-14T11:24:29.471877+0000","last_clean":"2024-05-14T11:24:29.471877+0000","last_became_active":"2024-05-14T11:24:28.477392+0000","last_became_peered":"2024-05-14T11:24:28.477392+0000","last_unstale":"2024-05-14T11:24:29.471877+0000","last_undegraded":"2024-05-14T11:24:29.471877+0000","last_fullsized":"2024-05-14T11:24:29.471877+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:17.986233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495884+0000","last_change":"2024-05-14T11:24:29.494436+0000","last_active":"2024-05-14T11:24:29.495884+0000","last_peered":"2024-05-14T11:24:29.495884+0000","last_clean":"2024-05-14T11:24:29.495884+0000","last_became_active":"2024-05-14T11:24:29.493953+0000","last_became_peered":"2024-05-14T11:24:29.493953+0000","last_unstale":"2024-05-14T11:24:29.495884+0000","last_undegraded":"2024-05-14T11:24:29.495884+0000","last_fullsized":"2024-05-14T11:24:29.495884+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:14:41.122327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.471699+0000","last_change":"2024-05-14T11:24:28.484318+0000","last_active":"2024-05-14T11:24:29.471699+0000","last_peered":"2024-05-14T11:24:29.471699+0000","last_clean":"2024-05-14T11:24:29.471699+0000","last_became_active":"2024-05-14T11:24:28.484005+0000","last_became_peered":"2024-05-14T11:24:28.484005+0000","last_unstale":"2024-05-14T11:24:29.471699+0000","last_undegraded":"2024-05-14T11:24:29.471699+0000","last_fullsized":"2024-05-14T11:24:29.471699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:05:51.761563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.495345+0000","last_change":"2024-05-14T11:24:29.490790+0000","last_active":"2024-05-14T11:24:29.495345+0000","last_peered":"2024-05-14T11:24:29.495345+0000","last_clean":"2024-05-14T11:24:29.495345+0000","last_became_active":"2024-05-14T11:24:29.490490+0000","last_became_peered":"2024-05-14T11:24:29.490490+0000","last_unstale":"2024-05-14T11:24:29.495345+0000","last_undegraded":"2024-05-14T11:24:29.495345+0000","last_fullsized":"2024-05-14T11:24:29.495345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:11:30.360961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.474277+0000","last_change":"2024-05-14T11:24:28.479358+0000","last_active":"2024-05-14T11:24:29.474277+0000","last_peered":"2024-05-14T11:24:29.474277+0000","last_clean":"2024-05-14T11:24:29.474277+0000","last_became_active":"2024-05-14T11:24:28.479052+0000","last_became_peered":"2024-05-14T11:24:28.479052+0000","last_unstale":"2024-05-14T11:24:29.474277+0000","last_undegraded":"2024-05-14T11:24:29.474277+0000","last_fullsized":"2024-05-14T11:24:29.474277+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:04:36.505749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.496424+0000","last_change":"2024-05-14T11:24:29.494580+0000","last_active":"2024-05-14T11:24:29.496424+0000","last_peered":"2024-05-14T11:24:29.496424+0000","last_clean":"2024-05-14T11:24:29.496424+0000","last_became_active":"2024-05-14T11:24:29.494207+0000","last_became_peered":"2024-05-14T11:24:29.494207+0000","last_unstale":"2024-05-14T11:24:29.496424+0000","last_undegraded":"2024-05-14T11:24:29.496424+0000","last_fullsized":"2024-05-14T11:24:29.496424+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:28.447761+0000","last_clean_scrub_stamp":"2024-05-14T11:24:28.447761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:08:10.006732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.507618+0000","last_change":"2024-05-14T11:24:28.485089+0000","last_active":"2024-05-14T11:24:29.507618+0000","last_peered":"2024-05-14T11:24:29.507618+0000","last_clean":"2024-05-14T11:24:29.507618+0000","last_became_active":"2024-05-14T11:24:28.484228+0000","last_became_peered":"2024-05-14T11:24:28.484228+0000","last_unstale":"2024-05-14T11:24:29.507618+0000","last_undegraded":"2024-05-14T11:24:29.507618+0000","last_fullsized":"2024-05-14T11:24:29.507618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:24:27.433121+0000","last_clean_scrub_stamp":"2024-05-14T11:24:27.433121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:53.962180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":253,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-14T11:24:29.477789+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:24:29.477789+0000","last_peered":"2024-05-14T11:24:29.477789+0000","last_clean":"2024-05-14T11:24:29.477789+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:24:29.477789+0000","last_undegraded":"2024-05-14T11:24:29.477789+0000","last_fullsized":"2024-05-14T11:24:29.477789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1818624,"data_stored":1152659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1236992,"data_stored":564347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1216512,"data_stored":562131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":562223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":562713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":562165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1220608,"data_stored":562165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1826816,"data_stored":1154201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:24:33.996 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:24:33.996 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:24:34.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 Processor -- start 2024-05-14T11:24:34.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 -- start start 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d806cc10 con 0x7f17d815c1d0 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d806cef0 con 0x7f17d80ce450 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17dd400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f17d806d790 con 0x7f17d80d0c50 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d6800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f17d815c1d0 0x7f17d80cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52368/0 (socket says 172.21.15.116:52368) 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d6800640 1 -- 172.21.15.116:0/3308495390 learned_addr learned my addr 172.21.15.116:0/3308495390 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:34.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d7200640 1 --2- 172.21.15.116:0/3308495390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1808785616 0 0) 0x7f17d806cc10 con 0x7f17d815c1d0 2024-05-14T11:24:34.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17bc003530 con 0x7f17d815c1d0 2024-05-14T11:24:34.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928997162 0 0) 0x7f17d806cef0 con 0x7f17d80ce450 2024-05-14T11:24:34.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17d806cc10 con 0x7f17d80ce450 2024-05-14T11:24:34.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d7200640 1 -- 172.21.15.116:0/3308495390 >> v1:172.21.15.149:6789/0 conn(0x7f17d80ce450 legacy=0x7f17d80cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d7200640 1 -- 172.21.15.116:0/3308495390 >> v1:172.21.15.116:6789/0 conn(0x7f17d815c1d0 legacy=0x7f17d80cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 172493076 0 0) 0x7f17bc003530 con 0x7f17d815c1d0 2024-05-14T11:24:34.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.191+0000 7f17d7200640 1 -- 172.21.15.116:0/3308495390 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d806d1d0 con 0x7f17d80d0c50 2024-05-14T11:24:34.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17d7200640 1 --2- 172.21.15.116:0/3308495390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d80cc470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17d000aed0 tx=0x7f17d001d940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f404c06b77da6e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8001700 con 0x7f17d80d0c50 2024-05-14T11:24:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d000a0d0 con 0x7f17d80d0c50 2024-05-14T11:24:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17d5400640 1 -- 172.21.15.116:0/3308495390 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00a6890 con 0x7f17d80d0c50 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 -- 172.21.15.116:0/3308495390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 msgr2=0x7f17d80cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 --2- 172.21.15.116:0/3308495390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d80cc470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17d000aed0 tx=0x7f17d001d940 comp rx=0 tx=0).stop 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 -- 172.21.15.116:0/3308495390 shutdown_connections 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 --2- 172.21.15.116:0/3308495390 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d80cc470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 -- 172.21.15.116:0/3308495390 >> 172.21.15.116:0/3308495390 conn(0x7f17d80b4720 msgr2=0x7f17d80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 -- 172.21.15.116:0/3308495390 shutdown_connections 2024-05-14T11:24:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.195+0000 7f17dd400640 1 -- 172.21.15.116:0/3308495390 wait complete. 2024-05-14T11:24:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 Processor -- start 2024-05-14T11:24:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 -- start start 2024-05-14T11:24:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d81f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d815c1d0 0x7f17d8166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80d18b0 con 0x7f17d815d060 2024-05-14T11:24:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80d1730 con 0x7f17d815c1d0 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17dd400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f17d80d1a30 con 0x7f17d80d0c50 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54004/0 (socket says 172.21.15.116:54004) 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 -- 172.21.15.116:0/1209586496 learned_addr learned my addr 172.21.15.116:0/1209586496 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d6800640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d81f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d5e00640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d815c1d0 0x7f17d8166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 -- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 msgr2=0x7f17d81f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d81f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 -- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d815c1d0 msgr2=0x7f17d8166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d815c1d0 0x7f17d8166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.199+0000 7f17d7200640 1 -- 172.21.15.116:0/1209586496 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d81fca40 con 0x7f17d815d060 2024-05-14T11:24:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17d7200640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f17d00b0040 tx=0x7f17d80cd1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00bc020 con 0x7f17d815d060 2024-05-14T11:24:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0008850 con 0x7f17d815d060 2024-05-14T11:24:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d81f9d30 con 0x7f17d815d060 2024-05-14T11:24:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000a560 con 0x7f17d815d060 2024-05-14T11:24:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81fa2f0 con 0x7f17d815d060 2024-05-14T11:24:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d806a9c0 con 0x7f17d815d060 2024-05-14T11:24:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.203+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17d00bf020 con 0x7f17d815d060 2024-05-14T11:24:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.207+0000 7f17cea00640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 0x7f17b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.207+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f17d0129d90 con 0x7f17d815d060 2024-05-14T11:24:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.207+0000 7f17d6800640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 0x7f17b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.207+0000 7f17d6800640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 0x7f17b8071910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17c8001390 tx=0x7f17c8001730 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.211+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f17d00c6030 con 0x7f17d815d060 2024-05-14T11:24:34.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.423+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f17d8069de0 con 0x7f17d815d060 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17cea00640 1 -- 172.21.15.116:0/1209586496 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f17d00175c0 con 0x7f17d815d060 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:24:34.427 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:24:34.428 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 msgr2=0x7f17b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 0x7f17b8071910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17c8001390 tx=0x7f17c8001730 comp rx=0 tx=0).stop 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 msgr2=0x7f17d81fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f17d00b0040 tx=0x7f17d80cd1c0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 shutdown_connections 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f17b806f400 0x7f17b8071910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17d815c1d0 0x7f17d8166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17d815d060 0x7f17d81fd220 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 --2- 172.21.15.116:0/1209586496 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f17d80d0c50 0x7f17d81f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 >> 172.21.15.116:0/1209586496 conn(0x7f17d80b4720 msgr2=0x7f17d806fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:34.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 shutdown_connections 2024-05-14T11:24:34.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.427+0000 7f17dd400640 1 -- 172.21.15.116:0/1209586496 wait complete. 2024-05-14T11:24:34.448 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:34.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 Processor -- start 2024-05-14T11:24:34.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 -- start start 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa7880d1510 0x7fa7880d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880d1e60 con 0x7fa788160080 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa7880b2f40 con 0x7fa7880d1510 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa78e800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa7880b3220 con 0x7fa7880b3fe0 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa787e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa77ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa7880d1510 0x7fa7880d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa787e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:36476/0 (socket says 172.21.15.116:36476) 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.595+0000 7fa787e00640 1 -- 172.21.15.116:0/3452298296 learned_addr learned my addr 172.21.15.116:0/3452298296 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa787e00640 1 -- 172.21.15.116:0/3452298296 >> v2:172.21.15.149:3300/0 conn(0x7fa7880d1510 msgr2=0x7fa7880d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa787e00640 1 --2- 172.21.15.116:0/3452298296 >> v2:172.21.15.149:3300/0 conn(0x7fa7880d1510 0x7fa7880d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa787e00640 1 -- 172.21.15.116:0/3452298296 >> v1:172.21.15.149:6789/0 conn(0x7fa788160080 legacy=0x7fa7880cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa787e00640 1 -- 172.21.15.116:0/3452298296 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880b3500 con 0x7fa7880b3fe0 2024-05-14T11:24:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa787e00640 1 --2- 172.21.15.116:0/3452298296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa780009e50 tx=0x7fa78001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b707988dcf903 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa786a00640 1 -- 172.21.15.116:0/3452298296 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7800a57a0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa786a00640 1 -- 172.21.15.116:0/3452298296 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7800a5940 con 0x7fa7880b3fe0 2024-05-14T11:24:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa786a00640 1 -- 172.21.15.116:0/3452298296 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7800a5d10 con 0x7fa7880b3fe0 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa78e800640 1 -- 172.21.15.116:0/3452298296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 msgr2=0x7fa7880b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.599+0000 7fa78e800640 1 --2- 172.21.15.116:0/3452298296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa780009e50 tx=0x7fa78001c8e0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- 172.21.15.116:0/3452298296 shutdown_connections 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 --2- 172.21.15.116:0/3452298296 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880b3fe0 0x7fa7880b43f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 --2- 172.21.15.116:0/3452298296 >> v2:172.21.15.149:3300/0 conn(0x7fa7880d1510 0x7fa7880d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- 172.21.15.116:0/3452298296 >> 172.21.15.116:0/3452298296 conn(0x7fa7880d0770 msgr2=0x7fa7880d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- 172.21.15.116:0/3452298296 shutdown_connections 2024-05-14T11:24:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- 172.21.15.116:0/3452298296 wait complete. 2024-05-14T11:24:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 Processor -- start 2024-05-14T11:24:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- start start 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880d1510 0x7fa78815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 0x7fa78815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa78806fdd0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880ce690 con 0x7fa788160080 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.603+0000 7fa78e800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa78806ff50 con 0x7fa7880d1510 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54008/0 (socket says 172.21.15.116:54008) 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 -- 172.21.15.116:0/4237429632 learned_addr learned my addr 172.21.15.116:0/4237429632 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787e00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 0x7fa78815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa77ea00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880d1510 0x7fa78815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 -- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880d1510 msgr2=0x7fa78815bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880d1510 0x7fa78815bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 -- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 msgr2=0x7fa78815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 0x7fa78815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 -- 172.21.15.116:0/4237429632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7881508a0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787e00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 0x7fa78815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa787400640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c420 tx=0x7fa77000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770017070 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7700052d0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa788150b80 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770003840 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa788152770 con 0x7fa7880b3fe0 2024-05-14T11:24:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78806a9c0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.607+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa77001a070 con 0x7fa7880b3fe0 2024-05-14T11:24:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.611+0000 7fa77fe00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.611+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fa77008e670 con 0x7fa7880b3fe0 2024-05-14T11:24:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.611+0000 7fa77ea00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.611+0000 7fa77ea00640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7880cd770 tx=0x7fa778008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:34.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.615+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa7880d1e60 con 0x7fa7880b3fe0 2024-05-14T11:24:34.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.803+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa78815d0b0 con 0x7fa7880b3fe0 2024-05-14T11:24:34.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa77fe00640 1 -- 172.21.15.116:0/4237429632 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5183 (secure 0 0 0) 0x7fa770005480 con 0x7fa7880b3fe0 2024-05-14T11:24:34.807 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:34.807 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:31.883949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7638":{"gid":7638,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3421632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3421632},{"type":"v1","addr":"172.21.15.116:6835","nonce":3421632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7638,"qdb_cluster":[7638]},"id":7}]} 2024-05-14T11:24:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 83 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 msgr2=0x7fa764071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7880cd770 tx=0x7fa778008040 comp rx=0 tx=0).stop 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 msgr2=0x7fa78815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c420 tx=0x7fa77000c8e0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 shutdown_connections 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa788160080 0x7fa78815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa7880d1510 0x7fa78815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 --2- 172.21.15.116:0/4237429632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7880b3fe0 0x7fa78815b7e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 >> 172.21.15.116:0/4237429632 conn(0x7fa7880d0770 msgr2=0x7fa788156520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 shutdown_connections 2024-05-14T11:24:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:34.807+0000 7fa78e800640 1 -- 172.21.15.116:0/4237429632 wait complete. 2024-05-14T11:24:35.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:24:28.859727+0000', 'modified': '2024-05-14T11:24:31.883949+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7638}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7638': {'gid': 7638, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3421632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3421632}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3421632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7638, 'qdb_cluster': [7638]} 2024-05-14T11:24:35.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:24:35.233 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-14T11:24:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 Processor -- start 2024-05-14T11:24:35.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 -- start start 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60d415db30 0x7f60d40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f60d4161600 0x7f60d4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f60d40ae100 con 0x7f60d415db30 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f60d40ae870 con 0x7f60d4161600 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60da000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60d406ff20 con 0x7f60d40d1670 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60d415db30 0x7f60d40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60d415db30 0x7f60d40aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54024/0 (socket says 172.21.15.116:54024) 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 -- 172.21.15.116:0/2452514416 learned_addr learned my addr 172.21.15.116:0/2452514416 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60caa00640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2452514416 >> v2:172.21.15.149:3300/0 conn(0x7f60d4161600 0x7f60d4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 -- 172.21.15.116:0/2452514416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 msgr2=0x7f60d40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 -- 172.21.15.116:0/2452514416 >> v2:172.21.15.149:3300/0 conn(0x7f60d4161600 msgr2=0x7f60d4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 --2- 172.21.15.116:0/2452514416 >> v2:172.21.15.149:3300/0 conn(0x7f60d4161600 0x7f60d4161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.383+0000 7f60d3400640 1 -- 172.21.15.116:0/2452514416 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d40d1fc0 con 0x7f60d415db30 2024-05-14T11:24:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60d3400640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d415db30 0x7f60d40aeaa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f60d40b41d0 tx=0x7f60c401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7754bd3adb55a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60d2a00640 1 -- 172.21.15.116:0/2452514416 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40a56d0 con 0x7f60d415db30 2024-05-14T11:24:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60d2a00640 1 -- 172.21.15.116:0/2452514416 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40a5870 con 0x7f60d415db30 2024-05-14T11:24:35.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60d2a00640 1 -- 172.21.15.116:0/2452514416 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40a5c40 con 0x7f60d415db30 2024-05-14T11:24:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 -- 172.21.15.116:0/2452514416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d415db30 msgr2=0x7f60d40aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d415db30 0x7f60d40aeaa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f60d40b41d0 tx=0x7f60c401c960 comp rx=0 tx=0).stop 2024-05-14T11:24:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 -- 172.21.15.116:0/2452514416 shutdown_connections 2024-05-14T11:24:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 --2- 172.21.15.116:0/2452514416 >> v2:172.21.15.149:3300/0 conn(0x7f60d4161600 0x7f60d4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 --2- 172.21.15.116:0/2452514416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d415db30 0x7f60d40aeaa0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 -- 172.21.15.116:0/2452514416 >> 172.21.15.116:0/2452514416 conn(0x7f60d40b4620 msgr2=0x7f60d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 -- 172.21.15.116:0/2452514416 shutdown_connections 2024-05-14T11:24:35.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.387+0000 7f60da000640 1 -- 172.21.15.116:0/2452514416 wait complete. 2024-05-14T11:24:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 Processor -- start 2024-05-14T11:24:35.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- start start 2024-05-14T11:24:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d415db30 0x7f60d414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4162360 con 0x7f60d4161600 2024-05-14T11:24:35.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60d40aefe0 con 0x7f60d415db30 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f60d415df40 con 0x7f60d40d1670 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54030/0 (socket says 172.21.15.116:54030) 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 -- 172.21.15.116:0/2481554636 learned_addr learned my addr 172.21.15.116:0/2481554636 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60caa00640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d415db30 0x7f60d414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3400640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 -- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 msgr2=0x7f60d4159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d4159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 -- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d415db30 msgr2=0x7f60d414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d415db30 0x7f60d414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 -- 172.21.15.116:0/2481554636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d4158190 con 0x7f60d4161600 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00d500 tx=0x7f60cc00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60cbe00640 1 -- 172.21.15.116:0/2481554636 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0125f0 con 0x7f60d4161600 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60cbe00640 1 -- 172.21.15.116:0/2481554636 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc019070 con 0x7f60d4161600 2024-05-14T11:24:35.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60cbe00640 1 -- 172.21.15.116:0/2481554636 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0129d0 con 0x7f60d4161600 2024-05-14T11:24:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d4158470 con 0x7f60d4161600 2024-05-14T11:24:35.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.391+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d4158a30 con 0x7f60d4161600 2024-05-14T11:24:35.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60cbe00640 1 -- 172.21.15.116:0/2481554636 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60cc012b70 con 0x7f60d4161600 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60cbe00640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 0x7f60b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60cbe00640 1 -- 172.21.15.116:0/2481554636 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f60cc08f360 con 0x7f60d4161600 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60d3400640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 0x7f60b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60e1687000 1 Processor -- start 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.395+0000 7f60e1687000 1 -- start start 2024-05-14T11:24:35.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 0x7f60b0071940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ae000 tx=0x7f60c40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561424b276b0 0x561424c8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561424b2c620 con 0x5614247557c0 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561424a47740 con 0x561424b3df80 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561424a95090 con 0x561424b276b0 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54034/0 (socket says 172.21.15.116:54034) 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 -- 172.21.15.116:0/125451955 learned_addr learned my addr 172.21.15.116:0/125451955 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 -- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561424b276b0 msgr2=0x561424c8fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561424b276b0 0x561424c8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 -- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 msgr2=0x561424c8f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 -- 172.21.15.116:0/125451955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561424c290a0 con 0x5614247557c0 2024-05-14T11:24:35.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60d3400640 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f60c40af040 tx=0x7f60c4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2feef7a523fffc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60c3400640 1 -- 172.21.15.116:0/125451955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40c6070 con 0x5614247557c0 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60c3400640 1 -- 172.21.15.116:0/125451955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40a5e20 con 0x5614247557c0 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60c3400640 1 -- 172.21.15.116:0/125451955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40c2620 con 0x5614247557c0 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- 172.21.15.116:0/125451955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 msgr2=0x561424c8f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f60c40af040 tx=0x7f60c4004ac0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- 172.21.15.116:0/125451955 shutdown_connections 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c8f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561424b276b0 0x561424c8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 --2- 172.21.15.116:0/125451955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614247557c0 0x561424c8f320 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- 172.21.15.116:0/125451955 >> 172.21.15.116:0/125451955 conn(0x561424c14340 msgr2=0x561424c43d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- 172.21.15.116:0/125451955 shutdown_connections 2024-05-14T11:24:35.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.399+0000 7f60e1687000 1 -- 172.21.15.116:0/125451955 wait complete. 2024-05-14T11:24:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- start start 2024-05-14T11:24:35.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 Processor -- start 2024-05-14T11:24:35.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c4a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c74770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561424a91050 con 0x561424b276b0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561424a4dec0 con 0x561424b3df80 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561424a34c90 con 0x5614247557c0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 10 client.0 _ll_register_callbacks cb 0x561424c30e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54042/0 (socket says 172.21.15.116:54042) 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 -- 172.21.15.116:0/2503178665 learned_addr learned my addr 172.21.15.116:0/2503178665 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c74770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60d3400640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c4a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 -- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 msgr2=0x561424c4a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c4a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 -- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 msgr2=0x561424c74770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c74770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 -- 172.21.15.116:0/2503178665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561424b3f6f0 con 0x561424b276b0 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60d3e00640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c74770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60caa00640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f60bc006d50 tx=0x7f60bc0108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79479b86b3751eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc018070 con 0x561424b276b0 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f60bc004020 con 0x561424b276b0 2024-05-14T11:24:35.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c40a5e20 con 0x561424b276b0 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f60bc022050 con 0x561424b276b0 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 10 client.7779 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x561424b3b6a0 con 0x561424b276b0 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40a5e20 con 0x561424b276b0 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f60bc025020 con 0x561424b276b0 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 10 client.7779 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:35.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 10 client.7779 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 20 client.7779 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.403+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56142473a7d0 0x561424ca08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 4 client.7779 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561424a2dbe0 con 0x56142473a7d0 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60d3400640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56142473a7d0 0x561424ca08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60d3400640 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56142473a7d0 0x561424ca08e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a584b38cc065357 server_cookie=82bf4037a9274833 in_seq=0 out_seq=0 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60b7e00640 10 client.7779 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60b7e00640 1 -- 172.21.15.116:0/2503178665 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561424a2dbe0 con 0x56142473a7d0 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60b7e00640 10 client.7779 handle_command_reply: tid=0 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 client.7779 shutdown 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 msgr2=0x561424c4a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f60bc006d50 tx=0x7f60bc0108a0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 shutdown_connections 2024-05-14T11:24:35.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56142473a7d0 0x561424ca08e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b3df80 0x561424c74770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424c4a890 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60b7e00640 0 client.7779 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 --2- 172.21.15.116:0/2503178665 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c4a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 >> 172.21.15.116:0/2503178665 conn(0x561424c14340 msgr2=0x561424c44490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 shutdown_connections 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 1 -- 172.21.15.116:0/2503178665 wait complete. 2024-05-14T11:24:35.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.407+0000 7f60e1687000 20 client.7779 trim_cache size 0 max 16384 2024-05-14T11:24:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 Processor -- start 2024-05-14T11:24:35.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 -- start start 2024-05-14T11:24:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c74690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b276b0 0x561424c442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4160eb0 con 0x561424c14340 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4202d40 con 0x561424b276b0 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561424a2a7d0 con 0x5614247557c0 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54046/0 (socket says 172.21.15.116:54046) 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 -- 172.21.15.116:0/264151062 learned_addr learned my addr 172.21.15.116:0/264151062 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60caa00640 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c74690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3e00640 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b276b0 0x561424c442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 -- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 msgr2=0x561424c74690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c74690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 -- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b276b0 msgr2=0x561424c442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b276b0 0x561424c442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 -- 172.21.15.116:0/264151062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561424a6eaf0 con 0x561424c14340 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.427+0000 7f60d3400640 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f60c40093e0 tx=0x7f60c40b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842e0ac1c4d3fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60c2a00640 1 -- 172.21.15.116:0/264151062 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40dc070 con 0x561424c14340 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60c2a00640 1 -- 172.21.15.116:0/264151062 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40c3ab0 con 0x561424c14340 2024-05-14T11:24:35.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60c2a00640 1 -- 172.21.15.116:0/264151062 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40ba040 con 0x561424c14340 2024-05-14T11:24:35.432 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f07b0c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- 172.21.15.116:0/264151062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 msgr2=0x561424c74150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f60c40093e0 tx=0x7f60c40b7000 comp rx=0 tx=0).stop 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- 172.21.15.116:0/264151062 shutdown_connections 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424b276b0 0x561424c442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424c74690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- 172.21.15.116:0/264151062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424c14340 0x561424c74150 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- 172.21.15.116:0/264151062 >> 172.21.15.116:0/264151062 conn(0x561424c72b40 msgr2=0x561424c9bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- 172.21.15.116:0/264151062 shutdown_connections 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- 172.21.15.116:0/264151062 wait complete. 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- start start 2024-05-14T11:24:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 Processor -- start 2024-05-14T11:24:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424d02170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424c14340 0x561424cbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561424a92020 con 0x561424b276b0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561424a2f220 con 0x561424c14340 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561424a387d0 con 0x5614247557c0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.431+0000 7f60e1687000 10 client.0 _ll_register_callbacks cb 0x7f60bc007470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54052/0 (socket says 172.21.15.116:54052) 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 -- 172.21.15.116:0/1143843404 learned_addr learned my addr 172.21.15.116:0/1143843404 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60d3400640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424d02170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60d3e00640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424c14340 0x561424cbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 -- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 msgr2=0x561424d02170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424d02170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 -- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424c14340 msgr2=0x561424cbd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424c14340 0x561424cbd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 -- 172.21.15.116:0/1143843404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561424a6de40 con 0x561424b276b0 2024-05-14T11:24:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60caa00640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f60bc010e20 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3679fcb62bd1f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc018070 con 0x561424b276b0 2024-05-14T11:24:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f60bc014c80 con 0x561424b276b0 2024-05-14T11:24:35.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c40c3ab0 con 0x561424b276b0 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f60bc030050 con 0x561424b276b0 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 10 client.7785 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x561424b154e0 con 0x561424b276b0 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40c3ab0 con 0x561424b276b0 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f60bc033020 con 0x561424b276b0 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 10 client.7785 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 10 client.7785 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 20 client.7785 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x561424753e80 0x561424cbe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 4 client.7785 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "status"}) -- 0x561424aa2ca0 con 0x561424753e80 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60d3400640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x561424753e80 0x561424cbe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 10 client.7785 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60d3400640 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x561424753e80 0x561424cbe430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6c7e28d1ec852b server_cookie=803d24285b2f3aa5 in_seq=0 out_seq=0 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 1 -- 172.21.15.116:0/1143843404 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561424aa2ca0 con 0x561424753e80 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 10 client.7785 handle_command_reply: tid=0 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 client.7785 shutdown 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 msgr2=0x561424d026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f60bc010e20 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 shutdown_connections 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x561424753e80 0x561424cbe430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561424c14340 0x561424cbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561424b276b0 0x561424d026b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 --2- 172.21.15.116:0/1143843404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5614247557c0 0x561424d02170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60b7e00640 0 client.7785 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 >> 172.21.15.116:0/1143843404 conn(0x561424c72b40 msgr2=0x561424c74150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 shutdown_connections 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 1 -- 172.21.15.116:0/1143843404 wait complete. 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.435+0000 7f60e1687000 20 client.7785 trim_cache size 0 max 16384 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7638, 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:24:35.441 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 5.9643668569999999, 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 83, 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 99, 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 99, 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 10.808111200999999 2024-05-14T11:24:35.442 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 msgr2=0x7f60b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 0x7f60b0071940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ae000 tx=0x7f60c40a7070 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 msgr2=0x7f60d4156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00d500 tx=0x7f60cc00d9c0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 shutdown_connections 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f60b006f430 0x7f60b0071940 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d415db30 0x7f60d414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d4161600 0x7f60d4156a50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 --2- 172.21.15.116:0/2481554636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f60d40d1670 0x7f60d4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 >> 172.21.15.116:0/2481554636 conn(0x7f60d40b4620 msgr2=0x7f60d415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 shutdown_connections 2024-05-14T11:24:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.439+0000 7f60da000640 1 -- 172.21.15.116:0/2481554636 wait complete. 2024-05-14T11:24:35.459 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:24:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 Processor -- start 2024-05-14T11:24:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 -- start start 2024-05-14T11:24:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7feb400d1670 0x7feb400d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4006ff20 con 0x7feb4015db30 2024-05-14T11:24:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb400d1fc0 con 0x7feb4006c700 2024-05-14T11:24:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb47c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7feb400ae100 con 0x7feb400d1670 2024-05-14T11:24:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb45200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feb4015db30 0x7feb400aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52370/0 (socket says 172.21.15.116:52370) 2024-05-14T11:24:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb45200640 1 -- 172.21.15.116:0/69642869 learned_addr learned my addr 172.21.15.116:0/69642869 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3fe00640 1 --2- 172.21.15.116:0/69642869 >> v2:172.21.15.149:3300/0 conn(0x7feb400d1670 0x7feb400d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3128409844 0 0) 0x7feb4006ff20 con 0x7feb4015db30 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb28003530 con 0x7feb4015db30 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685421723 0 0) 0x7feb380014a0 con 0x7feb4006c700 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4006ff20 con 0x7feb4006c700 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3fe00640 1 -- 172.21.15.116:0/69642869 >> v1:172.21.15.149:6789/0 conn(0x7feb4006c700 legacy=0x7feb4006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3fe00640 1 -- 172.21.15.116:0/69642869 >> v1:172.21.15.116:6789/0 conn(0x7feb4015db30 legacy=0x7feb400aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.607+0000 7feb3fe00640 1 -- 172.21.15.116:0/69642869 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb40070470 con 0x7feb400d1670 2024-05-14T11:24:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb3fe00640 1 --2- 172.21.15.116:0/69642869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb400d1a80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb400ae280 tx=0x7feb2c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce58b43289dd523f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b0070 con 0x7feb400d1670 2024-05-14T11:24:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c0ab600 con 0x7feb400d1670 2024-05-14T11:24:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb3f400640 1 -- 172.21.15.116:0/69642869 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0ab9d0 con 0x7feb400d1670 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 -- 172.21.15.116:0/69642869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 msgr2=0x7feb400d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 --2- 172.21.15.116:0/69642869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb400d1a80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb400ae280 tx=0x7feb2c01c380 comp rx=0 tx=0).stop 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 -- 172.21.15.116:0/69642869 shutdown_connections 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 --2- 172.21.15.116:0/69642869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb400d1a80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 -- 172.21.15.116:0/69642869 >> 172.21.15.116:0/69642869 conn(0x7feb400b4620 msgr2=0x7feb400b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 -- 172.21.15.116:0/69642869 shutdown_connections 2024-05-14T11:24:35.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.611+0000 7feb47c00640 1 -- 172.21.15.116:0/69642869 wait complete. 2024-05-14T11:24:35.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 Processor -- start 2024-05-14T11:24:35.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 -- start start 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb4006c700 0x7feb4014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb4015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb4016bf90 con 0x7feb4015db30 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb4015df40 con 0x7feb400d1670 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb47c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feb4016c110 con 0x7feb4006c700 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb45c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb45c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54062/0 (socket says 172.21.15.116:54062) 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb45c00640 1 -- 172.21.15.116:0/3425054377 learned_addr learned my addr 172.21.15.116:0/3425054377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb45200640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb4006c700 0x7feb4014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.615+0000 7feb3fe00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb4015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 -- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb4006c700 msgr2=0x7feb4014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb4006c700 0x7feb4014d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 -- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 msgr2=0x7feb4015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb4015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 -- 172.21.15.116:0/3425054377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4015eb60 con 0x7feb4015db30 2024-05-14T11:24:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb45c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feb3800c700 tx=0x7feb3800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38003b80 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38003d20 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3800aa90 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4015ee40 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4015f400 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4006a9c0 con 0x7feb4015db30 2024-05-14T11:24:35.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.619+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb3800ac30 con 0x7feb4015db30 2024-05-14T11:24:35.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.623+0000 7feb3cc00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 0x7feb20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:35.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.623+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7feb3808f290 con 0x7feb4015db30 2024-05-14T11:24:35.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.623+0000 7feb45200640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 0x7feb20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:35.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.623+0000 7feb45200640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 0x7feb20071910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c10 tx=0x7feb34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:35.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.627+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feb400d1fc0 con 0x7feb4015db30 2024-05-14T11:24:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feb400b3bf0 con 0x7feb4015db30 2024-05-14T11:24:35.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb3cc00640 1 -- 172.21.15.116:0/3425054377 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7feb38060e20 con 0x7feb4015db30 2024-05-14T11:24:35.820 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 msgr2=0x7feb20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 0x7feb20071910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7feb34000c10 tx=0x7feb34009040 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 msgr2=0x7feb4015e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feb3800c700 tx=0x7feb3800cbc0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 shutdown_connections 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7feb2006f400 0x7feb20071910 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb400d1670 0x7feb4015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb4015db30 0x7feb4015e0c0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 --2- 172.21.15.116:0/3425054377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feb4006c700 0x7feb4014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 >> 172.21.15.116:0/3425054377 conn(0x7feb400b4620 msgr2=0x7feb400b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.819+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 shutdown_connections 2024-05-14T11:24:35.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:35.823+0000 7feb47c00640 1 -- 172.21.15.116:0/3425054377 wait complete. 2024-05-14T11:24:35.842 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:24:35.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:24:35.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:24:35.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:24:35.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:35.843 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:24:35.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:24:35.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:24:35.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:35.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:35.843 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:24:35.852 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41927sec preferred_lft 41927sec 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:24:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:35.855 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:24:35.855 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:35.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:35.857 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:24:35.920 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:24:35.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:35.922 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:35.982 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:24:35.983 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:24:36.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:24:36.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:36.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.115 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:24:36.124 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:24:36.124 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:24:36.124 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:36.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.124 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:24:36.186 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:24:36.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:24:36.186 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:36.234 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:36.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:24:36.235 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:36.286 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:24:36.350 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:24:36.404 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:24:36.405 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:36.406 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:36.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.407 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:24:36.458 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:24:36.458 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:24:36.505 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:24:36.526 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:24:36.584 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:24:36.585 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:36.586 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:36.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.587 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:24:36.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:24:36.607+0000 7fb3b042b580 -1 init, newargv = 0x7fb39401b4e0 newargc=15 2024-05-14T11:24:36.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[46925]: starting ceph client 2024-05-14T11:24:36.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[46925]: starting fuse 2024-05-14T11:24:36.619 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:24:36.619 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:24:36.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:24:36.620 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:24:36.621 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:36.746 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.46925.asok 2024-05-14T11:24:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:36.757 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46925.asok 2024-05-14T11:24:36.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.758 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46925.asok status 2024-05-14T11:24:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "46925", 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:24:36.934 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7800, 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 7800 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2950235296 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2950235296 2024-05-14T11:24:36.935 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.7800 192.168.0.1:0/2950235296", 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/2950235296", 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 83, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 99, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:24:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:36.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.950 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:24:36.950 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:24:36.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:36.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:36.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:37.022 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:37.022 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:37.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:37.074 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-14T11:24:37.216 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records in 2024-05-14T11:24:37.216 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records out 2024-05-14T11:24:37.217 INFO:teuthology.orchestra.run.smithi116.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0825942 s, 76.2 MB/s 2024-05-14T11:24:37.218 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:37.218 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:37.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:24:37.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 Processor -- start 2024-05-14T11:24:37.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 -- start start 2024-05-14T11:24:37.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5c00c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c0164750 con 0x7fb5c00d1340 2024-05-14T11:24:37.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c0164470 con 0x7fb5c00ceb30 2024-05-14T11:24:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5c5000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00cde20 con 0x7fb5c00cf820 2024-05-14T11:24:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.415+0000 7fb5be800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb5c00d1340 0x7fb5c00d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52378/0 (socket says 172.21.15.116:52378) 2024-05-14T11:24:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5be800640 1 -- 172.21.15.116:0/2538222946 learned_addr learned my addr 172.21.15.116:0/2538222946 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5c00c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130143394 0 0) 0x7fb5c0164750 con 0x7fb5c00d1340 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5a4003530 con 0x7fb5c00d1340 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179352120 0 0) 0x7fb5c0164470 con 0x7fb5c00ceb30 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5c0164750 con 0x7fb5c00ceb30 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918463326 0 0) 0x7fb5a4003530 con 0x7fb5c00d1340 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 msgr2=0x7fb5c00c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5c00c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 >> v1:172.21.15.149:6789/0 conn(0x7fb5c00ceb30 legacy=0x7fb5c00cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00ce050 con 0x7fb5c00d1340 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fb5b8001a80 con 0x7fb5c00d1340 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 >> v1:172.21.15.116:6789/0 conn(0x7fb5c00d1340 legacy=0x7fb5c00d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5a4003c20 0x7fb5a4004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a4004da0 0x7fb5a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a400bc40 con 0x7fb5a4003c20 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a400bdf0 con 0x7fb5a4004da0 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a400bff0 con 0x7fb5c00cf820 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5a4003c20 0x7fb5a4004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5be800640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a4004da0 0x7fb5a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5b5400640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 -- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 msgr2=0x7fb5a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5c00cf820 0x7fb5a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 -- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a4004da0 msgr2=0x7fb5a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a4004da0 0x7fb5a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5bde00640 1 -- 172.21.15.116:0/2538222946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a4003530 con 0x7fb5a4003c20 2024-05-14T11:24:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.419+0000 7fb5be800640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a4004da0 0x7fb5a4005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5bde00640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5a4003c20 0x7fb5a4004070 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009ae0 tx=0x7fb5b001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43d082f14d7d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b2070 con 0x7fb5a4003c20 2024-05-14T11:24:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b00a5600 con 0x7fb5a4003c20 2024-05-14T11:24:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5bd400640 1 -- 172.21.15.116:0/2538222946 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00096b0 con 0x7fb5a4003c20 2024-05-14T11:24:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 -- 172.21.15.116:0/2538222946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5a4003c20 msgr2=0x7fb5a4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 --2- 172.21.15.116:0/2538222946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5a4003c20 0x7fb5a4004070 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009ae0 tx=0x7fb5b001ca90 comp rx=0 tx=0).stop 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5be800640 1 -- 172.21.15.116:0/2538222946 reap_dead start 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 -- 172.21.15.116:0/2538222946 shutdown_connections 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 -- 172.21.15.116:0/2538222946 >> 172.21.15.116:0/2538222946 conn(0x7fb5c00cffe0 msgr2=0x7fb5c00d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 -- 172.21.15.116:0/2538222946 shutdown_connections 2024-05-14T11:24:37.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.423+0000 7fb5c5000640 1 -- 172.21.15.116:0/2538222946 wait complete. 2024-05-14T11:24:37.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 Processor -- start 2024-05-14T11:24:37.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- start start 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5a4003c20 0x7fb5c0162e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c01684a0 0x7fb5c0161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c0164220 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c006e7b0 con 0x7fb5c01684a0 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00ce6a0 con 0x7fb5a4003c20 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54076/0 (socket says 172.21.15.116:54076) 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 -- 172.21.15.116:0/1296001835 learned_addr learned my addr 172.21.15.116:0/1296001835 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5bde00640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5a4003c20 0x7fb5c0162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5b5400640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c01684a0 0x7fb5c0161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 -- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5a4003c20 msgr2=0x7fb5c0162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5a4003c20 0x7fb5c0162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 -- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c01684a0 msgr2=0x7fb5c0161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c01684a0 0x7fb5c0161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 -- 172.21.15.116:0/1296001835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00c6630 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5be800640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80012f0 tx=0x7fb5b8007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800ea20 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8018070 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800edc0 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c00c6b30 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.427+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c01617d0 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c00c68a0 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5b801d460 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5b6a00640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 0x7fb59c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fb5b808f560 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5bde00640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 0x7fb59c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.431+0000 7fb5bde00640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 0x7fb59c071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00af000 tx=0x7fb5b00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.439+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb5b805bc60 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5c015ead0 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5b6a00640 1 -- 172.21.15.116:0/1296001835 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7fb5b80642a0 con 0x7fb5c00cc9b0 2024-05-14T11:24:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:24:37.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 msgr2=0x7fb59c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 0x7fb59c071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00af000 tx=0x7fb5b00046d0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 msgr2=0x7fb5c00ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80012f0 tx=0x7fb5b8007bc0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 shutdown_connections 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb59c06f400 0x7fb59c071910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c01684a0 0x7fb5c0161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c00cc9b0 0x7fb5c00ccdc0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 --2- 172.21.15.116:0/1296001835 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5a4003c20 0x7fb5c0162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 >> 172.21.15.116:0/1296001835 conn(0x7fb5c00cffe0 msgr2=0x7fb5c00c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 shutdown_connections 2024-05-14T11:24:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.615+0000 7fb5c5000640 1 -- 172.21.15.116:0/1296001835 wait complete. 2024-05-14T11:24:37.638 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:37.638 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:37.647 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:24:37.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.835+0000 7f80c8a00640 1 Processor -- start 2024-05-14T11:24:37.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.835+0000 7f80c8a00640 1 -- start start 2024-05-14T11:24:37.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f80c40b70a0 0x7f80c40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 0x7f80c4163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f80c40b94d0 con 0x7f80c40b70a0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f80c40cfeb0 con 0x7f80c41638a0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c8a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f80c40cf110 con 0x7f80c415fe00 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f80c40b70a0 0x7f80c40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f80c40b70a0 0x7f80c40b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32768/0 (socket says 172.21.15.116:32768) 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 -- 172.21.15.116:0/406125322 learned_addr learned my addr 172.21.15.116:0/406125322 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80b9400640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c2800640 1 --2- 172.21.15.116:0/406125322 >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 0x7f80c4163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 -- 172.21.15.116:0/406125322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 msgr2=0x7f80c40b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40b8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 -- 172.21.15.116:0/406125322 >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 msgr2=0x7f80c4163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 --2- 172.21.15.116:0/406125322 >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 0x7f80c4163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c1e00640 1 -- 172.21.15.116:0/406125322 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c40b6d80 con 0x7f80c40b70a0 2024-05-14T11:24:37.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.839+0000 7f80c2800640 1 --2- 172.21.15.116:0/406125322 >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 0x7f80c4163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:37.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c1e00640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c40b70a0 0x7f80c40b8a50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f80c40b3170 tx=0x7f80b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965d8deb56acee5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c1400640 1 -- 172.21.15.116:0/406125322 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b40b1070 con 0x7f80c40b70a0 2024-05-14T11:24:37.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c1400640 1 -- 172.21.15.116:0/406125322 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b40ac630 con 0x7f80c40b70a0 2024-05-14T11:24:37.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c1400640 1 -- 172.21.15.116:0/406125322 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b40a8050 con 0x7f80c40b70a0 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- 172.21.15.116:0/406125322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c40b70a0 msgr2=0x7f80c40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c40b70a0 0x7f80c40b8a50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f80c40b3170 tx=0x7f80b401c520 comp rx=0 tx=0).stop 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- 172.21.15.116:0/406125322 shutdown_connections 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 --2- 172.21.15.116:0/406125322 >> v2:172.21.15.149:3300/0 conn(0x7f80c41638a0 0x7f80c4163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 --2- 172.21.15.116:0/406125322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c40b70a0 0x7f80c40b8a50 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- 172.21.15.116:0/406125322 >> 172.21.15.116:0/406125322 conn(0x7f80c40d3740 msgr2=0x7f80c40d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:37.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- 172.21.15.116:0/406125322 shutdown_connections 2024-05-14T11:24:37.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- 172.21.15.116:0/406125322 wait complete. 2024-05-14T11:24:37.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 Processor -- start 2024-05-14T11:24:37.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.843+0000 7f80c8a00640 1 -- start start 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 0x7f80c4156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f80c40b69c0 con 0x7f80c41638a0 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4164600 con 0x7f80c40b70a0 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f80c40b6b40 con 0x7f80c415fe00 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32776/0 (socket says 172.21.15.116:32776) 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 -- 172.21.15.116:0/435718218 learned_addr learned my addr 172.21.15.116:0/435718218 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c1e00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 0x7f80c4156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80b9400640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 -- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 msgr2=0x7f80c40c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 -- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 msgr2=0x7f80c4156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 0x7f80c4156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 -- 172.21.15.116:0/435718218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c41578c0 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c1e00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 0x7f80c4156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c2800640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00cd20 tx=0x7f80bc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00f680 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc00f840 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00fc30 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c4157ba0 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c4160570 con 0x7f80c41638a0 2024-05-14T11:24:37.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.847+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c406a9c0 con 0x7f80c41638a0 2024-05-14T11:24:37.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.851+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f80bc00e3e0 con 0x7f80c41638a0 2024-05-14T11:24:37.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.851+0000 7f80baa00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 0x7f80a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:37.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.851+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f80bc090290 con 0x7f80c41638a0 2024-05-14T11:24:37.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.851+0000 7f80c1e00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 0x7f80a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:37.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.851+0000 7f80c1e00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 0x7f80a0071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f80b40ae000 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:37.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:37.855+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f80bc014030 con 0x7f80c41638a0 2024-05-14T11:24:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80c4068a80 con 0x7f80c41638a0 2024-05-14T11:24:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80baa00640 1 -- 172.21.15.116:0/435718218 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7f80bc05d0f0 con 0x7f80c41638a0 2024-05-14T11:24:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:24:38.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 msgr2=0x7f80a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:38.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 0x7f80a0071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f80b40ae000 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 msgr2=0x7f80c40c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00cd20 tx=0x7f80bc00a650 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 shutdown_connections 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f80a006f400 0x7f80a0071910 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80c41638a0 0x7f80c40c75e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80c415fe00 0x7f80c40c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 --2- 172.21.15.116:0/435718218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80c40b70a0 0x7f80c4156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 >> 172.21.15.116:0/435718218 conn(0x7f80c40d3740 msgr2=0x7f80c415d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 shutdown_connections 2024-05-14T11:24:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:38.023+0000 7f80c8a00640 1 -- 172.21.15.116:0/435718218 wait complete. 2024-05-14T11:24:38.046 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:24:38.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:38.046 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:38.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[46925]: fuse finished with error 0 and tester_r 0 2024-05-14T11:24:38.067 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:43.350 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:24:43.354 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:24:44.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:24:44.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:44.169 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:24:44.180 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:44.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.375+0000 7f1dd6e00640 1 Processor -- start 2024-05-14T11:24:44.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.375+0000 7f1dd6e00640 1 -- start start 2024-05-14T11:24:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd6e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1dd00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd6e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1dd006c700 0x7f1dd006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd6e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd006ff20 con 0x7f1dd015db30 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd6e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd00ae100 con 0x7f1dd006c700 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd6e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd00ae870 con 0x7f1dd00d1670 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcfe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1dd015db30 0x7f1dd00aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45892/0 (socket says 172.21.15.116:45892) 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcfe00640 1 -- 172.21.15.116:0/1425493493 learned_addr learned my addr 172.21.15.116:0/1425493493 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcf400640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1dd00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1425493493 >> v2:172.21.15.149:3300/0 conn(0x7f1dd006c700 0x7f1dd006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884240175 0 0) 0x7f1dd006ff20 con 0x7f1dd015db30 2024-05-14T11:24:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1db4003530 con 0x7f1dd015db30 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1997229146 0 0) 0x7f1db4003530 con 0x7f1dd015db30 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 msgr2=0x7f1dd00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1dd00d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 >> v2:172.21.15.149:3300/0 conn(0x7f1dd006c700 msgr2=0x7f1dd006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 --2- 172.21.15.116:0/1425493493 >> v2:172.21.15.149:3300/0 conn(0x7f1dd006c700 0x7f1dd006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00d1fc0 con 0x7f1dd015db30 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f1dc0001a80 con 0x7f1dd015db30 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcf400640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1dd00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 >> v1:172.21.15.116:6789/0 conn(0x7f1dd015db30 legacy=0x7f1dd00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db4003a10 0x7f1db4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1db4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1db4004d50 0x7f1db4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00ae100 con 0x7f1db4004d50 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00ae870 con 0x7f1db4003a10 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1db400bb50 con 0x7f1dd00d1670 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1db4004d50 0x7f1db4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcfe00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db4003a10 0x7f1db4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dcf400640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1db4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 msgr2=0x7f1db4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd00d1670 0x7f1db4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db4003a10 msgr2=0x7f1db4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db4003a10 0x7f1db4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.379+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1425493493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db4003530 con 0x7f1db4004d50 2024-05-14T11:24:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1db4004d50 0x7f1db4005160 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00b2dd0 tx=0x7f1dc801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf5a05afa12b187 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80096b0 con 0x7f1db4004d50 2024-05-14T11:24:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc80a8450 con 0x7f1db4004d50 2024-05-14T11:24:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dcea00640 1 -- 172.21.15.116:0/1425493493 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80a8840 con 0x7f1db4004d50 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1425493493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1db4004d50 msgr2=0x7f1db4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1425493493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1db4004d50 0x7f1db4005160 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00b2dd0 tx=0x7f1dc801ca50 comp rx=0 tx=0).stop 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1425493493 reap_dead start 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1425493493 shutdown_connections 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1425493493 >> 172.21.15.116:0/1425493493 conn(0x7f1dd00b4620 msgr2=0x7f1dd00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1425493493 shutdown_connections 2024-05-14T11:24:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.383+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1425493493 wait complete. 2024-05-14T11:24:44.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 Processor -- start 2024-05-14T11:24:44.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 -- start start 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd01567f0 0x7f1dd0152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd009eac0 0x7f1dd01fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0168760 con 0x7f1dd009eed0 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd015df40 con 0x7f1dd009eac0 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd6e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd01688e0 con 0x7f1dd01567f0 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32796/0 (socket says 172.21.15.116:32796) 2024-05-14T11:24:44.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1570043578 learned_addr learned my addr 172.21.15.116:0/1570043578 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dcfe00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd01567f0 0x7f1dd0152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dcf400640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd009eac0 0x7f1dd01fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd01567f0 msgr2=0x7f1dd0152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.387+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd01567f0 0x7f1dd0152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd009eac0 msgr2=0x7f1dd01fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd009eac0 0x7f1dd01fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd4e00640 1 -- 172.21.15.116:0/1570043578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00b3920 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd4e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8098200 tx=0x7f1db4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80096b0 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8000e50 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd00b3360 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80a8da0 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01fca00 con 0x7f1dd009eed0 2024-05-14T11:24:44.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd006a9c0 con 0x7f1dd009eed0 2024-05-14T11:24:44.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1dc80a6040 con 0x7f1dd009eed0 2024-05-14T11:24:44.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 0x7f1db0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.391+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f1dc8128b90 con 0x7f1dd009eed0 2024-05-14T11:24:44.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.395+0000 7f1dcfe00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 0x7f1db0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.395+0000 7f1dcfe00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 0x7f1db0071910 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0168370 tx=0x7f1dc0006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.399+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1dc80b5070 con 0x7f1dd009eed0 2024-05-14T11:24:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.591+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd0069000 con 0x7f1dd009eed0 2024-05-14T11:24:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dc7e00640 1 -- 172.21.15.116:0/1570043578 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5183 (secure 0 0 0) 0x7f1dc80ac030 con 0x7f1dd009eed0 2024-05-14T11:24:44.595 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:44.595 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:31.883949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7638":{"gid":7638,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3421632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3421632},{"type":"v1","addr":"172.21.15.116:6835","nonce":3421632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7638,"qdb_cluster":[7638]},"id":7}]} 2024-05-14T11:24:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 83 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 msgr2=0x7f1db0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 0x7f1db0071910 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0168370 tx=0x7f1dc0006040 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 msgr2=0x7f1dd01626f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8098200 tx=0x7f1db4005840 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 shutdown_connections 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1db006f400 0x7f1db0071910 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd009eac0 0x7f1dd01fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dd009eed0 0x7f1dd01626f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 --2- 172.21.15.116:0/1570043578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1dd01567f0 0x7f1dd0152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 >> 172.21.15.116:0/1570043578 conn(0x7f1dd00b4620 msgr2=0x7f1dd0163510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 shutdown_connections 2024-05-14T11:24:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.595+0000 7f1dd6e00640 1 -- 172.21.15.116:0/1570043578 wait complete. 2024-05-14T11:24:44.617 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-14T11:24:44.791 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:24:44.792 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:24:44.792 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:44.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.935+0000 7fd294a00640 1 Processor -- start 2024-05-14T11:24:44.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.935+0000 7fd294a00640 1 -- start start 2024-05-14T11:24:44.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd294a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 0x7fd29006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd294a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd2900af380 0x7fd2900af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd294a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd290160510 con 0x7fd2900d1670 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd294a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd2901573b0 con 0x7fd2900af380 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd294a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd290161120 con 0x7fd290156fa0 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28d400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd2900d1670 0x7fd2900d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45902/0 (socket says 172.21.15.116:45902) 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28d400640 1 -- 172.21.15.116:0/1626628934 learned_addr learned my addr 172.21.15.116:0/1626628934 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28de00640 1 --2- 172.21.15.116:0/1626628934 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 0x7fd29006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28e800640 1 --2- 172.21.15.116:0/1626628934 >> v2:172.21.15.149:3300/0 conn(0x7fd2900af380 0x7fd2900af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28ca00640 1 -- 172.21.15.116:0/1626628934 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117568821 0 0) 0x7fd290160510 con 0x7fd2900d1670 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28ca00640 1 -- 172.21.15.116:0/1626628934 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd274003530 con 0x7fd2900d1670 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28de00640 1 -- 172.21.15.116:0/1626628934 >> v2:172.21.15.149:3300/0 conn(0x7fd2900af380 msgr2=0x7fd2900af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28de00640 1 --2- 172.21.15.116:0/1626628934 >> v2:172.21.15.149:3300/0 conn(0x7fd2900af380 0x7fd2900af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28de00640 1 -- 172.21.15.116:0/1626628934 >> v1:172.21.15.116:6789/0 conn(0x7fd2900d1670 legacy=0x7fd2900d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.939+0000 7fd28de00640 1 -- 172.21.15.116:0/1626628934 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2901607f0 con 0x7fd290156fa0 2024-05-14T11:24:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd28de00640 1 --2- 172.21.15.116:0/1626628934 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 0x7fd29006ffe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2800097c0 tx=0x7fd28001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23e83b9237d1636 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd28ca00640 1 -- 172.21.15.116:0/1626628934 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2800a7070 con 0x7fd290156fa0 2024-05-14T11:24:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd28ca00640 1 -- 172.21.15.116:0/1626628934 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2800b0070 con 0x7fd290156fa0 2024-05-14T11:24:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd28ca00640 1 -- 172.21.15.116:0/1626628934 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2800ab870 con 0x7fd290156fa0 2024-05-14T11:24:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 -- 172.21.15.116:0/1626628934 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 msgr2=0x7fd29006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 --2- 172.21.15.116:0/1626628934 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 0x7fd29006ffe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd2800097c0 tx=0x7fd28001c470 comp rx=0 tx=0).stop 2024-05-14T11:24:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 -- 172.21.15.116:0/1626628934 shutdown_connections 2024-05-14T11:24:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 --2- 172.21.15.116:0/1626628934 >> v2:172.21.15.149:3300/0 conn(0x7fd2900af380 0x7fd2900af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 --2- 172.21.15.116:0/1626628934 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd290156fa0 0x7fd29006ffe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 -- 172.21.15.116:0/1626628934 >> 172.21.15.116:0/1626628934 conn(0x7fd2900b4620 msgr2=0x7fd2900b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 -- 172.21.15.116:0/1626628934 shutdown_connections 2024-05-14T11:24:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.943+0000 7fd294a00640 1 -- 172.21.15.116:0/1626628934 wait complete. 2024-05-14T11:24:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 Processor -- start 2024-05-14T11:24:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- start start 2024-05-14T11:24:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2900d1670 0x7fd2900ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 0x7fd29015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900b0660 con 0x7fd2900af380 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900b04e0 con 0x7fd290156fa0 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd290169810 con 0x7fd2900d1670 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32810/0 (socket says 172.21.15.116:32810) 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 -- 172.21.15.116:0/2696429423 learned_addr learned my addr 172.21.15.116:0/2696429423 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28d400640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2900d1670 0x7fd2900ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28e800640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 0x7fd29015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 -- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2900d1670 msgr2=0x7fd2900ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2900d1670 0x7fd2900ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 -- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 msgr2=0x7fd29015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 0x7fd29015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 -- 172.21.15.116:0/2696429423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900d0b50 con 0x7fd2900af380 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28e800640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 0x7fd29015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd28de00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd2800ad040 tx=0x7fd280019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280003af0 con 0x7fd2900af380 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd280003c90 con 0x7fd2900af380 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2900d0080 con 0x7fd2900af380 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2800c0680 con 0x7fd2900af380 2024-05-14T11:24:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.947+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2900d0870 con 0x7fd2900af380 2024-05-14T11:24:44.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29006a9c0 con 0x7fd2900af380 2024-05-14T11:24:44.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2800c0820 con 0x7fd2900af380 2024-05-14T11:24:44.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd286000640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 0x7fd270071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:44.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fd28012a150 con 0x7fd2900af380 2024-05-14T11:24:44.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd28d400640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 0x7fd270071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:44.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.951+0000 7fd28d400640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 0x7fd270071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd27c007a70 tx=0x7fd27c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:44.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:44.955+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd280016090 con 0x7fd2900af380 2024-05-14T11:24:45.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2900ce580 con 0x7fd2900af380 2024-05-14T11:24:45.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd286000640 1 -- 172.21.15.116:0/2696429423 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5183 (secure 0 0 0) 0x7fd2800f6f30 con 0x7fd2900af380 2024-05-14T11:24:45.148 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:45.148 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:31.883949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7638":{"gid":7638,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3421632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3421632},{"type":"v1","addr":"172.21.15.116:6835","nonce":3421632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7638,"qdb_cluster":[7638]},"id":7}]} 2024-05-14T11:24:45.148 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 83 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 msgr2=0x7fd270071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 0x7fd270071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd27c007a70 tx=0x7fd27c009040 comp rx=0 tx=0).stop 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 msgr2=0x7fd2900cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd2800ad040 tx=0x7fd280019210 comp rx=0 tx=0).stop 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 shutdown_connections 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd27006f430 0x7fd270071940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd290156fa0 0x7fd29015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2900d1670 0x7fd2900ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 --2- 172.21.15.116:0/2696429423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2900af380 0x7fd2900cfb40 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 >> 172.21.15.116:0/2696429423 conn(0x7fd2900b4620 msgr2=0x7fd290153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:45.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 shutdown_connections 2024-05-14T11:24:45.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.147+0000 7fd294a00640 1 -- 172.21.15.116:0/2696429423 wait complete. 2024-05-14T11:24:45.169 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-14T11:24:45.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 Processor -- start 2024-05-14T11:24:45.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- start start 2024-05-14T11:24:45.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720420 0x55e7237a9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723722ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e72367faa0 con 0x55e723720fa0 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e723683ae0 con 0x55e723720420 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55e723709b60 con 0x55e7237ac980 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32832/0 (socket says 172.21.15.116:32832) 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 -- 172.21.15.116:0/795634687 learned_addr learned my addr 172.21.15.116:0/795634687 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623c00640 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720420 0x55e7237a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe624600640 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723722ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 -- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 msgr2=0x55e723722ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723722ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 -- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720420 msgr2=0x55e7237a9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720420 0x55e7237a9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 -- 172.21.15.116:0/795634687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7237aa3e0 con 0x55e723720fa0 2024-05-14T11:24:45.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe623200640 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe61001b7b0 tx=0x7fe610003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85c11d7e7c28154 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe621e00640 1 -- 172.21.15.116:0/795634687 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100be070 con 0x55e723720fa0 2024-05-14T11:24:45.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe621e00640 1 -- 172.21.15.116:0/795634687 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100b5040 con 0x55e723720fa0 2024-05-14T11:24:45.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe621e00640 1 -- 172.21.15.116:0/795634687 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100b4070 con 0x55e723720fa0 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- 172.21.15.116:0/795634687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 msgr2=0x55e723704c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe61001b7b0 tx=0x7fe610003030 comp rx=0 tx=0).stop 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- 172.21.15.116:0/795634687 shutdown_connections 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723722ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720fa0 0x55e723704c60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 --2- 172.21.15.116:0/795634687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720420 0x55e7237a9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- 172.21.15.116:0/795634687 >> 172.21.15.116:0/795634687 conn(0x55e7236c4e10 msgr2=0x55e7237afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- 172.21.15.116:0/795634687 shutdown_connections 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.227+0000 7fe62aa90080 1 -- 172.21.15.116:0/795634687 wait complete. 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 Processor -- start 2024-05-14T11:24:45.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- start start 2024-05-14T11:24:45.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 0x55e723703250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720fa0 0x55e723722ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723723220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7237aa680 con 0x55e723720420 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e7237055e0 con 0x55e723720fa0 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55e7237aa920 con 0x55e7237ac980 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 0x55e723703250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 0x55e723703250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32848/0 (socket says 172.21.15.116:32848) 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 -- 172.21.15.116:0/3765319339 learned_addr learned my addr 172.21.15.116:0/3765319339 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe624600640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723723220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623200640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720fa0 0x55e723722ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 -- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 msgr2=0x55e723723220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55e7237ac980 0x55e723723220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 -- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720fa0 msgr2=0x55e723722ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720fa0 0x55e723722ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 -- 172.21.15.116:0/3765319339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7236fb080 con 0x55e723720420 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623200640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e723720fa0 0x55e723722ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:45.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe623c00640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 0x55e723703250 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c4f0 tx=0x7fe61400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe620a00640 1 -- 172.21.15.116:0/3765319339 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614010600 con 0x55e723720420 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe620a00640 1 -- 172.21.15.116:0/3765319339 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614015020 con 0x55e723720420 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe620a00640 1 -- 172.21.15.116:0/3765319339 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140109e0 con 0x55e723720420 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e7236fb320 con 0x55e723720420 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.231+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7236fbf10 con 0x55e723720420 2024-05-14T11:24:45.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe620a00640 1 -- 172.21.15.116:0/3765319339 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe614010b80 con 0x55e723720420 2024-05-14T11:24:45.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe620a00640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60406dde0 0x7fe6040702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe620a00640 1 -- 172.21.15.116:0/3765319339 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fe61408ea20 con 0x55e723720420 2024-05-14T11:24:45.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe623200640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60406dde0 0x7fe6040702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe62aa90080 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55e72371bce0 0x55e72371c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55e72371c630 con 0x55e72371bce0 2024-05-14T11:24:45.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe624600640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55e72371bce0 0x55e72371c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe623200640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60406dde0 0x7fe6040702f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe6100bb000 tx=0x7fe6100a9000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.235+0000 7fe624600640 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55e72371bce0 0x55e72371c0f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe624600640 1 -- 172.21.15.116:0/3765319339 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fe61c002580 con 0x55e72371bce0 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55e72371bce0 msgr2=0x55e72371c0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55e72371bce0 0x55e72371c0f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60406dde0 msgr2=0x7fe6040702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe60406dde0 0x7fe6040702f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe6100bb000 tx=0x7fe6100a9000 comp rx=0 tx=0).stop 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 msgr2=0x55e723703250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 --2- 172.21.15.116:0/3765319339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e723720420 0x55e723703250 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c4f0 tx=0x7fe61400c9b0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe624600640 1 -- 172.21.15.116:0/3765319339 reap_dead start 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 shutdown_connections 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 >> 172.21.15.116:0/3765319339 conn(0x55e7236c4e10 msgr2=0x55e7237a9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 shutdown_connections 2024-05-14T11:24:45.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.239+0000 7fe62aa90080 1 -- 172.21.15.116:0/3765319339 wait complete. 2024-05-14T11:24:45.244 INFO:teuthology.orchestra.run.smithi116.stdout:����� 2024-05-14T11:24:45.244 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:24:45.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.387+0000 7f42c4e00640 1 Processor -- start 2024-05-14T11:24:45.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.387+0000 7f42c4e00640 1 -- start start 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42c4e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f42c0156fa0 0x7f42c006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00af380 0x7f42c00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42c4e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c0160510 con 0x7f42c00d1670 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42c4e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f42c01573b0 con 0x7f42c0156fa0 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42c4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0161120 con 0x7f42c00af380 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42b7e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f42c00d1670 0x7f42c00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45924/0 (socket says 172.21.15.116:45924) 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42b7e00640 1 -- 172.21.15.116:0/1281862098 learned_addr learned my addr 172.21.15.116:0/1281862098 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bde00640 1 --2- 172.21.15.116:0/1281862098 >> v2:172.21.15.149:3300/0 conn(0x7f42c0156fa0 0x7f42c006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42be800640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00af380 0x7f42c00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bd400640 1 -- 172.21.15.116:0/1281862098 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3078976595 0 0) 0x7f42c0160510 con 0x7f42c00d1670 2024-05-14T11:24:45.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bd400640 1 -- 172.21.15.116:0/1281862098 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42a4003530 con 0x7f42c00d1670 2024-05-14T11:24:45.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bde00640 1 -- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00af380 msgr2=0x7f42c00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:45.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bde00640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00af380 0x7f42c00af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bde00640 1 -- 172.21.15.116:0/1281862098 >> v1:172.21.15.116:6789/0 conn(0x7f42c00d1670 legacy=0x7f42c00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.391+0000 7f42bde00640 1 -- 172.21.15.116:0/1281862098 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01607f0 con 0x7f42c0156fa0 2024-05-14T11:24:45.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42bde00640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c0156fa0 0x7f42c006ffe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b00097c0 tx=0x7f42b001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0c2e1d8d1fd104 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42bd400640 1 -- 172.21.15.116:0/1281862098 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b00a7070 con 0x7f42c0156fa0 2024-05-14T11:24:45.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42bd400640 1 -- 172.21.15.116:0/1281862098 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b00b0070 con 0x7f42c0156fa0 2024-05-14T11:24:45.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42bd400640 1 -- 172.21.15.116:0/1281862098 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b00ab890 con 0x7f42c0156fa0 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 -- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c0156fa0 msgr2=0x7f42c006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c0156fa0 0x7f42c006ffe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b00097c0 tx=0x7f42b001c470 comp rx=0 tx=0).stop 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 -- 172.21.15.116:0/1281862098 shutdown_connections 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00af380 0x7f42c00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1281862098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c0156fa0 0x7f42c006ffe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 -- 172.21.15.116:0/1281862098 >> 172.21.15.116:0/1281862098 conn(0x7f42c00b4620 msgr2=0x7f42c00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:45.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 -- 172.21.15.116:0/1281862098 shutdown_connections 2024-05-14T11:24:45.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.395+0000 7f42c4e00640 1 -- 172.21.15.116:0/1281862098 wait complete. 2024-05-14T11:24:45.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 Processor -- start 2024-05-14T11:24:45.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 -- start start 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c00af380 0x7f42c014c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00d1670 0x7f42c0147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00b0660 con 0x7f42c0156fa0 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00b04e0 con 0x7f42c00af380 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0169810 con 0x7f42c00d1670 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32852/0 (socket says 172.21.15.116:32852) 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 -- 172.21.15.116:0/1602070196 learned_addr learned my addr 172.21.15.116:0/1602070196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42b7e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00d1670 0x7f42c0147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42bde00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c00af380 0x7f42c014c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 -- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00d1670 msgr2=0x7f42c0147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00d1670 0x7f42c0147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 -- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c00af380 msgr2=0x7f42c014c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c00af380 0x7f42c014c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 -- 172.21.15.116:0/1602070196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01562d0 con 0x7f42c0156fa0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42be800640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f42b800efc0 tx=0x7f42b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8018020 con 0x7f42c0156fa0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b8014480 con 0x7f42c0156fa0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c0155d70 con 0x7f42c0156fa0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.399+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8014870 con 0x7f42c0156fa0 2024-05-14T11:24:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c0155a90 con 0x7f42c0156fa0 2024-05-14T11:24:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c006a9c0 con 0x7f42c0156fa0 2024-05-14T11:24:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f42b801b020 con 0x7f42c0156fa0 2024-05-14T11:24:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42b6000640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 0x7f429c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f42b808fd60 con 0x7f42c0156fa0 2024-05-14T11:24:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42bde00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 0x7f429c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.403+0000 7f42bde00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 0x7f429c071910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f42b00af000 tx=0x7f42b00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:45.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.407+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f42b8022070 con 0x7f42c0156fa0 2024-05-14T11:24:45.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.611+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42c00b4d80 con 0x7f42c0156fa0 2024-05-14T11:24:45.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.611+0000 7f42b6000640 1 -- 172.21.15.116:0/1602070196 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f42b8010040 con 0x7f42c0156fa0 2024-05-14T11:24:45.613 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:24:45.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 msgr2=0x7f429c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 0x7f429c071910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f42b00af000 tx=0x7f42b00a7000 comp rx=0 tx=0).stop 2024-05-14T11:24:45.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 msgr2=0x7f42c015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:45.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f42b800efc0 tx=0x7f42b800c490 comp rx=0 tx=0).stop 2024-05-14T11:24:45.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 shutdown_connections 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f429c06f400 0x7f429c071910 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42c0156fa0 0x7f42c015bd70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42c00d1670 0x7f42c0147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 --2- 172.21.15.116:0/1602070196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c00af380 0x7f42c014c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 >> 172.21.15.116:0/1602070196 conn(0x7f42c00b4620 msgr2=0x7f42c006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 shutdown_connections 2024-05-14T11:24:45.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45.615+0000 7f42c4e00640 1 -- 172.21.15.116:0/1602070196 wait complete. 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:24:45.635 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:45.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:45.636 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.645 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41917sec preferred_lft 41917sec 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.646 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:24:45.647 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:24:45.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:45.648 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:24:45.707 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:24:45.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:45.710 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:24:45.765 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:24:45.766 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:24:45.767 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:24:45.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:24:45.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:45.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:45.899 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:24:45.907 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:24:45.907 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:24:45.908 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:45.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:45.908 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:24:45.974 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:24:45.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:24:45.974 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:24:46.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:24:46.022 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:46.073 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:24:46.134 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:24:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:24:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:24:46.184 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:46.186 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:46.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.186 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:24:46.241 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:24:46.242 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:24:46.288 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:24:46.309 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:24:46.364 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:24:46.364 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:24:46.365 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:24:46.366 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:46.367 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:24:46.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.368 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:24:46.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:24:46.391+0000 7fb1b3680580 -1 init, newargv = 0x7fb198001280 newargc=15 2024-05-14T11:24:46.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[47171]: starting ceph client 2024-05-14T11:24:46.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[47171]: starting fuse 2024-05-14T11:24:46.403 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:24:46.403 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:24:46.403 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:24:46.403 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:24:46.403 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:24:46.403 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:46.403 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:24:46.404 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:24:46.405 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:46.527 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.47171.asok 2024-05-14T11:24:46.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:46.540 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47171.asok 2024-05-14T11:24:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.540 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47171.asok status 2024-05-14T11:24:46.717 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:46.717 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:24:46.717 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:24:46.717 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "47171", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7848, 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 7848 2024-05-14T11:24:46.718 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1944932905 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1944932905 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.7848 192.168.0.1:0/1944932905", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/1944932905", 2024-05-14T11:24:46.719 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 83, 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 99, 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:24:46.720 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:46.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.733 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:24:46.733 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:46.742 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:24:46.742 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:46.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.743 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:46.806 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:46.806 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:46.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.855 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-14T11:24:46.984 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records in 2024-05-14T11:24:46.984 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records out 2024-05-14T11:24:46.984 INFO:teuthology.orchestra.run.smithi116.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0724414 s, 86.8 MB/s 2024-05-14T11:24:46.985 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:46.985 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:46.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:46.993 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-14T11:24:47.114 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records in 2024-05-14T11:24:47.114 INFO:teuthology.orchestra.run.smithi116.stderr:6+0 records out 2024-05-14T11:24:47.115 INFO:teuthology.orchestra.run.smithi116.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0694541 s, 90.6 MB/s 2024-05-14T11:24:47.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:47.116 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-14T11:24:47.128 INFO:teuthology.orchestra.run.smithi116.stdout:./ 2024-05-14T11:24:47.128 INFO:teuthology.orchestra.run.smithi116.stdout:./file2_sixmegs 2024-05-14T11:24:47.128 INFO:teuthology.orchestra.run.smithi116.stdout:./file3_sixmegs 2024-05-14T11:24:47.128 INFO:teuthology.orchestra.run.smithi116.stdout:./file1_sixmegs 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-14T11:24:47.129 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:47.233 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:24:47.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-14T11:24:47.243 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:47.350 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:24:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-14T11:24:47.359 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:47.466 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627778 2024-05-14T11:24:47.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-14T11:24:47.475 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:24:47.583 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:24:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:24:47.593 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:47.593 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:47.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:24:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.831+0000 7fb595e00640 1 Processor -- start 2024-05-14T11:24:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.831+0000 7fb595e00640 1 -- start start 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb595e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb590166750 0x7fb590168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb595e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59006ff20 con 0x7fb59015db30 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb595e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb590070470 con 0x7fb5900d1670 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb595e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb59006d710 con 0x7fb590166750 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb59015db30 0x7fb5900aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50378/0 (socket says 172.21.15.116:50378) 2024-05-14T11:24:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58ea00640 1 -- 172.21.15.116:0/1720286546 learned_addr learned my addr 172.21.15.116:0/1720286546 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58f400640 1 --2- 172.21.15.116:0/1720286546 >> v2:172.21.15.149:3300/0 conn(0x7fb590166750 0x7fb590168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705415360 0 0) 0x7fb59006ff20 con 0x7fb59015db30 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb574003530 con 0x7fb59015db30 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3590256143 0 0) 0x7fb574003530 con 0x7fb59015db30 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58f400640 1 -- 172.21.15.116:0/1720286546 >> v1:172.21.15.149:6789/0 conn(0x7fb5900d1670 legacy=0x7fb5900d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58f400640 1 -- 172.21.15.116:0/1720286546 >> v1:172.21.15.116:6789/0 conn(0x7fb59015db30 legacy=0x7fb5900aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:47.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.835+0000 7fb58f400640 1 -- 172.21.15.116:0/1720286546 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb590070750 con 0x7fb590166750 2024-05-14T11:24:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb58f400640 1 --2- 172.21.15.116:0/1720286546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590168b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb588009d90 tx=0x7fb58801c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a9b5d43b1bbbce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578001700 con 0x7fb590166750 2024-05-14T11:24:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5880a8420 con 0x7fb590166750 2024-05-14T11:24:47.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb58d600640 1 -- 172.21.15.116:0/1720286546 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5880a87f0 con 0x7fb590166750 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 -- 172.21.15.116:0/1720286546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 msgr2=0x7fb590168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 --2- 172.21.15.116:0/1720286546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590168b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb588009d90 tx=0x7fb58801c770 comp rx=0 tx=0).stop 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 -- 172.21.15.116:0/1720286546 shutdown_connections 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 --2- 172.21.15.116:0/1720286546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590168b70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 -- 172.21.15.116:0/1720286546 >> 172.21.15.116:0/1720286546 conn(0x7fb5900b4620 msgr2=0x7fb5900b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 -- 172.21.15.116:0/1720286546 shutdown_connections 2024-05-14T11:24:47.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.839+0000 7fb595e00640 1 -- 172.21.15.116:0/1720286546 wait complete. 2024-05-14T11:24:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 Processor -- start 2024-05-14T11:24:47.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- start start 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5900d1670 0x7fb59014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb59016bae0 con 0x7fb59015db30 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5900ae870 con 0x7fb590166750 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb59016bc60 con 0x7fb5900d1670 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45968/0 (socket says 172.21.15.116:45968) 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 -- 172.21.15.116:0/3320981042 learned_addr learned my addr 172.21.15.116:0/3320981042 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58ea00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5900d1670 0x7fb59014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:47.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58f400640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 -- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5900d1670 msgr2=0x7fb59014d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5900d1670 0x7fb59014d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 -- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 msgr2=0x7fb590167bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590167bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 -- 172.21.15.116:0/3320981042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5900b16a0 con 0x7fb59015db30 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb58e000640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c420 tx=0x7fb58000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580017070 con 0x7fb59015db30 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5900b1b40 con 0x7fb59015db30 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5800052d0 con 0x7fb59015db30 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800054c0 con 0x7fb59015db30 2024-05-14T11:24:47.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.843+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5900b2100 con 0x7fb59015db30 2024-05-14T11:24:47.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5900b2820 con 0x7fb59015db30 2024-05-14T11:24:47.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb58001a070 con 0x7fb59015db30 2024-05-14T11:24:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb587400640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fb58008e780 con 0x7fb59015db30 2024-05-14T11:24:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb58ea00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.847+0000 7fb58ea00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 0x7fb56c071910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb5780013d0 tx=0x7fb578001be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:47.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:47.851+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb58005ae90 con 0x7fb59015db30 2024-05-14T11:24:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.019+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb59006a9c0 con 0x7fb59015db30 2024-05-14T11:24:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.019+0000 7fb587400640 1 -- 172.21.15.116:0/3320981042 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7fb58005f190 con 0x7fb59015db30 2024-05-14T11:24:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:24:48.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 msgr2=0x7fb56c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 0x7fb56c071910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb5780013d0 tx=0x7fb578001be0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 msgr2=0x7fb590164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c420 tx=0x7fb58000c8e0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 shutdown_connections 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb56c06f400 0x7fb56c071910 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb590166750 0x7fb590167bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb59015db30 0x7fb590164400 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 --2- 172.21.15.116:0/3320981042 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb5900d1670 0x7fb59014d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 >> 172.21.15.116:0/3320981042 conn(0x7fb5900b4620 msgr2=0x7fb59015c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 shutdown_connections 2024-05-14T11:24:48.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.023+0000 7fb595e00640 1 -- 172.21.15.116:0/3320981042 wait complete. 2024-05-14T11:24:48.043 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:48.043 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:48.052 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:24:48.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 Processor -- start 2024-05-14T11:24:48.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 -- start start 2024-05-14T11:24:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2c180af380 0x7f2c180af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c18160510 con 0x7f2c180d1670 2024-05-14T11:24:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f2c181573b0 con 0x7f2c180af380 2024-05-14T11:24:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.239+0000 7f2c1d000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c18161120 con 0x7f2c18156fa0 2024-05-14T11:24:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c16800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2c180af380 0x7f2c180af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:35980/0 (socket says 172.21.15.116:35980) 2024-05-14T11:24:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 -- 172.21.15.116:0/1522407469 learned_addr learned my addr 172.21.15.116:0/1522407469 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c0d400640 1 --1- 172.21.15.116:0/1522407469 >> v1:172.21.15.149:6789/0 conn(0x7f2c180d1670 0x7f2c180d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:35272/0 (socket says 172.21.15.116:35272) 2024-05-14T11:24:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 -- 172.21.15.116:0/1522407469 >> v2:172.21.15.149:3300/0 conn(0x7f2c180af380 msgr2=0x7f2c180af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 --2- 172.21.15.116:0/1522407469 >> v2:172.21.15.149:3300/0 conn(0x7f2c180af380 0x7f2c180af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 -- 172.21.15.116:0/1522407469 >> v1:172.21.15.149:6789/0 conn(0x7f2c180d1670 legacy=0x7f2c180d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 -- 172.21.15.116:0/1522407469 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181607f0 con 0x7f2c18156fa0 2024-05-14T11:24:48.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15e00640 1 --2- 172.21.15.116:0/1522407469 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009870 tx=0x7f2c0801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0bbe91ddeb93b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15400640 1 -- 172.21.15.116:0/1522407469 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080a95f0 con 0x7f2c18156fa0 2024-05-14T11:24:48.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15400640 1 -- 172.21.15.116:0/1522407469 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c080a9790 con 0x7f2c18156fa0 2024-05-14T11:24:48.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.243+0000 7f2c15400640 1 -- 172.21.15.116:0/1522407469 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080a9b60 con 0x7f2c18156fa0 2024-05-14T11:24:48.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- 172.21.15.116:0/1522407469 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 msgr2=0x7f2c1806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 --2- 172.21.15.116:0/1522407469 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009870 tx=0x7f2c0801c530 comp rx=0 tx=0).stop 2024-05-14T11:24:48.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- 172.21.15.116:0/1522407469 shutdown_connections 2024-05-14T11:24:48.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 --2- 172.21.15.116:0/1522407469 >> v2:172.21.15.149:3300/0 conn(0x7f2c180af380 0x7f2c180af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 --2- 172.21.15.116:0/1522407469 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806ffe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- 172.21.15.116:0/1522407469 >> 172.21.15.116:0/1522407469 conn(0x7f2c180b4620 msgr2=0x7f2c180b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:48.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- 172.21.15.116:0/1522407469 shutdown_connections 2024-05-14T11:24:48.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- 172.21.15.116:0/1522407469 wait complete. 2024-05-14T11:24:48.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 Processor -- start 2024-05-14T11:24:48.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.247+0000 7f2c1d000640 1 -- start start 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c180d1670 0x7f2c1809a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b0660 con 0x7f2c180af380 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b04e0 con 0x7f2c180d1670 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c18169810 con 0x7f2c18156fa0 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45974/0 (socket says 172.21.15.116:45974) 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 -- 172.21.15.116:0/2942621130 learned_addr learned my addr 172.21.15.116:0/2942621130 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c0d400640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c180d1670 0x7f2c1809a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c16800640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 -- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 msgr2=0x7f2c1806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 -- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c180d1670 msgr2=0x7f2c1809a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c180d1670 0x7f2c1809a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 -- 172.21.15.116:0/2942621130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181fa290 con 0x7f2c180af380 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c15e00640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2c0801abe0 tx=0x7f2c080192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080bb640 con 0x7f2c180af380 2024-05-14T11:24:48.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c080bb7e0 con 0x7f2c180af380 2024-05-14T11:24:48.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c180cfc00 con 0x7f2c180af380 2024-05-14T11:24:48.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080bbbd0 con 0x7f2c180af380 2024-05-14T11:24:48.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c180d01c0 con 0x7f2c180af380 2024-05-14T11:24:48.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.251+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1806a9c0 con 0x7f2c180af380 2024-05-14T11:24:48.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.255+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2c080b13f0 con 0x7f2c180af380 2024-05-14T11:24:48.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.255+0000 7f2c0ea00640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 0x7f2bf4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.255+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f2c0812a080 con 0x7f2c180af380 2024-05-14T11:24:48.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.255+0000 7f2c0d400640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 0x7f2bf4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.255+0000 7f2c0d400640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 0x7f2bf4071910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2c1806d280 tx=0x7f2c00007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.259+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2c080c2030 con 0x7f2c180af380 2024-05-14T11:24:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c180b3050 con 0x7f2c180af380 2024-05-14T11:24:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c0ea00640 1 -- 172.21.15.116:0/2942621130 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7f2c080ad020 con 0x7f2c180af380 2024-05-14T11:24:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:24:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 msgr2=0x7f2bf4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 0x7f2bf4071910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2c1806d280 tx=0x7f2c00007e50 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 msgr2=0x7f2c181f9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2c0801abe0 tx=0x7f2c080192d0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 shutdown_connections 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2bf406f400 0x7f2bf4071910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c180d1670 0x7f2c1809a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c18156fa0 0x7f2c1806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 --2- 172.21.15.116:0/2942621130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c180af380 0x7f2c181f9d50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 >> 172.21.15.116:0/2942621130 conn(0x7f2c180b4620 msgr2=0x7f2c180b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 shutdown_connections 2024-05-14T11:24:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.427+0000 7f2c1d000640 1 -- 172.21.15.116:0/2942621130 wait complete. 2024-05-14T11:24:48.449 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:24:48.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:48.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:48.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[47171]: fuse finished with error 0 and tester_r 0 2024-05-14T11:24:48.473 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:24:48.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:24:48.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:48.567 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:24:48.576 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:48.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 Processor -- start 2024-05-14T11:24:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 -- start start 2024-05-14T11:24:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff6240d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6240d1b20 0x7ff6240d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6240d1f30 con 0x7ff6240d08a0 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff6240ce4e0 con 0x7ff6240d1b20 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff62a200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff6240cd8d0 con 0x7ff62415feb0 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff623e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6240d1b20 0x7ff6240d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff623e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6240d1b20 0x7ff6240d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45980/0 (socket says 172.21.15.116:45980) 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.763+0000 7ff623e00640 1 -- 172.21.15.116:0/402131574 learned_addr learned my addr 172.21.15.116:0/402131574 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff628e00640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff6240d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623e00640 1 -- 172.21.15.116:0/402131574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 msgr2=0x7ff6240d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623e00640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff6240d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623e00640 1 -- 172.21.15.116:0/402131574 >> v1:172.21.15.149:6789/0 conn(0x7ff6240d08a0 legacy=0x7ff62406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623e00640 1 -- 172.21.15.116:0/402131574 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6240cdbb0 con 0x7ff6240d1b20 2024-05-14T11:24:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623400640 1 -- 172.21.15.116:0/402131574 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053869740 0 0) 0x7ff61c001120 con 0x7ff6240d08a0 2024-05-14T11:24:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623e00640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff6240d0360 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff610009870 tx=0x7ff61001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b4f1e05215e17e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623400640 1 -- 172.21.15.116:0/402131574 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100ab650 con 0x7ff6240d1b20 2024-05-14T11:24:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623400640 1 -- 172.21.15.116:0/402131574 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6100b0070 con 0x7ff6240d1b20 2024-05-14T11:24:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.767+0000 7ff623400640 1 -- 172.21.15.116:0/402131574 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100aba10 con 0x7ff6240d1b20 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- 172.21.15.116:0/402131574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 msgr2=0x7ff6240d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff6240d0360 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff610009870 tx=0x7ff61001c530 comp rx=0 tx=0).stop 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- 172.21.15.116:0/402131574 shutdown_connections 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff6240d0360 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 --2- 172.21.15.116:0/402131574 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff6240d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- 172.21.15.116:0/402131574 >> 172.21.15.116:0/402131574 conn(0x7ff6240b4530 msgr2=0x7ff6240b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- 172.21.15.116:0/402131574 shutdown_connections 2024-05-14T11:24:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- 172.21.15.116:0/402131574 wait complete. 2024-05-14T11:24:48.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 Processor -- start 2024-05-14T11:24:48.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.771+0000 7ff62a200640 1 -- start start 2024-05-14T11:24:48.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6240d08a0 0x7ff6240caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff62415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff6240d0e30 con 0x7ff6240d1b20 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6240d0cb0 con 0x7ff6240d08a0 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff62416c870 con 0x7ff62415feb0 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45982/0 (socket says 172.21.15.116:45982) 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 -- 172.21.15.116:0/2203007277 learned_addr learned my addr 172.21.15.116:0/2203007277 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff628e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6240d08a0 0x7ff6240caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff629800640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff62415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 -- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 msgr2=0x7ff62415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff62415e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 -- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6240d08a0 msgr2=0x7ff6240caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6240d08a0 0x7ff6240caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 -- 172.21.15.116:0/2203007277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff624157400 con 0x7ff6240d1b20 2024-05-14T11:24:48.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff623e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff61001a160 tx=0x7ff61001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100be5f0 con 0x7ff6240d1b20 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6100be790 con 0x7ff6240d1b20 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100beb80 con 0x7ff6240d1b20 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6241575f0 con 0x7ff6240d1b20 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff624157bb0 con 0x7ff6240d1b20 2024-05-14T11:24:48.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6240cb090 con 0x7ff6240d1b20 2024-05-14T11:24:48.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.775+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff6100b43f0 con 0x7ff6240d1b20 2024-05-14T11:24:48.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.779+0000 7ff620c00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:48.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.779+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7ff610129ce0 con 0x7ff6240d1b20 2024-05-14T11:24:48.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.779+0000 7ff628e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:48.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.779+0000 7ff628e00640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 0x7ff604071910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff614011340 tx=0x7ff61400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:48.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.783+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff610016150 con 0x7ff6240d1b20 2024-05-14T11:24:48.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.971+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff624050880 con 0x7ff6240d1b20 2024-05-14T11:24:48.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.971+0000 7ff620c00640 1 -- 172.21.15.116:0/2203007277 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5183 (secure 0 0 0) 0x7ff6100f6350 con 0x7ff6240d1b20 2024-05-14T11:24:48.974 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:48.974 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:31.883949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7638":{"gid":7638,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3421632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3421632},{"type":"v1","addr":"172.21.15.116:6835","nonce":3421632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7638,"qdb_cluster":[7638]},"id":7}]} 2024-05-14T11:24:48.974 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 83 2024-05-14T11:24:48.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 msgr2=0x7ff604071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 0x7ff604071910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff614011340 tx=0x7ff61400b040 comp rx=0 tx=0).stop 2024-05-14T11:24:48.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 msgr2=0x7ff624156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff61001a160 tx=0x7ff61001ad70 comp rx=0 tx=0).stop 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 shutdown_connections 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff62415feb0 0x7ff62415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6240d1b20 0x7ff624156ec0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 --2- 172.21.15.116:0/2203007277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6240d08a0 0x7ff6240caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 >> 172.21.15.116:0/2203007277 conn(0x7ff6240b4530 msgr2=0x7ff6240b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 shutdown_connections 2024-05-14T11:24:48.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:48.975+0000 7ff62a200640 1 -- 172.21.15.116:0/2203007277 wait complete. 2024-05-14T11:24:48.996 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-14T11:24:49.169 INFO:teuthology.orchestra.run.smithi116.stdout:{"message":"","return_code":0} 2024-05-14T11:24:49.169 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-14T11:24:49.169 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:24:49.170 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:24:49.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:24:49.179 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi116.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-14T11:24:49.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:24:49.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:49.179 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:24:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-14T11:24:49.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:24:49.234 DEBUG:teuthology.orchestra.run.smithi116:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-14T11:24:49.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:49.285 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-14T11:24:49.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.435+0000 7fd85fc00640 1 Processor -- start 2024-05-14T11:24:49.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 -- start start 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd858071ab0 0x7fd858160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd85815d250 0x7fd85815d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8580b6e80 con 0x7fd8580ceec0 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd8580cf2d0 con 0x7fd858071ab0 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd8580b77b0 con 0x7fd85815d250 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd858071ab0 0x7fd858160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd85dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd85815d250 0x7fd85815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd858071ab0 0x7fd858160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:37908/0 (socket says 172.21.15.116:37908) 2024-05-14T11:24:49.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 -- 172.21.15.116:0/503176137 learned_addr learned my addr 172.21.15.116:0/503176137 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 -- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd85815d250 msgr2=0x7fd85815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 --2- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd85815d250 0x7fd85815d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 -- 172.21.15.116:0/503176137 >> v1:172.21.15.149:6789/0 conn(0x7fd8580ceec0 legacy=0x7fd8580713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.439+0000 7fd857e00640 1 -- 172.21.15.116:0/503176137 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8580b7160 con 0x7fd858071ab0 2024-05-14T11:24:49.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd857e00640 1 --2- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd858071ab0 0x7fd858160fe0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84401c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be76ce0bda070a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd857400640 1 -- 172.21.15.116:0/503176137 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440b0070 con 0x7fd858071ab0 2024-05-14T11:24:49.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd857400640 1 -- 172.21.15.116:0/503176137 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8440ab600 con 0x7fd858071ab0 2024-05-14T11:24:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd857400640 1 -- 172.21.15.116:0/503176137 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440ab9d0 con 0x7fd858071ab0 2024-05-14T11:24:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 -- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd858071ab0 msgr2=0x7fd858160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 --2- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd858071ab0 0x7fd858160fe0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84401c500 comp rx=0 tx=0).stop 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 -- 172.21.15.116:0/503176137 shutdown_connections 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 --2- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd85815d250 0x7fd85815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 --2- 172.21.15.116:0/503176137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd858071ab0 0x7fd858160fe0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 -- 172.21.15.116:0/503176137 >> 172.21.15.116:0/503176137 conn(0x7fd8580d24b0 msgr2=0x7fd8580d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 -- 172.21.15.116:0/503176137 shutdown_connections 2024-05-14T11:24:49.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.443+0000 7fd85fc00640 1 -- 172.21.15.116:0/503176137 wait complete. 2024-05-14T11:24:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 Processor -- start 2024-05-14T11:24:49.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 -- start start 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 0x7fd858149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 0x7fd8580b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd85815a940 con 0x7fd858149ed0 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd858167fb0 con 0x7fd85815d250 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd85815c550 con 0x7fd8580ceec0 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45988/0 (socket says 172.21.15.116:45988) 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 -- 172.21.15.116:0/1781337533 learned_addr learned my addr 172.21.15.116:0/1781337533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd857e00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 0x7fd8580b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85d200640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 0x7fd858149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 -- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 msgr2=0x7fd858149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 0x7fd858149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 -- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 msgr2=0x7fd8580b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 0x7fd8580b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 -- 172.21.15.116:0/1781337533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85814a740 con 0x7fd858149ed0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd857e00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 0x7fd8580b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85d200640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 0x7fd858149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.447+0000 7fd85dc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd850009cb0 tx=0x7fd85000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 -- 172.21.15.116:0/1781337533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850018020 con 0x7fd858149ed0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 -- 172.21.15.116:0/1781337533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850014480 con 0x7fd858149ed0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd858159d00 con 0x7fd858149ed0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 -- 172.21.15.116:0/1781337533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850014870 con 0x7fd858149ed0 2024-05-14T11:24:49.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd858159740 con 0x7fd858149ed0 2024-05-14T11:24:49.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 -- 172.21.15.116:0/1781337533 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd85001b020 con 0x7fd858149ed0 2024-05-14T11:24:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 0x7fd838071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd854c00640 1 -- 172.21.15.116:0/1781337533 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fd8500900c0 con 0x7fd858149ed0 2024-05-14T11:24:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd85d200640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 0x7fd838071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.451+0000 7fd8671ac000 1 Processor -- start 2024-05-14T11:24:49.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- start start 2024-05-14T11:24:49.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85d200640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 0x7fd838071940 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd8580d2390 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185fef0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fd899f0 0x56185fef1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56185fd8e960 con 0x56185f977630 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56185fcf6200 con 0x56185fda02c0 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56185fcaeda0 con 0x56185fd899f0 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45990/0 (socket says 172.21.15.116:45990) 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 -- 172.21.15.116:0/2733964499 learned_addr learned my addr 172.21.15.116:0/2733964499 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85d200640 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185fef0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd857e00640 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fd899f0 0x56185fef1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 -- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fd899f0 msgr2=0x56185fef1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fd899f0 0x56185fef1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 -- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 msgr2=0x56185fef0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185fef0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 -- 172.21.15.116:0/2733964499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185fcb3900 con 0x56185f977630 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd85dc00640 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd850059a10 tx=0x7fd85005bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8056edb414c62138 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd849600640 1 -- 172.21.15.116:0/2733964499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850075020 con 0x56185f977630 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd849600640 1 -- 172.21.15.116:0/2733964499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850063070 con 0x56185f977630 2024-05-14T11:24:49.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd849600640 1 -- 172.21.15.116:0/2733964499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850071d80 con 0x56185f977630 2024-05-14T11:24:49.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- 172.21.15.116:0/2733964499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 msgr2=0x56185fef18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd850059a10 tx=0x7fd85005bf10 comp rx=0 tx=0).stop 2024-05-14T11:24:49.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- 172.21.15.116:0/2733964499 shutdown_connections 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185f977630 0x56185fef18c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fd899f0 0x56185fef1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 --2- 172.21.15.116:0/2733964499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185fef0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- 172.21.15.116:0/2733964499 >> 172.21.15.116:0/2733964499 conn(0x56185fdad760 msgr2=0x56185fea5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- 172.21.15.116:0/2733964499 shutdown_connections 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.455+0000 7fd8671ac000 1 -- 172.21.15.116:0/2733964499 wait complete. 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 -- start start 2024-05-14T11:24:49.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 Processor -- start 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185feacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185feb37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56185fcd0df0 con 0x56185fd899f0 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56185fcb2180 con 0x56185f977630 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56185fcf5510 con 0x56185fda02c0 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd8671ac000 10 client.0 _ll_register_callbacks cb 0x56185fda9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46004/0 (socket says 172.21.15.116:46004) 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 -- 172.21.15.116:0/1198732002 learned_addr learned my addr 172.21.15.116:0/1198732002 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd85d200640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185feacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd85dc00640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185feb37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 -- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 msgr2=0x56185feb37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185feb37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 -- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 msgr2=0x56185feacf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185feacf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 -- 172.21.15.116:0/1198732002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185fca0960 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd857e00640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ae040 tx=0x7fd84401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a37965305ff5021 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.459+0000 7fd83e000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440bb070 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fd8440abb20 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd850063070 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd8440c6050 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 10 client.7872 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x56185fcaeff0 con 0x56185fd899f0 2024-05-14T11:24:49.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 Processor -- start 2024-05-14T11:24:49.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850063070 con 0x56185fd899f0 2024-05-14T11:24:49.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fd8440c9020 con 0x56185fd899f0 2024-05-14T11:24:49.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 10 client.7872 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:49.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 10 client.7872 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:49.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 20 client.7872 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:49.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 -- start start 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f99bb80 0x56185fef7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 4 client.7872 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56185fc8eef0 con 0x56185f99bb80 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd85d200640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f99bb80 0x56185fef7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 10 client.7872 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd85d200640 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f99bb80 0x56185fef7e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936c636f94392f7e server_cookie=d30bab95fa30c454 in_seq=0 out_seq=0 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 1 -- 172.21.15.116:0/1198732002 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56185fc8eef0 con 0x56185f99bb80 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 10 client.7872 handle_command_reply: tid=0 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 client.7872 shutdown 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 msgr2=0x56185feb3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd8440ae040 tx=0x7fd84401ada0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 shutdown_connections 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f99bb80 0x56185fef7e00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185feb37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185feb3270 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 --2- 172.21.15.116:0/1198732002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185feacf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 >> 172.21.15.116:0/1198732002 conn(0x56185fdad760 msgr2=0x56185fea61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd83e000640 0 client.7872 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 shutdown_connections 2024-05-14T11:24:49.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 1 -- 172.21.15.116:0/1198732002 wait complete. 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 0x7f3c200ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c20160950 0x7f3c2006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200cf430 0x7f3c2006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3c200b5e60 con 0x7f3c20160950 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3c200b6140 con 0x7f3c20160540 2024-05-14T11:24:49.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c28800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3c200b6420 con 0x7f3c200cf430 2024-05-14T11:24:49.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7fd8671ac000 20 client.7872 trim_cache size 0 max 16384 2024-05-14T11:24:49.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c20160950 0x7f3c2006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c20160950 0x7f3c2006d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46012/0 (socket says 172.21.15.116:46012) 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 -- 172.21.15.116:0/1003298873 learned_addr learned my addr 172.21.15.116:0/1003298873 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 -- 172.21.15.116:0/1003298873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200cf430 msgr2=0x7f3c2006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 --2- 172.21.15.116:0/1003298873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200cf430 0x7f3c2006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25400640 1 -- 172.21.15.116:0/1003298873 >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 msgr2=0x7f3c200ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.463+0000 7f3c25e00640 1 --2- 172.21.15.116:0/1003298873 >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 0x7f3c200ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c25400640 1 --2- 172.21.15.116:0/1003298873 >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 0x7f3c200ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c25400640 1 -- 172.21.15.116:0/1003298873 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c200b6700 con 0x7f3c20160950 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c25e00640 1 --2- 172.21.15.116:0/1003298873 >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 0x7f3c200ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c25400640 1 --2- 172.21.15.116:0/1003298873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2006d710 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c200b5a50 tx=0x7f3c1001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e40a7a6d4b895f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c24a00640 1 -- 172.21.15.116:0/1003298873 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c100ab660 con 0x7f3c20160950 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c24a00640 1 -- 172.21.15.116:0/1003298873 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c100b0070 con 0x7f3c20160950 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.467+0000 7f3c24a00640 1 -- 172.21.15.116:0/1003298873 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c100aba20 con 0x7f3c20160950 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- 172.21.15.116:0/1003298873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 msgr2=0x7f3c2006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 --2- 172.21.15.116:0/1003298873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2006d710 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c200b5a50 tx=0x7f3c1001c540 comp rx=0 tx=0).stop 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- 172.21.15.116:0/1003298873 shutdown_connections 2024-05-14T11:24:49.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 --2- 172.21.15.116:0/1003298873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2006d710 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 --2- 172.21.15.116:0/1003298873 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c200cf430 0x7f3c2006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 --2- 172.21.15.116:0/1003298873 >> v2:172.21.15.149:3300/0 conn(0x7f3c20160540 0x7f3c200ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- 172.21.15.116:0/1003298873 >> 172.21.15.116:0/1003298873 conn(0x7f3c20072fd0 msgr2=0x7f3c200733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- 172.21.15.116:0/1003298873 shutdown_connections 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- 172.21.15.116:0/1003298873 wait complete. 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 Processor -- start 2024-05-14T11:24:49.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.471+0000 7f3c28800640 1 -- start start 2024-05-14T11:24:49.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200cf430 0x7f3c2014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c20160540 0x7f3c2014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200b5c00 con 0x7f3c20160950 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200b5a80 con 0x7f3c200cf430 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2015dca0 con 0x7f3c20160540 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2015b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46016/0 (socket says 172.21.15.116:46016) 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 -- 172.21.15.116:0/3868516444 learned_addr learned my addr 172.21.15.116:0/3868516444 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c25400640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c20160540 0x7f3c2014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c25e00640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200cf430 0x7f3c2014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 -- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c20160540 msgr2=0x7f3c2014d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3c20160540 0x7f3c2014d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 -- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200cf430 msgr2=0x7f3c2014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200cf430 0x7f3c2014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 -- 172.21.15.116:0/3868516444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2006fb40 con 0x7f3c20160950 2024-05-14T11:24:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c25e00640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c200cf430 0x7f3c2014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:49.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c26800640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c20160950 0x7f3c2015b820 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00ba10 tx=0x7f3c1c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c019070 con 0x7f3c20160950 2024-05-14T11:24:49.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c00cb40 con 0x7f3c20160950 2024-05-14T11:24:49.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c014440 con 0x7f3c20160950 2024-05-14T11:24:49.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- 172.21.15.116:0/3868516444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2015fed0 con 0x7f3c20160950 2024-05-14T11:24:49.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- 172.21.15.116:0/3868516444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2014e1b0 con 0x7f3c20160950 2024-05-14T11:24:49.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.475+0000 7f3c28800640 1 -- 172.21.15.116:0/3868516444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3c2006aa30 con 0x7f3c20160950 2024-05-14T11:24:49.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c1c01c070 con 0x7f3c20160950 2024-05-14T11:24:49.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c1a000640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3c0406f430 0x7f3c04071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f3c1c08efc0 con 0x7f3c20160950 2024-05-14T11:24:49.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c25e00640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3c0406f430 0x7f3c04071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c1a000640 1 -- 172.21.15.116:0/3868516444 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f3c1c091640 con 0x7f3c20160950 2024-05-14T11:24:49.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c28800640 1 -- 172.21.15.116:0/3868516444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f3c2015fc20 con 0x7f3c20160950 2024-05-14T11:24:49.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.479+0000 7f3c25e00640 1 --2- 172.21.15.116:0/3868516444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3c0406f430 0x7f3c04071940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400a9b0 tx=0x7f3c14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.483+0000 7fd8671ac000 1 Processor -- start 2024-05-14T11:24:49.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.483+0000 7fd8671ac000 1 -- start start 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185ff07ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185fef9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd858205760 con 0x56185fd899f0 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd85815c110 con 0x56185f977630 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd858205110 con 0x56185fda02c0 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46024/0 (socket says 172.21.15.116:46024) 2024-05-14T11:24:49.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 -- 172.21.15.116:0/183602500 learned_addr learned my addr 172.21.15.116:0/183602500 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 -- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 msgr2=0x56185fef9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185fef9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 -- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 msgr2=0x56185ff07ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185ff07ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 -- 172.21.15.116:0/183602500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185fd77970 con 0x56185fd899f0 2024-05-14T11:24:49.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd857e00640 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd8440b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7730d215ef9fbedd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd848c00640 1 -- 172.21.15.116:0/183602500 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440bb040 con 0x56185fd899f0 2024-05-14T11:24:49.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd848c00640 1 -- 172.21.15.116:0/183602500 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844000e50 con 0x56185fd899f0 2024-05-14T11:24:49.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd848c00640 1 -- 172.21.15.116:0/183602500 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440b0040 con 0x56185fd899f0 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- 172.21.15.116:0/183602500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 msgr2=0x56185ff08400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd8440b4000 comp rx=0 tx=0).stop 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- 172.21.15.116:0/183602500 shutdown_connections 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185fda02c0 0x56185fef9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff08400 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 --2- 172.21.15.116:0/183602500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185f977630 0x56185ff07ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- 172.21.15.116:0/183602500 >> 172.21.15.116:0/183602500 conn(0x56185fdad760 msgr2=0x56185feb32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- 172.21.15.116:0/183602500 shutdown_connections 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.487+0000 7fd8671ac000 1 -- 172.21.15.116:0/183602500 wait complete. 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- start start 2024-05-14T11:24:49.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 Processor -- start 2024-05-14T11:24:49.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185f977630 0x56185fefddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185ff20d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56185fd03600 con 0x56185fd899f0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56185fcfa830 con 0x56185fda02c0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56185fcab140 con 0x56185f977630 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 10 client.0 _ll_register_callbacks cb 0x7fd8500095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46034/0 (socket says 172.21.15.116:46034) 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 -- 172.21.15.116:0/741836205 learned_addr learned my addr 172.21.15.116:0/741836205 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd85d200640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185f977630 0x56185fefddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd85dc00640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185ff20d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 -- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185f977630 msgr2=0x56185fefddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185f977630 0x56185fefddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 -- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 msgr2=0x56185ff20d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185ff20d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 -- 172.21.15.116:0/741836205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185fc929e0 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd857e00640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd84401c9e0 tx=0x7fd8440a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a7e49a6c84e517 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440b0070 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fd8440abbb0 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd844000e50 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd8440bb040 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 10 client.7884 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x56185f9af470 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844000e50 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fd8440c9030 con 0x56185fd899f0 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 10 client.7884 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 10 client.7884 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 20 client.7884 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:49.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f989be0 0x56185ff64070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 4 client.7884 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-14T11:24:49.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x56185fd03e10 con 0x56185f989be0 2024-05-14T11:24:49.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.491+0000 7fd85d200640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f989be0 0x56185ff64070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85d200640 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f989be0 0x56185ff64070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbaf8d0e95f0410c server_cookie=677532c260dba471 in_seq=0 out_seq=0 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd83e000640 10 client.7884 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd83e000640 1 -- 172.21.15.116:0/741836205 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56185fd03e10 con 0x56185f989be0 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd83e000640 10 client.7884 handle_command_reply: tid=0 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 client.7884 shutdown 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 msgr2=0x56185ff20820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd84401c9e0 tx=0x7fd8440a6000 comp rx=0 tx=0).stop 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 shutdown_connections 2024-05-14T11:24:49.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x56185f989be0 0x56185ff64070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56185fda02c0 0x56185ff20d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56185fd899f0 0x56185ff20820 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 --2- 172.21.15.116:0/741836205 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56185f977630 0x56185fefddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 >> 172.21.15.116:0/741836205 conn(0x56185fdad760 msgr2=0x56185ff06eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd83e000640 0 client.7884 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 shutdown_connections 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 1 -- 172.21.15.116:0/741836205 wait complete. 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd8671ac000 20 client.7884 trim_cache size 0 max 16384 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:49.499 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:24:49.500 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "9e9066dd-5c7b-4b4a-bd81-7a26277c8e43", 2024-05-14T11:24:49.500 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:24:49.500 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:49.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 msgr2=0x7fd838071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 0x7fd838071940 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd8580d2390 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 msgr2=0x7fd8580b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd850009cb0 tx=0x7fd85000c490 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 shutdown_connections 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd83806f430 0x7fd838071940 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd85815d250 0x7fd8580b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd858149ed0 0x7fd8580b2f80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 --2- 172.21.15.116:0/1781337533 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd8580ceec0 0x7fd858149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 >> 172.21.15.116:0/1781337533 conn(0x7fd8580d24b0 msgr2=0x7fd85806cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 shutdown_connections 2024-05-14T11:24:49.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.495+0000 7fd85fc00640 1 -- 172.21.15.116:0/1781337533 wait complete. 2024-05-14T11:24:49.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:49.516 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-14T11:24:49.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.663+0000 7fbb32400640 1 Processor -- start 2024-05-14T11:24:49.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.663+0000 7fbb32400640 1 -- start start 2024-05-14T11:24:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb32400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb2c0d1670 0x7fbb2c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb32400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c06c700 0x7fbb2c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb32400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2c06ff20 con 0x7fbb2c15db30 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb32400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2c0ae100 con 0x7fbb2c0d1670 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb32400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c0ae870 con 0x7fbb2c06c700 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb2c0d1670 0x7fbb2c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbb2c15db30 0x7fbb2c0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50394/0 (socket says 172.21.15.116:50394) 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2b400640 1 -- 172.21.15.116:0/1251290453 learned_addr learned my addr 172.21.15.116:0/1251290453 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2be00640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c06c700 0x7fbb2c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2aa00640 1 -- 172.21.15.116:0/1251290453 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c06c700 msgr2=0x7fbb2c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c06c700 0x7fbb2c06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2aa00640 1 -- 172.21.15.116:0/1251290453 >> v1:172.21.15.116:6789/0 conn(0x7fbb2c15db30 legacy=0x7fbb2c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2aa00640 1 -- 172.21.15.116:0/1251290453 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c0d1fc0 con 0x7fbb2c0d1670 2024-05-14T11:24:49.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.667+0000 7fbb2a000640 1 -- 172.21.15.116:0/1251290453 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423023629 0 0) 0x7fbb2c06ff20 con 0x7fbb2c15db30 2024-05-14T11:24:49.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c0d1670 0x7fbb2c0d1a80 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097c0 tx=0x7fbb1801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9abb9d2ebbe71d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb2a000640 1 -- 172.21.15.116:0/1251290453 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180b0070 con 0x7fbb2c0d1670 2024-05-14T11:24:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb2a000640 1 -- 172.21.15.116:0/1251290453 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb180ab600 con 0x7fbb2c0d1670 2024-05-14T11:24:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb2a000640 1 -- 172.21.15.116:0/1251290453 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180ab9d0 con 0x7fbb2c0d1670 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 -- 172.21.15.116:0/1251290453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c0d1670 msgr2=0x7fbb2c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c0d1670 0x7fbb2c0d1a80 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097c0 tx=0x7fbb1801c340 comp rx=0 tx=0).stop 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 -- 172.21.15.116:0/1251290453 shutdown_connections 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c06c700 0x7fbb2c06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 --2- 172.21.15.116:0/1251290453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c0d1670 0x7fbb2c0d1a80 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 -- 172.21.15.116:0/1251290453 >> 172.21.15.116:0/1251290453 conn(0x7fbb2c0b4620 msgr2=0x7fbb2c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 -- 172.21.15.116:0/1251290453 shutdown_connections 2024-05-14T11:24:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.671+0000 7fbb32400640 1 -- 172.21.15.116:0/1251290453 wait complete. 2024-05-14T11:24:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 Processor -- start 2024-05-14T11:24:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- start start 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 0x7fbb2c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 0x7fbb2c0740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c16a770 con 0x7fbb2c06c700 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c15df40 con 0x7fbb2c15db30 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c16a8f0 con 0x7fbb2c0d1670 2024-05-14T11:24:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46048/0 (socket says 172.21.15.116:46048) 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 -- 172.21.15.116:0/735686802 learned_addr learned my addr 172.21.15.116:0/735686802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 0x7fbb2c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2be00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 0x7fbb2c0740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 -- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 msgr2=0x7fbb2c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 0x7fbb2c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 -- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 msgr2=0x7fbb2c0740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 0x7fbb2c0740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 -- 172.21.15.116:0/735686802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c14eb80 con 0x7fbb2c06c700 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2be00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 0x7fbb2c0740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 0x7fbb2c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb2b400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00c770 tx=0x7fbb1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb23400640 1 -- 172.21.15.116:0/735686802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c018070 con 0x7fbb2c06c700 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb23400640 1 -- 172.21.15.116:0/735686802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00a640 con 0x7fbb2c06c700 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb23400640 1 -- 172.21.15.116:0/735686802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00aa30 con 0x7fbb2c06c700 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c15f8e0 con 0x7fbb2c06c700 2024-05-14T11:24:49.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.675+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c1569e0 con 0x7fbb2c06c700 2024-05-14T11:24:49.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.679+0000 7fbb23400640 1 -- 172.21.15.116:0/735686802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbb1c00abd0 con 0x7fbb2c06c700 2024-05-14T11:24:49.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.679+0000 7fbb23400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 0x7fbb0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.679+0000 7fbb23400640 1 -- 172.21.15.116:0/735686802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fbb1c08ea40 con 0x7fbb2c06c700 2024-05-14T11:24:49.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.679+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 0x7fbb0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.679+0000 7fbb399f3000 1 Processor -- start 2024-05-14T11:24:49.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- start start 2024-05-14T11:24:49.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 0x7fbb0c071910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb180ad000 tx=0x7fbb18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138c5f00 0x562c13a3f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13a3fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c13844100 con 0x562c134c53e0 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562c137fcca0 con 0x562c138c5f00 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562c138400c0 con 0x562c138d8120 2024-05-14T11:24:49.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13a3fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138c5f00 0x562c13a3f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:36110/0 (socket says 172.21.15.116:36110) 2024-05-14T11:24:49.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 -- 172.21.15.116:0/238314638 learned_addr learned my addr 172.21.15.116:0/238314638 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 -- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138c5f00 msgr2=0x562c13a3f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138c5f00 0x562c13a3f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 -- 172.21.15.116:0/238314638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 msgr2=0x562c13a3fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13a3fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 -- 172.21.15.116:0/238314638 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c13801800 con 0x562c138d8120 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb2b400640 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c05af90 tx=0x7fbb1c05a830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb86482d4bc3a7bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb17e00640 1 -- 172.21.15.116:0/238314638 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c05f3b0 con 0x562c138d8120 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb17e00640 1 -- 172.21.15.116:0/238314638 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c063e60 con 0x562c138d8120 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb17e00640 1 -- 172.21.15.116:0/238314638 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c05a460 con 0x562c138d8120 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 msgr2=0x562c13a3f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c05af90 tx=0x7fbb1c05a830 comp rx=0 tx=0).stop 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- 172.21.15.116:0/238314638 shutdown_connections 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138c5f00 0x562c13a3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13a3fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 --2- 172.21.15.116:0/238314638 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a3f350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- 172.21.15.116:0/238314638 >> 172.21.15.116:0/238314638 conn(0x562c13505400 msgr2=0x562c13a22310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- 172.21.15.116:0/238314638 shutdown_connections 2024-05-14T11:24:49.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.683+0000 7fbb399f3000 1 -- 172.21.15.116:0/238314638 wait complete. 2024-05-14T11:24:49.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 -- start start 2024-05-14T11:24:49.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 Processor -- start 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c134c53e0 0x562c13a45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c139f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c1390f2a0 con 0x562c138c5f00 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562c138fff40 con 0x562c138d8120 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562c13800080 con 0x562c134c53e0 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 10 client.0 _ll_register_callbacks cb 0x562c139e0c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46076/0 (socket says 172.21.15.116:46076) 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 -- 172.21.15.116:0/2783462598 learned_addr learned my addr 172.21.15.116:0/2783462598 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2be00640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c139f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c134c53e0 0x562c13a45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 -- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c134c53e0 msgr2=0x562c13a45dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c134c53e0 0x562c13a45dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 -- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 msgr2=0x562c139f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c139f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 -- 172.21.15.116:0/2783462598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c137ee860 con 0x562c138c5f00 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2be00640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c139f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fbb18008010 tx=0x7fbb180046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3706e1c56d860f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb14c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180b0040 con 0x562c138c5f00 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fbb180b6b70 con 0x562c138c5f00 2024-05-14T11:24:49.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.687+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c063e60 con 0x562c138c5f00 2024-05-14T11:24:49.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbb180cf050 con 0x562c138c5f00 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 10 client.7899 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x562c137fcef0 con 0x562c138c5f00 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c063e60 con 0x562c138c5f00 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fbb180d2020 con 0x562c138c5f00 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 10 client.7899 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 10 client.7899 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 20 client.7899 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134e9930 0x562c139f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 4 client.7899 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c137dcdf0 con 0x562c134e9930 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134e9930 0x562c139f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134e9930 0x562c139f9d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452465191c4251bf server_cookie=7189316e85f59225 in_seq=0 out_seq=0 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 10 client.7899 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 1 -- 172.21.15.116:0/2783462598 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c137dcdf0 con 0x562c134e9930 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 10 client.7899 handle_command_reply: tid=0 2024-05-14T11:24:49.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 client.7899 shutdown 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 msgr2=0x562c13a46310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fbb18008010 tx=0x7fbb180046d0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 shutdown_connections 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134e9930 0x562c139f9d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c139f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a46310 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb14c00640 0 client.7899 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2783462598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c134c53e0 0x562c13a45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 >> 172.21.15.116:0/2783462598 conn(0x562c13505400 msgr2=0x562c13a22200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 shutdown_connections 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 1 -- 172.21.15.116:0/2783462598 wait complete. 2024-05-14T11:24:49.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.691+0000 7fbb399f3000 20 client.7899 trim_cache size 0 max 16384 2024-05-14T11:24:49.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.707+0000 7fbb399f3000 1 Processor -- start 2024-05-14T11:24:49.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- start start 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c134c53e0 0x562c13a4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a48140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c205170 con 0x562c138c5f00 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c155f30 con 0x562c134c53e0 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c156580 con 0x562c138d8120 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46088/0 (socket says 172.21.15.116:46088) 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 -- 172.21.15.116:0/4107703040 learned_addr learned my addr 172.21.15.116:0/4107703040 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2be00640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a48140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2b400640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c134c53e0 0x562c13a4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 -- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 msgr2=0x562c13a48140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a48140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 -- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c134c53e0 msgr2=0x562c13a4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c134c53e0 0x562c13a4b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 -- 172.21.15.116:0/4107703040 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c139f7070 con 0x562c138c5f00 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2be00640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a48140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbb18097d00 tx=0x7fbb18009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e656dfd089a8222d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb17400640 1 -- 172.21.15.116:0/4107703040 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180d7070 con 0x562c138c5f00 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb17400640 1 -- 172.21.15.116:0/4107703040 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb180c7070 con 0x562c138c5f00 2024-05-14T11:24:49.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb17400640 1 -- 172.21.15.116:0/4107703040 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180b0040 con 0x562c138c5f00 2024-05-14T11:24:49.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- 172.21.15.116:0/4107703040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 msgr2=0x562c13a47c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbb18097d00 tx=0x7fbb18009190 comp rx=0 tx=0).stop 2024-05-14T11:24:49.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- 172.21.15.116:0/4107703040 shutdown_connections 2024-05-14T11:24:49.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138d8120 0x562c13a48140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c138c5f00 0x562c13a47c00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 --2- 172.21.15.116:0/4107703040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c134c53e0 0x562c13a4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- 172.21.15.116:0/4107703040 >> 172.21.15.116:0/4107703040 conn(0x562c13505400 msgr2=0x562c13a20d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- 172.21.15.116:0/4107703040 shutdown_connections 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.711+0000 7fbb399f3000 1 -- 172.21.15.116:0/4107703040 wait complete. 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- start start 2024-05-14T11:24:49.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 Processor -- start 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138c5f00 0x562c13ab2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c13a49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c13848730 con 0x562c134c53e0 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562c13843410 con 0x562c138d8120 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562c13851500 con 0x562c138c5f00 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 10 client.0 _ll_register_callbacks cb 0x7fbb24009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46096/0 (socket says 172.21.15.116:46096) 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 -- 172.21.15.116:0/2095675359 learned_addr learned my addr 172.21.15.116:0/2095675359 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2be00640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c13a49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138c5f00 0x562c13ab2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 -- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138c5f00 msgr2=0x562c13ab2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138c5f00 0x562c13ab2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 -- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 msgr2=0x562c13a49790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c13a49790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 -- 172.21.15.116:0/2095675359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c138c5340 con 0x562c134c53e0 2024-05-14T11:24:49.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb2b400640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c058270 tx=0x7fbb1c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244da517b9ed67d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c057650 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fbb1c05a460 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb180c7070 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c063e60 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 10 client.7905 fetch_fsmap learned FSMap version 83 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x562c137e08e0 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb180c7070 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fbb1c056970 con 0x562c134c53e0 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 10 client.7905 fetch_fsmap finished waiting for FSMap version 83 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 10 client.7905 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 20 client.7905 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134d7990 0x562c13a4a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 4 client.7905 mds_command: new command op to 7638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.715+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 --> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c13851d10 con 0x562c134d7990 2024-05-14T11:24:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134d7990 0x562c13a4a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb2aa00640 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134d7990 0x562c13a4a120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69ad62ee45e6d06 server_cookie=f813a54c4347be37 in_seq=0 out_seq=0 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb14c00640 10 client.7905 ms_handle_connect on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb14c00640 1 -- 172.21.15.116:0/2095675359 <== mds.0 v2:172.21.15.116:6834/3421632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562c13851d10 con 0x562c134d7990 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb14c00640 10 client.7905 handle_command_reply: tid=0 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 client.7905 shutdown 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 msgr2=0x562c13ab1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c058270 tx=0x7fbb1c001260 comp rx=0 tx=0).stop 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 shutdown_connections 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:6834/3421632,v1:172.21.15.116:6835/3421632] conn(0x562c134d7990 0x562c13a4a120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c138d8120 0x562c13a49790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562c138c5f00 0x562c13ab2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb14c00640 0 client.7905 ms_handle_reset on v2:172.21.15.116:6834/3421632 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 --2- 172.21.15.116:0/2095675359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c134c53e0 0x562c13ab1d40 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 >> 172.21.15.116:0/2095675359 conn(0x562c13505400 msgr2=0x562c13a20300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 shutdown_connections 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 1 -- 172.21.15.116:0/2095675359 wait complete. 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb399f3000 20 client.7905 trim_cache size 0 max 16384 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:24:49.723 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 msgr2=0x7fbb0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 0x7fbb0c071910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb180ad000 tx=0x7fbb18008040 comp rx=0 tx=0).stop 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 msgr2=0x7fbb2c15ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00c770 tx=0x7fbb1c00cc30 comp rx=0 tx=0).stop 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 shutdown_connections 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbb0c06f400 0x7fbb0c071910 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb2c15db30 0x7fbb2c0740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb2c0d1670 0x7fbb2c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 --2- 172.21.15.116:0/735686802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb2c06c700 0x7fbb2c15ef20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 >> 172.21.15.116:0/735686802 conn(0x7fbb2c0b4620 msgr2=0x7fbb2c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 shutdown_connections 2024-05-14T11:24:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.719+0000 7fbb32400640 1 -- 172.21.15.116:0/735686802 wait complete. 2024-05-14T11:24:49.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:24:49.840 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:24:49.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 Processor -- start 2024-05-14T11:24:49.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 -- start start 2024-05-14T11:24:49.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f440b6dc0 0x7f0f4415c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f44070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4406edb0 con 0x7f0f4406e7e0 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4406f090 con 0x7f0f440b6dc0 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f4b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4406f370 con 0x7f0f4406e3d0 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f440b6dc0 0x7f0f4415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f440b6dc0 0x7f0f4415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46106/0 (socket says 172.21.15.116:46106) 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 -- 172.21.15.116:0/214355324 learned_addr learned my addr 172.21.15.116:0/214355324 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49400640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f44070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 -- 172.21.15.116:0/214355324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 msgr2=0x7f0f44070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f44070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 -- 172.21.15.116:0/214355324 >> v1:172.21.15.149:6789/0 conn(0x7f0f4406e7e0 legacy=0x7f0f440d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.987+0000 7f0f49e00640 1 -- 172.21.15.116:0/214355324 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4415cd90 con 0x7f0f440b6dc0 2024-05-14T11:24:49.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f49e00640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f440b6dc0 0x7f0f4415c850 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009870 tx=0x7f0f3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738d0e3d15111367 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:49.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f48a00640 1 -- 172.21.15.116:0/214355324 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380aa650 con 0x7f0f440b6dc0 2024-05-14T11:24:49.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f48a00640 1 -- 172.21.15.116:0/214355324 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f380af070 con 0x7f0f440b6dc0 2024-05-14T11:24:49.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f48a00640 1 -- 172.21.15.116:0/214355324 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380aaa10 con 0x7f0f440b6dc0 2024-05-14T11:24:49.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 -- 172.21.15.116:0/214355324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f440b6dc0 msgr2=0x7f0f4415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:49.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f440b6dc0 0x7f0f4415c850 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009870 tx=0x7f0f3801c4d0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 -- 172.21.15.116:0/214355324 shutdown_connections 2024-05-14T11:24:49.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f44070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 --2- 172.21.15.116:0/214355324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f440b6dc0 0x7f0f4415c850 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:49.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 -- 172.21.15.116:0/214355324 >> 172.21.15.116:0/214355324 conn(0x7f0f440cf440 msgr2=0x7f0f440cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:49.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 -- 172.21.15.116:0/214355324 shutdown_connections 2024-05-14T11:24:49.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.991+0000 7f0f4b200640 1 -- 172.21.15.116:0/214355324 wait complete. 2024-05-14T11:24:49.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 Processor -- start 2024-05-14T11:24:49.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 -- start start 2024-05-14T11:24:49.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f440cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 0x7f0f44155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f440b6dc0 0x7f0f440b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44168230 con 0x7f0f4406e7e0 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0f441680b0 con 0x7f0f440b6dc0 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f4b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0f441683b0 con 0x7f0f4406e3d0 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f49400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 0x7f0f44155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f49400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 0x7f0f44155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46122/0 (socket says 172.21.15.116:46122) 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f440cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.995+0000 7f0f49400640 1 -- 172.21.15.116:0/309302277 learned_addr learned my addr 172.21.15.116:0/309302277 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f4a800640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f440b6dc0 0x7f0f440b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 -- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f440b6dc0 msgr2=0x7f0f440b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f440b6dc0 0x7f0f440b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 -- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 msgr2=0x7f0f44155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 0x7f0f44155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 -- 172.21.15.116:0/309302277 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f44158020 con 0x7f0f4406e3d0 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f49e00640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f440cdbf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f380ab000 tx=0x7f0f3801ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380b3800 con 0x7f0f4406e3d0 2024-05-14T11:24:50.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f380a6040 con 0x7f0f4406e3d0 2024-05-14T11:24:50.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f44158300 con 0x7f0f4406e3d0 2024-05-14T11:24:50.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f380b3be0 con 0x7f0f4406e3d0 2024-05-14T11:24:50.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f441588c0 con 0x7f0f4406e3d0 2024-05-14T11:24:50.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f44050880 con 0x7f0f4406e3d0 2024-05-14T11:24:50.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:49.999+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f380b2380 con 0x7f0f4406e3d0 2024-05-14T11:24:50.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.003+0000 7f0f3e000640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 0x7f0f28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.003+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f0f381296a0 con 0x7f0f4406e3d0 2024-05-14T11:24:50.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.003+0000 7f0f49400640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 0x7f0f28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.003+0000 7f0f49400640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 0x7f0f28071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f44153230 tx=0x7f0f34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.007+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0f380c3060 con 0x7f0f4406e3d0 2024-05-14T11:24:50.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.195+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0f4415cb70 con 0x7f0f4406e3d0 2024-05-14T11:24:50.198 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:24:50.195+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:24:50.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.203+0000 7f0f3e000640 1 -- 172.21.15.116:0/309302277 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7f0f38016150 con 0x7f0f4406e3d0 2024-05-14T11:24:50.207 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 msgr2=0x7f0f28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 0x7f0f28071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f44153230 tx=0x7f0f34008040 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 msgr2=0x7f0f440cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f440cdbf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f380ab000 tx=0x7f0f3801ad10 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 shutdown_connections 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0f2806f400 0x7f0f28071910 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f440b6dc0 0x7f0f440b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f4406e7e0 0x7f0f44155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 --2- 172.21.15.116:0/309302277 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0f4406e3d0 0x7f0f440cdbf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 >> 172.21.15.116:0/309302277 conn(0x7f0f440cf440 msgr2=0x7f0f440d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 shutdown_connections 2024-05-14T11:24:50.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.207+0000 7f0f4b200640 1 -- 172.21.15.116:0/309302277 wait complete. 2024-05-14T11:24:50.212 INFO:tasks.ceph.mds.a.smithi116.stderr: -4882> 2024-05-14T11:24:35.431+0000 7f07b0c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:24:50.230 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-14T11:24:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- start start 2024-05-14T11:24:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 Processor -- start 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 0x55ce7f02c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e322400 0x55ce7f02cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce7f03c600 con 0x55ce7e321680 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce7f03c480 con 0x55ce7e322400 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce7f03c300 con 0x55ce7e321b00 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46140/0 (socket says 172.21.15.116:46140) 2024-05-14T11:24:50.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 -- 172.21.15.116:0/2644154753 learned_addr learned my addr 172.21.15.116:0/2644154753 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa511600640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 0x55ce7f02c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512a00640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e322400 0x55ce7f02cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 -- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 msgr2=0x55ce7f02c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 0x55ce7f02c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 -- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e322400 msgr2=0x55ce7f02cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e322400 0x55ce7f02cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 -- 172.21.15.116:0/2644154753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7f026960 con 0x55ce7e321680 2024-05-14T11:24:50.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa511600640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 0x55ce7f02c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa512000640 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x55ce7f0b58c0 tx=0x55ce7e364600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca49afe32e51609a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa50ee00640 1 -- 172.21.15.116:0/2644154753 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e2804e0 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa50ee00640 1 -- 172.21.15.116:0/2644154753 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce7e294540 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce7f026d20 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ce7f026f00 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 4 waiting for MDS map... 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa50ee00640 1 -- 172.21.15.116:0/2644154753 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e280b60 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa50ee00640 1 -- 172.21.15.116:0/2644154753 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x55ce7f176600 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa50ee00640 1 -- 172.21.15.116:0/2644154753 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x55ce7e2a1180 con 0x55ce7e321680 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.291+0000 7fa513f08400 4 Got MDS map 84 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 10 main: JournalTool::main 2024-05-14T11:24:50.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 4 main: JournalTool: connecting to RADOS... 2024-05-14T11:24:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 Processor -- start 2024-05-14T11:24:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 -- start start 2024-05-14T11:24:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce7f03c180 con 0x55ce7e320480 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce7e2aa480 con 0x55ce7e320900 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce7e2aa900 con 0x55ce7e320d80 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46144/0 (socket says 172.21.15.116:46144) 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 -- 172.21.15.116:0/2162150185 learned_addr learned my addr 172.21.15.116:0/2162150185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa512000640 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa512a00640 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 -- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320d80 msgr2=0x55ce7f02d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 -- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320900 msgr2=0x55ce7f02d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 -- 172.21.15.116:0/2162150185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7f026d20 con 0x55ce7e320480 2024-05-14T11:24:50.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa511600640 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55ce7f0b4f90 tx=0x55ce7e3648a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2767eaea91409301 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa50c600640 1 -- 172.21.15.116:0/2162150185 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e2809c0 con 0x55ce7e320480 2024-05-14T11:24:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa50c600640 1 -- 172.21.15.116:0/2162150185 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce7e294a80 con 0x55ce7e320480 2024-05-14T11:24:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.295+0000 7fa50c600640 1 -- 172.21.15.116:0/2162150185 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e2804e0 con 0x55ce7e320480 2024-05-14T11:24:50.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- 172.21.15.116:0/2162150185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 msgr2=0x55ce7f02db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55ce7f0b4f90 tx=0x55ce7e3648a0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- 172.21.15.116:0/2162150185 shutdown_connections 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- 172.21.15.116:0/2162150185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- 172.21.15.116:0/2162150185 >> 172.21.15.116:0/2162150185 conn(0x55ce7e321200 msgr2=0x55ce7a246800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- 172.21.15.116:0/2162150185 shutdown_connections 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- 172.21.15.116:0/2162150185 wait complete. 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 Processor -- start 2024-05-14T11:24:50.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- start start 2024-05-14T11:24:50.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce7f03d200 con 0x55ce7e320d80 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce7f03c180 con 0x55ce7e320480 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa513f08400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ce7f03d380 con 0x55ce7e320900 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa511600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa511600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 0x55ce7f02d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46154/0 (socket says 172.21.15.116:46154) 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 learned_addr learned my addr 172.21.15.116:0/1320535973 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.299+0000 7fa512000640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320900 msgr2=0x55ce7f02d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e320900 0x55ce7f02d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320480 msgr2=0x55ce7f02db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7f027a40 con 0x55ce7e320d80 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e320480 0x55ce7f02db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa511600640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 0x55ce7f02d080 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55ce7f164d20 tx=0x55ce7e364960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 -- 172.21.15.116:0/1320535973 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e280b60 con 0x55ce7e320d80 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 -- 172.21.15.116:0/1320535973 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce7e295340 con 0x55ce7e320d80 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce7f027c20 con 0x55ce7e320d80 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 -- 172.21.15.116:0/1320535973 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce7e280000 con 0x55ce7e320d80 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce7f0261e0 con 0x55ce7e320d80 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 4 main: JournalTool: resolving pool 15 2024-05-14T11:24:50.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 -- 172.21.15.116:0/1320535973 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55ce7e58ac00 con 0x55ce7e320d80 2024-05-14T11:24:50.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ce7e323180 0x55ce7f02e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa50a800640 1 -- 172.21.15.116:0/1320535973 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x55ce7e58e600 con 0x55ce7e320d80 2024-05-14T11:24:50.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 4 main: JournalTool: creating IoCtx.. 2024-05-14T11:24:50.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 4 main: Executing for rank 0 2024-05-14T11:24:50.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 4 Filtering by inode '1099511627777' 2024-05-14T11:24:50.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ce7e320000 0x55ce7f02e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ce7e323180 0x55ce7f02e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.303+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ce7e68c000 con 0x55ce7e320000 2024-05-14T11:24:50.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512000640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ce7e320000 0x55ce7f02e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ce7e323180 0x55ce7f02e100 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55ce7e39ade0 tx=0x55ce7f185f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512000640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ce7e320000 0x55ce7f02e680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512000640 1 -- 172.21.15.116:0/1320535973 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55ce7e34a000 con 0x55ce7e320000 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 4 Pointer 400.00000000 is readable 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ce7e68c400 con 0x55ce7e320000 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512000640 1 -- 172.21.15.116:0/1320535973 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55ce7e34a000 con 0x55ce7e320000 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Header 0x400000 0x405f0e 0x40628a 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Starting journal scan from offset 0x405f0e 2024-05-14T11:24:50.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ce7e68d000 con 0x55ce7e320000 2024-05-14T11:24:50.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512000640 1 -- 172.21.15.116:0/1320535973 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55ce7e34a000 con 0x55ce7e320000 2024-05-14T11:24:50.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Parsing data, 0x37c bytes available 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Attempting decode at 0x405f0e 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Consumed 0x37c bytes 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 10 Valid entry at 0x405f0e 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ce7e322880 0x55ce7f02ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ce7e68d400 con 0x55ce7e322880 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ce7e322880 0x55ce7f02ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.307+0000 7fa512a00640 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ce7e322880 0x55ce7f02ec00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa512a00640 1 -- 172.21.15.116:0/1320535973 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ce7e34a500 con 0x55ce7e322880 2024-05-14T11:24:50.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 Reached end of journal objects 2024-05-14T11:24:50.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 Scanned objects, 0 missing, 1 valid 2024-05-14T11:24:50.312 INFO:teuthology.orchestra.run.smithi116.stdout:Events by type: 2024-05-14T11:24:50.312 INFO:teuthology.orchestra.run.smithi116.stdout:Errors: 0 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 Events scanned, 0 gaps 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 Found 1 valid events 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 Selected 0 events events for processing 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 4 0 events 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ce7e320000 msgr2=0x55ce7f02e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ce7e320000 0x55ce7f02e680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ce7e322880 msgr2=0x55ce7f02ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ce7e322880 0x55ce7f02ec00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ce7e323180 msgr2=0x55ce7f02e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ce7e323180 0x55ce7f02e100 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55ce7e39ade0 tx=0x55ce7f185f80 comp rx=0 tx=0).stop 2024-05-14T11:24:50.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 msgr2=0x55ce7f02d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/1320535973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e320d80 0x55ce7f02d080 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55ce7f164d20 tx=0x55ce7e364960 comp rx=0 tx=0).stop 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 reap_dead start 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa511600640 1 -- 172.21.15.116:0/1320535973 reap_dead start 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 shutdown_connections 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 >> 172.21.15.116:0/1320535973 conn(0x55ce7e321200 msgr2=0x55ce7a246800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 shutdown_connections 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/1320535973 wait complete. 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 msgr2=0x55ce7f02c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x55ce7f0b58c0 tx=0x55ce7e364600 comp rx=0 tx=0).stop 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 shutdown_connections 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce7e322400 0x55ce7f02cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ce7e321b00 0x55ce7f02c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 --2- 172.21.15.116:0/2644154753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce7e321680 0x55ce7f02c000 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 >> 172.21.15.116:0/2644154753 conn(0x55ce7e321f80 msgr2=0x55ce7a247800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 shutdown_connections 2024-05-14T11:24:50.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.311+0000 7fa513f08400 1 -- 172.21.15.116:0/2644154753 wait complete. 2024-05-14T11:24:50.319 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.089608, result: Events by type: Errors: 0 2024-05-14T11:24:50.319 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-14T11:24:50.337 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:24:50.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- start start 2024-05-14T11:24:50.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 Processor -- start 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50091680 0x55ea50d98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 0x55ea50d98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc780 con 0x55ea50091b00 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc600 con 0x55ea50091680 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ea50dbc480 con 0x55ea50092400 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46174/0 (socket says 172.21.15.116:46174) 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 -- 172.21.15.116:0/2584933634 learned_addr learned my addr 172.21.15.116:0/2584933634 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939e00640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50091680 0x55ea50d98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93a800640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 0x55ea50d98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 -- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 msgr2=0x55ea50d98b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 0x55ea50d98b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 -- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50091680 msgr2=0x55ea50d98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50091680 0x55ea50d98580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 -- 172.21.15.116:0/2584933634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea50d96960 con 0x55ea50091b00 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93a800640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 0x55ea50d98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa939400640 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x55ea50e31890 tx=0x55ea500d44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341f22e8ffde0b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa936c00640 1 -- 172.21.15.116:0/2584933634 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff04e0 con 0x55ea50091b00 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa936c00640 1 -- 172.21.15.116:0/2584933634 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea500041c0 con 0x55ea50091b00 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa936c00640 1 -- 172.21.15.116:0/2584933634 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff0b60 con 0x55ea50091b00 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea50d96d20 con 0x55ea50091b00 2024-05-14T11:24:50.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ea50d96f00 con 0x55ea50091b00 2024-05-14T11:24:50.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa93cc2c400 4 waiting for MDS map... 2024-05-14T11:24:50.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.379+0000 7fa936c00640 1 -- 172.21.15.116:0/2584933634 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x55ea50ee2600 con 0x55ea50091b00 2024-05-14T11:24:50.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa936c00640 1 -- 172.21.15.116:0/2584933634 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x55ea50011180 con 0x55ea50091b00 2024-05-14T11:24:50.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 4 Got MDS map 84 2024-05-14T11:24:50.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 10 main: JournalTool::main 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 4 main: JournalTool: connecting to RADOS... 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 Processor -- start 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 -- start start 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc300 con 0x55ea50090d80 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc000 con 0x55ea50090480 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ea5001a900 con 0x55ea50090900 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46180/0 (socket says 172.21.15.116:46180) 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 -- 172.21.15.116:0/3850166717 learned_addr learned my addr 172.21.15.116:0/3850166717 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 -- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090900 msgr2=0x55ea50d99600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 -- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090480 msgr2=0x55ea50d99b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 -- 172.21.15.116:0/3850166717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea50d96d20 con 0x55ea50090d80 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa93a800640 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef0c90 tx=0x55ea50ef8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9730630d77028b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa934400640 1 -- 172.21.15.116:0/3850166717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff04e0 con 0x55ea50090d80 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa934400640 1 -- 172.21.15.116:0/3850166717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea50004e00 con 0x55ea50090d80 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.383+0000 7fa934400640 1 -- 172.21.15.116:0/3850166717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff0b60 con 0x55ea50090d80 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- 172.21.15.116:0/3850166717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 msgr2=0x55ea50d99080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef0c90 tx=0x55ea50ef8c00 comp rx=0 tx=0).stop 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- 172.21.15.116:0/3850166717 shutdown_connections 2024-05-14T11:24:50.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/3850166717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- 172.21.15.116:0/3850166717 >> 172.21.15.116:0/3850166717 conn(0x55ea50091200 msgr2=0x55ea502e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- 172.21.15.116:0/3850166717 shutdown_connections 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- 172.21.15.116:0/3850166717 wait complete. 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 Processor -- start 2024-05-14T11:24:50.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- start start 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc300 con 0x55ea50090d80 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea50dbc000 con 0x55ea50090900 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93cc2c400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55ea50dbd500 con 0x55ea50090480 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46196/0 (socket says 172.21.15.116:46196) 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 -- 172.21.15.116:0/316378236 learned_addr learned my addr 172.21.15.116:0/316378236 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939e00640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090480 msgr2=0x55ea50d99b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090900 msgr2=0x55ea50d99600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50090900 0x55ea50d99600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea50d97c20 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50090480 0x55ea50d99b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa939400640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef1950 tx=0x55ea500d4900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa932600640 1 -- 172.21.15.116:0/316378236 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff0340 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa932600640 1 -- 172.21.15.116:0/316378236 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea50004a80 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.387+0000 7fa932600640 1 -- 172.21.15.116:0/316378236 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea4fff04e0 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea50d97e00 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea503081e0 con 0x55ea50090d80 2024-05-14T11:24:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 4 main: JournalTool: resolving pool 15 2024-05-14T11:24:50.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa932600640 1 -- 172.21.15.116:0/316378236 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55ea50f16c00 con 0x55ea50090d80 2024-05-14T11:24:50.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa932600640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ea50093180 0x55ea50d9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa932600640 1 -- 172.21.15.116:0/316378236 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x55ea50304600 con 0x55ea50090d80 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 4 main: JournalTool: creating IoCtx.. 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 4 main: Executing for rank 0 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 4 Filtering by inode '1099511627778' 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ea50090000 0x55ea50d9a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ea50093180 0x55ea50d9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea502f7800 con 0x55ea50090000 2024-05-14T11:24:50.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa939e00640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ea50090000 0x55ea50d9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ea50093180 0x55ea50d9a100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef0c90 tx=0x55ea500d4540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.391+0000 7fa939e00640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ea50090000 0x55ea50d9a680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa939e00640 1 -- 172.21.15.116:0/316378236 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55ea500ba000 con 0x55ea50090000 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Pointer 400.00000000 is readable 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea502f7c00 con 0x55ea50090000 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa939e00640 1 -- 172.21.15.116:0/316378236 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55ea500ba000 con 0x55ea50090000 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Header 0x400000 0x405f0e 0x40628a 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Starting journal scan from offset 0x405f0e 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea4bf8bc00 con 0x55ea50090000 2024-05-14T11:24:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa939e00640 1 -- 172.21.15.116:0/316378236 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55ea500ba000 con 0x55ea50090000 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Parsing data, 0x37c bytes available 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Attempting decode at 0x405f0e 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Consumed 0x37c bytes 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 10 Valid entry at 0x405f0e 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ea50092880 0x55ea50d9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea4bf8b400 con 0x55ea50092880 2024-05-14T11:24:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ea50092880 0x55ea50d9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93a800640 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ea50092880 0x55ea50d9ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93a800640 1 -- 172.21.15.116:0/316378236 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ea500ba500 con 0x55ea50092880 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Reached end of journal objects 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Scanned objects, 0 missing, 1 valid 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stdout:Events by type: 2024-05-14T11:24:50.398 INFO:teuthology.orchestra.run.smithi116.stdout:Errors: 0 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Events scanned, 0 gaps 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Found 1 valid events 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 Selected 0 events events for processing 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 4 0 events 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ea50090000 msgr2=0x55ea50d9a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55ea50090000 0x55ea50d9a680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ea50092880 msgr2=0x55ea50d9ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55ea50092880 0x55ea50d9ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ea50093180 msgr2=0x55ea50d9a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55ea50093180 0x55ea50d9a100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef0c90 tx=0x55ea500d4540 comp rx=0 tx=0).stop 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 msgr2=0x55ea50d99080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/316378236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50090d80 0x55ea50d99080 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x55ea50ef1950 tx=0x55ea500d4900 comp rx=0 tx=0).stop 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa939e00640 1 -- 172.21.15.116:0/316378236 reap_dead start 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 shutdown_connections 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 >> 172.21.15.116:0/316378236 conn(0x55ea50091200 msgr2=0x55ea502e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 shutdown_connections 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/316378236 wait complete. 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 msgr2=0x55ea50d98000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x55ea50e31890 tx=0x55ea500d44e0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 shutdown_connections 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55ea50092400 0x55ea50d98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea50091b00 0x55ea50d98000 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 --2- 172.21.15.116:0/2584933634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea50091680 0x55ea50d98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 >> 172.21.15.116:0/2584933634 conn(0x55ea50091f80 msgr2=0x55ea4bfb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 shutdown_connections 2024-05-14T11:24:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.395+0000 7fa93cc2c400 1 -- 172.21.15.116:0/2584933634 wait complete. 2024-05-14T11:24:50.403 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.083206, result: Events by type: Errors: 0 2024-05-14T11:24:50.403 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-14T11:24:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 Processor -- start 2024-05-14T11:24:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 -- start start 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ecff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e3f080 0x55d820ec4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d820df6d60 con 0x55d820e40570 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d820d9eaa0 con 0x55d820e3f080 2024-05-14T11:24:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f08571ad080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d820df6ee0 con 0x55d820dfb5c0 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46208/0 (socket says 172.21.15.116:46208) 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 -- 172.21.15.116:0/765759808 learned_addr learned my addr 172.21.15.116:0/765759808 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f084fa00640 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e3f080 0x55d820ec4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850400640 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ecff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 -- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 msgr2=0x55d820ecff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ecff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 -- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e3f080 msgr2=0x55d820ec4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e3f080 0x55d820ec4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.507+0000 7f0850e00640 1 -- 172.21.15.116:0/765759808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d820ecf070 con 0x55d820e40570 2024-05-14T11:24:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f0850e00640 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f08480015f0 tx=0x7f0848001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0891f29d2d0bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f084e600640 1 -- 172.21.15.116:0/765759808 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848016070 con 0x55d820e40570 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f084e600640 1 -- 172.21.15.116:0/765759808 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084800f710 con 0x55d820e40570 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f084e600640 1 -- 172.21.15.116:0/765759808 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084800fb00 con 0x55d820e40570 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- 172.21.15.116:0/765759808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 msgr2=0x55d820eca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f08480015f0 tx=0x7f0848001ab0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- 172.21.15.116:0/765759808 shutdown_connections 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e3f080 0x55d820ec4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e40570 0x55d820eca0b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- 172.21.15.116:0/765759808 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ecff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- 172.21.15.116:0/765759808 >> 172.21.15.116:0/765759808 conn(0x55d820de3e10 msgr2=0x55d820e40db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- 172.21.15.116:0/765759808 shutdown_connections 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- 172.21.15.116:0/765759808 wait complete. 2024-05-14T11:24:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 Processor -- start 2024-05-14T11:24:50.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- start start 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ece2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 0x55d820ebf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e40570 0x55d820ebfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d820ecf5b0 con 0x55d820e3f080 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d820ecf310 con 0x55d820e40570 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.511+0000 7f08571ad080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d820ecf850 con 0x55d820dfb5c0 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 0x55d820ebf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 0x55d820ebf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46212/0 (socket says 172.21.15.116:46212) 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 -- 172.21.15.116:0/4110745097 learned_addr learned my addr 172.21.15.116:0/4110745097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f0850e00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e40570 0x55d820ebfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f0850400640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ece2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 -- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 msgr2=0x55d820ece2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d820dfb5c0 0x55d820ece2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 -- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e40570 msgr2=0x55d820ebfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d820e40570 0x55d820ebfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 -- 172.21.15.116:0/4110745097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d820eb95e0 con 0x55d820e3f080 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084fa00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 0x55d820ebf580 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f083c0ac040 tx=0x7f083c019c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084d200640 1 -- 172.21.15.116:0/4110745097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0c2070 con 0x55d820e3f080 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084d200640 1 -- 172.21.15.116:0/4110745097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c0aaab0 con 0x55d820e3f080 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d820eb9880 con 0x55d820e3f080 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084d200640 1 -- 172.21.15.116:0/4110745097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0aae60 con 0x55d820e3f080 2024-05-14T11:24:50.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d820eb9dc0 con 0x55d820e3f080 2024-05-14T11:24:50.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.515+0000 7f084d200640 1 -- 172.21.15.116:0/4110745097 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f083c0c5070 con 0x55d820e3f080 2024-05-14T11:24:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f084d200640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f083006dde0 0x7f08300702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f084d200640 1 -- 172.21.15.116:0/4110745097 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f083c128df0 con 0x55d820e3f080 2024-05-14T11:24:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f0850400640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f083006dde0 0x7f08300702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f08571ad080 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55d820ece800 0x55d820f6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55d820eada00 con 0x55d820ece800 2024-05-14T11:24:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f0850e00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55d820ece800 0x55d820f6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f0850400640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f083006dde0 0x7f08300702f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0840005fd0 tx=0x7f084000e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.519+0000 7f0850e00640 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55d820ece800 0x55d820f6bc30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f0850e00640 1 -- 172.21.15.116:0/4110745097 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v100'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f084800f070 con 0x55d820ece800 2024-05-14T11:24:50.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55d820ece800 msgr2=0x55d820f6bc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55d820ece800 0x55d820f6bc30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f083006dde0 msgr2=0x7f08300702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f083006dde0 0x7f08300702f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0840005fd0 tx=0x7f084000e040 comp rx=0 tx=0).stop 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 msgr2=0x55d820ebf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 --2- 172.21.15.116:0/4110745097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d820e3f080 0x55d820ebf580 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f083c0ac040 tx=0x7f083c019c60 comp rx=0 tx=0).stop 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f0850e00640 1 -- 172.21.15.116:0/4110745097 reap_dead start 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 shutdown_connections 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 >> 172.21.15.116:0/4110745097 conn(0x55d820de3e10 msgr2=0x55d820eca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 shutdown_connections 2024-05-14T11:24:50.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.523+0000 7f08571ad080 1 -- 172.21.15.116:0/4110745097 wait complete. 2024-05-14T11:24:50.529 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:24:50.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 Processor -- start 2024-05-14T11:24:50.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 -- start start 2024-05-14T11:24:50.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 0x7ffb040cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb0406cc10 con 0x7ffb040ce450 2024-05-14T11:24:50.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb0406cef0 con 0x7ffb0415c1d0 2024-05-14T11:24:50.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb09800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0406d790 con 0x7ffb040d0c50 2024-05-14T11:24:50.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb02000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffb040ce450 0x7ffb040cbd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50434/0 (socket says 172.21.15.116:50434) 2024-05-14T11:24:50.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb02000640 1 -- 172.21.15.116:0/3123913013 learned_addr learned my addr 172.21.15.116:0/3123913013 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb03400640 1 --2- 172.21.15.116:0/3123913013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 0x7ffb040cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470476435 0 0) 0x7ffb0406cc10 con 0x7ffb040ce450 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffae8003530 con 0x7ffb040ce450 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823544044 0 0) 0x7ffb0406cef0 con 0x7ffb0415c1d0 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.719+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb0406cc10 con 0x7ffb0415c1d0 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb03400640 1 -- 172.21.15.116:0/3123913013 >> v1:172.21.15.149:6789/0 conn(0x7ffb0415c1d0 legacy=0x7ffb040cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb03400640 1 -- 172.21.15.116:0/3123913013 >> v1:172.21.15.116:6789/0 conn(0x7ffb040ce450 legacy=0x7ffb040cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb03400640 1 -- 172.21.15.116:0/3123913013 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0406d1d0 con 0x7ffb040d0c50 2024-05-14T11:24:50.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb03400640 1 --2- 172.21.15.116:0/3123913013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 0x7ffb040cc470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009cb0 tx=0x7ffafc01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7994f2085e37f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0b2070 con 0x7ffb040d0c50 2024-05-14T11:24:50.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffafc0a58e0 con 0x7ffb040d0c50 2024-05-14T11:24:50.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb01600640 1 -- 172.21.15.116:0/3123913013 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0a5cb0 con 0x7ffb040d0c50 2024-05-14T11:24:50.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb09800640 1 -- 172.21.15.116:0/3123913013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 msgr2=0x7ffb040cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.723+0000 7ffb09800640 1 --2- 172.21.15.116:0/3123913013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 0x7ffb040cc470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009cb0 tx=0x7ffafc01c910 comp rx=0 tx=0).stop 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 -- 172.21.15.116:0/3123913013 shutdown_connections 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 --2- 172.21.15.116:0/3123913013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb040d0c50 0x7ffb040cc470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 -- 172.21.15.116:0/3123913013 >> 172.21.15.116:0/3123913013 conn(0x7ffb040b4720 msgr2=0x7ffb040b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 -- 172.21.15.116:0/3123913013 shutdown_connections 2024-05-14T11:24:50.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 -- 172.21.15.116:0/3123913013 wait complete. 2024-05-14T11:24:50.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 Processor -- start 2024-05-14T11:24:50.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 -- start start 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.727+0000 7ffb09800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0415c1d0 0x7ffb0415cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb041f9390 0x7ffb0406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb040d18b0 con 0x7ffb040d0c50 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb040d1730 con 0x7ffb0415c1d0 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffb040d1a30 con 0x7ffb041f9390 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46222/0 (socket says 172.21.15.116:46222) 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 -- 172.21.15.116:0/2450426591 learned_addr learned my addr 172.21.15.116:0/2450426591 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:50.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02000640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0415c1d0 0x7ffb0415cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb03400640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb041f9390 0x7ffb0406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 -- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb041f9390 msgr2=0x7ffb0406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb041f9390 0x7ffb0406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 -- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0415c1d0 msgr2=0x7ffb0415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0415c1d0 0x7ffb0415cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 -- 172.21.15.116:0/2450426591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb040cb7a0 con 0x7ffb040d0c50 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb02a00640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c430 tx=0x7ffaf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4017070 con 0x7ffb040d0c50 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf40052d0 con 0x7ffb040d0c50 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003840 con 0x7ffb040d0c50 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb040690f0 con 0x7ffb040d0c50 2024-05-14T11:24:50.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb04166200 con 0x7ffb040d0c50 2024-05-14T11:24:50.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0406a9c0 con 0x7ffb040d0c50 2024-05-14T11:24:50.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.731+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffaf401a070 con 0x7ffb040d0c50 2024-05-14T11:24:50.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.735+0000 7ffafaa00640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 0x7ffae4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:50.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.735+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ffaf408e7f0 con 0x7ffb040d0c50 2024-05-14T11:24:50.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.735+0000 7ffb02000640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 0x7ffae4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:50.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.735+0000 7ffb02000640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 0x7ffae4071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0000c10 tx=0x7ffaf0007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.739+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffaf4005480 con 0x7ffb040d0c50 2024-05-14T11:24:50.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:50.927+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ffb04069de0 con 0x7ffb040d0c50 2024-05-14T11:24:51.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.203+0000 7ffafaa00640 1 -- 172.21.15.116:0/2450426591 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) ==== 162+0+0 (secure 0 0 0) 0x7ffaf405aea0 con 0x7ffb040d0c50 2024-05-14T11:24:51.206 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:24:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 msgr2=0x7ffae4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 0x7ffae4071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0000c10 tx=0x7ffaf0007ef0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 msgr2=0x7ffb0409a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c430 tx=0x7ffaf400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 shutdown_connections 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffae406f400 0x7ffae4071910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0415c1d0 0x7ffb0415cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffb041f9390 0x7ffb0406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 --2- 172.21.15.116:0/2450426591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb040d0c50 0x7ffb0409a940 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 >> 172.21.15.116:0/2450426591 conn(0x7ffb040b4720 msgr2=0x7ffb040b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 shutdown_connections 2024-05-14T11:24:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.207+0000 7ffb09800640 1 -- 172.21.15.116:0/2450426591 wait complete. 2024-05-14T11:24:51.232 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 Processor -- start 2024-05-14T11:24:51.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 -- start start 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fea3015db30 0x7fea300aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300d1670 0x7fea300d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fea300ae100 con 0x7fea300d1670 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fea300ae870 con 0x7fea3015db30 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea38000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fea3006ff20 con 0x7fea30161600 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300d1670 0x7fea300d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300d1670 0x7fea300d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46234/0 (socket says 172.21.15.116:46234) 2024-05-14T11:24:51.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 -- 172.21.15.116:0/1304285329 learned_addr learned my addr 172.21.15.116:0/1304285329 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea35600640 1 --2- 172.21.15.116:0/1304285329 >> v2:172.21.15.149:3300/0 conn(0x7fea3015db30 0x7fea300aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea36000640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 -- 172.21.15.116:0/1304285329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 msgr2=0x7fea30161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 -- 172.21.15.116:0/1304285329 >> v2:172.21.15.149:3300/0 conn(0x7fea3015db30 msgr2=0x7fea300aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 --2- 172.21.15.116:0/1304285329 >> v2:172.21.15.149:3300/0 conn(0x7fea3015db30 0x7fea300aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.379+0000 7fea34c00640 1 -- 172.21.15.116:0/1304285329 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300d1fc0 con 0x7fea300d1670 2024-05-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea34c00640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300d1670 0x7fea300d1a80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fea30071f20 tx=0x7fea1c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1736c47dbc8e806 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea2fe00640 1 -- 172.21.15.116:0/1304285329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b0070 con 0x7fea300d1670 2024-05-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea2fe00640 1 -- 172.21.15.116:0/1304285329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c0ab600 con 0x7fea300d1670 2024-05-14T11:24:51.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea2fe00640 1 -- 172.21.15.116:0/1304285329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0ab9d0 con 0x7fea300d1670 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 -- 172.21.15.116:0/1304285329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300d1670 msgr2=0x7fea300d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300d1670 0x7fea300d1a80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fea30071f20 tx=0x7fea1c01c490 comp rx=0 tx=0).stop 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 -- 172.21.15.116:0/1304285329 shutdown_connections 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300d1670 0x7fea300d1a80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 --2- 172.21.15.116:0/1304285329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 --2- 172.21.15.116:0/1304285329 >> v2:172.21.15.149:3300/0 conn(0x7fea3015db30 0x7fea300aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 -- 172.21.15.116:0/1304285329 >> 172.21.15.116:0/1304285329 conn(0x7fea300b4620 msgr2=0x7fea300b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 -- 172.21.15.116:0/1304285329 shutdown_connections 2024-05-14T11:24:51.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.383+0000 7fea38000640 1 -- 172.21.15.116:0/1304285329 wait complete. 2024-05-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 Processor -- start 2024-05-14T11:24:51.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- start start 2024-05-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea300d1670 0x7fea30147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea30162360 con 0x7fea3015db30 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea300aefe0 con 0x7fea300d1670 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fea3015df40 con 0x7fea30161600 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46250/0 (socket says 172.21.15.116:46250) 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 -- 172.21.15.116:0/1545475232 learned_addr learned my addr 172.21.15.116:0/1545475232 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea35600640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea300d1670 0x7fea30147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea36000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 -- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 msgr2=0x7fea30159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 -- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea300d1670 msgr2=0x7fea30147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea300d1670 0x7fea30147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 -- 172.21.15.116:0/1545475232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea30149d10 con 0x7fea3015db30 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea34c00640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01cd30 tx=0x7fea1c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:51.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b9070 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c0b4620 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3015ed20 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b0040 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.387+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea30148380 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea30148660 con 0x7fea3015db30 2024-05-14T11:24:51.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fea1c0bc070 con 0x7fea3015db30 2024-05-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea2d600640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 0x7fea10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fea1c128bc0 con 0x7fea3015db30 2024-05-14T11:24:51.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea35600640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 0x7fea10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:51.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.391+0000 7fea35600640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 0x7fea10071910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea24009ac0 tx=0x7fea24009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:51.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.395+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fea1c0c3020 con 0x7fea3015db30 2024-05-14T11:24:51.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.587+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea3006a9c0 con 0x7fea3015db30 2024-05-14T11:24:51.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea2d600640 1 -- 172.21.15.116:0/1545475232 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5184 (secure 0 0 0) 0x7fea300d1fc0 con 0x7fea3015db30 2024-05-14T11:24:51.591 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:51.591 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:51.212464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"a","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.116:6835/288710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":288710459},{"type":"v1","addr":"172.21.15.116:6835","nonce":288710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:24:51.591 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 86 2024-05-14T11:24:51.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 msgr2=0x7fea10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 0x7fea10071910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea24009ac0 tx=0x7fea24009190 comp rx=0 tx=0).stop 2024-05-14T11:24:51.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 msgr2=0x7fea30158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:51.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01cd30 tx=0x7fea1c019230 comp rx=0 tx=0).stop 2024-05-14T11:24:51.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 shutdown_connections 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fea1006f400 0x7fea10071910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fea30161600 0x7fea30159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea3015db30 0x7fea30158dc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 --2- 172.21.15.116:0/1545475232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea300d1670 0x7fea30147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 >> 172.21.15.116:0/1545475232 conn(0x7fea300b4620 msgr2=0x7fea300b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 shutdown_connections 2024-05-14T11:24:51.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:51.591+0000 7fea38000640 1 -- 172.21.15.116:0/1545475232 wait complete. 2024-05-14T11:24:52.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:24:28.859727+0000', 'modified': '2024-05-14T11:24:51.212464+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7932': {'gid': 7932, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.116:6835/288710459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 288710459}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 288710459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:24:52.016 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7932:up:replay 2024-05-14T11:24:53.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:53.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 Processor -- start 2024-05-14T11:24:53.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 -- start start 2024-05-14T11:24:53.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 0x7ff7bc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7bc0b3950 0x7ff7bc0b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7bc0cec80 con 0x7ff7bc0cdde0 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff7bc162630 con 0x7ff7bc0b3950 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c3200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff7bc162910 con 0x7ff7bc0ce330 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c1200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7bc0b3950 0x7ff7bc0b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7bbe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff7bc0cdde0 0x7ff7bc15de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50442/0 (socket says 172.21.15.116:50442) 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7bbe00640 1 -- 172.21.15.116:0/4182081022 learned_addr learned my addr 172.21.15.116:0/4182081022 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7bb400640 1 --2- 172.21.15.116:0/4182081022 >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 0x7ff7bc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:53.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c1200640 1 -- 172.21.15.116:0/4182081022 >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 msgr2=0x7ff7bc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:53.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c1200640 1 --2- 172.21.15.116:0/4182081022 >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 0x7ff7bc0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c1200640 1 -- 172.21.15.116:0/4182081022 >> v1:172.21.15.116:6789/0 conn(0x7ff7bc0cdde0 legacy=0x7ff7bc15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7baa00640 1 -- 172.21.15.116:0/4182081022 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662140797 0 0) 0x7ff7ac0014a0 con 0x7ff7bc0cdde0 2024-05-14T11:24:53.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7c1200640 1 -- 172.21.15.116:0/4182081022 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc0b2e70 con 0x7ff7bc0b3950 2024-05-14T11:24:53.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.175+0000 7ff7bb400640 1 --2- 172.21.15.116:0/4182081022 >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 0x7ff7bc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c1200640 1 --2- 172.21.15.116:0/4182081022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc0b3d60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0b9500 tx=0x7ff7b401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de95b924f54882 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7baa00640 1 -- 172.21.15.116:0/4182081022 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b1070 con 0x7ff7bc0b3950 2024-05-14T11:24:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7baa00640 1 -- 172.21.15.116:0/4182081022 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b40ac620 con 0x7ff7bc0b3950 2024-05-14T11:24:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7baa00640 1 -- 172.21.15.116:0/4182081022 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40ac9f0 con 0x7ff7bc0b3950 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 -- 172.21.15.116:0/4182081022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0b3950 msgr2=0x7ff7bc0b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 --2- 172.21.15.116:0/4182081022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc0b3d60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0b9500 tx=0x7ff7b401c3c0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 -- 172.21.15.116:0/4182081022 shutdown_connections 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 --2- 172.21.15.116:0/4182081022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc0b3d60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 --2- 172.21.15.116:0/4182081022 >> v2:172.21.15.149:3300/0 conn(0x7ff7bc0ce330 0x7ff7bc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 -- 172.21.15.116:0/4182081022 >> 172.21.15.116:0/4182081022 conn(0x7ff7bc0d3670 msgr2=0x7ff7bc0d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 -- 172.21.15.116:0/4182081022 shutdown_connections 2024-05-14T11:24:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.179+0000 7ff7c3200640 1 -- 172.21.15.116:0/4182081022 wait complete. 2024-05-14T11:24:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 Processor -- start 2024-05-14T11:24:53.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 -- start start 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 0x7ff7bc1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0cf070 con 0x7ff7bc0ce330 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0ceef0 con 0x7ff7bc0b3950 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c3200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc16ae90 con 0x7ff7bc0cdde0 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46272/0 (socket says 172.21.15.116:46272) 2024-05-14T11:24:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 -- 172.21.15.116:0/3784693617 learned_addr learned my addr 172.21.15.116:0/3784693617 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7bb400640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 0x7ff7bc1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 -- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 msgr2=0x7ff7bc1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 0x7ff7bc1593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 -- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7bc0b3950 msgr2=0x7ff7bc152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.183+0000 7ff7c1200640 1 -- 172.21.15.116:0/3784693617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc15ce30 con 0x7ff7bc0ce330 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7bb400640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 0x7ff7bc1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7c1200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4018fd0 tx=0x7ff7b401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b1040 con 0x7ff7bc0ce330 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b4004030 con 0x7ff7bc0ce330 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc15c070 con 0x7ff7bc0ce330 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4006e20 con 0x7ff7bc0ce330 2024-05-14T11:24:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc15c630 con 0x7ff7bc0ce330 2024-05-14T11:24:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc06a9c0 con 0x7ff7bc0ce330 2024-05-14T11:24:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff7b4003620 con 0x7ff7bc0ce330 2024-05-14T11:24:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 0x7ff798071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff7b4129ad0 con 0x7ff7bc0ce330 2024-05-14T11:24:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.187+0000 7ff7bbe00640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 0x7ff798071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:53.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.191+0000 7ff7bbe00640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 0x7ff798071940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc074330 tx=0x7ff7ac007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.199+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff7b40c7020 con 0x7ff7bc0ce330 2024-05-14T11:24:53.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.387+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7bc168220 con 0x7ff7bc0ce330 2024-05-14T11:24:53.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.387+0000 7ff7b3e00640 1 -- 172.21.15.116:0/3784693617 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5184 (secure 0 0 0) 0x7ff7b4016070 con 0x7ff7bc0ce330 2024-05-14T11:24:53.390 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:53.390 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:52.218516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"a","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.116:6835/288710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":288710459},{"type":"v1","addr":"172.21.15.116:6835","nonce":288710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:24:53.391 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 88 2024-05-14T11:24:53.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 msgr2=0x7ff798071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 0x7ff798071940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc074330 tx=0x7ff7ac007cf0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 msgr2=0x7ff7bc15fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4018fd0 tx=0x7ff7b401ac00 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 shutdown_connections 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff79806f430 0x7ff798071940 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7bc0ce330 0x7ff7bc15fb40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff7bc0cdde0 0x7ff7bc1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 --2- 172.21.15.116:0/3784693617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7bc0b3950 0x7ff7bc152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 >> 172.21.15.116:0/3784693617 conn(0x7ff7bc0d3670 msgr2=0x7ff7bc155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 shutdown_connections 2024-05-14T11:24:53.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:53.391+0000 7ff7c3200640 1 -- 172.21.15.116:0/3784693617 wait complete. 2024-05-14T11:24:53.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:24:28.859727+0000', 'modified': '2024-05-14T11:24:52.218516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7932': {'gid': 7932, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.116:6835/288710459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 288710459}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 288710459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:24:53.814 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7932:up:rejoin 2024-05-14T11:24:54.815 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:24:54.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.975+0000 7f1873400640 1 Processor -- start 2024-05-14T11:24:54.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.975+0000 7f1873400640 1 -- start start 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186c156fa0 0x7f186c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f186c0d1670 0x7f186c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c06e9e0 0x7f186c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f186c1573b0 con 0x7f186c156fa0 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f186c161120 con 0x7f186c0d1670 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1873400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0d1fc0 con 0x7f186c06e9e0 2024-05-14T11:24:54.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186c156fa0 0x7f186c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186c156fa0 0x7f186c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46284/0 (socket says 172.21.15.116:46284) 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 -- 172.21.15.116:0/215113869 learned_addr learned my addr 172.21.15.116:0/215113869 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f186be00640 1 --2- 172.21.15.116:0/215113869 >> v2:172.21.15.149:3300/0 conn(0x7f186c0d1670 0x7f186c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1871400640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c06e9e0 0x7f186c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 -- 172.21.15.116:0/215113869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c06e9e0 msgr2=0x7f186c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c06e9e0 0x7f186c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 -- 172.21.15.116:0/215113869 >> v2:172.21.15.149:3300/0 conn(0x7f186c0d1670 msgr2=0x7f186c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 --2- 172.21.15.116:0/215113869 >> v2:172.21.15.149:3300/0 conn(0x7f186c0d1670 0x7f186c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 -- 172.21.15.116:0/215113869 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c160510 con 0x7f186c156fa0 2024-05-14T11:24:54.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.979+0000 7f1870a00640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c06ffe0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f186c070c70 tx=0x7f185c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e0e4c650157a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:54.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f186b400640 1 -- 172.21.15.116:0/215113869 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0aa610 con 0x7f186c156fa0 2024-05-14T11:24:54.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f186b400640 1 -- 172.21.15.116:0/215113869 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c0af070 con 0x7f186c156fa0 2024-05-14T11:24:54.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f186b400640 1 -- 172.21.15.116:0/215113869 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0aa9d0 con 0x7f186c156fa0 2024-05-14T11:24:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- 172.21.15.116:0/215113869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 msgr2=0x7f186c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c06ffe0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f186c070c70 tx=0x7f185c01c4f0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- 172.21.15.116:0/215113869 shutdown_connections 2024-05-14T11:24:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c06e9e0 0x7f186c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 --2- 172.21.15.116:0/215113869 >> v2:172.21.15.149:3300/0 conn(0x7f186c0d1670 0x7f186c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 --2- 172.21.15.116:0/215113869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c06ffe0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- 172.21.15.116:0/215113869 >> 172.21.15.116:0/215113869 conn(0x7f186c0b4620 msgr2=0x7f186c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- 172.21.15.116:0/215113869 shutdown_connections 2024-05-14T11:24:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- 172.21.15.116:0/215113869 wait complete. 2024-05-14T11:24:54.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 Processor -- start 2024-05-14T11:24:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.983+0000 7f1873400640 1 -- start start 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c06e9e0 0x7f186c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c0d1670 0x7f186c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f186c070520 con 0x7f186c156fa0 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f186c06f740 con 0x7f186c06e9e0 2024-05-14T11:24:54.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f186c0706a0 con 0x7f186c0d1670 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46290/0 (socket says 172.21.15.116:46290) 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 -- 172.21.15.116:0/273268474 learned_addr learned my addr 172.21.15.116:0/273268474 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f186be00640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c0d1670 0x7f186c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1870a00640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c06e9e0 0x7f186c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 -- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c0d1670 msgr2=0x7f186c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c0d1670 0x7f186c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 -- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c06e9e0 msgr2=0x7f186c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c06e9e0 0x7f186c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:54.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 -- 172.21.15.116:0/273268474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c151660 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1871400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1864007e60 tx=0x7f186400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864014420 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864019070 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c151940 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864014820 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c151f00 con 0x7f186c156fa0 2024-05-14T11:24:54.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.987+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c06a9c0 con 0x7f186c156fa0 2024-05-14T11:24:54.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.991+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18640149c0 con 0x7f186c156fa0 2024-05-14T11:24:54.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.991+0000 7f1868c00640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 0x7f1848071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:54.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.991+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6216+0+0 (secure 0 0 0) 0x7f186408fc70 con 0x7f186c156fa0 2024-05-14T11:24:54.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.991+0000 7f1870a00640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 0x7f1848071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:54.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.991+0000 7f1870a00640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 0x7f1848071910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f185c0ae000 tx=0x7f185c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:54.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:54.995+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f186405c270 con 0x7f186c156fa0 2024-05-14T11:24:55.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.187+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f186c1675c0 con 0x7f186c156fa0 2024-05-14T11:24:55.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.187+0000 7f1868c00640 1 -- 172.21.15.116:0/273268474 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5191 (secure 0 0 0) 0x7f1864060570 con 0x7f186c156fa0 2024-05-14T11:24:55.185 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:55.185 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:24:54.222806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/288710459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":288710459},{"type":"v1","addr":"172.21.15.116:6835","nonce":288710459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7932,"qdb_cluster":[7932]},"id":7}]} 2024-05-14T11:24:55.185 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 89 2024-05-14T11:24:55.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 msgr2=0x7f1848071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 0x7f1848071910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f185c0ae000 tx=0x7f185c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 msgr2=0x7f186c14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1864007e60 tx=0x7f186400e4b0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 shutdown_connections 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f184806f400 0x7f1848071910 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c156fa0 0x7f186c14f7e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f186c0d1670 0x7f186c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 --2- 172.21.15.116:0/273268474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c06e9e0 0x7f186c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 >> 172.21.15.116:0/273268474 conn(0x7f186c0b4620 msgr2=0x7f186c166eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 shutdown_connections 2024-05-14T11:24:55.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.191+0000 7f1873400640 1 -- 172.21.15.116:0/273268474 wait complete. 2024-05-14T11:24:55.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:24:28.859727+0000', 'modified': '2024-05-14T11:24:54.222806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7932': {'gid': 7932, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/288710459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 288710459}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 288710459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7932, 'qdb_cluster': [7932]} 2024-05-14T11:24:55.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:24:55.609 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-14T11:24:55.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 Processor -- start 2024-05-14T11:24:55.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 -- start start 2024-05-14T11:24:55.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8fd006c700 0x7f8fd006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd006ff20 con 0x7f8fd015db30 2024-05-14T11:24:55.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd00ae100 con 0x7f8fd006c700 2024-05-14T11:24:55.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd00ae870 con 0x7f8fd00d1670 2024-05-14T11:24:55.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.759+0000 7f8fd5000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8fd015db30 0x7f8fd00aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50454/0 (socket says 172.21.15.116:50454) 2024-05-14T11:24:55.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5000640 1 -- 172.21.15.116:0/540650371 learned_addr learned my addr 172.21.15.116:0/540650371 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/540650371 >> v2:172.21.15.149:3300/0 conn(0x7f8fd006c700 0x7f8fd006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797710084 0 0) 0x7f8fd006ff20 con 0x7f8fd015db30 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fb4003530 con 0x7f8fd015db30 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 -- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 msgr2=0x7f8fd00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1247274467 0 0) 0x7f8fb4003530 con 0x7f8fd015db30 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 -- 172.21.15.116:0/540650371 >> v1:172.21.15.116:6789/0 conn(0x7f8fd015db30 legacy=0x7f8fd00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 -- 172.21.15.116:0/540650371 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd00d1fc0 con 0x7f8fd006c700 2024-05-14T11:24:55.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.763+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd006c700 0x7f8fd006cb10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009d90 tx=0x7f8fc801c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a4209e4a23f61f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80ac070 con 0x7f8fd006c700 2024-05-14T11:24:55.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc80a6430 con 0x7f8fd006c700 2024-05-14T11:24:55.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fcfe00640 1 -- 172.21.15.116:0/540650371 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0001700 con 0x7f8fd006c700 2024-05-14T11:24:55.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd006c700 msgr2=0x7f8fd006cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd006c700 0x7f8fd006cb10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009d90 tx=0x7f8fc801c730 comp rx=0 tx=0).stop 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- 172.21.15.116:0/540650371 shutdown_connections 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd006c700 0x7f8fd006cb10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/540650371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- 172.21.15.116:0/540650371 >> 172.21.15.116:0/540650371 conn(0x7f8fd00b4620 msgr2=0x7f8fd00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- 172.21.15.116:0/540650371 shutdown_connections 2024-05-14T11:24:55.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- 172.21.15.116:0/540650371 wait complete. 2024-05-14T11:24:55.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 Processor -- start 2024-05-14T11:24:55.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.767+0000 7f8fd7a00640 1 -- start start 2024-05-14T11:24:55.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 0x7f8fd00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd016a770 con 0x7f8fd006c700 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd015df40 con 0x7f8fd015db30 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd016a8f0 con 0x7f8fd00d1670 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46294/0 (socket says 172.21.15.116:46294) 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 -- 172.21.15.116:0/3141039955 learned_addr learned my addr 172.21.15.116:0/3141039955 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 0x7f8fd00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 -- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 msgr2=0x7f8fd00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 -- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 msgr2=0x7f8fd00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 0x7f8fd00b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 -- 172.21.15.116:0/3141039955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd009bb70 con 0x7f8fd006c700 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:55.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 0x7f8fd00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd5000640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c510 tx=0x7f8fc000c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fcd600640 1 -- 172.21.15.116:0/3141039955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0005cf0 con 0x7f8fd006c700 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fcd600640 1 -- 172.21.15.116:0/3141039955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc0007070 con 0x7f8fd006c700 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd00cffd0 con 0x7f8fd006c700 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fcd600640 1 -- 172.21.15.116:0/3141039955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0007460 con 0x7f8fd006c700 2024-05-14T11:24:55.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.771+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd00cf210 con 0x7f8fd006c700 2024-05-14T11:24:55.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fcd600640 1 -- 172.21.15.116:0/3141039955 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8fc0007600 con 0x7f8fd006c700 2024-05-14T11:24:55.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fcd600640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 0x7f8fac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fcd600640 1 -- 172.21.15.116:0/3141039955 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f8fc008f290 con 0x7f8fd006c700 2024-05-14T11:24:55.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 0x7f8fac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 Processor -- start 2024-05-14T11:24:55.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 -- start start 2024-05-14T11:24:55.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 0x7f8fac071910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8fd009eda0 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 0x558ea0da9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558ea0c46620 con 0x558ea0c416b0 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ea0b61740 con 0x558ea0c57f80 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558ea0baf090 con 0x558ea086f7c0 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46308/0 (socket says 172.21.15.116:46308) 2024-05-14T11:24:55.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 -- 172.21.15.116:0/1697367060 learned_addr learned my addr 172.21.15.116:0/1697367060 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 -- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 msgr2=0x558ea0da9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 0x558ea0da9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 -- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 msgr2=0x558ea0da9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 0x558ea0da9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5a00640 1 -- 172.21.15.116:0/1697367060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea0d430a0 con 0x558ea0c416b0 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.775+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 0x558ea0da9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8fc801c770 tx=0x7f8fc8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396678780748900 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fc5600640 1 -- 172.21.15.116:0/1697367060 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80c2070 con 0x558ea0c416b0 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fc5600640 1 -- 172.21.15.116:0/1697367060 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc80b2560 con 0x558ea0c416b0 2024-05-14T11:24:55.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fc5600640 1 -- 172.21.15.116:0/1697367060 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80ac040 con 0x558ea0c416b0 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1697367060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 msgr2=0x558ea0da9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8fc801c770 tx=0x7f8fc8004ac0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1697367060 shutdown_connections 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c57f80 0x558ea0da9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0c416b0 0x558ea0da9da0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1697367060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1697367060 >> 172.21.15.116:0/1697367060 conn(0x558ea0d2e340 msgr2=0x558ea0d5dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1697367060 shutdown_connections 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1697367060 wait complete. 2024-05-14T11:24:55.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 -- start start 2024-05-14T11:24:55.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.779+0000 7f8fdf00c000 1 Processor -- start 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0d92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea0c57f80 0x558ea0d930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558ea0bab050 con 0x558ea086f7c0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ea0b67ec0 con 0x558ea0c416b0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558ea0b4ec90 con 0x558ea0c57f80 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 10 client.0 _ll_register_callbacks cb 0x558ea0d4ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46310/0 (socket says 172.21.15.116:46310) 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 -- 172.21.15.116:0/1596279682 learned_addr learned my addr 172.21.15.116:0/1596279682 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0d92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea0c57f80 0x558ea0d930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 -- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea0c57f80 msgr2=0x558ea0d930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea0c57f80 0x558ea0d930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 -- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 msgr2=0x558ea0d92b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0d92b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 -- 172.21.15.116:0/1596279682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea0c596f0 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0d92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8fc005b4e0 tx=0x7f8fc005be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ea320fb9510f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc008b060 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f8fc0018b40 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc80b2560 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8fc0079050 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 10 client.7992 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x558ea0c556a0 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc80b2560 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f8fc005cb10 con 0x558ea086f7c0 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 10 client.7992 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 10 client.7992 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 20 client.7992 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:55.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea08547d0 0x558ea0dbedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 4 client.7992 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:55.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ea0b47be0 con 0x558ea08547d0 2024-05-14T11:24:55.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea08547d0 0x558ea0dbedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea08547d0 0x558ea0dbedf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfefa98dbf22137 server_cookie=2b4ad540cec255d9 in_seq=0 out_seq=0 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 10 client.7992 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 1 -- 172.21.15.116:0/1596279682 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ea0b47be0 con 0x558ea08547d0 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fbe000640 10 client.7992 handle_command_reply: tid=0 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 client.7992 shutdown 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 msgr2=0x558ea0d64790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.783+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8fc005b4e0 tx=0x7f8fc005be00 comp rx=0 tx=0).stop 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 shutdown_connections 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0d92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea08547d0 0x558ea0dbedf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea0c57f80 0x558ea0d930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1596279682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea086f7c0 0x558ea0d64790 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fbe000640 0 client.7992 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:55.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 >> 172.21.15.116:0/1596279682 conn(0x558ea0d2e340 msgr2=0x558ea0d5e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 shutdown_connections 2024-05-14T11:24:55.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1596279682 wait complete. 2024-05-14T11:24:55.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.787+0000 7f8fdf00c000 20 client.7992 trim_cache size 0 max 16384 2024-05-14T11:24:55.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.803+0000 7f8fdf00c000 1 Processor -- start 2024-05-14T11:24:55.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.803+0000 7f8fdf00c000 1 -- start start 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0da7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0074480 con 0x558ea0d2e340 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd009dde0 con 0x558ea0c416b0 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd0073e30 con 0x558ea086f7c0 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46316/0 (socket says 172.21.15.116:46316) 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 -- 172.21.15.116:0/1975222268 learned_addr learned my addr 172.21.15.116:0/1975222268 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0da7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 -- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 msgr2=0x558ea0da79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 -- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 msgr2=0x558ea0da7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0da7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 -- 172.21.15.116:0/1975222268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea0b88af0 con 0x558ea0d2e340 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fd5000640 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008d3c0 tx=0x7f8fc005b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd9512d295f983a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fc4c00640 1 -- 172.21.15.116:0/1975222268 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0074070 con 0x558ea0d2e340 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fc4c00640 1 -- 172.21.15.116:0/1975222268 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc008f070 con 0x558ea0d2e340 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fc4c00640 1 -- 172.21.15.116:0/1975222268 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc005b130 con 0x558ea0d2e340 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1975222268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 msgr2=0x558ea0da8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008d3c0 tx=0x7f8fc005b080 comp rx=0 tx=0).stop 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1975222268 shutdown_connections 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0da7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0da79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/1975222268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0da8b30 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1975222268 >> 172.21.15.116:0/1975222268 conn(0x558ea0d61dc0 msgr2=0x558ea0d8dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1975222268 shutdown_connections 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- 172.21.15.116:0/1975222268 wait complete. 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 -- start start 2024-05-14T11:24:55.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.807+0000 7f8fdf00c000 1 Processor -- start 2024-05-14T11:24:55.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0e1ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0e20460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558ea0bac020 con 0x558ea0d2e340 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ea0b527d0 con 0x558ea0c416b0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558ea0bbc490 con 0x558ea086f7c0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 10 client.0 _ll_register_callbacks cb 0x7f8fc0005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46326/0 (socket says 172.21.15.116:46326) 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 -- 172.21.15.116:0/2160048103 learned_addr learned my addr 172.21.15.116:0/2160048103 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5000640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0e1ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fc7e00640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0e20460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 -- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 msgr2=0x558ea0e1ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0e1ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 -- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 msgr2=0x558ea0e20460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0e20460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 -- 172.21.15.116:0/2160048103 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea0b87e40 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5a00640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8fc801c770 tx=0x7f8fc801af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6eefce10fa5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80ca070 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f8fc80040c0 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ea0bbcca0 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8fc80d5050 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 10 client.7998 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x558ea0c2f4e0 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558ea0bbcca0 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f8fc80d8020 con 0x558ea0d2e340 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 10 client.7998 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 10 client.7998 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-14T11:24:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 20 client.7998 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea086de80 0x558ea0d602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 4 client.7998 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:24:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "status"}) -- 0x558ea0bb6b10 con 0x558ea086de80 2024-05-14T11:24:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5000640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea086de80 0x558ea0d602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fd5000640 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea086de80 0x558ea0d602c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4241b146c4dc556 server_cookie=5c3f85e6abe34eff in_seq=0 out_seq=0 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.811+0000 7f8fbe000640 10 client.7998 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fbe000640 1 -- 172.21.15.116:0/2160048103 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x558ea0bb6b10 con 0x558ea086de80 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fbe000640 10 client.7998 handle_command_reply: tid=0 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 client.7998 shutdown 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 msgr2=0x558ea0d5f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8fc801c770 tx=0x7f8fc801af70 comp rx=0 tx=0).stop 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 shutdown_connections 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ea0c416b0 0x558ea0e20460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x558ea086de80 0x558ea0d602c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558ea0d2e340 0x558ea0d5f900 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 --2- 172.21.15.116:0/2160048103 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558ea086f7c0 0x558ea0e1ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fbe000640 0 client.7998 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 >> 172.21.15.116:0/2160048103 conn(0x558ea0d61dc0 msgr2=0x558ea0db93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 shutdown_connections 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 1 -- 172.21.15.116:0/2160048103 wait complete. 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fdf00c000 20 client.7998 trim_cache size 0 max 16384 2024-05-14T11:24:55.814 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 7932, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.5962020739999998, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 89, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 102, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 100, 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 5.4696845979999997 2024-05-14T11:24:55.815 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 msgr2=0x7f8fac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 0x7f8fac071910 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8fd009eda0 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 msgr2=0x7f8fd009b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c510 tx=0x7f8fc000c9d0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 shutdown_connections 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8fac06f400 0x7f8fac071910 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fd015db30 0x7f8fd00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8fd00d1670 0x7f8fd00d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 --2- 172.21.15.116:0/3141039955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd006c700 0x7f8fd009b630 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 >> 172.21.15.116:0/3141039955 conn(0x7f8fd00b4620 msgr2=0x7f8fd01600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 shutdown_connections 2024-05-14T11:24:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.815+0000 7f8fd7a00640 1 -- 172.21.15.116:0/3141039955 wait complete. 2024-05-14T11:24:55.831 DEBUG:teuthology.orchestra.run.smithi116:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-14T11:24:55.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:55.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-14T11:24:55.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.991+0000 7f991cc00640 1 Processor -- start 2024-05-14T11:24:55.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.991+0000 7f991cc00640 1 -- start start 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f991cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180b5f30 0x7f99180b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f991cc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99180cfda0 0x7f9918167070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f991cc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99180b6880 con 0x7f99180cf280 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f991cc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f991815d0f0 con 0x7f99180cfda0 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f991cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f991815c800 con 0x7f99180b5f30 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99180cfda0 0x7f9918167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99180cfda0 0x7f9918167070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46334/0 (socket says 172.21.15.116:46334) 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 -- 172.21.15.116:0/3347124649 learned_addr learned my addr 172.21.15.116:0/3347124649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9915400640 1 --1- 172.21.15.116:0/3347124649 >> v1:172.21.15.116:6789/0 conn(0x7f99180cf280 0x7f99180cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50458/0 (socket says 172.21.15.116:50458) 2024-05-14T11:24:55.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9915e00640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180b5f30 0x7f99180b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:55.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 -- 172.21.15.116:0/3347124649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180b5f30 msgr2=0x7f99180b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:55.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180b5f30 0x7f99180b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 -- 172.21.15.116:0/3347124649 >> v1:172.21.15.116:6789/0 conn(0x7f99180cf280 legacy=0x7f99180cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9916800640 1 -- 172.21.15.116:0/3347124649 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991815caa0 con 0x7f99180cfda0 2024-05-14T11:24:55.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.995+0000 7f9914a00640 1 -- 172.21.15.116:0/3347124649 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426372655 0 0) 0x7f9904001120 con 0x7f99180cf280 2024-05-14T11:24:55.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f9916800640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cfda0 0x7f9918167070 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9910009e50 tx=0x7f991001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ea1e07410b4259 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:55.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f9914a00640 1 -- 172.21.15.116:0/3347124649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100a5810 con 0x7f99180cfda0 2024-05-14T11:24:55.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f9914a00640 1 -- 172.21.15.116:0/3347124649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99100a59b0 con 0x7f99180cfda0 2024-05-14T11:24:55.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f9914a00640 1 -- 172.21.15.116:0/3347124649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100a5d80 con 0x7f99180cfda0 2024-05-14T11:24:55.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 -- 172.21.15.116:0/3347124649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cfda0 msgr2=0x7f9918167070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:55.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cfda0 0x7f9918167070 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9910009e50 tx=0x7f991001ca30 comp rx=0 tx=0).stop 2024-05-14T11:24:55.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 -- 172.21.15.116:0/3347124649 shutdown_connections 2024-05-14T11:24:55.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cfda0 0x7f9918167070 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 --2- 172.21.15.116:0/3347124649 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180b5f30 0x7f99180b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:55.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 -- 172.21.15.116:0/3347124649 >> 172.21.15.116:0/3347124649 conn(0x7f99180cde80 msgr2=0x7f99180ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:55.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 -- 172.21.15.116:0/3347124649 shutdown_connections 2024-05-14T11:24:55.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:55.999+0000 7f991cc00640 1 -- 172.21.15.116:0/3347124649 wait complete. 2024-05-14T11:24:55.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 Processor -- start 2024-05-14T11:24:55.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- start start 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99180b5f30 0x7f99181525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180cfda0 0x7f99181559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f99180b5380 con 0x7f99180cf280 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99180b5200 con 0x7f99180b5f30 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f991806d710 con 0x7f99180cfda0 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46342/0 (socket says 172.21.15.116:46342) 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 -- 172.21.15.116:0/3318603657 learned_addr learned my addr 172.21.15.116:0/3318603657 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9916800640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180cfda0 0x7f99181559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915e00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99180b5f30 0x7f99181525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 -- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180cfda0 msgr2=0x7f99181559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180cfda0 0x7f99181559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 -- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99180b5f30 msgr2=0x7f99181525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99180b5f30 0x7f99181525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 -- 172.21.15.116:0/3318603657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991814f780 con 0x7f99180cf280 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f9915400640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f990400c390 tx=0x7f990400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f990e000640 1 -- 172.21.15.116:0/3318603657 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040074e0 con 0x7f99180cf280 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f990e000640 1 -- 172.21.15.116:0/3318603657 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99040047f0 con 0x7f99180cf280 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f990e000640 1 -- 172.21.15.116:0/3318603657 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904004be0 con 0x7f99180cf280 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991814fa60 con 0x7f99180cf280 2024-05-14T11:24:56.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.003+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9918150020 con 0x7f99180cf280 2024-05-14T11:24:56.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.007+0000 7f990e000640 1 -- 172.21.15.116:0/3318603657 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9904011070 con 0x7f99180cf280 2024-05-14T11:24:56.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.007+0000 7f990e000640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 0x7f98f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.007+0000 7f990e000640 1 -- 172.21.15.116:0/3318603657 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f990408ee80 con 0x7f99180cf280 2024-05-14T11:24:56.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.007+0000 7f9915e00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 0x7f98f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.007+0000 7f9924377000 1 Processor -- start 2024-05-14T11:24:56.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- start start 2024-05-14T11:24:56.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915e00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 0x7f98f8071940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9908009ac0 tx=0x7f9908009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f782c0 0x5568730cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556872f66960 con 0x556872f619f0 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556872ece200 con 0x556872f782c0 2024-05-14T11:24:56.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556872e86da0 con 0x556872b4f630 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46350/0 (socket says 172.21.15.116:46350) 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 -- 172.21.15.116:0/385015846 learned_addr learned my addr 172.21.15.116:0/385015846 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9916800640 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915e00640 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f782c0 0x5568730cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 -- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 msgr2=0x5568730cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 -- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f782c0 msgr2=0x5568730cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f782c0 0x5568730cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 -- 172.21.15.116:0/385015846 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556872e8b900 con 0x556872f619f0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9915400640 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9904057590 tx=0x7f990405b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc26c74cbb35d80a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9902600640 1 -- 172.21.15.116:0/385015846 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904064230 con 0x556872f619f0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9902600640 1 -- 172.21.15.116:0/385015846 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904001090 con 0x556872f619f0 2024-05-14T11:24:56.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9902600640 1 -- 172.21.15.116:0/385015846 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990405a770 con 0x556872f619f0 2024-05-14T11:24:56.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- 172.21.15.116:0/385015846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 msgr2=0x5568730cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9904057590 tx=0x7f990405b020 comp rx=0 tx=0).stop 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- 172.21.15.116:0/385015846 shutdown_connections 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730cd7e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 --2- 172.21.15.116:0/385015846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f782c0 0x5568730cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- 172.21.15.116:0/385015846 >> 172.21.15.116:0/385015846 conn(0x556872f85760 msgr2=0x55687307da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- 172.21.15.116:0/385015846 shutdown_connections 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.011+0000 7f9924377000 1 -- 172.21.15.116:0/385015846 wait complete. 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 -- start start 2024-05-14T11:24:56.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 Processor -- start 2024-05-14T11:24:56.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7fefa0c00640 1 Processor -- start 2024-05-14T11:24:56.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556872ea8df0 con 0x556872f619f0 2024-05-14T11:24:56.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556872e8a180 con 0x556872b4f630 2024-05-14T11:24:56.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556872ecd510 con 0x556872f782c0 2024-05-14T11:24:56.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7fefa0c00640 1 -- start start 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 10 client.0 _ll_register_callbacks cb 0x556872f81530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46352/0 (socket says 172.21.15.116:46352) 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 -- 172.21.15.116:0/3316045707 learned_addr learned my addr 172.21.15.116:0/3316045707 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915e00640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9916800640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 -- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 msgr2=0x5568730b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 -- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 msgr2=0x5568730d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 -- 172.21.15.116:0/3316045707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556872e78960 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9915400640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9904000f70 tx=0x7f9904000fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2cf5d690ce9254 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f98f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904064230 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f9904074bc0 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.015+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9904001090 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9904062b00 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 10 client.8013 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x556872e86ff0 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904001090 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f9904062ca0 con 0x556872f619f0 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 10 client.8013 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:56.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 10 client.8013 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 20 client.8013 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b73b80 0x5568730b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 4 client.8013 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556872e66ef0 con 0x556872b73b80 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fefa0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fefa0c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fef9c0b6c30 0x7fef9c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fefa0c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9c071c10 con 0x7fef9c071800 2024-05-14T11:24:56.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fefa0c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fef9c0ceb50 con 0x7fef9c0b6c30 2024-05-14T11:24:56.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fefa0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0702b0 con 0x7fef9c0ce740 2024-05-14T11:24:56.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9915e00640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b73b80 0x5568730b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:36396/0 (socket says 172.21.15.116:36396) 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 -- 172.21.15.116:0/2464837548 learned_addr learned my addr 172.21.15.116:0/2464837548 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99400640 1 --2- 172.21.15.116:0/2464837548 >> v2:172.21.15.149:3300/0 conn(0x7fef9c0b6c30 0x7fef9c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9915e00640 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b73b80 0x5568730b8370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3e10840c68169e server_cookie=3f141252012e05b4 in_seq=0 out_seq=0 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 10 client.8013 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 -- 172.21.15.116:0/2464837548 >> v2:172.21.15.149:3300/0 conn(0x7fef9c0b6c30 msgr2=0x7fef9c0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 --2- 172.21.15.116:0/2464837548 >> v2:172.21.15.149:3300/0 conn(0x7fef9c0b6c30 0x7fef9c0712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 -- 172.21.15.116:0/2464837548 >> v1:172.21.15.149:6789/0 conn(0x7fef9c071800 legacy=0x7fef9c0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7fef99e00640 1 -- 172.21.15.116:0/2464837548 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c070500 con 0x7fef9c0ce740 2024-05-14T11:24:56.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 1 -- 172.21.15.116:0/3316045707 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556872e66ef0 con 0x556872b73b80 2024-05-14T11:24:56.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 10 client.8013 handle_command_reply: tid=0 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 client.8013 shutdown 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 msgr2=0x5568730d3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9904000f70 tx=0x7f9904000fc0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 shutdown_connections 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b73b80 0x5568730b8370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730d3710 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 --2- 172.21.15.116:0/3316045707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f98f6a00640 0 client.8013 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 >> 172.21.15.116:0/3316045707 conn(0x556872f85760 msgr2=0x556873084730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 shutdown_connections 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 1 -- 172.21.15.116:0/3316045707 wait complete. 2024-05-14T11:24:56.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.019+0000 7f9924377000 20 client.8013 trim_cache size 0 max 16384 2024-05-14T11:24:56.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fef99e00640 1 --2- 172.21.15.116:0/2464837548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853952e1d7fdade1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fef98a00640 1 -- 172.21.15.116:0/2464837548 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0b2070 con 0x7fef9c0ce740 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fef98a00640 1 -- 172.21.15.116:0/2464837548 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c0a9070 con 0x7fef9c0ce740 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fef98a00640 1 -- 172.21.15.116:0/2464837548 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ad880 con 0x7fef9c0ce740 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 -- 172.21.15.116:0/2464837548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 msgr2=0x7fef9c0b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 --2- 172.21.15.116:0/2464837548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c01c560 comp rx=0 tx=0).stop 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 -- 172.21.15.116:0/2464837548 shutdown_connections 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 --2- 172.21.15.116:0/2464837548 >> v2:172.21.15.149:3300/0 conn(0x7fef9c0b6c30 0x7fef9c0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 --2- 172.21.15.116:0/2464837548 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c0ce740 0x7fef9c0b66f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 -- 172.21.15.116:0/2464837548 >> 172.21.15.116:0/2464837548 conn(0x7fef9c0d35e0 msgr2=0x7fef9c0d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 -- 172.21.15.116:0/2464837548 shutdown_connections 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.023+0000 7fefa0c00640 1 -- 172.21.15.116:0/2464837548 wait complete. 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 Processor -- start 2024-05-14T11:24:56.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- start start 2024-05-14T11:24:56.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c071800 0x7fef9c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef9c0b6c30 0x7fef9c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c0ce740 0x7fef9c0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c0d26c0 con 0x7fef9c0b6c30 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c070e20 con 0x7fef9c0ce740 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0d2840 con 0x7fef9c071800 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef9c0b6c30 0x7fef9c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef9c0b6c30 0x7fef9c0c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46354/0 (socket says 172.21.15.116:46354) 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 -- 172.21.15.116:0/112121364 learned_addr learned my addr 172.21.15.116:0/112121364 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef9a800640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c0ce740 0x7fef9c0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99e00640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c071800 0x7fef9c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 -- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c071800 msgr2=0x7fef9c154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fef9c071800 0x7fef9c154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 -- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c0ce740 msgr2=0x7fef9c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c0ce740 0x7fef9c0cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 -- 172.21.15.116:0/112121364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c155650 con 0x7fef9c0b6c30 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef9a800640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c0ce740 0x7fef9c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:24:56.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef99400640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef9c0b6c30 0x7fef9c0c8de0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fef8800e840 tx=0x7fef8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88017070 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8800c730 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8800cb00 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- 172.21.15.116:0/112121364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c155930 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.027+0000 7fefa0c00640 1 -- 172.21.15.116:0/112121364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c155ef0 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fefa0c00640 1 -- 172.21.15.116:0/112121364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fef9c06aa30 con 0x7fef9c0b6c30 2024-05-14T11:24:56.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fef8801a070 con 0x7fef9c0b6c30 2024-05-14T11:24:56.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef92000640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef7c06f400 0x7fef7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7fef8808f600 con 0x7fef9c0b6c30 2024-05-14T11:24:56.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef99e00640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef7c06f400 0x7fef7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+514 (secure 0 0 0) 0x7fef88091ba0 con 0x7fef9c0b6c30 2024-05-14T11:24:56.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fefa0c00640 1 -- 172.21.15.116:0/112121364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fef9c0ccd90 con 0x7fef9c0b6c30 2024-05-14T11:24:56.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 7 ==== log(3 entries from seq 936 at 2024-05-14T11:24:54.222856+0000) ==== 649+0+0 (secure 0 0 0) 0x7fef8808d9a0 con 0x7fef9c0b6c30 2024-05-14T11:24:56.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.031+0000 7fef99e00640 1 --2- 172.21.15.116:0/112121364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fef7c06f400 0x7fef7c071910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0af000 tx=0x7fef8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: cluster: 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: id: 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: health: HEALTH_OK 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: services: 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: mgr: y(active, since 9m), standbys: x 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: mds: 1/1 daemons up, 3 standby 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 9m) 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: data: 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: volumes: 1/1 healthy 2024-05-14T11:24:56.031 INFO:teuthology.orchestra.run.smithi116.stdout: pools: 3 pools, 97 pgs 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: objects: 30 objects, 19 MiB 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: pgs: 97 active+clean 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: io: 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: client: 1.4 KiB/s rd, 1.5 MiB/s wr, 1 op/s rd, 3 op/s wr 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:54.222856+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T11:24:56.032 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:54.222878+0000 mon.a [INF] Cluster is now healthy 2024-05-14T11:24:56.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.039+0000 7f9924377000 1 Processor -- start 2024-05-14T11:24:56.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.039+0000 7f9924377000 1 -- start start 2024-05-14T11:24:56.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f619f0 0x5568730e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f99181f8b20 con 0x556872f782c0 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9918159f50 con 0x556872f619f0 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9918158b50 con 0x556872b4f630 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46362/0 (socket says 172.21.15.116:46362) 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 -- 172.21.15.116:0/3257973034 learned_addr learned my addr 172.21.15.116:0/3257973034 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9915e00640 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9915400640 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f619f0 0x5568730e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 -- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 msgr2=0x5568730e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 -- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f619f0 msgr2=0x5568730e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f619f0 0x5568730e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 -- 172.21.15.116:0/3257973034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556872f4f970 con 0x556872f782c0 2024-05-14T11:24:56.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9916800640 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f991001a660 tx=0x7f9910004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc07b2904aaf990 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9901c00640 1 -- 172.21.15.116:0/3257973034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100a5810 con 0x556872f782c0 2024-05-14T11:24:56.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9901c00640 1 -- 172.21.15.116:0/3257973034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99100a59b0 con 0x556872f782c0 2024-05-14T11:24:56.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9901c00640 1 -- 172.21.15.116:0/3257973034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100a5d80 con 0x556872f782c0 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- 172.21.15.116:0/3257973034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 msgr2=0x5568730d51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f991001a660 tx=0x7f9910004ac0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- 172.21.15.116:0/3257973034 shutdown_connections 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872f619f0 0x5568730e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f782c0 0x5568730d51e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 --2- 172.21.15.116:0/3257973034 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872b4f630 0x5568730e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- 172.21.15.116:0/3257973034 >> 172.21.15.116:0/3257973034 conn(0x556872f85760 msgr2=0x55687308a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- 172.21.15.116:0/3257973034 shutdown_connections 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- 172.21.15.116:0/3257973034 wait complete. 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 -- start start 2024-05-14T11:24:56.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.043+0000 7f9924377000 1 Processor -- start 2024-05-14T11:24:56.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556872edb600 con 0x556872f619f0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556872ed2830 con 0x556872b4f630 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556872e83140 con 0x556872f782c0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 10 client.0 _ll_register_callbacks cb 0x7f9908009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46370/0 (socket says 172.21.15.116:46370) 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 -- 172.21.15.116:0/1174865327 learned_addr learned my addr 172.21.15.116:0/1174865327 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9916800640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915e00640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 -- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 msgr2=0x5568730fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 -- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 msgr2=0x5568730d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 -- 172.21.15.116:0/1174865327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556872e6a9e0 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915400640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f990405b4a0 tx=0x7f9904000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772777d6a3bd6b9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904064230 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f990405a980 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99100a59b0 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f990405a3f0 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 10 client.8022 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x556872b87470 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99100a59b0 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f99040609a0 con 0x556872f619f0 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 10 client.8022 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 10 client.8022 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 20 client.8022 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b61be0 0x55687313ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 4 client.8022 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-14T11:24:56.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x556872edbe10 con 0x556872b61be0 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915e00640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b61be0 0x55687313ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f9915e00640 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b61be0 0x55687313ff70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd469aa0578ca9d4 server_cookie=d0c8ccf99af25345 in_seq=0 out_seq=0 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.047+0000 7f98f6a00640 10 client.8022 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f98f6a00640 1 -- 172.21.15.116:0/1174865327 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556872edbe10 con 0x556872b61be0 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f98f6a00640 10 client.8022 handle_command_reply: tid=0 2024-05-14T11:24:56.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 client.8022 shutdown 2024-05-14T11:24:56.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 msgr2=0x5568730fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f990405b4a0 tx=0x7f9904000ee0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.049 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-14T11:24:56.049 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-14T11:24:56.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 shutdown_connections 2024-05-14T11:24:56.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x556872b61be0 0x55687313ff70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556872f782c0 0x5568730fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f98f6a00640 0 client.8022 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556872f619f0 0x5568730fbcd0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 --2- 172.21.15.116:0/1174865327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556872b4f630 0x5568730d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 >> 172.21.15.116:0/1174865327 conn(0x556872f85760 msgr2=0x55687308ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.050 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-14T11:24:56.050 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 shutdown_connections 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 1 -- 172.21.15.116:0/1174865327 wait complete. 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f9924377000 20 client.8022 trim_cache size 0 max 16384 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "4478e34e-a5b8-47be-9903-df3feb791b97", 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-05-14T11:24:56.051 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 msgr2=0x7f98f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 0x7f98f8071940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9908009ac0 tx=0x7f9908009190 comp rx=0 tx=0).stop 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 msgr2=0x7f991814b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f990400c390 tx=0x7f990400c850 comp rx=0 tx=0).stop 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 shutdown_connections 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f98f806f430 0x7f98f8071940 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f99180cfda0 0x7f99181559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99180cf280 0x7f991814b8b0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 --2- 172.21.15.116:0/3318603657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99180b5f30 0x7f99181525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 >> 172.21.15.116:0/3318603657 conn(0x7f99180cde80 msgr2=0x7f99180b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 shutdown_connections 2024-05-14T11:24:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.051+0000 7f991cc00640 1 -- 172.21.15.116:0/3318603657 wait complete. 2024-05-14T11:24:56.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:24:56.069 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-14T11:24:56.216 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-05-14T11:24:56.219+0000 7ff952a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:24:56.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.219+0000 7fee57e00640 1 Processor -- start 2024-05-14T11:24:56.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.219+0000 7fee57e00640 1 -- start start 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fee580d1bd0 0x7fee580c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee5815e080 con 0x7fee580d01b0 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee580d0e30 con 0x7fee5815eb20 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fee5815e690 con 0x7fee580d1bd0 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fee580d1bd0 0x7fee580c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee4fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fee580d01b0 0x7fee580d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50470/0 (socket says 172.21.15.116:50470) 2024-05-14T11:24:56.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee4fe00640 1 -- 172.21.15.116:0/88996328 learned_addr learned my addr 172.21.15.116:0/88996328 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:24:56.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57400640 1 -- 172.21.15.116:0/88996328 >> v1:172.21.15.149:6789/0 conn(0x7fee5815eb20 legacy=0x7fee58157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57400640 1 -- 172.21.15.116:0/88996328 >> v1:172.21.15.116:6789/0 conn(0x7fee580d01b0 legacy=0x7fee580d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57400640 1 -- 172.21.15.116:0/88996328 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580d0b50 con 0x7fee580d1bd0 2024-05-14T11:24:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.223+0000 7fee57400640 1 --2- 172.21.15.116:0/88996328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580d1bd0 0x7fee580c6f90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee5001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e08bf15a888498 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee56000640 1 -- 172.21.15.116:0/88996328 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500b2070 con 0x7fee580d1bd0 2024-05-14T11:24:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee56000640 1 -- 172.21.15.116:0/88996328 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500a5820 con 0x7fee580d1bd0 2024-05-14T11:24:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee56000640 1 -- 172.21.15.116:0/88996328 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500a5bf0 con 0x7fee580d1bd0 2024-05-14T11:24:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- 172.21.15.116:0/88996328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580d1bd0 msgr2=0x7fee580c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 --2- 172.21.15.116:0/88996328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580d1bd0 0x7fee580c6f90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee5001c850 comp rx=0 tx=0).stop 2024-05-14T11:24:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- 172.21.15.116:0/88996328 shutdown_connections 2024-05-14T11:24:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 --2- 172.21.15.116:0/88996328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580d1bd0 0x7fee580c6f90 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- 172.21.15.116:0/88996328 >> 172.21.15.116:0/88996328 conn(0x7fee580b4510 msgr2=0x7fee580b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- 172.21.15.116:0/88996328 shutdown_connections 2024-05-14T11:24:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- 172.21.15.116:0/88996328 wait complete. 2024-05-14T11:24:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 Processor -- start 2024-05-14T11:24:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.227+0000 7fee57e00640 1 -- start start 2024-05-14T11:24:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 0x7fee580b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 0x7fee580b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee58162070 con 0x7fee580af450 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee58161ef0 con 0x7fee5815eb20 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fee581621f0 con 0x7fee580d01b0 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46396/0 (socket says 172.21.15.116:46396) 2024-05-14T11:24:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 -- 172.21.15.116:0/1200406695 learned_addr learned my addr 172.21.15.116:0/1200406695 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 -- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 msgr2=0x7fee580b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee56a00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 0x7fee580b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee4fe00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 0x7fee580b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 0x7fee580b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 -- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 msgr2=0x7fee580b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 0x7fee580b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 -- 172.21.15.116:0/1200406695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5806db80 con 0x7fee580af450 2024-05-14T11:24:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee4fe00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 0x7fee580b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee56a00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 0x7fee580b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57400640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee500195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee4ea00640 1 -- 172.21.15.116:0/1200406695 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500bd020 con 0x7fee580af450 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee4ea00640 1 -- 172.21.15.116:0/1200406695 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500b8670 con 0x7fee580af450 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee4ea00640 1 -- 172.21.15.116:0/1200406695 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500b2040 con 0x7fee580af450 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee580afda0 con 0x7fee580af450 2024-05-14T11:24:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.231+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5814d9b0 con 0x7fee580af450 2024-05-14T11:24:56.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee4ea00640 1 -- 172.21.15.116:0/1200406695 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee500c0020 con 0x7fee580af450 2024-05-14T11:24:56.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee64cb0000 1 Processor -- start 2024-05-14T11:24:56.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee4ea00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 0x7fee34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee64cb0000 1 -- start start 2024-05-14T11:24:56.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee4ea00640 1 -- 172.21.15.116:0/1200406695 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7fee5012a4d0 con 0x7fee580af450 2024-05-14T11:24:56.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee56a00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 0x7fee34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.235+0000 7fee56a00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 0x7fee34071940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee5806d050 tx=0x7fee48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d993b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d993bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0d973c100 con 0x55a0d93bd3e0 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0d96f4ca0 con 0x55a0d97bdf00 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a0d97380c0 con 0x55a0d97d0120 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46402/0 (socket says 172.21.15.116:46402) 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 -- 172.21.15.116:0/4270917350 learned_addr learned my addr 172.21.15.116:0/4270917350 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee56a00640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d993b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee4fe00640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d993bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 -- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 msgr2=0x55a0d993b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d993b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 -- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 msgr2=0x55a0d993bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d993bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 -- 172.21.15.116:0/4270917350 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d96f9800 con 0x55a0d93bd3e0 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee56a00640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d993b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee4fe00640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d993bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee57400640 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fee500b0040 tx=0x7fee500f6370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29357a2b46f9feff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee47000640 1 -- 172.21.15.116:0/4270917350 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500ff1d0 con 0x55a0d93bd3e0 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee47000640 1 -- 172.21.15.116:0/4270917350 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500f7120 con 0x55a0d93bd3e0 2024-05-14T11:24:56.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee47000640 1 -- 172.21.15.116:0/4270917350 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500f6a20 con 0x55a0d93bd3e0 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- 172.21.15.116:0/4270917350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 msgr2=0x55a0d993c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fee500b0040 tx=0x7fee500f6370 comp rx=0 tx=0).stop 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- 172.21.15.116:0/4270917350 shutdown_connections 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d993bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d93bd3e0 0x55a0d993c230 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 --2- 172.21.15.116:0/4270917350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d993b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- 172.21.15.116:0/4270917350 >> 172.21.15.116:0/4270917350 conn(0x55a0d93fd400 msgr2=0x55a0d991a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- 172.21.15.116:0/4270917350 shutdown_connections 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- 172.21.15.116:0/4270917350 wait complete. 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 -- start start 2024-05-14T11:24:56.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.239+0000 7fee64cb0000 1 Processor -- start 2024-05-14T11:24:56.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9942c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97bdf00 0x55a0d9943150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0d98072a0 con 0x55a0d97d0120 2024-05-14T11:24:56.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0d97f7f40 con 0x55a0d93bd3e0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a0d96f8080 con 0x55a0d97bdf00 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 10 client.0 _ll_register_callbacks cb 0x55a0d98d8c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46414/0 (socket says 172.21.15.116:46414) 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 -- 172.21.15.116:0/7406564 learned_addr learned my addr 172.21.15.116:0/7406564 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee56a00640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9942c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee4fe00640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97bdf00 0x55a0d9943150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 -- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97bdf00 msgr2=0x55a0d9943150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97bdf00 0x55a0d9943150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 -- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 msgr2=0x55a0d9942c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9942c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 -- 172.21.15.116:0/7406564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d96e6860 con 0x55a0d97d0120 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee57400640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fee500f6720 tx=0x7fee5001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ded2076fe449b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500fadd0 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7fee500ff3f0 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee500f7120 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fee500ff880 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 10 client.8037 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55a0d96f4ef0 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500f7120 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fee50128bc0 con 0x55a0d97d0120 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 10 client.8037 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 10 client.8037 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 20 client.8037 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93e1930 0x55a0d98faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 4 client.8037 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d96d4df0 con 0x55a0d93e1930 2024-05-14T11:24:56.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee56a00640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93e1930 0x55a0d98faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee56a00640 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93e1930 0x55a0d98faa10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca44ee5bbde8d75 server_cookie=fba07477d7fee6ee in_seq=0 out_seq=0 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.243+0000 7fee3b400640 10 client.8037 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee3b400640 1 -- 172.21.15.116:0/7406564 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0d96d4df0 con 0x55a0d93e1930 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee3b400640 10 client.8037 handle_command_reply: tid=0 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 client.8037 shutdown 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 msgr2=0x55a0d98f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fee500f6720 tx=0x7fee5001c850 comp rx=0 tx=0).stop 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 shutdown_connections 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93e1930 0x55a0d98faa10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d98f9ca0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97bdf00 0x55a0d9943150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 --2- 172.21.15.116:0/7406564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9942c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee3b400640 0 client.8037 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 >> 172.21.15.116:0/7406564 conn(0x55a0d93fd400 msgr2=0x55a0d991a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 shutdown_connections 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 1 -- 172.21.15.116:0/7406564 wait complete. 2024-05-14T11:24:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.247+0000 7fee64cb0000 20 client.8037 trim_cache size 0 max 16384 2024-05-14T11:24:56.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 Processor -- start 2024-05-14T11:24:56.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 -- start start 2024-05-14T11:24:56.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9952db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d9947a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee580cca40 con 0x55a0d97bdf00 2024-05-14T11:24:56.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee5806f590 con 0x55a0d93bd3e0 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.263+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fee5814f510 con 0x55a0d97d0120 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46420/0 (socket says 172.21.15.116:46420) 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 -- 172.21.15.116:0/3053663375 learned_addr learned my addr 172.21.15.116:0/3053663375 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee56a00640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9952db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee57400640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d9947a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 -- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 msgr2=0x55a0d9947a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d9947a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 -- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 msgr2=0x55a0d9952db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9952db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 -- 172.21.15.116:0/3053663375 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d97bd340 con 0x55a0d97bdf00 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee57400640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d9947a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee4fe00640 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fee40005e20 tx=0x7fee40007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc09b85e9444ff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee46600640 1 -- 172.21.15.116:0/3053663375 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40019020 con 0x55a0d97bdf00 2024-05-14T11:24:56.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee46600640 1 -- 172.21.15.116:0/3053663375 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee40011830 con 0x55a0d97bdf00 2024-05-14T11:24:56.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee46600640 1 -- 172.21.15.116:0/3053663375 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40011c20 con 0x55a0d97bdf00 2024-05-14T11:24:56.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- 172.21.15.116:0/3053663375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 msgr2=0x55a0d99532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fee40005e20 tx=0x7fee40007420 comp rx=0 tx=0).stop 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- 172.21.15.116:0/3053663375 shutdown_connections 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d97d0120 0x55a0d9947a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97bdf00 0x55a0d99532f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 --2- 172.21.15.116:0/3053663375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d93bd3e0 0x55a0d9952db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- 172.21.15.116:0/3053663375 >> 172.21.15.116:0/3053663375 conn(0x55a0d93fd400 msgr2=0x55a0d9916af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- 172.21.15.116:0/3053663375 shutdown_connections 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- 172.21.15.116:0/3053663375 wait complete. 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 -- start start 2024-05-14T11:24:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.267+0000 7fee64cb0000 1 Processor -- start 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d93bd3e0 0x55a0d9952df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d9953330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0d9740730 con 0x55a0d97d0120 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0d973b410 con 0x55a0d97bdf00 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55a0d9749500 con 0x55a0d93bd3e0 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 10 client.0 _ll_register_callbacks cb 0x7fee50009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46432/0 (socket says 172.21.15.116:46432) 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 -- 172.21.15.116:0/2164594253 learned_addr learned my addr 172.21.15.116:0/2164594253 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:24:56.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee4fe00640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d9953330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee56a00640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d93bd3e0 0x55a0d9952df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 -- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d93bd3e0 msgr2=0x55a0d9952df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d93bd3e0 0x55a0d9952df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 -- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 msgr2=0x55a0d9953330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d9953330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 -- 172.21.15.116:0/2164594253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d96d88e0 con 0x55a0d97d0120 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee57400640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fee5001d0f0 tx=0x7fee50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb636920a98db4e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500f6a20 con 0x55a0d97d0120 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7fee50115ce0 con 0x55a0d97d0120 2024-05-14T11:24:56.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee40011830 con 0x55a0d97d0120 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fee500f6190 con 0x55a0d97d0120 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 10 client.8043 fetch_fsmap learned FSMap version 89 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55a0d93f5220 con 0x55a0d97d0120 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee40011830 con 0x55a0d97d0120 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7fee500f5c80 con 0x55a0d97d0120 2024-05-14T11:24:56.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 10 client.8043 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 10 client.8043 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 20 client.8043 populate_metadata read hostname 'smithi116' 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93cf990 0x55a0d9944f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 4 client.8043 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0d9749d10 con 0x55a0d93cf990 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee56a00640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93cf990 0x55a0d9944f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee56a00640 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93cf990 0x55a0d9944f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9010a77eaa888562 server_cookie=6d1efaee655e4c9e in_seq=0 out_seq=0 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 10 client.8043 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 1 -- 172.21.15.116:0/2164594253 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55a0d9749d10 con 0x55a0d93cf990 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 10 client.8043 handle_command_reply: tid=0 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 client.8043 shutdown 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 msgr2=0x55a0d9944a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fee5001d0f0 tx=0x7fee50009750 comp rx=0 tx=0).stop 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 shutdown_connections 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d97bdf00 0x55a0d9953330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55a0d93cf990 0x55a0d9944f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d97d0120 0x55a0d9944a40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 --2- 172.21.15.116:0/2164594253 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55a0d93bd3e0 0x55a0d9952df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 >> 172.21.15.116:0/2164594253 conn(0x55a0d93fd400 msgr2=0x55a0d9917280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee3b400640 0 client.8043 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 shutdown_connections 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 1 -- 172.21.15.116:0/2164594253 wait complete. 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.271+0000 7fee64cb0000 20 client.8043 trim_cache size 0 max 16384 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "4478e34e-a5b8-47be-9903-df3feb791b97": { 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "4478e34e-a5b8-47be-9903-df3feb791b97", 2024-05-14T11:24:56.273 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,repair" 2024-05-14T11:24:56.274 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:56.274 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:24:56.274 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:24:56.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 msgr2=0x7fee34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 0x7fee34071940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee5806d050 tx=0x7fee48009040 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 msgr2=0x7fee580af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee500195f0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 shutdown_connections 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fee3406f430 0x7fee34071940 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee5815eb20 0x7fee580b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee580af450 0x7fee580af860 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 --2- 172.21.15.116:0/1200406695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee580d01b0 0x7fee580b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 >> 172.21.15.116:0/1200406695 conn(0x7fee580b4510 msgr2=0x7fee58152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 shutdown_connections 2024-05-14T11:24:56.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:56.275+0000 7fee57e00640 1 -- 172.21.15.116:0/1200406695 wait complete. 2024-05-14T11:24:56.290 INFO:tasks.cephfs.filesystem:scrub status for tag:4478e34e-a5b8-47be-9903-df3feb791b97 - {'path': '/', 'tag': '4478e34e-a5b8-47be-9903-df3feb791b97', 'options': 'recursive,repair'} 2024-05-14T11:24:57.237 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.051346+0000 mds.a [INF] scrub queued for path: / 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.051349+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.051354+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.052027+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.052032+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:24:57.239+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-14T11:24:56.051346+0000) ==== 1407+0+0 (secure 0 0 0) 0x7fef8800cca0 con 0x7fef9c0b6c30 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.052299+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-14T11:24:57.238 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.052305+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-05-14T11:24:57.239 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:24:56.053223+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-14T11:25:02.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:02.251+0000 7fef92000640 1 -- 172.21.15.116:0/112121364 <== mon.0 v2:172.21.15.116:3300/0 9 ==== log(1 entries from seq 9 at 2024-05-14T11:25:00.348932+0000) ==== 189+0+0 (secure 0 0 0) 0x7fef8805be60 con 0x7fef9c0b6c30 2024-05-14T11:25:02.248 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:25:00.348932+0000 mds.a [INF] scrub summary: idle 2024-05-14T11:25:13.689 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:25:13.693 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:25:26.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:26.291 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-14T11:25:26.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.454+0000 7ffbc8a00640 1 Processor -- start 2024-05-14T11:25:26.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.454+0000 7ffbc8a00640 1 -- start start 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc8a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc8a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc406ff20 con 0x7ffbc40d1670 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc8a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc40ae100 con 0x7ffbc415db30 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc40ae870 con 0x7ffbc406c700 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc1e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc1e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:46478/0 (socket says 172.21.15.116:46478) 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:33674/0 (socket says 172.21.15.116:33674) 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc1e00640 1 -- 172.21.15.116:0/1165425755 learned_addr learned my addr 172.21.15.116:0/1165425755 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc1400640 1 --1- 172.21.15.116:0/1165425755 >> v1:172.21.15.149:6789/0 conn(0x7ffbc40d1670 0x7ffbc40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:58320/0 (socket says 172.21.15.116:58320) 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 -- 172.21.15.116:0/1165425755 >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 msgr2=0x7ffbc40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 --2- 172.21.15.116:0/1165425755 >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 -- 172.21.15.116:0/1165425755 >> v1:172.21.15.149:6789/0 conn(0x7ffbc40d1670 legacy=0x7ffbc40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc2800640 1 -- 172.21.15.116:0/1165425755 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc4070470 con 0x7ffbc406c700 2024-05-14T11:25:26.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.458+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/1165425755 >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:26.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc2800640 1 --2- 172.21.15.116:0/1165425755 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0097c0 tx=0x7ffbbc01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae0a1dd77c6f600 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc0a00640 1 -- 172.21.15.116:0/1165425755 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b0070 con 0x7ffbc406c700 2024-05-14T11:25:26.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc0a00640 1 -- 172.21.15.116:0/1165425755 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0ab600 con 0x7ffbc406c700 2024-05-14T11:25:26.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc0a00640 1 -- 172.21.15.116:0/1165425755 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0ab9d0 con 0x7ffbc406c700 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 -- 172.21.15.116:0/1165425755 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 msgr2=0x7ffbc406cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/1165425755 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0097c0 tx=0x7ffbbc01c3d0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 -- 172.21.15.116:0/1165425755 shutdown_connections 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/1165425755 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc406cb10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/1165425755 >> v2:172.21.15.149:3300/0 conn(0x7ffbc415db30 0x7ffbc40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 -- 172.21.15.116:0/1165425755 >> 172.21.15.116:0/1165425755 conn(0x7ffbc40b4620 msgr2=0x7ffbc40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 -- 172.21.15.116:0/1165425755 shutdown_connections 2024-05-14T11:25:26.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.462+0000 7ffbc8a00640 1 -- 172.21.15.116:0/1165425755 wait complete. 2024-05-14T11:25:26.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 Processor -- start 2024-05-14T11:25:26.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 -- start start 2024-05-14T11:25:26.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbc415db30 0x7ffbc41501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc416a770 con 0x7ffbc40d1670 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc415df40 con 0x7ffbc415db30 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc416a8f0 con 0x7ffbc406c700 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39364/0 (socket says 172.21.15.116:39364) 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 -- 172.21.15.116:0/693934104 learned_addr learned my addr 172.21.15.116:0/693934104 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc2800640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbc415db30 0x7ffbc41501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 -- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 msgr2=0x7ffbc415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 -- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbc415db30 msgr2=0x7ffbc41501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbc415db30 0x7ffbc41501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 -- 172.21.15.116:0/693934104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc40cf580 con 0x7ffbc40d1670 2024-05-14T11:25:26.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc1400640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c430 tx=0x7ffbac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbba000640 1 -- 172.21.15.116:0/693934104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0074e0 con 0x7ffbc40d1670 2024-05-14T11:25:26.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbba000640 1 -- 172.21.15.116:0/693934104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac007680 con 0x7ffbc40d1670 2024-05-14T11:25:26.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbba000640 1 -- 172.21.15.116:0/693934104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac004a10 con 0x7ffbc40d1670 2024-05-14T11:25:26.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.466+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc40cf860 con 0x7ffbc40d1670 2024-05-14T11:25:26.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc40cfe20 con 0x7ffbc40d1670 2024-05-14T11:25:26.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbba000640 1 -- 172.21.15.116:0/693934104 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffbac01a020 con 0x7ffbc40d1670 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbba000640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 0x7ffba0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbba000640 1 -- 172.21.15.116:0/693934104 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7ffbac08eb90 con 0x7ffbc40d1670 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 0x7ffba0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbd014c000 1 Processor -- start 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.470+0000 7ffbd014c000 1 -- start start 2024-05-14T11:25:26.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 0x7ffba0071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40072c0 tx=0x7ffbb400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a085af00 0x55d9a09d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a045a3e0 0x55d9a09d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d9a07d9100 con 0x55d9a086d120 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d9a0791ca0 con 0x55d9a045a3e0 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d9a07d50c0 con 0x55d9a085af00 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39372/0 (socket says 172.21.15.116:39372) 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 -- 172.21.15.116:0/1659722185 learned_addr learned my addr 172.21.15.116:0/1659722185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1400640 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a085af00 0x55d9a09d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc2800640 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a045a3e0 0x55d9a09d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 -- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a085af00 msgr2=0x55d9a09d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a085af00 0x55d9a09d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 -- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a045a3e0 msgr2=0x55d9a09d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a045a3e0 0x55d9a09d4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 -- 172.21.15.116:0/1659722185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a0796800 con 0x55d9a086d120 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009380 tx=0x7ffbb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca6213b8b09dc65 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbb2600640 1 -- 172.21.15.116:0/1659722185 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb401b070 con 0x55d9a086d120 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbb2600640 1 -- 172.21.15.116:0/1659722185 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4027070 con 0x55d9a086d120 2024-05-14T11:25:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbb2600640 1 -- 172.21.15.116:0/1659722185 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4015a20 con 0x55d9a086d120 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- 172.21.15.116:0/1659722185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 msgr2=0x55d9a09d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009380 tx=0x7ffbb4014000 comp rx=0 tx=0).stop 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- 172.21.15.116:0/1659722185 shutdown_connections 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a045a3e0 0x55d9a09d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a085af00 0x55d9a09d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 --2- 172.21.15.116:0/1659722185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a09d4350 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- 172.21.15.116:0/1659722185 >> 172.21.15.116:0/1659722185 conn(0x55d9a049a400 msgr2=0x55d9a09b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- 172.21.15.116:0/1659722185 shutdown_connections 2024-05-14T11:25:26.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.474+0000 7ffbd014c000 1 -- 172.21.15.116:0/1659722185 wait complete. 2024-05-14T11:25:26.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- start start 2024-05-14T11:25:26.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 Processor -- start 2024-05-14T11:25:26.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09dadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a098dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d9a08a42a0 con 0x55d9a085af00 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d9a0894f40 con 0x55d9a086d120 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d9a0795080 con 0x55d9a045a3e0 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 10 client.0 _ll_register_callbacks cb 0x55d9a0975c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39386/0 (socket says 172.21.15.116:39386) 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 -- 172.21.15.116:0/99786849 learned_addr learned my addr 172.21.15.116:0/99786849 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09dadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc2800640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a098dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 -- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 msgr2=0x55d9a09dadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09dadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 -- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 msgr2=0x55d9a098dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a098dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 -- 172.21.15.116:0/99786849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a0783860 con 0x55d9a085af00 2024-05-14T11:25:26.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbc1400640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0572a0 tx=0x7ffbac05b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596bbcc47abe809f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac063cb0 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7ffbac014cb0 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb4027070 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ffbac063f40 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 10 client.8052 fetch_fsmap learned FSMap version 89 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55d9a0791ef0 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4027070 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7ffbac004bb0 con 0x55d9a085af00 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 10 client.8052 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 10 client.8052 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 20 client.8052 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a047e930 0x55d9a098ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 4 client.8052 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:25:26.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.478+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a0771df0 con 0x55d9a047e930 2024-05-14T11:25:26.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a047e930 0x55d9a098ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffba6a00640 10 client.8052 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a047e930 0x55d9a098ed30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce57e1609df89e7 server_cookie=a497741e9435e256 in_seq=0 out_seq=0 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffba6a00640 1 -- 172.21.15.116:0/99786849 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d9a0771df0 con 0x55d9a047e930 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffba6a00640 10 client.8052 handle_command_reply: tid=0 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 client.8052 shutdown 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 msgr2=0x55d9a09db310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0572a0 tx=0x7ffbac05b840 comp rx=0 tx=0).stop 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 shutdown_connections 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a098dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a047e930 0x55d9a098ed30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a09db310 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 --2- 172.21.15.116:0/99786849 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09dadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 >> 172.21.15.116:0/99786849 conn(0x55d9a049a400 msgr2=0x55d9a09b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffba6a00640 0 client.8052 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 shutdown_connections 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 1 -- 172.21.15.116:0/99786849 wait complete. 2024-05-14T11:25:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.482+0000 7ffbd014c000 20 client.8052 trim_cache size 0 max 16384 2024-05-14T11:25:26.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.498+0000 7ffbd014c000 1 Processor -- start 2024-05-14T11:25:26.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.498+0000 7ffbd014c000 1 -- start start 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a0a42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a0a437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc4205320 con 0x55d9a085af00 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc41675d0 con 0x55d9a086d120 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc4204d00 con 0x55d9a045a3e0 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39392/0 (socket says 172.21.15.116:39392) 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 -- 172.21.15.116:0/883572527 learned_addr learned my addr 172.21.15.116:0/883572527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a0a42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc2800640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a0a437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 -- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 msgr2=0x55d9a0a42d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a0a42d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 -- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 msgr2=0x55d9a0a437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a0a437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 -- 172.21.15.116:0/883572527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a098c070 con 0x55d9a085af00 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc2800640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a0a437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a0a42d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbc1400640 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c400 tx=0x7ffbac05e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcd2c00e6d48fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbb1c00640 1 -- 172.21.15.116:0/883572527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac063ab0 con 0x55d9a085af00 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbb1c00640 1 -- 172.21.15.116:0/883572527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac063c50 con 0x55d9a085af00 2024-05-14T11:25:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbb1c00640 1 -- 172.21.15.116:0/883572527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac05a680 con 0x55d9a085af00 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- 172.21.15.116:0/883572527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 msgr2=0x55d9a0a43270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c400 tx=0x7ffbac05e000 comp rx=0 tx=0).stop 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- 172.21.15.116:0/883572527 shutdown_connections 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a086d120 0x55d9a0a437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a085af00 0x55d9a0a43270 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 --2- 172.21.15.116:0/883572527 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a0a42d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- 172.21.15.116:0/883572527 >> 172.21.15.116:0/883572527 conn(0x55d9a049a400 msgr2=0x55d9a098cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- 172.21.15.116:0/883572527 shutdown_connections 2024-05-14T11:25:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.502+0000 7ffbd014c000 1 -- 172.21.15.116:0/883572527 wait complete. 2024-05-14T11:25:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- start start 2024-05-14T11:25:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 Processor -- start 2024-05-14T11:25:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a085af00 0x55d9a0a038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d9a07dd730 con 0x55d9a086d120 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d9a07d8410 con 0x55d9a085af00 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d9a07e6500 con 0x55d9a045a3e0 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 10 client.0 _ll_register_callbacks cb 0x7ffbac0578f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc1e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39402/0 (socket says 172.21.15.116:39402) 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 -- 172.21.15.116:0/361825420 learned_addr learned my addr 172.21.15.116:0/361825420 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc1400640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a085af00 0x55d9a0a038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 -- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 msgr2=0x55d9a09df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 -- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a085af00 msgr2=0x55d9a0a038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a085af00 0x55d9a0a038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 -- 172.21.15.116:0/361825420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a085a340 con 0x55d9a086d120 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbc2800640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc008010 tx=0x7ffbbc01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c7cf0776d833b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffba6a00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0bc070 con 0x55d9a086d120 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbac063c50 con 0x55d9a086d120 2024-05-14T11:25:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7ffbbc0c1cb0 con 0x55d9a086d120 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc0b0040 con 0x55d9a086d120 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 10 client.8058 fetch_fsmap learned FSMap version 89 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55d9a07758e0 con 0x55d9a086d120 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.506+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac063c50 con 0x55d9a086d120 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7ffbbc0abbf0 con 0x55d9a086d120 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 10 client.8058 fetch_fsmap finished waiting for FSMap version 89 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 10 client.8058 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 20 client.8058 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a046c990 0x55d9a0a04350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 4 client.8058 mds_command: new command op to 7932 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 --> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9a07e6d10 con 0x55d9a046c990 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a046c990 0x55d9a0a04350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc1e00640 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a046c990 0x55d9a0a04350 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb2af0eef3f0e51 server_cookie=776fe52048999fa7 in_seq=0 out_seq=0 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffba6a00640 10 client.8058 ms_handle_connect on v2:172.21.15.116:6834/288710459 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffba6a00640 1 -- 172.21.15.116:0/361825420 <== mds.0 v2:172.21.15.116:6834/288710459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d9a07e6d10 con 0x55d9a046c990 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffba6a00640 10 client.8058 handle_command_reply: tid=0 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 client.8058 shutdown 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 msgr2=0x55d9a0a03e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc008010 tx=0x7ffbbc01ac10 comp rx=0 tx=0).stop 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 shutdown_connections 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d9a085af00 0x55d9a0a038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:6834/288710459,v1:172.21.15.116:6835/288710459] conn(0x55d9a046c990 0x55d9a0a04350 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d9a086d120 0x55d9a0a03e10 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 --2- 172.21.15.116:0/361825420 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d9a045a3e0 0x55d9a09df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffba6a00640 0 client.8058 ms_handle_reset on v2:172.21.15.116:6834/288710459 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 >> 172.21.15.116:0/361825420 conn(0x55d9a049a400 msgr2=0x55d9a098d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 shutdown_connections 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 1 -- 172.21.15.116:0/361825420 wait complete. 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbd014c000 20 client.8058 trim_cache size 0 max 16384 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-05-14T11:25:26.511 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 msgr2=0x7ffba0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 0x7ffba0071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40072c0 tx=0x7ffbb400b040 comp rx=0 tx=0).stop 2024-05-14T11:25:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 msgr2=0x7ffbc414e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c430 tx=0x7ffbac00c8f0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 shutdown_connections 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ffba006f400 0x7ffba0071910 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbc415db30 0x7ffbc41501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc40d1670 0x7ffbc414e8d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 --2- 172.21.15.116:0/693934104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbc406c700 0x7ffbc415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 >> 172.21.15.116:0/693934104 conn(0x7ffbc40b4620 msgr2=0x7ffbc415f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 shutdown_connections 2024-05-14T11:25:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.510+0000 7ffbc8a00640 1 -- 172.21.15.116:0/693934104 wait complete. 2024-05-14T11:25:26.528 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T11:25:26.629 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:25:26.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.778+0000 7f2d9de00640 1 Processor -- start 2024-05-14T11:25:26.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.778+0000 7f2d9de00640 1 -- start start 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d9de00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d98166750 0x7f2d98168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d9de00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9806ff20 con 0x7f2d980d1670 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d9de00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d98070470 con 0x7f2d9815db30 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d9de00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9806d710 con 0x7f2d98166750 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d97400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d98166750 0x7f2d98168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d96000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2d980d1670 0x7f2d980d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:51634/0 (socket says 172.21.15.116:51634) 2024-05-14T11:25:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d96000640 1 -- 172.21.15.116:0/1619492360 learned_addr learned my addr 172.21.15.116:0/1619492360 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d97400640 1 -- 172.21.15.116:0/1619492360 >> v1:172.21.15.149:6789/0 conn(0x7f2d9815db30 legacy=0x7f2d980aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d97400640 1 -- 172.21.15.116:0/1619492360 >> v1:172.21.15.116:6789/0 conn(0x7f2d980d1670 legacy=0x7f2d980d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.782+0000 7f2d97400640 1 -- 172.21.15.116:0/1619492360 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d98070750 con 0x7f2d98166750 2024-05-14T11:25:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d97400640 1 --2- 172.21.15.116:0/1619492360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d98166750 0x7f2d98168b70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097c0 tx=0x7f2d9001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c7dfc43348a4b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d95600640 1 -- 172.21.15.116:0/1619492360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900b0070 con 0x7f2d98166750 2024-05-14T11:25:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d95600640 1 -- 172.21.15.116:0/1619492360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d900ab600 con 0x7f2d98166750 2024-05-14T11:25:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d95600640 1 -- 172.21.15.116:0/1619492360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900ab9d0 con 0x7f2d98166750 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 -- 172.21.15.116:0/1619492360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d98166750 msgr2=0x7f2d98168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 --2- 172.21.15.116:0/1619492360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d98166750 0x7f2d98168b70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097c0 tx=0x7f2d9001c380 comp rx=0 tx=0).stop 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 -- 172.21.15.116:0/1619492360 shutdown_connections 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 --2- 172.21.15.116:0/1619492360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d98166750 0x7f2d98168b70 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 -- 172.21.15.116:0/1619492360 >> 172.21.15.116:0/1619492360 conn(0x7f2d980b4620 msgr2=0x7f2d980b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 -- 172.21.15.116:0/1619492360 shutdown_connections 2024-05-14T11:25:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.786+0000 7f2d9de00640 1 -- 172.21.15.116:0/1619492360 wait complete. 2024-05-14T11:25:26.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 Processor -- start 2024-05-14T11:25:26.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- start start 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d9815db30 0x7f2d981fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d98166750 0x7f2d98073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9816bae0 con 0x7f2d980d1670 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980ae870 con 0x7f2d98166750 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9816bc60 con 0x7f2d9815db30 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39428/0 (socket says 172.21.15.116:39428) 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 -- 172.21.15.116:0/3537604694 learned_addr learned my addr 172.21.15.116:0/3537604694 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d97400640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d98166750 0x7f2d98073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96000640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d9815db30 0x7f2d981fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 -- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d9815db30 msgr2=0x7f2d981fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d9815db30 0x7f2d981fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 -- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d98166750 msgr2=0x7f2d98073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d98166750 0x7f2d98073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 -- 172.21.15.116:0/3537604694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981f9d80 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d96a00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c760 tx=0x7f2d8800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88003b80 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d88003d40 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800aa90 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981fa280 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.790+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d98163c10 con 0x7f2d980d1670 2024-05-14T11:25:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9806a9c0 con 0x7f2d980d1670 2024-05-14T11:25:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2d8800ac30 con 0x7f2d980d1670 2024-05-14T11:25:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d8f400640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 0x7f2d78071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6216+0+0 (secure 0 0 0) 0x7f2d8808f400 con 0x7f2d980d1670 2024-05-14T11:25:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d96000640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 0x7f2d78071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.794+0000 7f2d96000640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 0x7f2d78071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007fd0 tx=0x7f2d84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.798+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2d8805c0e0 con 0x7f2d980d1670 2024-05-14T11:25:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.990+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2d98168cb0 con 0x7f2d980d1670 2024-05-14T11:25:26.988 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:25:26.990+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:25:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.990+0000 7f2d8f400640 1 -- 172.21.15.116:0/3537604694 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f2d880640d0 con 0x7f2d980d1670 2024-05-14T11:25:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:25:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 msgr2=0x7f2d78071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 0x7f2d78071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007fd0 tx=0x7f2d84009040 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 msgr2=0x7f2d981f9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c760 tx=0x7f2d8800cc20 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 shutdown_connections 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2d7806f400 0x7f2d78071910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d98166750 0x7f2d98073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d9815db30 0x7f2d981fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 --2- 172.21.15.116:0/3537604694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d980d1670 0x7f2d981f9840 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 >> 172.21.15.116:0/3537604694 conn(0x7f2d980b4620 msgr2=0x7f2d9815b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 shutdown_connections 2024-05-14T11:25:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:26.994+0000 7f2d9de00640 1 -- 172.21.15.116:0/3537604694 wait complete. 2024-05-14T11:25:27.000 INFO:tasks.ceph.mds.a.smithi116.stderr: -1163> 2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-14T11:25:27.000 INFO:tasks.ceph.mds.a.smithi116.stderr: -1160> 2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-14T11:25:27.000 INFO:tasks.ceph.mds.a.smithi116.stderr: -1133> 2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-14T11:25:27.000 INFO:tasks.ceph.mds.a.smithi116.stderr: -1130> 2024-05-14T11:24:56.051+0000 7ff94da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-14T11:25:27.000 INFO:tasks.ceph.mds.a.smithi116.stderr: -1036> 2024-05-14T11:24:56.219+0000 7ff952a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:25:27.015 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-14T11:25:27.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.074+0000 7f8e9b616400 1 -- start start 2024-05-14T11:25:27.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.074+0000 7f8e9b616400 1 Processor -- start 2024-05-14T11:25:27.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 0x55bfca3e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f0000 0x55bfca3e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96de180 con 0x55bfc9677a80 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96de300 con 0x55bfc9677180 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bfc96de480 con 0x55bfca3f0000 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39446/0 (socket says 172.21.15.116:39446) 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 -- 172.21.15.116:0/2936614362 learned_addr learned my addr 172.21.15.116:0/2936614362 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e98c00640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 0x55bfca3e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9a000640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f0000 0x55bfca3e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 -- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f0000 msgr2=0x55bfca3e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f0000 0x55bfca3e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 -- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 msgr2=0x55bfca3e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 0x55bfca3e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 -- 172.21.15.116:0/2936614362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfca3deb40 con 0x55bfc9677a80 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e98c00640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 0x55bfca3e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:27.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e99600640 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x55bfca4873b0 tx=0x55bfc971c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186bfcbcd18c2056 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e96400640 1 -- 172.21.15.116:0/2936614362 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963c4e0 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e96400640 1 -- 172.21.15.116:0/2936614362 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bfc9650540 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e96400640 1 -- 172.21.15.116:0/2936614362 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963cb60 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfca3ded20 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55bfca3def00 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 4 waiting for MDS map... 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e96400640 1 -- 172.21.15.116:0/2936614362 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x55bfca530600 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e96400640 1 -- 172.21.15.116:0/2936614362 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 90) ==== 2885+0+0 (secure 0 0 0) 0x55bfc965d180 con 0x55bfc9677a80 2024-05-14T11:25:27.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 4 Got MDS map 90 2024-05-14T11:25:27.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 10 main: main 2024-05-14T11:25:27.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.078+0000 7f8e9b616400 4 main: connecting to RADOS... 2024-05-14T11:25:27.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 Processor -- start 2024-05-14T11:25:27.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- start start 2024-05-14T11:25:27.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96de600 con 0x55bfca3f0900 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96de780 con 0x55bfca3f0d80 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bfc96de900 con 0x55bfca3f1200 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39454/0 (socket says 172.21.15.116:39454) 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 -- 172.21.15.116:0/2398158058 learned_addr learned my addr 172.21.15.116:0/2398158058 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e99600640 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e98c00640 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 -- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 msgr2=0x55bfca3e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 -- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 msgr2=0x55bfca3e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 -- 172.21.15.116:0/2398158058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfca3ded20 con 0x55bfca3f0900 2024-05-14T11:25:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9a000640 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x55bfca53e660 tx=0x55bfca541800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5b50768097679 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e93c00640 1 -- 172.21.15.116:0/2398158058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963c4e0 con 0x55bfca3f0900 2024-05-14T11:25:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e93c00640 1 -- 172.21.15.116:0/2398158058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bfc9650e00 con 0x55bfca3f0900 2024-05-14T11:25:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e93c00640 1 -- 172.21.15.116:0/2398158058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963cb60 con 0x55bfca3f0900 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- 172.21.15.116:0/2398158058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 msgr2=0x55bfca3e5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x55bfca53e660 tx=0x55bfca541800 comp rx=0 tx=0).stop 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- 172.21.15.116:0/2398158058 shutdown_connections 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2398158058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- 172.21.15.116:0/2398158058 >> 172.21.15.116:0/2398158058 conn(0x55bfca3f0480 msgr2=0x55bfc9928000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- 172.21.15.116:0/2398158058 shutdown_connections 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.082+0000 7f8e9b616400 1 -- 172.21.15.116:0/2398158058 wait complete. 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 Processor -- start 2024-05-14T11:25:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- start start 2024-05-14T11:25:27.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96dfc80 con 0x55bfca3f0900 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfc96dfb00 con 0x55bfca3f0d80 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55bfc96df980 con 0x55bfca3f1200 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39470/0 (socket says 172.21.15.116:39470) 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 -- 172.21.15.116:0/3245343166 learned_addr learned my addr 172.21.15.116:0/3245343166 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e99600640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e98c00640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 -- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 msgr2=0x55bfca3e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f1200 0x55bfca3e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 -- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 msgr2=0x55bfca3e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfca3f0d80 0x55bfca3e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 -- 172.21.15.116:0/3245343166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfca3dfc20 con 0x55bfca3f0900 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9a000640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x55bfca53f920 tx=0x55bfc971cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e91e00640 1 -- 172.21.15.116:0/3245343166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963c000 con 0x55bfca3f0900 2024-05-14T11:25:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e91e00640 1 -- 172.21.15.116:0/3245343166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bfc9650c40 con 0x55bfca3f0900 2024-05-14T11:25:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e91e00640 1 -- 172.21.15.116:0/3245343166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bfc963c4e0 con 0x55bfca3f0900 2024-05-14T11:25:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bfca3dfe00 con 0x55bfca3f0900 2024-05-14T11:25:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfca3dfc20 con 0x55bfca3f0900 2024-05-14T11:25:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.086+0000 7f8e9b616400 4 main: resolving pool 15 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e91e00640 1 -- 172.21.15.116:0/3245343166 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55bfc9916c00 con 0x55bfca3f0900 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e91e00640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bfca3f3600 0x55bfca3e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e91e00640 1 -- 172.21.15.116:0/3245343166 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x55bfc9952400 con 0x55bfca3f0900 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e9b616400 4 main: creating IoCtx.. 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e99600640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bfca3f3600 0x55bfca3e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e9b616400 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bfca3f1680 0x55bfca3e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103) -- 0x55bfc993c800 con 0x55bfca3f1680 2024-05-14T11:25:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e98c00640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bfca3f1680 0x55bfca3e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e99600640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bfca3f3600 0x55bfca3e6100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55bfc9702750 tx=0x55bfc971c600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.090+0000 7f8e98c00640 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bfca3f1680 0x55bfca3e6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e98c00640 1 -- 172.21.15.116:0/3245343166 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55bfc9752000 con 0x55bfca3f1680 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: "0": { 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: "data": { 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 5, 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: "inotable": { 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: "projected_free": [ 2024-05-14T11:25:27.093 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "start": 1099511627779, 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "len": 1099511627773 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "free": [ 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "start": 1099511627779, 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "len": 1099511627773 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: "result": 0 2024-05-14T11:25:27.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:27.095 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:27.095 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bfca3f1680 msgr2=0x55bfca3e6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55bfca3f1680 0x55bfca3e6680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bfca3f3600 msgr2=0x55bfca3e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55bfca3f3600 0x55bfca3e6100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55bfc9702750 tx=0x55bfc971c600 comp rx=0 tx=0).stop 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 msgr2=0x55bfca3e5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/3245343166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfca3f0900 0x55bfca3e5080 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x55bfca53f920 tx=0x55bfc971cb40 comp rx=0 tx=0).stop 2024-05-14T11:25:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e98c00640 1 -- 172.21.15.116:0/3245343166 reap_dead start 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 shutdown_connections 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 >> 172.21.15.116:0/3245343166 conn(0x55bfca3f0480 msgr2=0x55bfc9928000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 shutdown_connections 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/3245343166 wait complete. 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 msgr2=0x55bfca3e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x55bfca4873b0 tx=0x55bfc971c4e0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 shutdown_connections 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc9677180 0x55bfca3e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55bfca3f0000 0x55bfca3e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 --2- 172.21.15.116:0/2936614362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bfc9677a80 0x55bfca3e4000 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 >> 172.21.15.116:0/2936614362 conn(0x55bfc9677600 msgr2=0x55bfc5605800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 shutdown_connections 2024-05-14T11:25:27.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.094+0000 7f8e9b616400 1 -- 172.21.15.116:0/2936614362 wait complete. 2024-05-14T11:25:27.099 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.084412, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-14T11:25:27.100 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:25:27.100 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:25:27.124 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:25:27.181 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:25:27.219 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:25:27.219 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:25:27.219 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:27.219 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:27.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:25:27.230 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:25:27.230 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:25:27.230 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:27.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:25:27.268 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:25:27.268 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:27.269 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:27.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:25:27.281 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:25:27.282 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:25:27.282 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:27.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:25:27.324 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:27.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 Processor -- start 2024-05-14T11:25:27.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 -- start start 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f446415fec0 0x7f44640d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446406ec60 con 0x7f44640cedd0 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f446406ef40 con 0x7f446415fec0 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4468e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f446406f220 con 0x7f44641602d0 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f446415fec0 0x7f44640d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4462800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4462800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:33770/0 (socket says 172.21.15.116:33770) 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f44640cedd0 0x7f446406fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:58360/0 (socket says 172.21.15.116:58360) 2024-05-14T11:25:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4462800640 1 -- 172.21.15.116:0/907365091 learned_addr learned my addr 172.21.15.116:0/907365091 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4460a00640 1 -- 172.21.15.116:0/907365091 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180126078 0 0) 0x7f446406ec60 con 0x7f44640cedd0 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4460a00640 1 -- 172.21.15.116:0/907365091 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4448003530 con 0x7f44640cedd0 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461400640 1 -- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 msgr2=0x7f44640d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461400640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461400640 1 -- 172.21.15.116:0/907365091 >> v1:172.21.15.149:6789/0 conn(0x7f44640cedd0 legacy=0x7f446406fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4461400640 1 -- 172.21.15.116:0/907365091 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446406f500 con 0x7f446415fec0 2024-05-14T11:25:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.474+0000 7f4462800640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:27.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4461400640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f44640d1b20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44500097c0 tx=0x7f445001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b03f8aff768a24c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4460a00640 1 -- 172.21.15.116:0/907365091 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44500b0070 con 0x7f446415fec0 2024-05-14T11:25:27.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4460a00640 1 -- 172.21.15.116:0/907365091 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44500ab600 con 0x7f446415fec0 2024-05-14T11:25:27.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4460a00640 1 -- 172.21.15.116:0/907365091 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44500ab9d0 con 0x7f446415fec0 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 -- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 msgr2=0x7f44640d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f44640d1b20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44500097c0 tx=0x7f445001c350 comp rx=0 tx=0).stop 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 -- 172.21.15.116:0/907365091 shutdown_connections 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f44640d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 --2- 172.21.15.116:0/907365091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f44640d1b20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 -- 172.21.15.116:0/907365091 >> 172.21.15.116:0/907365091 conn(0x7f44640d3670 msgr2=0x7f44640d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 -- 172.21.15.116:0/907365091 shutdown_connections 2024-05-14T11:25:27.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.478+0000 7f4468e00640 1 -- 172.21.15.116:0/907365091 wait complete. 2024-05-14T11:25:27.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 Processor -- start 2024-05-14T11:25:27.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 -- start start 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f4464156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f4464159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f446416d430 con 0x7f44640cedd0 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f446416d2b0 con 0x7f446415fec0 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4468e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f446416d5b0 con 0x7f44641602d0 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4461e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4461e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39488/0 (socket says 172.21.15.116:39488) 2024-05-14T11:25:27.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4461e00640 1 -- 172.21.15.116:0/2420087090 learned_addr learned my addr 172.21.15.116:0/2420087090 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4462800640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f4464159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.482+0000 7f4461400640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f4464156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 -- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 msgr2=0x7f4464159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f4464159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 -- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 msgr2=0x7f4464156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f4464156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 -- 172.21.15.116:0/2420087090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4464163950 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461400640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f4464156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4461e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f445400c460 tx=0x7f445400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454007480 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4454007640 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004a30 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4464156f30 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641574f0 con 0x7f44640cedd0 2024-05-14T11:25:27.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446406a9c0 con 0x7f44640cedd0 2024-05-14T11:25:27.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f445401a020 con 0x7f44640cedd0 2024-05-14T11:25:27.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.486+0000 7f445a000640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 0x7f4444071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.490+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f445408ec50 con 0x7f44640cedd0 2024-05-14T11:25:27.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.490+0000 7f4461400640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 0x7f4444071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.490+0000 7f4461400640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 0x7f4444071910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f44500ad000 tx=0x7f4450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.494+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4454094020 con 0x7f44640cedd0 2024-05-14T11:25:27.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f446406e770 con 0x7f44640cedd0 2024-05-14T11:25:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f445a000640 1 -- 172.21.15.116:0/2420087090 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+4351 (secure 0 0 0) 0x7f445405b8f0 con 0x7f44640cedd0 2024-05-14T11:25:27.679 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:27.679 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:25:26.993007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:25:27.680 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 90 2024-05-14T11:25:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 msgr2=0x7f4444071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 0x7f4444071910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f44500ad000 tx=0x7f4450008040 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 msgr2=0x7f4464155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f445400c460 tx=0x7f445400c920 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 shutdown_connections 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f444406f400 0x7f4444071910 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f446415fec0 0x7f4464156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44641602d0 0x7f4464159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 --2- 172.21.15.116:0/2420087090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44640cedd0 0x7f4464155d20 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 >> 172.21.15.116:0/2420087090 conn(0x7f44640d3670 msgr2=0x7f446415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 shutdown_connections 2024-05-14T11:25:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.682+0000 7f4468e00640 1 -- 172.21.15.116:0/2420087090 wait complete. 2024-05-14T11:25:27.701 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:27.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 Processor -- start 2024-05-14T11:25:27.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 -- start start 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fad080d0230 0x7fad080d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7fad080b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad080ceb20 con 0x7fad080b78d0 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fad080b4e40 con 0x7fad080d0230 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0e200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fad0815e010 con 0x7fad080d2560 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7facffe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fad080b78d0 0x7fad080b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46566/0 (socket says 172.21.15.116:46566) 2024-05-14T11:25:27.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7facffe00640 1 -- 172.21.15.116:0/2403384770 learned_addr learned my addr 172.21.15.116:0/2403384770 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:27.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0d800640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7fad080b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad0ce00640 1 --2- 172.21.15.116:0/2403384770 >> v2:172.21.15.149:3300/0 conn(0x7fad080d0230 0x7fad080d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970932050 0 0) 0x7facf0001120 con 0x7fad080b78d0 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facec003530 con 0x7fad080b78d0 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303348118 0 0) 0x7facec003530 con 0x7fad080b78d0 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 msgr2=0x7fad080b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7fad080b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 >> v2:172.21.15.149:3300/0 conn(0x7fad080d0230 msgr2=0x7fad080d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 --2- 172.21.15.116:0/2403384770 >> v2:172.21.15.149:3300/0 conn(0x7fad080d0230 0x7fad080d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080d0b80 con 0x7fad080b78d0 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7facf00018d0 con 0x7fad080b78d0 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 >> v1:172.21.15.116:6789/0 conn(0x7fad080b78d0 legacy=0x7fad080b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facec003a10 0x7facec003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7facec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec004d50 0x7facec005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad080b4e40 con 0x7facec004d50 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad0815e010 con 0x7facec003a10 2024-05-14T11:25:27.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.850+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7facec00ba80 con 0x7fad080d2560 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7facffe00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7facec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec004d50 0x7facec005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0ce00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facec003a10 0x7facec003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 -- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 msgr2=0x7facec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7facec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 -- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facec003a10 msgr2=0x7facec003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7facec003a10 0x7facec003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 -- 172.21.15.116:0/2403384770 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec003530 con 0x7facec004d50 2024-05-14T11:25:27.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7facffe00640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad080d2560 0x7facec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:27.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad0d800640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec004d50 0x7facec005160 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fad00009f90 tx=0x7fad0001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df468baba47d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000b2070 con 0x7facec004d50 2024-05-14T11:25:27.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad000a5c70 con 0x7facec004d50 2024-05-14T11:25:27.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.854+0000 7fad07e00640 1 -- 172.21.15.116:0/2403384770 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00009c10 con 0x7facec004d50 2024-05-14T11:25:27.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- 172.21.15.116:0/2403384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec004d50 msgr2=0x7facec005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 --2- 172.21.15.116:0/2403384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec004d50 0x7facec005160 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fad00009f90 tx=0x7fad0001cc60 comp rx=0 tx=0).stop 2024-05-14T11:25:27.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0d800640 1 -- 172.21.15.116:0/2403384770 reap_dead start 2024-05-14T11:25:27.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- 172.21.15.116:0/2403384770 shutdown_connections 2024-05-14T11:25:27.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- 172.21.15.116:0/2403384770 >> 172.21.15.116:0/2403384770 conn(0x7fad080b53c0 msgr2=0x7fad080b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:27.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- 172.21.15.116:0/2403384770 shutdown_connections 2024-05-14T11:25:27.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- 172.21.15.116:0/2403384770 wait complete. 2024-05-14T11:25:27.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 Processor -- start 2024-05-14T11:25:27.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.858+0000 7fad0e200640 1 -- start start 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080c8e40 0x7fad080c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad08154d20 0x7fad08071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad080b70d0 con 0x7fad08154910 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad080b6df0 con 0x7fad080c8e40 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fad080b73b0 con 0x7fad08154d20 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46446/0 (socket says 172.21.15.116:46446) 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 -- 172.21.15.116:0/2111567388 learned_addr learned my addr 172.21.15.116:0/2111567388 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0ce00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080c8e40 0x7fad080c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0d800640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad08154d20 0x7fad08071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 -- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad08154d20 msgr2=0x7fad08071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad08154d20 0x7fad08071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 -- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080c8e40 msgr2=0x7fad080c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:27.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080c8e40 0x7fad080c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:27.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 -- 172.21.15.116:0/2111567388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080c7060 con 0x7fad08154910 2024-05-14T11:25:27.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7facffe00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7facf0007290 tx=0x7facf0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0001190 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf0018070 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081f4370 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf000cc90 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad080c7300 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.862+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad08050880 con 0x7fad08154910 2024-05-14T11:25:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.866+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7facf001d460 con 0x7fad08154910 2024-05-14T11:25:27.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.866+0000 7fad05600640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 0x7face4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:27.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.866+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7facf008f780 con 0x7fad08154910 2024-05-14T11:25:27.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.866+0000 7fad0ce00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 0x7face4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:27.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.866+0000 7fad0ce00640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 0x7face4071910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7facf8009c60 tx=0x7facf8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:27.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:27.870+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fad080d0b80 con 0x7fad08154910 2024-05-14T11:25:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad081f4540 con 0x7fad08154910 2024-05-14T11:25:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad05600640 1 -- 172.21.15.116:0/2111567388 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5179 (secure 0 0 0) 0x7fad080ceb20 con 0x7fad08154910 2024-05-14T11:25:28.059 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:28.059 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8088,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2140008008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2140008008},{"type":"v1","addr":"172.21.15.116:6835","nonce":2140008008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:25:26.993007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:25:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 91 2024-05-14T11:25:28.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 msgr2=0x7face4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 0x7face4071910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7facf8009c60 tx=0x7facf8009190 comp rx=0 tx=0).stop 2024-05-14T11:25:28.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 msgr2=0x7fad08071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7facf0007290 tx=0x7facf0004f90 comp rx=0 tx=0).stop 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 shutdown_connections 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7face406f400 0x7face4071910 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fad08154d20 0x7fad08071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad08154910 0x7fad08071430 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 --2- 172.21.15.116:0/2111567388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080c8e40 0x7fad080c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 >> 172.21.15.116:0/2111567388 conn(0x7fad080b53c0 msgr2=0x7fad080cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 shutdown_connections 2024-05-14T11:25:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.062+0000 7fad0e200640 1 -- 172.21.15.116:0/2111567388 wait complete. 2024-05-14T11:25:28.080 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:28.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 Processor -- start 2024-05-14T11:25:28.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 -- start start 2024-05-14T11:25:28.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2dc0d1fc0 con 0x7ff2dc06e9e0 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc1573b0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc161120 con 0x7ff2dc156fa0 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46452/0 (socket says 172.21.15.116:46452) 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2998421562 learned_addr learned my addr 172.21.15.116:0/2998421562 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e1400640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2998421562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 msgr2=0x7ff2dc06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:28.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2998421562 >> v1:172.21.15.149:6789/0 conn(0x7ff2dc06e9e0 legacy=0x7ff2dc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2998421562 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc160510 con 0x7ff2dc0d1670 2024-05-14T11:25:28.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.230+0000 7ff2dbe00640 1 -- 172.21.15.116:0/2998421562 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423686242 0 0) 0x7ff2d40014a0 con 0x7ff2dc06e9e0 2024-05-14T11:25:28.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e0a00640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4a35299ca89c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2dbe00640 1 -- 172.21.15.116:0/2998421562 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80aa430 con 0x7ff2dc0d1670 2024-05-14T11:25:28.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2dbe00640 1 -- 172.21.15.116:0/2998421562 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c80af070 con 0x7ff2dc0d1670 2024-05-14T11:25:28.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2dbe00640 1 -- 172.21.15.116:0/2998421562 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80aa7f0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2998421562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 msgr2=0x7ff2dc0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c801c4d0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2998421562 shutdown_connections 2024-05-14T11:25:28.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0d1a80 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2998421562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2998421562 >> 172.21.15.116:0/2998421562 conn(0x7ff2dc0b4620 msgr2=0x7ff2dc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2998421562 shutdown_connections 2024-05-14T11:25:28.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.234+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2998421562 wait complete. 2024-05-14T11:25:28.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 Processor -- start 2024-05-14T11:25:28.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- start start 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2dc06e9e0 0x7ff2dc0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc070520 con 0x7ff2dc0d1670 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0b07a0 con 0x7ff2dc06e9e0 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc0706a0 con 0x7ff2dc156fa0 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46454/0 (socket says 172.21.15.116:46454) 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2929771004 learned_addr learned my addr 172.21.15.116:0/2929771004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e1400640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2dc06e9e0 0x7ff2dc0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e1e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 msgr2=0x7ff2dc0b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc0b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2dc06e9e0 msgr2=0x7ff2dc0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2dc06e9e0 0x7ff2dc0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 -- 172.21.15.116:0/2929771004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc0cdd40 con 0x7ff2dc0d1670 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e0a00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8019140 tx=0x7ff2c801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80b9070 con 0x7ff2dc0d1670 2024-05-14T11:25:28.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c8004230 con 0x7ff2dc0d1670 2024-05-14T11:25:28.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80aa900 con 0x7ff2dc0d1670 2024-05-14T11:25:28.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1f78b0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.238+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc0b22b0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc06a9c0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2c80aaaa0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2d9600640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7ff2c8128d30 con 0x7ff2dc0d1670 2024-05-14T11:25:28.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2e1400640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.242+0000 7ff2e1400640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 0x7ff2bc071940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0b0d60 tx=0x7ff2d0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.246+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2c8016150 con 0x7ff2dc0d1670 2024-05-14T11:25:28.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.406+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2dc06f1f0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.406+0000 7ff2d9600640 1 -- 172.21.15.116:0/2929771004 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+17227 (secure 0 0 0) 0x7ff2c80f52e0 con 0x7ff2dc0d1670 2024-05-14T11:25:28.406 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:28.406 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":103,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:26.992788+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:24:26.553545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-14T11:24:27.820765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.3","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6835/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6834/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6835/3421632":"2024-05-15T11:24:50.202416+0000","172.21.15.116:6834/3421632":"2024-05-15T11:24:50.202416+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 msgr2=0x7ff2bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 0x7ff2bc071940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0b0d60 tx=0x7ff2d0009190 comp rx=0 tx=0).stop 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 msgr2=0x7ff2dc0cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8019140 tx=0x7ff2c801ad70 comp rx=0 tx=0).stop 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 shutdown_connections 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2dc156fa0 0x7ff2dc0b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff2dc0d1670 0x7ff2dc0cd800 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 --2- 172.21.15.116:0/2929771004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2dc06e9e0 0x7ff2dc0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 >> 172.21.15.116:0/2929771004 conn(0x7ff2dc0b4620 msgr2=0x7ff2dc15c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 shutdown_connections 2024-05-14T11:25:28.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.410+0000 7ff2e3e00640 1 -- 172.21.15.116:0/2929771004 wait complete. 2024-05-14T11:25:28.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:25:28.429 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 Processor -- start 2024-05-14T11:25:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 -- start start 2024-05-14T11:25:28.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69f01621b0 0x7f69f00d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f69f00b9020 0x7f69f00ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f00ce800 0x7f69f00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f69f00d1d40 con 0x7f69f01621b0 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f69f00cec10 con 0x7f69f00b9020 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69f6a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69f00d1f90 con 0x7f69f00ce800 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69efe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69f01621b0 0x7f69f00d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69efe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69f01621b0 0x7f69f00d2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46458/0 (socket says 172.21.15.116:46458) 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69efe00640 1 -- 172.21.15.116:0/1757369251 learned_addr learned my addr 172.21.15.116:0/1757369251 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69efe00640 1 -- 172.21.15.116:0/1757369251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f00ce800 msgr2=0x7f69f00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.578+0000 7f69efe00640 1 --2- 172.21.15.116:0/1757369251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f00ce800 0x7f69f00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69efe00640 1 -- 172.21.15.116:0/1757369251 >> v2:172.21.15.149:3300/0 conn(0x7f69f00b9020 msgr2=0x7f69f00ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69efe00640 1 --2- 172.21.15.116:0/1757369251 >> v2:172.21.15.149:3300/0 conn(0x7f69f00b9020 0x7f69f00ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69efe00640 1 -- 172.21.15.116:0/1757369251 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f00d2270 con 0x7f69f01621b0 2024-05-14T11:25:28.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69efe00640 1 --2- 172.21.15.116:0/1757369251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f01621b0 0x7f69f00d2d50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f69f00743b0 tx=0x7f69e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6908efddef65cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69ef400640 1 -- 172.21.15.116:0/1757369251 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00b0070 con 0x7f69f01621b0 2024-05-14T11:25:28.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69ef400640 1 -- 172.21.15.116:0/1757369251 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e00ab600 con 0x7f69f01621b0 2024-05-14T11:25:28.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69ef400640 1 -- 172.21.15.116:0/1757369251 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00ab9d0 con 0x7f69f01621b0 2024-05-14T11:25:28.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 -- 172.21.15.116:0/1757369251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f01621b0 msgr2=0x7f69f00d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 --2- 172.21.15.116:0/1757369251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f01621b0 0x7f69f00d2d50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f69f00743b0 tx=0x7f69e001c540 comp rx=0 tx=0).stop 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 -- 172.21.15.116:0/1757369251 shutdown_connections 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 --2- 172.21.15.116:0/1757369251 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f00ce800 0x7f69f00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 --2- 172.21.15.116:0/1757369251 >> v2:172.21.15.149:3300/0 conn(0x7f69f00b9020 0x7f69f00ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 --2- 172.21.15.116:0/1757369251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f01621b0 0x7f69f00d2d50 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 -- 172.21.15.116:0/1757369251 >> 172.21.15.116:0/1757369251 conn(0x7f69f00d0770 msgr2=0x7f69f00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 -- 172.21.15.116:0/1757369251 shutdown_connections 2024-05-14T11:25:28.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.582+0000 7f69f6a00640 1 -- 172.21.15.116:0/1757369251 wait complete. 2024-05-14T11:25:28.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 Processor -- start 2024-05-14T11:25:28.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 -- start start 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 0x7f69f015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f01621b0 0x7f69f015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69f015d3e0 con 0x7f69f00b9020 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69f015d260 con 0x7f69f00ce800 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69f6a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69f015d560 con 0x7f69f01621b0 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69efe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69efe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46474/0 (socket says 172.21.15.116:46474) 2024-05-14T11:25:28.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.586+0000 7f69efe00640 1 -- 172.21.15.116:0/3305030408 learned_addr learned my addr 172.21.15.116:0/3305030408 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69f4a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f01621b0 0x7f69f015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69e7400640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 0x7f69f015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 -- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f01621b0 msgr2=0x7f69f015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f01621b0 0x7f69f015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 -- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 msgr2=0x7f69f015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 0x7f69f015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 -- 172.21.15.116:0/3305030408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f0167bb0 con 0x7f69f00b9020 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69e7400640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 0x7f69f015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:28.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69efe00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f69e00ae040 tx=0x7f69e001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00b0040 con 0x7f69f00b9020 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e00bc070 con 0x7f69f00b9020 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f0167e90 con 0x7f69f00b9020 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00b6820 con 0x7f69f00b9020 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f015f890 con 0x7f69f00b9020 2024-05-14T11:25:28.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f006a9c0 con 0x7f69f00b9020 2024-05-14T11:25:28.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.590+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69e00b69c0 con 0x7f69f00b9020 2024-05-14T11:25:28.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.594+0000 7f69ecc00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 0x7f69cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.594+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f69e0129720 con 0x7f69f00b9020 2024-05-14T11:25:28.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.594+0000 7f69e7400640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 0x7f69cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.594+0000 7f69e7400640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 0x7f69cc071940 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006fd0 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.598+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f69e00c7060 con 0x7f69f00b9020 2024-05-14T11:25:28.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.802+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69f00b6fe0 con 0x7f69f00b9020 2024-05-14T11:25:28.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69ecc00640 1 -- 172.21.15.116:0/3305030408 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5179 (secure 0 0 0) 0x7f69e0016130 con 0x7f69f00b9020 2024-05-14T11:25:28.803 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:28.803 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8088,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2140008008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2140008008},{"type":"v1","addr":"172.21.15.116:6835","nonce":2140008008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:25:26.993007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:25:28.804 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 91 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 msgr2=0x7f69cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 0x7f69cc071940 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006fd0 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 msgr2=0x7f69f01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f69e00ae040 tx=0x7f69e001ad80 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 shutdown_connections 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f69cc06f430 0x7f69cc071940 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f00ce800 0x7f69f015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69f01621b0 0x7f69f015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 --2- 172.21.15.116:0/3305030408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69f00b9020 0x7f69f01565d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 >> 172.21.15.116:0/3305030408 conn(0x7f69f00d0770 msgr2=0x7f69f00b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 shutdown_connections 2024-05-14T11:25:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.806+0000 7f69f6a00640 1 -- 172.21.15.116:0/3305030408 wait complete. 2024-05-14T11:25:28.825 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:28.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 Processor -- start 2024-05-14T11:25:28.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 -- start start 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900d1c20 0x7f91900ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91900b40c0 0x7f91900b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91900b4a10 con 0x7f91900cdf80 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f91900b3060 con 0x7f91900b40c0 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f9196600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f91900b3300 con 0x7f91900d1c20 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91900b40c0 0x7f91900b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91900b40c0 0x7f91900b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46486/0 (socket says 172.21.15.116:46486) 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f91900cdf80 0x7f91900d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46580/0 (socket says 172.21.15.116:46580) 2024-05-14T11:25:28.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 -- 172.21.15.116:0/1693527560 learned_addr learned my addr 172.21.15.116:0/1693527560 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918ea00640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900d1c20 0x7f91900ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 -- 172.21.15.116:0/1693527560 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900d1c20 msgr2=0x7f91900ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:28.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900d1c20 0x7f91900ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 -- 172.21.15.116:0/1693527560 >> v1:172.21.15.116:6789/0 conn(0x7f91900cdf80 legacy=0x7f91900d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.974+0000 7f918fe00640 1 -- 172.21.15.116:0/1693527560 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91900b35e0 con 0x7f91900b40c0 2024-05-14T11:25:28.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f918fe00640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900b44d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e50 tx=0x7f918801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606f2656c0c103aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f918e000640 1 -- 172.21.15.116:0/1693527560 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880a58d0 con 0x7f91900b40c0 2024-05-14T11:25:28.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f918e000640 1 -- 172.21.15.116:0/1693527560 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91880a5a70 con 0x7f91900b40c0 2024-05-14T11:25:28.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f918e000640 1 -- 172.21.15.116:0/1693527560 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880a5e40 con 0x7f91900b40c0 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 -- 172.21.15.116:0/1693527560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 msgr2=0x7f91900b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900b44d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e50 tx=0x7f918801ca00 comp rx=0 tx=0).stop 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 -- 172.21.15.116:0/1693527560 shutdown_connections 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900b44d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 --2- 172.21.15.116:0/1693527560 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900d1c20 0x7f91900ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 -- 172.21.15.116:0/1693527560 >> 172.21.15.116:0/1693527560 conn(0x7f91900d0770 msgr2=0x7f91900d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 -- 172.21.15.116:0/1693527560 shutdown_connections 2024-05-14T11:25:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.978+0000 7f9196600640 1 -- 172.21.15.116:0/1693527560 wait complete. 2024-05-14T11:25:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 Processor -- start 2024-05-14T11:25:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 -- start start 2024-05-14T11:25:28.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91900cdf80 0x7f919015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900ccea0 0x7f919015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f919006ff40 con 0x7f91900b40c0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f919006fdc0 con 0x7f91900cdf80 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f9196600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f919016cd60 con 0x7f91900ccea0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f918f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f918f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46498/0 (socket says 172.21.15.116:46498) 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f918f400640 1 -- 172.21.15.116:0/2809574175 learned_addr learned my addr 172.21.15.116:0/2809574175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f918fe00640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900ccea0 0x7f919015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.982+0000 7f918ea00640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91900cdf80 0x7f919015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 -- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900ccea0 msgr2=0x7f919015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900ccea0 0x7f919015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 -- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91900cdf80 msgr2=0x7f919015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91900cdf80 0x7f919015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 -- 172.21.15.116:0/2809574175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91900cfa80 con 0x7f91900b40c0 2024-05-14T11:25:28.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918f400640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f917800c460 tx=0x7f917800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178018070 con 0x7f91900b40c0 2024-05-14T11:25:28.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9178007480 con 0x7f91900b40c0 2024-05-14T11:25:28.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780047f0 con 0x7f91900b40c0 2024-05-14T11:25:28.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91900cb170 con 0x7f91900b40c0 2024-05-14T11:25:28.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91900cb410 con 0x7f91900b40c0 2024-05-14T11:25:28.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919006a9c0 con 0x7f91900b40c0 2024-05-14T11:25:28.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f917801b070 con 0x7f91900b40c0 2024-05-14T11:25:28.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 0x7f916c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:28.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f917808e7d0 con 0x7f91900b40c0 2024-05-14T11:25:28.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.986+0000 7f918ea00640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 0x7f916c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:28.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.990+0000 7f918ea00640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 0x7f916c071940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f919015bfb0 tx=0x7f9180008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:28.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:28.990+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f917805b3d0 con 0x7f91900b40c0 2024-05-14T11:25:29.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.182+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91900b9590 con 0x7f91900b40c0 2024-05-14T11:25:29.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.182+0000 7f9187400640 1 -- 172.21.15.116:0/2809574175 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5179 (secure 0 0 0) 0x7f917805acd0 con 0x7f91900b40c0 2024-05-14T11:25:29.181 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:29.181 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8088,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2140008008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2140008008},{"type":"v1","addr":"172.21.15.116:6835","nonce":2140008008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:24:28.859727+0000","modified":"2024-05-14T11:25:26.993007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-14T11:25:29.182 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 91 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 msgr2=0x7f916c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 0x7f916c071940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f919015bfb0 tx=0x7f9180008040 comp rx=0 tx=0).stop 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 msgr2=0x7f91900cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f917800c460 tx=0x7f917800c920 comp rx=0 tx=0).stop 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 shutdown_connections 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f916c06f430 0x7f916c071940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91900cdf80 0x7f919015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f91900ccea0 0x7f919015aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 --2- 172.21.15.116:0/2809574175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91900b40c0 0x7f91900cc960 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 >> 172.21.15.116:0/2809574175 conn(0x7f91900d0770 msgr2=0x7f9190071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:29.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 shutdown_connections 2024-05-14T11:25:29.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.186+0000 7f9196600640 1 -- 172.21.15.116:0/2809574175 wait complete. 2024-05-14T11:25:29.203 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:29.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.350+0000 7f0212a00640 1 Processor -- start 2024-05-14T11:25:29.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.350+0000 7f0212a00640 1 -- start start 2024-05-14T11:25:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0212a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f020c0b0770 0x7f020c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0212a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c06cc40 con 0x7f020c06f730 2024-05-14T11:25:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0212a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c06cf10 con 0x7f020c072a30 2024-05-14T11:25:29.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0212a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f020c06db20 con 0x7f020c0b0770 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0203400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f020c06f730 0x7f020c06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46582/0 (socket says 172.21.15.116:46582) 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020be00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f020c0b0770 0x7f020c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f0203400640 1 -- 172.21.15.116:0/90603832 learned_addr learned my addr 172.21.15.116:0/90603832 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962604759 0 0) 0x7f020c06cc40 con 0x7f020c06f730 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f0005740 con 0x7f020c06f730 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020be00640 1 -- 172.21.15.116:0/90603832 >> v1:172.21.15.149:6789/0 conn(0x7f020c072a30 legacy=0x7f020c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612387108 0 0) 0x7f020c06cf10 con 0x7f020c072a30 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020be00640 1 -- 172.21.15.116:0/90603832 >> v1:172.21.15.116:6789/0 conn(0x7f020c06f730 legacy=0x7f020c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.354+0000 7f020be00640 1 -- 172.21.15.116:0/90603832 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c06d1f0 con 0x7f020c0b0770 2024-05-14T11:25:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f020be00640 1 --2- 172.21.15.116:0/90603832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c06c700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb09e7e242890e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0b2070 con 0x7f020c0b0770 2024-05-14T11:25:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0a9070 con 0x7f020c0b0770 2024-05-14T11:25:29.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f020b400640 1 -- 172.21.15.116:0/90603832 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0ad860 con 0x7f020c0b0770 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 -- 172.21.15.116:0/90603832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 msgr2=0x7f020c06c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 --2- 172.21.15.116:0/90603832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c06c700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc01c3e0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 -- 172.21.15.116:0/90603832 shutdown_connections 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 --2- 172.21.15.116:0/90603832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c06c700 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 -- 172.21.15.116:0/90603832 >> 172.21.15.116:0/90603832 conn(0x7f020c0b4620 msgr2=0x7f020c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:29.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 -- 172.21.15.116:0/90603832 shutdown_connections 2024-05-14T11:25:29.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.358+0000 7f0212a00640 1 -- 172.21.15.116:0/90603832 wait complete. 2024-05-14T11:25:29.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 Processor -- start 2024-05-14T11:25:29.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- start start 2024-05-14T11:25:29.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f020c06f730 0x7f020c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f020c073ae0 con 0x7f020c072a30 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f020c073960 con 0x7f020c0b0770 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f020c169fb0 con 0x7f020c06f730 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46502/0 (socket says 172.21.15.116:46502) 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 -- 172.21.15.116:0/818679624 learned_addr learned my addr 172.21.15.116:0/818679624 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f020be00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f020c06f730 0x7f020c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0210a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 -- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f020c06f730 msgr2=0x7f020c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f020c06f730 0x7f020c153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 -- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 msgr2=0x7f020c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 -- 172.21.15.116:0/818679624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c154590 con 0x7f020c072a30 2024-05-14T11:25:29.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0203400640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c420 tx=0x7f01f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4017070 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f4005cf0 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c153a10 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f40072d0 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.362+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c153fd0 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c06a9c0 con 0x7f020c072a30 2024-05-14T11:25:29.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01f401a070 con 0x7f020c072a30 2024-05-14T11:25:29.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f0208c00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 0x7f01e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f01f408e7e0 con 0x7f020c072a30 2024-05-14T11:25:29.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f020be00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 0x7f01e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.366+0000 7f020be00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 0x7f01e8071910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0af000 tx=0x7f01fc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.370+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f01f405ae20 con 0x7f020c072a30 2024-05-14T11:25:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.530+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f020c1537a0 con 0x7f020c072a30 2024-05-14T11:25:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0208c00640 1 -- 172.21.15.116:0/818679624 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+17227 (secure 0 0 0) 0x7f01f405f120 con 0x7f020c072a30 2024-05-14T11:25:29.531 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:29.531 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":103,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:26.992788+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:24:26.553545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-14T11:24:27.820765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.3","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6835/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6834/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6835/3421632":"2024-05-15T11:24:50.202416+0000","172.21.15.116:6834/3421632":"2024-05-15T11:24:50.202416+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 msgr2=0x7f01e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 0x7f01e8071910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0af000 tx=0x7f01fc0a7000 comp rx=0 tx=0).stop 2024-05-14T11:25:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 msgr2=0x7f020c0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c420 tx=0x7f01f400c8e0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 shutdown_connections 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f01e806f400 0x7f01e8071910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f020c0b0770 0x7f020c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f020c072a30 0x7f020c0cdac0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 --2- 172.21.15.116:0/818679624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f020c06f730 0x7f020c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 >> 172.21.15.116:0/818679624 conn(0x7f020c0b4620 msgr2=0x7f020c15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 shutdown_connections 2024-05-14T11:25:29.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.534+0000 7f0212a00640 1 -- 172.21.15.116:0/818679624 wait complete. 2024-05-14T11:25:29.553 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:25:29.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 Processor -- start 2024-05-14T11:25:29.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 -- start start 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f968006d110 0x7f96800cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96800cfa80 con 0x7f96800cf670 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96800700b0 con 0x7f96800cd230 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9685000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f968006e0b0 con 0x7f968006d110 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9676a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f96800cf670 0x7f968006f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46592/0 (socket says 172.21.15.116:46592) 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f9676a00640 1 -- 172.21.15.116:0/2624008244 learned_addr learned my addr 172.21.15.116:0/2624008244 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f967f400640 1 --2- 172.21.15.116:0/2624008244 >> v2:172.21.15.149:3300/0 conn(0x7f968006d110 0x7f96800cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1943348906 0 0) 0x7f9668001120 con 0x7f96800cf670 2024-05-14T11:25:29.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.702+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9660005740 con 0x7f96800cf670 2024-05-14T11:25:29.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967f400640 1 -- 172.21.15.116:0/2624008244 >> v1:172.21.15.149:6789/0 conn(0x7f96800cd230 legacy=0x7f968006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3077219934 0 0) 0x7f9660005740 con 0x7f96800cf670 2024-05-14T11:25:29.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967f400640 1 -- 172.21.15.116:0/2624008244 >> v1:172.21.15.116:6789/0 conn(0x7f96800cf670 legacy=0x7f968006f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967f400640 1 -- 172.21.15.116:0/2624008244 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800cba60 con 0x7f968006d110 2024-05-14T11:25:29.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967f400640 1 --2- 172.21.15.116:0/2624008244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f968006d110 0x7f96800cf130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f96700097c0 tx=0x7f967001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a0be78cdeeed499 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700b0070 con 0x7f968006d110 2024-05-14T11:25:29.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700ab400 con 0x7f968006d110 2024-05-14T11:25:29.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.706+0000 7f967ea00640 1 -- 172.21.15.116:0/2624008244 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96680018d0 con 0x7f968006d110 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- 172.21.15.116:0/2624008244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f968006d110 msgr2=0x7f96800cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 --2- 172.21.15.116:0/2624008244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f968006d110 0x7f96800cf130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f96700097c0 tx=0x7f967001c300 comp rx=0 tx=0).stop 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- 172.21.15.116:0/2624008244 shutdown_connections 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 --2- 172.21.15.116:0/2624008244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f968006d110 0x7f96800cf130 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- 172.21.15.116:0/2624008244 >> 172.21.15.116:0/2624008244 conn(0x7f96800b4820 msgr2=0x7f96800b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- 172.21.15.116:0/2624008244 shutdown_connections 2024-05-14T11:25:29.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- 172.21.15.116:0/2624008244 wait complete. 2024-05-14T11:25:29.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 Processor -- start 2024-05-14T11:25:29.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 -- start start 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f968006d110 0x7f968009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.710+0000 7f9685000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 0x7f9680154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f96800cbd00 con 0x7f96800cd230 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f968006ddd0 con 0x7f96800cf670 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f968006d520 con 0x7f968006d110 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46510/0 (socket says 172.21.15.116:46510) 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 -- 172.21.15.116:0/774825220 learned_addr learned my addr 172.21.15.116:0/774825220 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f967f400640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f968006d110 0x7f968009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f967fe00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 0x7f9680154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 -- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f968006d110 msgr2=0x7f968009a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f968006d110 0x7f968009a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 -- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 msgr2=0x7f9680154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 0x7f9680154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 -- 172.21.15.116:0/774825220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968009ae60 con 0x7f96800cd230 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f967fe00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 0x7f9680154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9676a00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f96680015a0 tx=0x7f966800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668005ae0 con 0x7f96800cd230 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668005ca0 con 0x7f96800cd230 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96680072b0 con 0x7f96800cd230 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9680153d60 con 0x7f96800cd230 2024-05-14T11:25:29.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801542e0 con 0x7f96800cd230 2024-05-14T11:25:29.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9680050880 con 0x7f96800cd230 2024-05-14T11:25:29.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.714+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9668007450 con 0x7f96800cd230 2024-05-14T11:25:29.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.718+0000 7f9677e00640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 0x7f965c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:29.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.718+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f966808f1d0 con 0x7f96800cd230 2024-05-14T11:25:29.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.718+0000 7f967f400640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 0x7f965c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:29.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.718+0000 7f967f400640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 0x7f965c071940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f96700ad000 tx=0x7f96700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:29.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.722+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f96800cfa80 con 0x7f96800cd230 2024-05-14T11:25:29.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:29.910+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f968015b8a0 con 0x7f96800cd230 2024-05-14T11:25:30.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.318+0000 7f9677e00640 1 -- 172.21.15.116:0/774825220 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 161+0+0 (secure 0 0 0) 0x7f966805be50 con 0x7f96800cd230 2024-05-14T11:25:30.317 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:25:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 msgr2=0x7f965c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 0x7f965c071940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f96700ad000 tx=0x7f96700a6000 comp rx=0 tx=0).stop 2024-05-14T11:25:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 msgr2=0x7f96800c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f96680015a0 tx=0x7f966800cb90 comp rx=0 tx=0).stop 2024-05-14T11:25:30.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 shutdown_connections 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f965c06f430 0x7f965c071940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96800cf670 0x7f9680154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f96800cd230 0x7f96800c5040 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 --2- 172.21.15.116:0/774825220 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f968006d110 0x7f968009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 >> 172.21.15.116:0/774825220 conn(0x7f96800b4820 msgr2=0x7f96800b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 shutdown_connections 2024-05-14T11:25:30.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.322+0000 7f9685000640 1 -- 172.21.15.116:0/774825220 wait complete. 2024-05-14T11:25:30.339 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:25:30.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 Processor -- start 2024-05-14T11:25:30.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 -- start start 2024-05-14T11:25:30.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 0x7f00581620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00580d1ec0 con 0x7f005815e1b0 2024-05-14T11:25:30.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f0058162630 con 0x7f00580c8bb0 2024-05-14T11:25:30.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.490+0000 7f005f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00580c76b0 con 0x7f0058161ce0 2024-05-14T11:25:30.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 0x7f00581620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f005ce00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f005ce00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:43664/0 (socket says 172.21.15.116:43664) 2024-05-14T11:25:30.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f005ce00640 1 -- 172.21.15.116:0/831436893 learned_addr learned my addr 172.21.15.116:0/831436893 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 -- 172.21.15.116:0/831436893 >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 msgr2=0x7f00581617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 --2- 172.21.15.116:0/831436893 >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 -- 172.21.15.116:0/831436893 >> v1:172.21.15.149:6789/0 conn(0x7f005815e1b0 legacy=0x7f005815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 -- 172.21.15.116:0/831436893 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00580c7990 con 0x7f0058161ce0 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f005ce00640 1 --2- 172.21.15.116:0/831436893 >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:30.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057400640 1 -- 172.21.15.116:0/831436893 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181585251 0 0) 0x7f00580d1ec0 con 0x7f005815e1b0 2024-05-14T11:25:30.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.494+0000 7f0057e00640 1 --2- 172.21.15.116:0/831436893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 0x7f00581620f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00440097c0 tx=0x7f004401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e86ecc8de35ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:30.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f0057400640 1 -- 172.21.15.116:0/831436893 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440b2070 con 0x7f0058161ce0 2024-05-14T11:25:30.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f0057400640 1 -- 172.21.15.116:0/831436893 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00440ad680 con 0x7f0058161ce0 2024-05-14T11:25:30.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f0057400640 1 -- 172.21.15.116:0/831436893 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440ada50 con 0x7f0058161ce0 2024-05-14T11:25:30.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- 172.21.15.116:0/831436893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 msgr2=0x7f00581620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 --2- 172.21.15.116:0/831436893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 0x7f00581620f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00440097c0 tx=0x7f004401c470 comp rx=0 tx=0).stop 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- 172.21.15.116:0/831436893 shutdown_connections 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 --2- 172.21.15.116:0/831436893 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0058161ce0 0x7f00581620f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 --2- 172.21.15.116:0/831436893 >> v2:172.21.15.149:3300/0 conn(0x7f00580c8bb0 0x7f00581617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- 172.21.15.116:0/831436893 >> 172.21.15.116:0/831436893 conn(0x7f0058071190 msgr2=0x7f00580715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- 172.21.15.116:0/831436893 shutdown_connections 2024-05-14T11:25:30.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- 172.21.15.116:0/831436893 wait complete. 2024-05-14T11:25:30.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 Processor -- start 2024-05-14T11:25:30.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.498+0000 7f005f800640 1 -- start start 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 0x7f005814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 0x7f0058160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00580b8050 con 0x7f005815e1b0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f00580b81d0 con 0x7f0058161ce0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00580c8fc0 con 0x7f00580c8bb0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46522/0 (socket says 172.21.15.116:46522) 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 -- 172.21.15.116:0/2332215637 learned_addr learned my addr 172.21.15.116:0/2332215637 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005ce00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 0x7f005814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005d800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 0x7f0058160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 -- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 msgr2=0x7f005814f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 0x7f005814f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 -- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 msgr2=0x7f0058160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 0x7f0058160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 -- 172.21.15.116:0/2332215637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005815f610 con 0x7f005815e1b0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005ce00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 0x7f005814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005d800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 0x7f0058160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0057e00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f00440b0040 tx=0x7f0044019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440b2040 con 0x7f005815e1b0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0044004220 con 0x7f005815e1b0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440ad8f0 con 0x7f005815e1b0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00580c9970 con 0x7f005815e1b0 2024-05-14T11:25:30.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005815fb60 con 0x7f005815e1b0 2024-05-14T11:25:30.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.502+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00580753e0 con 0x7f005815e1b0 2024-05-14T11:25:30.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.506+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00440c3060 con 0x7f005815e1b0 2024-05-14T11:25:30.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.506+0000 7f0054c00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 0x7f0038071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:30.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.506+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f0044128cf0 con 0x7f005815e1b0 2024-05-14T11:25:30.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.506+0000 7f005ce00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 0x7f0038071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:30.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.506+0000 7f005ce00640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 0x7f0038071910 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0048009d20 tx=0x7f004800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:30.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.510+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0044016090 con 0x7f005815e1b0 2024-05-14T11:25:30.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:30.702+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f005806a9c0 con 0x7f005815e1b0 2024-05-14T11:25:31.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.326+0000 7f0054c00640 1 -- 172.21.15.116:0/2332215637 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) ==== 104+0+0 (secure 0 0 0) 0x7f00440f58f0 con 0x7f005815e1b0 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 msgr2=0x7f0038071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 0x7f0038071910 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0048009d20 tx=0x7f004800b040 comp rx=0 tx=0).stop 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 msgr2=0x7f00580c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f00440b0040 tx=0x7f0044019210 comp rx=0 tx=0).stop 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 shutdown_connections 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f003806f400 0x7f0038071910 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0058161ce0 0x7f0058160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005815e1b0 0x7f00580c9430 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 --2- 172.21.15.116:0/2332215637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00580c8bb0 0x7f005814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 >> 172.21.15.116:0/2332215637 conn(0x7f0058071190 msgr2=0x7f00580b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:31.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 shutdown_connections 2024-05-14T11:25:31.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.330+0000 7f005f800640 1 -- 172.21.15.116:0/2332215637 wait complete. 2024-05-14T11:25:31.352 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:25:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 Processor -- start 2024-05-14T11:25:31.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 -- start start 2024-05-14T11:25:31.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9f0015dd10 0x7f9f0006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f0015e120 con 0x7f9f000ce400 2024-05-14T11:25:31.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f9f000cdc90 con 0x7f9f0015dd10 2024-05-14T11:25:31.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9eff200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9f000d08f0 con 0x7f9f000d1bc0 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efe800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9f0015dd10 0x7f9f0006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efe800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:56254/0 (socket says 172.21.15.116:56254) 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9ef7e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f9f000ce400 0x7f9f000d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:40792/0 (socket says 172.21.15.116:40792) 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efe800640 1 -- 172.21.15.116:0/2050815164 learned_addr learned my addr 172.21.15.116:0/2050815164 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 --2- 172.21.15.116:0/2050815164 >> v2:172.21.15.149:3300/0 conn(0x7f9f0015dd10 0x7f9f0006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:43690/0 (socket says 172.21.15.116:43690) 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 -- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 msgr2=0x7f9f00161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 -- 172.21.15.116:0/2050815164 >> v1:172.21.15.149:6789/0 conn(0x7f9f000ce400 legacy=0x7f9f000d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efde00640 1 -- 172.21.15.116:0/2050815164 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f000d0bd0 con 0x7f9f0015dd10 2024-05-14T11:25:31.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efd400640 1 -- 172.21.15.116:0/2050815164 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225614223 0 0) 0x7f9ee8001120 con 0x7f9f000ce400 2024-05-14T11:25:31.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.510+0000 7f9efe800640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:31.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9efde00640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f0006d9a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00097c0 tx=0x7f9ef001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b1b9c7c174daff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9efd400640 1 -- 172.21.15.116:0/2050815164 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00b2070 con 0x7f9f0015dd10 2024-05-14T11:25:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9efd400640 1 -- 172.21.15.116:0/2050815164 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef00ad640 con 0x7f9f0015dd10 2024-05-14T11:25:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9efd400640 1 -- 172.21.15.116:0/2050815164 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00ada10 con 0x7f9f0015dd10 2024-05-14T11:25:31.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 -- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 msgr2=0x7f9f0006d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f0006d9a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00097c0 tx=0x7f9ef001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 -- 172.21.15.116:0/2050815164 shutdown_connections 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000d1bc0 0x7f9f00161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 --2- 172.21.15.116:0/2050815164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f0006d9a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 -- 172.21.15.116:0/2050815164 >> 172.21.15.116:0/2050815164 conn(0x7f9f000b43a0 msgr2=0x7f9f000b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 -- 172.21.15.116:0/2050815164 shutdown_connections 2024-05-14T11:25:31.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.514+0000 7f9eff200640 1 -- 172.21.15.116:0/2050815164 wait complete. 2024-05-14T11:25:31.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 Processor -- start 2024-05-14T11:25:31.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 -- start start 2024-05-14T11:25:31.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000ce400 0x7f9f0014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f000b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0006e060 con 0x7f9f000af1c0 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0006dee0 con 0x7f9f0015dd10 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9eff200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0016b350 con 0x7f9f000ce400 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46534/0 (socket says 172.21.15.116:46534) 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 -- 172.21.15.116:0/3199458460 learned_addr learned my addr 172.21.15.116:0/3199458460 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9ef7e00640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f000b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efde00640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000ce400 0x7f9f0014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 -- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000ce400 msgr2=0x7f9f0014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000ce400 0x7f9f0014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 -- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 msgr2=0x7f9f000b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f000b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.518+0000 7f9efe800640 1 -- 172.21.15.116:0/3199458460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f000afb10 con 0x7f9f000af1c0 2024-05-14T11:25:31.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9efe800640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800ef90 tx=0x7f9ef800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800cd20 con 0x7f9f000af1c0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef80039a0 con 0x7f9f000af1c0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0009ccd0 con 0x7f9f000af1c0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8003d90 con 0x7f9f000af1c0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f000b2920 con 0x7f9f000af1c0 2024-05-14T11:25:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f00148880 con 0x7f9f000af1c0 2024-05-14T11:25:31.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ef8012050 con 0x7f9f000af1c0 2024-05-14T11:25:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 0x7f9edc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6302+0+0 (secure 0 0 0) 0x7f9ef808f880 con 0x7f9f000af1c0 2024-05-14T11:25:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.522+0000 7f9efde00640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 0x7f9edc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.526+0000 7f9efde00640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 0x7f9edc071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00af000 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:31.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.526+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9ef8022020 con 0x7f9f000af1c0 2024-05-14T11:25:31.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:31.686+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9f00050880 con 0x7f9f000af1c0 2024-05-14T11:25:32.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.330+0000 7f9ef6000640 1 -- 172.21.15.116:0/3199458460 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f9f000afb10 con 0x7f9f000af1c0 2024-05-14T11:25:32.328 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 msgr2=0x7f9edc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 0x7f9edc071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00af000 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 msgr2=0x7f9f000af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800ef90 tx=0x7f9ef800c490 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 shutdown_connections 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9edc06f400 0x7f9edc071910 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f0015dd10 0x7f9f000b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f000af1c0 0x7f9f000af5d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 --2- 172.21.15.116:0/3199458460 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9f000ce400 0x7f9f0014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 >> 172.21.15.116:0/3199458460 conn(0x7f9f000b43a0 msgr2=0x7f9f001669c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 shutdown_connections 2024-05-14T11:25:32.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.334+0000 7f9eff200640 1 -- 172.21.15.116:0/3199458460 wait complete. 2024-05-14T11:25:32.359 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:25:32.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.514+0000 7fa7a1800640 1 Processor -- start 2024-05-14T11:25:32.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 -- start start 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa79c0cda30 0x7fa79c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa79c161710 0x7fa79c161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa79c0d0270 con 0x7fa79c0cda30 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa79c0b0c30 con 0x7fa79c161710 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa7a1800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c0d04c0 con 0x7fa79c0cd620 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa79c0cda30 0x7fa79c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa79c0cda30 0x7fa79c06fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46542/0 (socket says 172.21.15.116:46542) 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 -- 172.21.15.116:0/2910905486 learned_addr learned my addr 172.21.15.116:0/2910905486 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa79aa00640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa79b400640 1 --2- 172.21.15.116:0/2910905486 >> v2:172.21.15.149:3300/0 conn(0x7fa79c161710 0x7fa79c161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 -- 172.21.15.116:0/2910905486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 msgr2=0x7fa79c06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c06f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 -- 172.21.15.116:0/2910905486 >> v2:172.21.15.149:3300/0 conn(0x7fa79c161710 msgr2=0x7fa79c161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 --2- 172.21.15.116:0/2910905486 >> v2:172.21.15.149:3300/0 conn(0x7fa79c161710 0x7fa79c161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.518+0000 7fa792000640 1 -- 172.21.15.116:0/2910905486 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c0d07a0 con 0x7fa79c0cda30 2024-05-14T11:25:32.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa792000640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c0cda30 0x7fa79c06fbf0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0b44c0 tx=0x7fa78401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b8100819690f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:32.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa79a000640 1 -- 172.21.15.116:0/2910905486 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b0070 con 0x7fa79c0cda30 2024-05-14T11:25:32.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa79a000640 1 -- 172.21.15.116:0/2910905486 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7840ab600 con 0x7fa79c0cda30 2024-05-14T11:25:32.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa79a000640 1 -- 172.21.15.116:0/2910905486 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840ab9d0 con 0x7fa79c0cda30 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 -- 172.21.15.116:0/2910905486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c0cda30 msgr2=0x7fa79c06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c0cda30 0x7fa79c06fbf0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0b44c0 tx=0x7fa78401c520 comp rx=0 tx=0).stop 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 -- 172.21.15.116:0/2910905486 shutdown_connections 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c0cda30 0x7fa79c06fbf0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2910905486 >> v2:172.21.15.149:3300/0 conn(0x7fa79c161710 0x7fa79c161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2910905486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 -- 172.21.15.116:0/2910905486 >> 172.21.15.116:0/2910905486 conn(0x7fa79c0cde80 msgr2=0x7fa79c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 -- 172.21.15.116:0/2910905486 shutdown_connections 2024-05-14T11:25:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.522+0000 7fa7a1800640 1 -- 172.21.15.116:0/2910905486 wait complete. 2024-05-14T11:25:32.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 Processor -- start 2024-05-14T11:25:32.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- start start 2024-05-14T11:25:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79c149990 0x7fa79c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c06d090 con 0x7fa79c161710 2024-05-14T11:25:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c06cf10 con 0x7fa79c149990 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c06d210 con 0x7fa79c0cd620 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46558/0 (socket says 172.21.15.116:46558) 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 -- 172.21.15.116:0/2875657833 learned_addr learned my addr 172.21.15.116:0/2875657833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa79b400640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79c149990 0x7fa79c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa79aa00640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 -- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 msgr2=0x7fa79c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 -- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79c149990 msgr2=0x7fa79c0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79c149990 0x7fa79c0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 -- 172.21.15.116:0/2875657833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c156a20 con 0x7fa79c161710 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa792000640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa78401abd0 tx=0x7fa7840192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840ba070 con 0x7fa79c161710 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7840b4620 con 0x7fa79c161710 2024-05-14T11:25:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c156d00 con 0x7fa79c161710 2024-05-14T11:25:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b0040 con 0x7fa79c161710 2024-05-14T11:25:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.526+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1572c0 con 0x7fa79c161710 2024-05-14T11:25:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c06aa30 con 0x7fa79c161710 2024-05-14T11:25:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa7840bd070 con 0x7fa79c161710 2024-05-14T11:25:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa793400640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 0x7fa778071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5864+0+0 (secure 0 0 0) 0x7fa784128450 con 0x7fa79c161710 2024-05-14T11:25:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa79aa00640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 0x7fa778071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:32.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.530+0000 7fa79aa00640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 0x7fa778071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0045f0 tx=0x7fa78c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.534+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa784016110 con 0x7fa79c161710 2024-05-14T11:25:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:32.694+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa79c0cfb40 con 0x7fa79c161710 2024-05-14T11:25:33.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.350+0000 7fa793400640 1 -- 172.21.15.116:0/2875657833 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7fa7840f4c40 con 0x7fa79c161710 2024-05-14T11:25:33.350 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:25:33.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.354+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 msgr2=0x7fa778071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.354+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 0x7fa778071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0045f0 tx=0x7fa78c008040 comp rx=0 tx=0).stop 2024-05-14T11:25:33.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.354+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 msgr2=0x7fa79c1581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.354+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa78401abd0 tx=0x7fa7840192c0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 shutdown_connections 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa77806f400 0x7fa778071910 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79c149990 0x7fa79c0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa79c161710 0x7fa79c1581b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 --2- 172.21.15.116:0/2875657833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa79c0cd620 0x7fa79c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 >> 172.21.15.116:0/2875657833 conn(0x7fa79c0cde80 msgr2=0x7fa79c14ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 shutdown_connections 2024-05-14T11:25:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.358+0000 7fa7a1800640 1 -- 172.21.15.116:0/2875657833 wait complete. 2024-05-14T11:25:33.380 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:25:33.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 Processor -- start 2024-05-14T11:25:33.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 -- start start 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c0d1670 0x7f650c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f650c06c700 0x7f650c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650c06ff20 con 0x7f650c15db30 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f650c0ae100 con 0x7f650c06c700 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f6511600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0ae870 con 0x7f650c0d1670 2024-05-14T11:25:33.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f650c06c700 0x7f650c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f650c06c700 0x7f650c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46574/0 (socket says 172.21.15.116:46574) 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 -- 172.21.15.116:0/243852804 learned_addr learned my addr 172.21.15.116:0/243852804 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650a000640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c0d1670 0x7f650c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 -- 172.21.15.116:0/243852804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c0d1670 msgr2=0x7f650c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c0d1670 0x7f650c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 -- 172.21.15.116:0/243852804 >> v1:172.21.15.149:6789/0 conn(0x7f650c15db30 legacy=0x7f650c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.538+0000 7f650b400640 1 -- 172.21.15.116:0/243852804 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c0d1fc0 con 0x7f650c06c700 2024-05-14T11:25:33.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f650b400640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c06c700 0x7f650c06cb10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6504009870 tx=0x7f650401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c80813fbf54e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6509600640 1 -- 172.21.15.116:0/243852804 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b0070 con 0x7f650c06c700 2024-05-14T11:25:33.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6509600640 1 -- 172.21.15.116:0/243852804 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040ab600 con 0x7f650c06c700 2024-05-14T11:25:33.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6509600640 1 -- 172.21.15.116:0/243852804 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040ab9d0 con 0x7f650c06c700 2024-05-14T11:25:33.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 -- 172.21.15.116:0/243852804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c06c700 msgr2=0x7f650c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c06c700 0x7f650c06cb10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6504009870 tx=0x7f650401c400 comp rx=0 tx=0).stop 2024-05-14T11:25:33.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 -- 172.21.15.116:0/243852804 shutdown_connections 2024-05-14T11:25:33.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c06c700 0x7f650c06cb10 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 --2- 172.21.15.116:0/243852804 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c0d1670 0x7f650c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 -- 172.21.15.116:0/243852804 >> 172.21.15.116:0/243852804 conn(0x7f650c0b4620 msgr2=0x7f650c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:33.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 -- 172.21.15.116:0/243852804 shutdown_connections 2024-05-14T11:25:33.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.542+0000 7f6511600640 1 -- 172.21.15.116:0/243852804 wait complete. 2024-05-14T11:25:33.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 Processor -- start 2024-05-14T11:25:33.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 -- start start 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f650c06c700 0x7f650c099cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c15db30 0x7f650c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f650c16a770 con 0x7f650c0d1670 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f650c15df40 con 0x7f650c06c700 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.546+0000 7f6511600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f650c16a8f0 con 0x7f650c15db30 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46576/0 (socket says 172.21.15.116:46576) 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 -- 172.21.15.116:0/1576541607 learned_addr learned my addr 172.21.15.116:0/1576541607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650aa00640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f650c06c700 0x7f650c099cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650b400640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c15db30 0x7f650c0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 -- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c15db30 msgr2=0x7f650c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c15db30 0x7f650c0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 -- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f650c06c700 msgr2=0x7f650c099cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f650c06c700 0x7f650c099cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 -- 172.21.15.116:0/1576541607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c14b7c0 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f650a000640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d5f0 tx=0x7f64f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40125f0 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f4019070 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40129b0 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c0cf900 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c0cfec0 con 0x7f650c0d1670 2024-05-14T11:25:33.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c06a9c0 con 0x7f650c0d1670 2024-05-14T11:25:33.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.550+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64f4012b50 con 0x7f650c0d1670 2024-05-14T11:25:33.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.554+0000 7f6502a00640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.554+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7f64f408f520 con 0x7f650c0d1670 2024-05-14T11:25:33.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.554+0000 7f650aa00640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.554+0000 7f650aa00640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 0x7f64e8071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64fc001e40 tx=0x7f64fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.558+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f64f405be40 con 0x7f650c0d1670 2024-05-14T11:25:33.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.730+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f650c1604a0 con 0x7f650c0d1670 2024-05-14T11:25:33.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.734+0000 7f6502a00640 1 -- 172.21.15.116:0/1576541607 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f64f4060140 con 0x7f650c0d1670 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 msgr2=0x7f64e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 0x7f64e8071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64fc001e40 tx=0x7f64fc009040 comp rx=0 tx=0).stop 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 msgr2=0x7f650c0b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d5f0 tx=0x7f64f400dab0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 shutdown_connections 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f650c15db30 0x7f650c0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650c0d1670 0x7f650c0b34f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 --2- 172.21.15.116:0/1576541607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f650c06c700 0x7f650c099cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 >> 172.21.15.116:0/1576541607 conn(0x7f650c0b4620 msgr2=0x7f650c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:33.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 shutdown_connections 2024-05-14T11:25:33.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.738+0000 7f6511600640 1 -- 172.21.15.116:0/1576541607 wait complete. 2024-05-14T11:25:33.756 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-14T11:25:33.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 Processor -- start 2024-05-14T11:25:33.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 -- start start 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda900d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 0x7fda9006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9006ff20 con 0x7fda9015db30 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fda900ae100 con 0x7fda9006c700 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda97800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda900ae870 con 0x7fda900d1670 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda87e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda900d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda95800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 0x7fda9006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda94e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7fda9015db30 0x7fda900aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:40808/0 (socket says 172.21.15.116:40808) 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda94e00640 1 -- 172.21.15.116:0/403863451 learned_addr learned my addr 172.21.15.116:0/403863451 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda87e00640 1 -- 172.21.15.116:0/403863451 >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 msgr2=0x7fda9006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda87e00640 1 --2- 172.21.15.116:0/403863451 >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 0x7fda9006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda87e00640 1 -- 172.21.15.116:0/403863451 >> v1:172.21.15.149:6789/0 conn(0x7fda9015db30 legacy=0x7fda900aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda87e00640 1 -- 172.21.15.116:0/403863451 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda900d1fc0 con 0x7fda900d1670 2024-05-14T11:25:33.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.906+0000 7fda95800640 1 --2- 172.21.15.116:0/403863451 >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 0x7fda9006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:33.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda87e00640 1 --2- 172.21.15.116:0/403863451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda900d1a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda780097c0 tx=0x7fda7801c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c59557e6893d764 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda8fe00640 1 -- 172.21.15.116:0/403863451 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780b0070 con 0x7fda900d1670 2024-05-14T11:25:33.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda8fe00640 1 -- 172.21.15.116:0/403863451 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda780ab600 con 0x7fda900d1670 2024-05-14T11:25:33.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda8fe00640 1 -- 172.21.15.116:0/403863451 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda780ab9d0 con 0x7fda900d1670 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 -- 172.21.15.116:0/403863451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 msgr2=0x7fda900d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 --2- 172.21.15.116:0/403863451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda900d1a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda780097c0 tx=0x7fda7801c340 comp rx=0 tx=0).stop 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 -- 172.21.15.116:0/403863451 shutdown_connections 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 --2- 172.21.15.116:0/403863451 >> v2:172.21.15.149:3300/0 conn(0x7fda9006c700 0x7fda9006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 --2- 172.21.15.116:0/403863451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda900d1a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 -- 172.21.15.116:0/403863451 >> 172.21.15.116:0/403863451 conn(0x7fda900b4620 msgr2=0x7fda900b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:33.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 -- 172.21.15.116:0/403863451 shutdown_connections 2024-05-14T11:25:33.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.910+0000 7fda97800640 1 -- 172.21.15.116:0/403863451 wait complete. 2024-05-14T11:25:33.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 Processor -- start 2024-05-14T11:25:33.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 -- start start 2024-05-14T11:25:33.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda9014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda9015db30 0x7fda9014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda9016a770 con 0x7fda9006c700 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda9015df40 con 0x7fda9015db30 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda97800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda9016a8f0 con 0x7fda900d1670 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda94e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda94e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46592/0 (socket says 172.21.15.116:46592) 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda94e00640 1 -- 172.21.15.116:0/414000127 learned_addr learned my addr 172.21.15.116:0/414000127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.914+0000 7fda95800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda9015db30 0x7fda9014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda87e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda9014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 -- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 msgr2=0x7fda9014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda9014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 -- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda9015db30 msgr2=0x7fda9014c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:33.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda9015db30 0x7fda9014c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:33.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 -- 172.21.15.116:0/414000127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda900d0930 con 0x7fda9006c700 2024-05-14T11:25:33.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda94e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c460 tx=0x7fda8000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda80005d10 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda900cda30 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800072b0 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda900cdff0 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9006a9c0 con 0x7fda9006c700 2024-05-14T11:25:33.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda8001a070 con 0x7fda9006c700 2024-05-14T11:25:33.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.918+0000 7fda8d600640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:33.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.922+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7fda8008e420 con 0x7fda9006c700 2024-05-14T11:25:33.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.922+0000 7fda87e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:33.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.922+0000 7fda87e00640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 0x7fda6c071910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fda780ad000 tx=0x7fda78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:33.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:33.926+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda8005b440 con 0x7fda9006c700 2024-05-14T11:25:34.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.138+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fda900b29a0 con 0x7fda9006c700 2024-05-14T11:25:34.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda8d600640 1 -- 172.21.15.116:0/414000127 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v439) ==== 146+0+0 (secure 0 0 0) 0x7fda8005ad40 con 0x7fda9006c700 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 msgr2=0x7fda6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 0x7fda6c071910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fda780ad000 tx=0x7fda78008040 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 msgr2=0x7fda900ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c460 tx=0x7fda8000c920 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 shutdown_connections 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda9015db30 0x7fda9014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda900d1670 0x7fda9014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 --2- 172.21.15.116:0/414000127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9006c700 0x7fda900ce510 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 >> 172.21.15.116:0/414000127 conn(0x7fda900b4620 msgr2=0x7fda900b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:34.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 shutdown_connections 2024-05-14T11:25:34.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.354+0000 7fda97800640 1 -- 172.21.15.116:0/414000127 wait complete. 2024-05-14T11:25:34.373 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-14T11:25:34.374 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-14T11:25:34.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:34.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.522+0000 7fa96b200640 1 Processor -- start 2024-05-14T11:25:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.522+0000 7fa96b200640 1 -- start start 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa96b200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa96c15c080 0x7fa96c0af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa96b200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c15c490 con 0x7fa96c0afb40 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa96b200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c0ae560 con 0x7fa96c06c700 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa96b200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c0af170 con 0x7fa96c15c080 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa961400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa96c0afb40 0x7fa96c06e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46604/0 (socket says 172.21.15.116:46604) 2024-05-14T11:25:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa961400640 1 -- 172.21.15.116:0/2209922541 learned_addr learned my addr 172.21.15.116:0/2209922541 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969e00640 1 --2- 172.21.15.116:0/2209922541 >> v2:172.21.15.149:3300/0 conn(0x7fa96c15c080 0x7fa96c0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969400640 1 -- 172.21.15.116:0/2209922541 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2682503512 0 0) 0x7fa96c15c490 con 0x7fa96c0afb40 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969400640 1 -- 172.21.15.116:0/2209922541 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa950005740 con 0x7fa96c0afb40 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969e00640 1 -- 172.21.15.116:0/2209922541 >> v1:172.21.15.149:6789/0 conn(0x7fa96c06c700 legacy=0x7fa96c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969e00640 1 -- 172.21.15.116:0/2209922541 >> v1:172.21.15.116:6789/0 conn(0x7fa96c0afb40 legacy=0x7fa96c06e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.526+0000 7fa969e00640 1 -- 172.21.15.116:0/2209922541 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0ae840 con 0x7fa96c15c080 2024-05-14T11:25:34.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa969e00640 1 --2- 172.21.15.116:0/2209922541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c15c080 0x7fa96c0af600 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0097c0 tx=0x7fa95c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda5cb76490d958a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa969400640 1 -- 172.21.15.116:0/2209922541 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0ab5f0 con 0x7fa96c15c080 2024-05-14T11:25:34.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa969400640 1 -- 172.21.15.116:0/2209922541 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c0b0070 con 0x7fa96c15c080 2024-05-14T11:25:34.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa969400640 1 -- 172.21.15.116:0/2209922541 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0ab9b0 con 0x7fa96c15c080 2024-05-14T11:25:34.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 -- 172.21.15.116:0/2209922541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c15c080 msgr2=0x7fa96c0af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 --2- 172.21.15.116:0/2209922541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c15c080 0x7fa96c0af600 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0097c0 tx=0x7fa95c01c470 comp rx=0 tx=0).stop 2024-05-14T11:25:34.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 -- 172.21.15.116:0/2209922541 shutdown_connections 2024-05-14T11:25:34.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 --2- 172.21.15.116:0/2209922541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c15c080 0x7fa96c0af600 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 -- 172.21.15.116:0/2209922541 >> 172.21.15.116:0/2209922541 conn(0x7fa96c0b4530 msgr2=0x7fa96c0b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:34.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 -- 172.21.15.116:0/2209922541 shutdown_connections 2024-05-14T11:25:34.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.530+0000 7fa96b200640 1 -- 172.21.15.116:0/2209922541 wait complete. 2024-05-14T11:25:34.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 Processor -- start 2024-05-14T11:25:34.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- start start 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c06c700 0x7fa96c09ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 0x7fa96c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06f240 con 0x7fa96c0afb40 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06f0c0 con 0x7fa96c06c700 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c169f90 con 0x7fa96c15c080 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46602/0 (socket says 172.21.15.116:46602) 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 -- 172.21.15.116:0/3727556794 learned_addr learned my addr 172.21.15.116:0/3727556794 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa969e00640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c06c700 0x7fa96c09ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96a800640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 0x7fa96c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 -- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 msgr2=0x7fa96c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 0x7fa96c0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 -- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c06c700 msgr2=0x7fa96c09ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c06c700 0x7fa96c09ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 -- 172.21.15.116:0/3727556794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0c9100 con 0x7fa96c0afb40 2024-05-14T11:25:34.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96a800640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 0x7fa96c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa961400640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c430 tx=0x7fa95400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954017070 con 0x7fa96c0afb40 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa954005cf0 con 0x7fa96c0afb40 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9540072d0 con 0x7fa96c0afb40 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c15d610 con 0x7fa96c0afb40 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.534+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c15dbd0 con 0x7fa96c0afb40 2024-05-14T11:25:34.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c050880 con 0x7fa96c0afb40 2024-05-14T11:25:34.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa95401a070 con 0x7fa96c0afb40 2024-05-14T11:25:34.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa962a00640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 0x7fa948071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7fa95408e3f0 con 0x7fa96c0afb40 2024-05-14T11:25:34.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa969e00640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 0x7fa948071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.538+0000 7fa969e00640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 0x7fa948071940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0af000 tx=0x7fa95c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.542+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa95405ae30 con 0x7fa96c0afb40 2024-05-14T11:25:34.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.702+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96c0ceb30 con 0x7fa96c0afb40 2024-05-14T11:25:34.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa962a00640 1 -- 172.21.15.116:0/3727556794 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13470 (secure 0 0 0) 0x7fa95405f130 con 0x7fa96c0afb40 2024-05-14T11:25:34.703 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:34.703 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":105,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:33.346970+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6835/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6834/288710459":"2024-05-15T11:25:26.992758+0000","172.21.15.116:6835/3421632":"2024-05-15T11:24:50.202416+0000","172.21.15.116:6834/3421632":"2024-05-15T11:24:50.202416+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 msgr2=0x7fa948071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 0x7fa948071940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0af000 tx=0x7fa95c0a7000 comp rx=0 tx=0).stop 2024-05-14T11:25:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 msgr2=0x7fa96c0cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c430 tx=0x7fa95400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 shutdown_connections 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa94806f430 0x7fa948071940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa96c15c080 0x7fa96c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96c0afb40 0x7fa96c0cab60 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 --2- 172.21.15.116:0/3727556794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa96c06c700 0x7fa96c09ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 >> 172.21.15.116:0/3727556794 conn(0x7fa96c0b4530 msgr2=0x7fa96c0727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 shutdown_connections 2024-05-14T11:25:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.706+0000 7fa96b200640 1 -- 172.21.15.116:0/3727556794 wait complete. 2024-05-14T11:25:34.725 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T11:16:00.384207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T11:25:34.725 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T11:25:34.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.874+0000 7fd51cc00640 1 Processor -- start 2024-05-14T11:25:34.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.874+0000 7fd51cc00640 1 -- start start 2024-05-14T11:25:34.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd51cc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd51806f450 0x7fd51806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd51cc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5180cde70 0x7fd5181675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd51cc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51806f860 con 0x7fd5180d1730 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd51cc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd51806db80 con 0x7fd5180cde70 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd51cc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd51806fae0 con 0x7fd51806f450 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5180cde70 0x7fd5181675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5180cde70 0x7fd5181675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46610/0 (socket says 172.21.15.116:46610) 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 -- 172.21.15.116:0/323856986 learned_addr learned my addr 172.21.15.116:0/323856986 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd515e00640 1 --2- 172.21.15.116:0/323856986 >> v2:172.21.15.149:3300/0 conn(0x7fd51806f450 0x7fd51806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 -- 172.21.15.116:0/323856986 >> v2:172.21.15.149:3300/0 conn(0x7fd51806f450 msgr2=0x7fd51806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 --2- 172.21.15.116:0/323856986 >> v2:172.21.15.149:3300/0 conn(0x7fd51806f450 0x7fd51806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 -- 172.21.15.116:0/323856986 >> v1:172.21.15.149:6789/0 conn(0x7fd5180d1730 legacy=0x7fd5180d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:34.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.878+0000 7fd516800640 1 -- 172.21.15.116:0/323856986 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51806fdc0 con 0x7fd5180cde70 2024-05-14T11:25:34.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd516800640 1 --2- 172.21.15.116:0/323856986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180cde70 0x7fd5181675f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd5180d11d0 tx=0x7fd51001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe28d986df22ccdd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd515400640 1 -- 172.21.15.116:0/323856986 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5100b0070 con 0x7fd5180cde70 2024-05-14T11:25:34.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd515400640 1 -- 172.21.15.116:0/323856986 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5100ab600 con 0x7fd5180cde70 2024-05-14T11:25:34.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd515400640 1 -- 172.21.15.116:0/323856986 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5100ab9d0 con 0x7fd5180cde70 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- 172.21.15.116:0/323856986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180cde70 msgr2=0x7fd5181675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 --2- 172.21.15.116:0/323856986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180cde70 0x7fd5181675f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd5180d11d0 tx=0x7fd51001c340 comp rx=0 tx=0).stop 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- 172.21.15.116:0/323856986 shutdown_connections 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 --2- 172.21.15.116:0/323856986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180cde70 0x7fd5181675f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 --2- 172.21.15.116:0/323856986 >> v2:172.21.15.149:3300/0 conn(0x7fd51806f450 0x7fd51806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- 172.21.15.116:0/323856986 >> 172.21.15.116:0/323856986 conn(0x7fd5180b4720 msgr2=0x7fd5180b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- 172.21.15.116:0/323856986 shutdown_connections 2024-05-14T11:25:34.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- 172.21.15.116:0/323856986 wait complete. 2024-05-14T11:25:34.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 Processor -- start 2024-05-14T11:25:34.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.882+0000 7fd51cc00640 1 -- start start 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 0x7fd5180d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 0x7fd5181f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd51816a4d0 con 0x7fd5180d1730 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd51816a350 con 0x7fd5180cde70 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd51816a650 con 0x7fd51806f450 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46622/0 (socket says 172.21.15.116:46622) 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 -- 172.21.15.116:0/2187171096 learned_addr learned my addr 172.21.15.116:0/2187171096 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd515e00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 0x7fd5180d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd50d400640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 0x7fd5181f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 -- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 msgr2=0x7fd5180d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 0x7fd5180d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 -- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 msgr2=0x7fd5181f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 0x7fd5181f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 -- 172.21.15.116:0/2187171096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5180cf0a0 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd50d400640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 0x7fd5181f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd515e00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 0x7fd5180d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd516800640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd51001a9f0 tx=0x7fd5100190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5100ba070 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5100b4620 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5180cf380 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5100b0040 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5180cf940 con 0x7fd5180d1730 2024-05-14T11:25:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.886+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51806a9c0 con 0x7fd5180d1730 2024-05-14T11:25:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.890+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5100bd070 con 0x7fd5180d1730 2024-05-14T11:25:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.890+0000 7fd50ea00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 0x7fd4f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.890+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7fd5101285b0 con 0x7fd5180d1730 2024-05-14T11:25:34.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.890+0000 7fd515e00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 0x7fd4f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:34.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.890+0000 7fd515e00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 0x7fd4f4071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd5180c83b0 tx=0x7fd50800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:34.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:34.894+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd5100eff50 con 0x7fd5180d1730 2024-05-14T11:25:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.050+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5180d0c40 con 0x7fd5180d1730 2024-05-14T11:25:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.050+0000 7fd50ea00640 1 -- 172.21.15.116:0/2187171096 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fd51806f860 con 0x7fd5180d1730 2024-05-14T11:25:35.049 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 msgr2=0x7fd4f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 0x7fd4f4071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd5180c83b0 tx=0x7fd50800b040 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 msgr2=0x7fd518159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd51001a9f0 tx=0x7fd5100190e0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 shutdown_connections 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd4f406f400 0x7fd4f4071910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5180cde70 0x7fd5181f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5180d1730 0x7fd518159b70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 --2- 172.21.15.116:0/2187171096 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd51806f450 0x7fd5180d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 >> 172.21.15.116:0/2187171096 conn(0x7fd5180b4720 msgr2=0x7fd51815cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 shutdown_connections 2024-05-14T11:25:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.054+0000 7fd51cc00640 1 -- 172.21.15.116:0/2187171096 wait complete. 2024-05-14T11:25:35.071 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-14T11:25:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.218+0000 7f0239400640 1 Processor -- start 2024-05-14T11:25:35.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.218+0000 7f0239400640 1 -- start start 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0239400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f02340cdd30 0x7f02340b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0239400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02340ce5f0 con 0x7f02340ce140 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0239400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02340ce8d0 con 0x7f023415eb80 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0239400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f02340d1da0 con 0x7f02340cdd30 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f02340cdd30 0x7f02340b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f022b400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f02340ce140 0x7f02340d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46608/0 (socket says 172.21.15.116:46608) 2024-05-14T11:25:35.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f022b400640 1 -- 172.21.15.116:0/1097509536 learned_addr learned my addr 172.21.15.116:0/1097509536 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233e00640 1 -- 172.21.15.116:0/1097509536 >> v1:172.21.15.149:6789/0 conn(0x7f023415eb80 legacy=0x7f02340d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233e00640 1 -- 172.21.15.116:0/1097509536 >> v1:172.21.15.116:6789/0 conn(0x7f02340ce140 legacy=0x7f02340d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233e00640 1 -- 172.21.15.116:0/1097509536 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02340d1750 con 0x7f02340cdd30 2024-05-14T11:25:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233400640 1 -- 172.21.15.116:0/1097509536 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477378689 0 0) 0x7f02340ce5f0 con 0x7f02340ce140 2024-05-14T11:25:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.222+0000 7f0233e00640 1 --2- 172.21.15.116:0/1097509536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02340cdd30 0x7f02340b54b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f02340d1f20 tx=0x7f022401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c41b0e048a50cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0233400640 1 -- 172.21.15.116:0/1097509536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240b0070 con 0x7f02340cdd30 2024-05-14T11:25:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0233400640 1 -- 172.21.15.116:0/1097509536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02240ab600 con 0x7f02340cdd30 2024-05-14T11:25:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0233400640 1 -- 172.21.15.116:0/1097509536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240ab9d0 con 0x7f02340cdd30 2024-05-14T11:25:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- 172.21.15.116:0/1097509536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02340cdd30 msgr2=0x7f02340b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 --2- 172.21.15.116:0/1097509536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02340cdd30 0x7f02340b54b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f02340d1f20 tx=0x7f022401c3e0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- 172.21.15.116:0/1097509536 shutdown_connections 2024-05-14T11:25:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 --2- 172.21.15.116:0/1097509536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02340cdd30 0x7f02340b54b0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- 172.21.15.116:0/1097509536 >> 172.21.15.116:0/1097509536 conn(0x7f02340d0550 msgr2=0x7f02340d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- 172.21.15.116:0/1097509536 shutdown_connections 2024-05-14T11:25:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- 172.21.15.116:0/1097509536 wait complete. 2024-05-14T11:25:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 Processor -- start 2024-05-14T11:25:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.226+0000 7f0239400640 1 -- start start 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02340cdd30 0x7f023414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02340ce140 0x7f023414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f023416be40 con 0x7f023415eb80 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02340d3260 con 0x7f02340cdd30 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f023416bfc0 con 0x7f02340ce140 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46632/0 (socket says 172.21.15.116:46632) 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 -- 172.21.15.116:0/2158968818 learned_addr learned my addr 172.21.15.116:0/2158968818 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0233e00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02340cdd30 0x7f023414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f022b400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02340ce140 0x7f023414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 -- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02340ce140 msgr2=0x7f023414c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02340ce140 0x7f023414c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 -- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02340cdd30 msgr2=0x7f023414e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02340cdd30 0x7f023414e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 -- 172.21.15.116:0/2158968818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023414ece0 con 0x7f023415eb80 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0238a00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c740 tx=0x7f022c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c003b80 con 0x7f023415eb80 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c00ce50 con 0x7f023415eb80 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023414efc0 con 0x7f023415eb80 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c00a850 con 0x7f023415eb80 2024-05-14T11:25:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023414f580 con 0x7f023415eb80 2024-05-14T11:25:35.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.230+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023414ea50 con 0x7f023415eb80 2024-05-14T11:25:35.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.234+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f022c00a9f0 con 0x7f023415eb80 2024-05-14T11:25:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.234+0000 7f0230c00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 0x7f0210071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.234+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7f022c08eff0 con 0x7f023415eb80 2024-05-14T11:25:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.234+0000 7f0233e00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 0x7f0210071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:35.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.234+0000 7f0233e00640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 0x7f0210071940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02240ad000 tx=0x7f02240a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:35.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.238+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f022c05bfc0 con 0x7f023415eb80 2024-05-14T11:25:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:35.454+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f0234050880 con 0x7f023415eb80 2024-05-14T11:25:36.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.362+0000 7f0230c00640 1 -- 172.21.15.116:0/2158968818 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v441) ==== 147+0+0 (secure 0 0 0) 0x7f022c064040 con 0x7f023415eb80 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 msgr2=0x7f0210071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 0x7f0210071940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02240ad000 tx=0x7f02240a6000 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 msgr2=0x7f02340b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c740 tx=0x7f022c00cc00 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 shutdown_connections 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f021006f430 0x7f0210071940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f023415eb80 0x7f02340b8470 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02340ce140 0x7f023414c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 --2- 172.21.15.116:0/2158968818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02340cdd30 0x7f023414e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 >> 172.21.15.116:0/2158968818 conn(0x7f02340d0550 msgr2=0x7f0234070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 shutdown_connections 2024-05-14T11:25:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.366+0000 7f0239400640 1 -- 172.21.15.116:0/2158968818 wait complete. 2024-05-14T11:25:36.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-14T11:25:36.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 Processor -- start 2024-05-14T11:25:36.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 -- start start 2024-05-14T11:25:36.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe1806f450 0x7efe1806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7efe180d1730 0x7efe180d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1806f860 con 0x7efe180cce60 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efe1806db80 con 0x7efe1806f450 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe1e600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7efe180d2080 con 0x7efe180d1730 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe1806f450 0x7efe1806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe1806f450 0x7efe1806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46644/0 (socket says 172.21.15.116:46644) 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 -- 172.21.15.116:0/2503848691 learned_addr learned my addr 172.21.15.116:0/2503848691 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe16a00640 1 --2- 172.21.15.116:0/2503848691 >> v2:172.21.15.149:3300/0 conn(0x7efe180d1730 0x7efe180d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 -- 172.21.15.116:0/2503848691 >> v2:172.21.15.149:3300/0 conn(0x7efe180d1730 msgr2=0x7efe180d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:36.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 --2- 172.21.15.116:0/2503848691 >> v2:172.21.15.149:3300/0 conn(0x7efe180d1730 0x7efe180d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 -- 172.21.15.116:0/2503848691 >> v1:172.21.15.116:6789/0 conn(0x7efe180cce60 legacy=0x7efe180cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe17400640 1 -- 172.21.15.116:0/2503848691 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1806fae0 con 0x7efe1806f450 2024-05-14T11:25:36.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.546+0000 7efe16000640 1 -- 172.21.15.116:0/2503848691 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729762362 0 0) 0x7efe10001120 con 0x7efe180cce60 2024-05-14T11:25:36.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe17400640 1 --2- 172.21.15.116:0/2503848691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1806f450 0x7efe1806d640 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7efe180d2430 tx=0x7efe0801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981a4b976c4ac7db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:36.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe16000640 1 -- 172.21.15.116:0/2503848691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180ce190 con 0x7efe1806f450 2024-05-14T11:25:36.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe16000640 1 -- 172.21.15.116:0/2503848691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe080a57a0 con 0x7efe1806f450 2024-05-14T11:25:36.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe16000640 1 -- 172.21.15.116:0/2503848691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe080a5b70 con 0x7efe1806f450 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 -- 172.21.15.116:0/2503848691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1806f450 msgr2=0x7efe1806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 --2- 172.21.15.116:0/2503848691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1806f450 0x7efe1806d640 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7efe180d2430 tx=0x7efe0801c9f0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 -- 172.21.15.116:0/2503848691 shutdown_connections 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 --2- 172.21.15.116:0/2503848691 >> v2:172.21.15.149:3300/0 conn(0x7efe180d1730 0x7efe180d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 --2- 172.21.15.116:0/2503848691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1806f450 0x7efe1806d640 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 -- 172.21.15.116:0/2503848691 >> 172.21.15.116:0/2503848691 conn(0x7efe180b4720 msgr2=0x7efe180b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 -- 172.21.15.116:0/2503848691 shutdown_connections 2024-05-14T11:25:36.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.550+0000 7efe1e600640 1 -- 172.21.15.116:0/2503848691 wait complete. 2024-05-14T11:25:36.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 Processor -- start 2024-05-14T11:25:36.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- start start 2024-05-14T11:25:36.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 0x7efe18155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe180d1730 0x7efe181662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe1816a610 con 0x7efe180cce60 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe18070660 con 0x7efe1806f450 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe1816a790 con 0x7efe180d1730 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46656/0 (socket says 172.21.15.116:46656) 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 -- 172.21.15.116:0/670428689 learned_addr learned my addr 172.21.15.116:0/670428689 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe17400640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 0x7efe18155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe17e00640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe180d1730 0x7efe181662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 -- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe180d1730 msgr2=0x7efe181662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe180d1730 0x7efe181662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 -- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 msgr2=0x7efe18155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 0x7efe18155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 -- 172.21.15.116:0/670428689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe18160200 con 0x7efe180cce60 2024-05-14T11:25:36.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe17400640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 0x7efe18155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:36.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe16a00640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e8a0 tx=0x7efe0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:36.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00017070 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0000ee90 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe18167f30 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000cb40 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.554+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe18157e20 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1806a9c0 con 0x7efe180cce60 2024-05-14T11:25:36.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe0001a070 con 0x7efe180cce60 2024-05-14T11:25:36.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe0f400640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 0x7efdf4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:36.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7efe0008f480 con 0x7efe180cce60 2024-05-14T11:25:36.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe17400640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 0x7efdf4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:36.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.558+0000 7efe17400640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 0x7efdf4071910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe080ae000 tx=0x7efe080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:36.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.562+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efe0005c3f0 con 0x7efe180cce60 2024-05-14T11:25:36.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.722+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efe18069de0 con 0x7efe180cce60 2024-05-14T11:25:36.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.726+0000 7efe0f400640 1 -- 172.21.15.116:0/670428689 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7efe0005bcf0 con 0x7efe180cce60 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 msgr2=0x7efdf4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 0x7efdf4071910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe080ae000 tx=0x7efe080a7070 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 msgr2=0x7efe18165d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7efe0000e8a0 tx=0x7efe0000ed60 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 shutdown_connections 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efdf406f400 0x7efdf4071910 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe180d1730 0x7efe181662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180cce60 0x7efe18165d90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 --2- 172.21.15.116:0/670428689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe1806f450 0x7efe18155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 >> 172.21.15.116:0/670428689 conn(0x7efe180b4720 msgr2=0x7efe180b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:36.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 shutdown_connections 2024-05-14T11:25:36.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:36.730+0000 7efe1e600640 1 -- 172.21.15.116:0/670428689 wait complete. 2024-05-14T11:25:36.747 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:25:36.748 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:25:36.797 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:25:36.895 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:25:36.895 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:36.895 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:36.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:25:36.904 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:25:36.904 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:36.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:25:36.913 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:36.913 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:36.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:25:36.954 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:25:36.955 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:36.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:25:36.964 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-14T11:25:36.964 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.142+0000 7f5fc3200640 1 Processor -- start 2024-05-14T11:25:37.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.142+0000 7f5fc3200640 1 -- start start 2024-05-14T11:25:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc3200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5fbc0d1670 0x7f5fbc0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 0x7f5fbc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc3200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fbc0d1fc0 con 0x7f5fbc156fa0 2024-05-14T11:25:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc3200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc0706f0 con 0x7f5fbc0d1670 2024-05-14T11:25:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc3200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc1573b0 con 0x7f5fbc0af380 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc1200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 0x7f5fbc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbb400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f5fbc0d1670 0x7f5fbc0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbbe00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f5fbc156fa0 0x7f5fbc06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:40850/0 (socket says 172.21.15.116:40850) 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbbe00640 1 -- 172.21.15.116:0/2280668939 learned_addr learned my addr 172.21.15.116:0/2280668939 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbb400640 1 -- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 msgr2=0x7f5fbc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbb400640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 0x7f5fbc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbb400640 1 -- 172.21.15.116:0/2280668939 >> v1:172.21.15.149:6789/0 conn(0x7f5fbc156fa0 legacy=0x7f5fbc06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fbb400640 1 -- 172.21.15.116:0/2280668939 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc160510 con 0x7f5fbc0d1670 2024-05-14T11:25:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.146+0000 7f5fc1200640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 0x7f5fbc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:37.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fbb400640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0d1670 0x7f5fbc0d1a80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097c0 tx=0x7f5fa801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a27ab371c50e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fbaa00640 1 -- 172.21.15.116:0/2280668939 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80b2070 con 0x7f5fbc0d1670 2024-05-14T11:25:37.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fbaa00640 1 -- 172.21.15.116:0/2280668939 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa80ad640 con 0x7f5fbc0d1670 2024-05-14T11:25:37.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fbaa00640 1 -- 172.21.15.116:0/2280668939 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80ada10 con 0x7f5fbc0d1670 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 -- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0d1670 msgr2=0x7f5fbc0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0d1670 0x7f5fbc0d1a80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097c0 tx=0x7f5fa801c4a0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 -- 172.21.15.116:0/2280668939 shutdown_connections 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0af380 0x7f5fbc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2280668939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0d1670 0x7f5fbc0d1a80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 -- 172.21.15.116:0/2280668939 >> 172.21.15.116:0/2280668939 conn(0x7f5fbc0b4620 msgr2=0x7f5fbc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 -- 172.21.15.116:0/2280668939 shutdown_connections 2024-05-14T11:25:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.150+0000 7f5fc3200640 1 -- 172.21.15.116:0/2280668939 wait complete. 2024-05-14T11:25:37.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 Processor -- start 2024-05-14T11:25:37.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 -- start start 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0af380 0x7f5fbc14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 0x7f5fbc154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc0b0660 con 0x7f5fbc156fa0 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc0b04e0 con 0x7f5fbc0af380 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc3200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc169810 con 0x7f5fbc0d1670 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46660/0 (socket says 172.21.15.116:46660) 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 -- 172.21.15.116:0/2494605146 learned_addr learned my addr 172.21.15.116:0/2494605146 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fbb400640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 0x7f5fbc154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fbbe00640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0af380 0x7f5fbc14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 -- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 msgr2=0x7f5fbc154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 0x7f5fbc154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 -- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0af380 msgr2=0x7f5fbc14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0af380 0x7f5fbc14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 -- 172.21.15.116:0/2494605146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc14ea50 con 0x7f5fbc156fa0 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fbb400640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 0x7f5fbc154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:37.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.154+0000 7f5fc1200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400e960 tx=0x7f5fb400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4017070 con 0x7f5fbc156fa0 2024-05-14T11:25:37.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4004020 con 0x7f5fbc156fa0 2024-05-14T11:25:37.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc1f9f30 con 0x7f5fbc156fa0 2024-05-14T11:25:37.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4004430 con 0x7f5fbc156fa0 2024-05-14T11:25:37.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc0748b0 con 0x7f5fbc156fa0 2024-05-14T11:25:37.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc06a9c0 con 0x7f5fbc156fa0 2024-05-14T11:25:37.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5fb400c9a0 con 0x7f5fbc156fa0 2024-05-14T11:25:37.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 0x7f5f9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7f5fb408f340 con 0x7f5fbc156fa0 2024-05-14T11:25:37.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.158+0000 7f5fbbe00640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 0x7f5f9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.162+0000 7f5fbbe00640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 0x7f5f9c071910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0012f0 tx=0x7f5fac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.162+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5fbc14ea50 con 0x7f5fbc156fa0 2024-05-14T11:25:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.354+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fbc06d5f0 con 0x7f5fbc156fa0 2024-05-14T11:25:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.354+0000 7f5fb3e00640 1 -- 172.21.15.116:0/2494605146 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+3878 (secure 0 0 0) 0x7f5fb405c350 con 0x7f5fbc156fa0 2024-05-14T11:25:37.353 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:37.353 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/324141257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":324141257},{"type":"v1","addr":"172.21.15.149:6835","nonce":324141257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7315,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3833603797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3833603797},{"type":"v1","addr":"172.21.15.116:6837","nonce":3833603797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1347718562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1347718562},{"type":"v1","addr":"172.21.15.149:6833","nonce":1347718562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8088,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2140008008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2140008008},{"type":"v1","addr":"172.21.15.116:6835","nonce":2140008008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-05-14T11:25:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 93 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 msgr2=0x7f5f9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 0x7f5f9c071910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0012f0 tx=0x7f5fac009040 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 msgr2=0x7f5fbc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400e960 tx=0x7f5fb400ee20 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 shutdown_connections 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f5f9c06f400 0x7f5f9c071910 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fbc156fa0 0x7f5fbc154b70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5fbc0d1670 0x7f5fbc154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 --2- 172.21.15.116:0/2494605146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fbc0af380 0x7f5fbc14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 >> 172.21.15.116:0/2494605146 conn(0x7f5fbc0b4620 msgr2=0x7f5fbc0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 shutdown_connections 2024-05-14T11:25:37.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.358+0000 7f5fc3200640 1 -- 172.21.15.116:0/2494605146 wait complete. 2024-05-14T11:25:37.378 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-14T11:25:37.378 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-14T11:25:37.378 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:25:37.378 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:25:37.379 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-14T11:25:37.379 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-14T11:25:37.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:25:37.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:25:37.379 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-14T11:25:37.379 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-14T11:25:37.379 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:25:37.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:25:37.380 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-14T11:25:37.380 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-14T11:25:37.380 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:25:37.380 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:25:37.393 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:25:37.393 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-14T11:25:37.395 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:25:37.395 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-14T11:25:37.395 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:25:37.395 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-14T11:25:37.402 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:25:37.402 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-14T11:25:37.442 INFO:tasks.ceph.mds.d:Started 2024-05-14T11:25:37.443 DEBUG:teuthology.parallel:result is None 2024-05-14T11:25:37.444 INFO:tasks.ceph.mds.c:Started 2024-05-14T11:25:37.444 DEBUG:teuthology.parallel:result is None 2024-05-14T11:25:37.445 INFO:tasks.ceph.mds.b:Started 2024-05-14T11:25:37.445 DEBUG:teuthology.parallel:result is None 2024-05-14T11:25:37.446 INFO:tasks.ceph.mds.a:Started 2024-05-14T11:25:37.446 DEBUG:teuthology.parallel:result is None 2024-05-14T11:25:37.447 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-14T11:25:37.544 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-05-14T11:25:37.545 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:25:37.549 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-05-14T11:25:37.549 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-05-14T11:25:37.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 Processor -- start 2024-05-14T11:25:37.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 -- start start 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1ce40d1670 0x7f1ce40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1ce406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce406ff20 con 0x7f1ce415db30 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce40ae100 con 0x7f1ce40d1670 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cea000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce40ae870 con 0x7f1ce406c700 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1ce415db30 0x7f1ce40aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37276/0 (socket says 172.21.15.116:37276) 2024-05-14T11:25:37.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce3400640 1 -- 172.21.15.116:0/3665069791 learned_addr learned my addr 172.21.15.116:0/3665069791 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> v2:172.21.15.149:3300/0 conn(0x7f1ce40d1670 0x7f1ce40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce3e00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1ce406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419366508 0 0) 0x7f1ce406ff20 con 0x7f1ce415db30 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc8003530 con 0x7f1ce415db30 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 929155462 0 0) 0x7f1cc8003530 con 0x7f1ce415db30 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 msgr2=0x7f1ce406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1ce406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 >> v2:172.21.15.149:3300/0 conn(0x7f1ce40d1670 msgr2=0x7f1ce40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:37.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 --2- 172.21.15.116:0/3665069791 >> v2:172.21.15.149:3300/0 conn(0x7f1ce40d1670 0x7f1ce40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40d1fc0 con 0x7f1ce415db30 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f1cd4001700 con 0x7f1ce415db30 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> v2:172.21.15.149:3300/0 conn(0x7f1ce40d1670 0x7f1ce40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 >> v1:172.21.15.116:6789/0 conn(0x7f1ce415db30 legacy=0x7f1ce40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1cc8003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cc8003fe0 0x7f1cc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8004d50 0x7f1cc8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40ae100 con 0x7f1cc8003fe0 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40ae870 con 0x7f1cc8004d50 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc800bc10 con 0x7f1ce406c700 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cc8003fe0 0x7f1cc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce3400640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1cc8003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.606+0000 7f1ce3e00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8004d50 0x7f1cc8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 -- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 msgr2=0x7f1cc8003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:37.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce406c700 0x7f1cc8003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 -- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8004d50 msgr2=0x7f1cc8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:37.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8004d50 0x7f1cc8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 -- 172.21.15.116:0/3665069791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc8003530 con 0x7f1cc8003fe0 2024-05-14T11:25:37.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cc8003fe0 0x7f1cc8004810 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c80 tx=0x7f1ccc01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed42c1c03e750701 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b2070 con 0x7f1cc8003fe0 2024-05-14T11:25:37.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0a57e0 con 0x7f1cc8003fe0 2024-05-14T11:25:37.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.610+0000 7f1ce2a00640 1 -- 172.21.15.116:0/3665069791 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc009880 con 0x7f1cc8003fe0 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- 172.21.15.116:0/3665069791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cc8003fe0 msgr2=0x7f1cc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 --2- 172.21.15.116:0/3665069791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cc8003fe0 0x7f1cc8004810 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c80 tx=0x7f1ccc01cc50 comp rx=0 tx=0).stop 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1ce3e00640 1 -- 172.21.15.116:0/3665069791 reap_dead start 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- 172.21.15.116:0/3665069791 shutdown_connections 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- 172.21.15.116:0/3665069791 >> 172.21.15.116:0/3665069791 conn(0x7f1ce40b4620 msgr2=0x7f1ce40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- 172.21.15.116:0/3665069791 shutdown_connections 2024-05-14T11:25:37.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- 172.21.15.116:0/3665069791 wait complete. 2024-05-14T11:25:37.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 Processor -- start 2024-05-14T11:25:37.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.614+0000 7f1cea000640 1 -- start start 2024-05-14T11:25:37.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 0x7f1ce41f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce41fc940 0x7f1ce41ed470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce406d320 con 0x7f1ce41479a0 2024-05-14T11:25:37.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce415df40 con 0x7f1cc8003fe0 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce406d4a0 con 0x7f1ce41fc940 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43204/0 (socket says 172.21.15.116:43204) 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 -- 172.21.15.116:0/4212843791 learned_addr learned my addr 172.21.15.116:0/4212843791 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 -- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce41fc940 msgr2=0x7f1ce41ed470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1ce3400640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 0x7f1ce41f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce41fc940 0x7f1ce41ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 -- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 msgr2=0x7f1ce41f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 0x7f1ce41f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 -- 172.21.15.116:0/4212843791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce415f0c0 con 0x7f1ce41479a0 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1ce3400640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 0x7f1ce41f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:37.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdbe00640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0b0040 tx=0x7f1cc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0bd020 con 0x7f1ce41479a0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc004020 con 0x7f1ce41479a0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b2040 con 0x7f1ce41479a0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce41599b0 con 0x7f1ce41479a0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce4159c90 con 0x7f1ce41479a0 2024-05-14T11:25:37.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce406a9c0 con 0x7f1ce41479a0 2024-05-14T11:25:37.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.618+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ccc003620 con 0x7f1ce41479a0 2024-05-14T11:25:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.622+0000 7f1cdb400640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 0x7f1cc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:37.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.622+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5421+0+0 (secure 0 0 0) 0x7f1ccc128fe0 con 0x7f1ce41479a0 2024-05-14T11:25:37.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.622+0000 7f1ce3400640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 0x7f1cc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:37.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.622+0000 7f1ce3400640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 0x7f1cc0071910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4007e60 tx=0x7f1cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.626+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1ccc016960 con 0x7f1ce41479a0 2024-05-14T11:25:37.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:37.794+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1ce41564a0 con 0x7f1ce41479a0 2024-05-14T11:25:38.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cdb400640 1 -- 172.21.15.116:0/4212843791 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) ==== 97+0+0 (secure 0 0 0) 0x7f1ccc0f6000 con 0x7f1ce41479a0 2024-05-14T11:25:38.388 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 msgr2=0x7f1cc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 0x7f1cc0071910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4007e60 tx=0x7f1cd4009040 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 msgr2=0x7f1ce41fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0b0040 tx=0x7f1cc8005840 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 shutdown_connections 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1cc006f400 0x7f1cc0071910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ce41fc940 0x7f1ce41ed470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ce41479a0 0x7f1ce41fc400 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 --2- 172.21.15.116:0/4212843791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc8003fe0 0x7f1ce41f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.390+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 >> 172.21.15.116:0/4212843791 conn(0x7f1ce40b4620 msgr2=0x7f1ce4156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.394+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 shutdown_connections 2024-05-14T11:25:38.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.394+0000 7f1cea000640 1 -- 172.21.15.116:0/4212843791 wait complete. 2024-05-14T11:25:38.414 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-14T11:25:38.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.562+0000 7f02c9a00640 1 Processor -- start 2024-05-14T11:25:38.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.562+0000 7f02c9a00640 1 -- start start 2024-05-14T11:25:38.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f02c406ead0 0x7f02c406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c415ce20 con 0x7f02c40d15e0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f02c406f850 con 0x7f02c406ead0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f02c406fb30 con 0x7f02c40cbaf0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c3e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f02c40d15e0 0x7f02c40d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37280/0 (socket says 172.21.15.116:37280) 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c3e00640 1 -- 172.21.15.116:0/3513343700 learned_addr learned my addr 172.21.15.116:0/3513343700 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02bb400640 1 --2- 172.21.15.116:0/3513343700 >> v2:172.21.15.116:3300/0 conn(0x7f02c406ead0 0x7f02c406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c9000640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c3400640 1 -- 172.21.15.116:0/3513343700 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1629541398 0 0) 0x7f02c415ce20 con 0x7f02c40d15e0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02c3400640 1 -- 172.21.15.116:0/3513343700 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02a8003530 con 0x7f02c40d15e0 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02bb400640 1 -- 172.21.15.116:0/3513343700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 msgr2=0x7f02c40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02bb400640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02bb400640 1 -- 172.21.15.116:0/3513343700 >> v1:172.21.15.116:6789/0 conn(0x7f02c40d15e0 legacy=0x7f02c40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.566+0000 7f02bb400640 1 -- 172.21.15.116:0/3513343700 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c406fe10 con 0x7f02c406ead0 2024-05-14T11:25:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02bb400640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c406eee0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097c0 tx=0x7f02ac01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1dc2f859634cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c3400640 1 -- 172.21.15.116:0/3513343700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0ab610 con 0x7f02c406ead0 2024-05-14T11:25:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c3400640 1 -- 172.21.15.116:0/3513343700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ac0b0070 con 0x7f02c406ead0 2024-05-14T11:25:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c3400640 1 -- 172.21.15.116:0/3513343700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0ab9b0 con 0x7f02c406ead0 2024-05-14T11:25:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 -- 172.21.15.116:0/3513343700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 msgr2=0x7f02c406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c406eee0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097c0 tx=0x7f02ac01c410 comp rx=0 tx=0).stop 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 -- 172.21.15.116:0/3513343700 shutdown_connections 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3513343700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c406eee0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 -- 172.21.15.116:0/3513343700 >> 172.21.15.116:0/3513343700 conn(0x7f02c40b4530 msgr2=0x7f02c40b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 -- 172.21.15.116:0/3513343700 shutdown_connections 2024-05-14T11:25:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.570+0000 7f02c9a00640 1 -- 172.21.15.116:0/3513343700 wait complete. 2024-05-14T11:25:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 Processor -- start 2024-05-14T11:25:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- start start 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02c40d15e0 0x7f02c406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4169570 con 0x7f02c406ead0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02c406f630 con 0x7f02c40d15e0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f02c41696f0 con 0x7f02c40cbaf0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43226/0 (socket says 172.21.15.116:43226) 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 -- 172.21.15.116:0/3285966360 learned_addr learned my addr 172.21.15.116:0/3285966360 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02bb400640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9000640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02c40d15e0 0x7f02c406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 -- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 msgr2=0x7f02c406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 -- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02c40d15e0 msgr2=0x7f02c406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02c40d15e0 0x7f02c406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 -- 172.21.15.116:0/3285966360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c409ae80 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02bb400640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c3e00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c450 tx=0x7f02b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4017070 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b4005cf0 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4007270 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c40c51c0 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.574+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c415d430 con 0x7f02c406ead0 2024-05-14T11:25:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c4050880 con 0x7f02c406ead0 2024-05-14T11:25:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f02b401a070 con 0x7f02c406ead0 2024-05-14T11:25:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02c0c00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 0x7f02a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f02b408e2d0 con 0x7f02c406ead0 2024-05-14T11:25:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02bb400640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 0x7f02a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.578+0000 7f02bb400640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 0x7f02a0071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0af000 tx=0x7f02ac0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.582+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f02b405ac90 con 0x7f02c406ead0 2024-05-14T11:25:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f02c4074260 con 0x7f02c406ead0 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c0c00640 1 -- 172.21.15.116:0/3285966360 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f02b405ef90 con 0x7f02c406ead0 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.791 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmfZY7MhAARnSXa8z1QkVWztm9l2QeCg==", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nm7W8CBBAAzyf8awnQIy0rN+rHrSHGQA==", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:25:38.792 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmEM0MOhAAeBgRZiiy/CpolefxJK/Nsw==", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0Nmbz2kCxAAMVQhhCldV+Bs5Gaz0mWH8A==", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.793 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDER0NmILFUDBAAHIxW7qAaGZ/F1vmd+5Gjhg==", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.794 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDHR0NmxMBfFxAA/ynQin7l6FWcz4zGJF3rMA==", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKR0NmNW5gIRAAx+gA4epZAIzFfNIfS2gWtA==", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.795 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDNR0NmkKLfKhAAWg/JjZSmqjFRUTSTGheGiQ==", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDTR0NmGxo6IhAAyYKVbEPmcRpGnSVkiy04mg==", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.796 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDWR0NmkBkNLRAAP/+3clvD1D0nASVKY9TQlw==", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDZR0Nmj4ueNxAAFhMIYPOHtZm43DGYPGdcnw==", 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.797 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDdR0NmmGx8BxAA5JrL8OnWBw272J+U0YziBg==", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw==", 2024-05-14T11:25:38.798 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmlnY7GBAAT3rtBRj/yK0NjO+yC3lZxA==", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmRmggFBAAhT+adLBRsvtwTSny3Zd6hw==", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDBR0NmCDZEHBAAvSb5rMccqn6GYG3PFsUx3g==", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.800 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmdtVyFBAAJ+0qj2OxA1EixYTcFSm+Fw==", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0NmafZWLRAAhfZ8CMNL4KnKZALKGBBHiA==", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:25:38.801 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDAR0Nm0oqZKBAAE81S6asBULLpm1TBS15YKQ==", 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-05-14T11:25:38.802 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 msgr2=0x7f02a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 0x7f02a0071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0af000 tx=0x7f02ac0a7000 comp rx=0 tx=0).stop 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 msgr2=0x7f02c409a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c450 tx=0x7f02b400c910 comp rx=0 tx=0).stop 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 shutdown_connections 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f02a006f430 0x7f02a0071940 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02c40d15e0 0x7f02c406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f02c40cbaf0 0x7f02c406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 --2- 172.21.15.116:0/3285966360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02c406ead0 0x7f02c409a940 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 >> 172.21.15.116:0/3285966360 conn(0x7f02c40b4530 msgr2=0x7f02c40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:38.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 shutdown_connections 2024-05-14T11:25:38.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.794+0000 7f02c9a00640 1 -- 172.21.15.116:0/3285966360 wait complete. 2024-05-14T11:25:38.814 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-14T11:25:38.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 Processor -- start 2024-05-14T11:25:38.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 -- start start 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f8c8c0b70a0 0x7f8c8c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8c0cfeb0 con 0x7f8c8c15fe00 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8c1646c0 con 0x7f8c8c0b8f90 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c91e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8c0cf110 con 0x7f8c8c0b70a0 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c82000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8c8c15fe00 0x7f8c8c1638a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37284/0 (socket says 172.21.15.116:37284) 2024-05-14T11:25:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c82000640 1 -- 172.21.15.116:0/2833719763 learned_addr learned my addr 172.21.15.116:0/2833719763 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/2833719763 >> v2:172.21.15.149:3300/0 conn(0x7f8c8c0b70a0 0x7f8c8c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467386027 0 0) 0x7f8c8c0cfeb0 con 0x7f8c8c15fe00 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c70003530 con 0x7f8c8c15fe00 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657003996 0 0) 0x7f8c8c1646c0 con 0x7f8c8c0b8f90 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c8c0cfeb0 con 0x7f8c8c0b8f90 2024-05-14T11:25:38.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2799921205 0 0) 0x7f8c70003530 con 0x7f8c8c15fe00 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 >> v2:172.21.15.149:3300/0 conn(0x7f8c8c0b70a0 msgr2=0x7f8c8c0b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 --2- 172.21.15.116:0/2833719763 >> v2:172.21.15.149:3300/0 conn(0x7f8c8c0b70a0 0x7f8c8c0b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 >> v1:172.21.15.149:6789/0 conn(0x7f8c8c0b8f90 legacy=0x7f8c8c163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c0b6d80 con 0x7f8c8c15fe00 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f8c7c001700 con 0x7f8c8c15fe00 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/2833719763 >> v2:172.21.15.149:3300/0 conn(0x7f8c8c0b70a0 0x7f8c8c0b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 >> v1:172.21.15.116:6789/0 conn(0x7f8c8c15fe00 legacy=0x7f8c8c1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.962+0000 7f8c8a000640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c70003c40 0x7f8c70004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c70004810 0x7f8c70004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c0cf110 con 0x7f8c70005290 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7000c240 con 0x7f8c70004810 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7000c440 con 0x7f8c70003c40 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c70003c40 0x7f8c70004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c82000640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c70004810 0x7f8c70004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 -- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c70003c40 msgr2=0x7f8c70004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c70003c40 0x7f8c70004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 -- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c70004810 msgr2=0x7f8c70004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c70004810 0x7f8c70004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 -- 172.21.15.116:0/2833719763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70003530 con 0x7f8c70005290 2024-05-14T11:25:38.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 -- 172.21.15.116:0/2833719763 reap_dead start 2024-05-14T11:25:38.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8b400640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c7000b9d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8c84002080 tx=0x7f8c8401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254b13073270be93 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84001c50 con 0x7f8c70005290 2024-05-14T11:25:38.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c84003b80 con 0x7f8c70005290 2024-05-14T11:25:38.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.966+0000 7f8c8a000640 1 -- 172.21.15.116:0/2833719763 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840050d0 con 0x7f8c70005290 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 msgr2=0x7f8c7000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c7000b9d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8c84002080 tx=0x7f8c8401c900 comp rx=0 tx=0).stop 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- 172.21.15.116:0/2833719763 shutdown_connections 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 --2- 172.21.15.116:0/2833719763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c7000b9d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- 172.21.15.116:0/2833719763 >> 172.21.15.116:0/2833719763 conn(0x7f8c8c0d3740 msgr2=0x7f8c8c0d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- 172.21.15.116:0/2833719763 shutdown_connections 2024-05-14T11:25:38.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- 172.21.15.116:0/2833719763 wait complete. 2024-05-14T11:25:38.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 Processor -- start 2024-05-14T11:25:38.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.970+0000 7f8c91e00640 1 -- start start 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c8c160c20 0x7f8c8c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c8c1eb8b0 0x7f8c8c1ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c0cf990 con 0x7f8c70005290 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c0cf6b0 con 0x7f8c8c1eb8b0 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c0b93a0 con 0x7f8c8c160c20 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43250/0 (socket says 172.21.15.116:43250) 2024-05-14T11:25:38.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 -- 172.21.15.116:0/425133836 learned_addr learned my addr 172.21.15.116:0/425133836 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8b400640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c8c1eb8b0 0x7f8c8c1ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c82000640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c8c160c20 0x7f8c8c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 -- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c8c160c20 msgr2=0x7f8c8c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c8c160c20 0x7f8c8c15ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 -- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c8c1eb8b0 msgr2=0x7f8c8c1ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c8c1eb8b0 0x7f8c8c1ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 -- 172.21.15.116:0/425133836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c1ec200 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c8aa00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f8c74011a30 tx=0x7f8c74011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400fb50 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7400fcf0 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c0aead0 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400fea0 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c162020 con 0x7f8c70005290 2024-05-14T11:25:38.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c06a9c0 con 0x7f8c70005290 2024-05-14T11:25:38.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.974+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c74012040 con 0x7f8c70005290 2024-05-14T11:25:38.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.978+0000 7f8c83e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 0x7f8c68071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:38.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.978+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f8c7408f370 con 0x7f8c70005290 2024-05-14T11:25:38.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.978+0000 7f8c82000640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 0x7f8c68071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:38.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.978+0000 7f8c82000640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 0x7f8c68071940 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c001060 tx=0x7f8c7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:38.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:38.982+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8c8c1ec200 con 0x7f8c70005290 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8c8c06f200 con 0x7f8c70005290 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c83e00640 1 -- 172.21.15.116:0/425133836 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) ==== 84+0+65 (secure 0 0 0) 0x7f8c74019030 con 0x7f8c70005290 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-05-14T11:25:39.140 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-05-14T11:25:39.141 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-05-14T11:25:39.141 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:39.141 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-05-14T11:25:39.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 msgr2=0x7f8c68071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 0x7f8c68071940 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c001060 tx=0x7f8c7c009040 comp rx=0 tx=0).stop 2024-05-14T11:25:39.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 msgr2=0x7f8c8c0b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f8c74011a30 tx=0x7f8c74011ef0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 shutdown_connections 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8c6806f430 0x7f8c68071940 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c8c1eb8b0 0x7f8c8c1ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8c8c160c20 0x7f8c8c15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 --2- 172.21.15.116:0/425133836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c70005290 0x7f8c8c0b4a90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 >> 172.21.15.116:0/425133836 conn(0x7f8c8c0d3740 msgr2=0x7f8c8c06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.142+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 shutdown_connections 2024-05-14T11:25:39.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.146+0000 7f8c91e00640 1 -- 172.21.15.116:0/425133836 wait complete. 2024-05-14T11:25:39.162 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T11:25:39.163 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T11:25:39.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 Processor -- start 2024-05-14T11:25:39.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 -- start start 2024-05-14T11:25:39.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f61c80cd620 0x7f61c806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 0x7f61c806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c80d04c0 con 0x7f61c8161710 2024-05-14T11:25:39.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80d0270 con 0x7f61c80cd620 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f61c80b0c30 con 0x7f61c80cda30 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cd000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f61c8161710 0x7f61c8161b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37298/0 (socket says 172.21.15.116:37298) 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61cd000640 1 -- 172.21.15.116:0/271771472 learned_addr learned my addr 172.21.15.116:0/271771472 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61bf400640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 0x7f61c806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7e00640 1 --2- 172.21.15.116:0/271771472 >> v2:172.21.15.149:3300/0 conn(0x7f61c80cd620 0x7f61c806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7400640 1 -- 172.21.15.116:0/271771472 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771331087 0 0) 0x7f61c80d04c0 con 0x7f61c8161710 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7400640 1 -- 172.21.15.116:0/271771472 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61ac003530 con 0x7f61c8161710 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7e00640 1 -- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 msgr2=0x7f61c806fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7e00640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 0x7f61c806fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7e00640 1 -- 172.21.15.116:0/271771472 >> v1:172.21.15.116:6789/0 conn(0x7f61c8161710 legacy=0x7f61c8161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61c7e00640 1 -- 172.21.15.116:0/271771472 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80d07a0 con 0x7f61c80cd620 2024-05-14T11:25:39.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.314+0000 7f61bf400640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 0x7f61c806fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:39.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61c7e00640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c806f6b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009d90 tx=0x7f61b801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa606669b44f0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:39.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61c7400640 1 -- 172.21.15.116:0/271771472 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80b2070 con 0x7f61c80cd620 2024-05-14T11:25:39.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61c7400640 1 -- 172.21.15.116:0/271771472 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b80a57d0 con 0x7f61c80cd620 2024-05-14T11:25:39.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61c7400640 1 -- 172.21.15.116:0/271771472 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a5b80 con 0x7f61c80cd620 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 -- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 msgr2=0x7f61c806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c806f6b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009d90 tx=0x7f61b801c910 comp rx=0 tx=0).stop 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 -- 172.21.15.116:0/271771472 shutdown_connections 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c80cda30 0x7f61c806fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 --2- 172.21.15.116:0/271771472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c806f6b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 -- 172.21.15.116:0/271771472 >> 172.21.15.116:0/271771472 conn(0x7f61c80cde80 msgr2=0x7f61c80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:39.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 -- 172.21.15.116:0/271771472 shutdown_connections 2024-05-14T11:25:39.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.318+0000 7f61cf000640 1 -- 172.21.15.116:0/271771472 wait complete. 2024-05-14T11:25:39.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 Processor -- start 2024-05-14T11:25:39.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 -- start start 2024-05-14T11:25:39.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c8148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c8161710 0x7f61c80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61c816b650 con 0x7f61c80b5f30 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61c816b4d0 con 0x7f61c80cd620 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.322+0000 7f61cf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f61c816b7d0 con 0x7f61c8161710 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cd000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61bf400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c8161710 0x7f61c80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cd000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43256/0 (socket says 172.21.15.116:43256) 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c8148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cd000640 1 -- 172.21.15.116:0/328066376 learned_addr learned my addr 172.21.15.116:0/328066376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 -- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c8161710 msgr2=0x7f61c80b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c8161710 0x7f61c80b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 -- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 msgr2=0x7f61c815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 -- 172.21.15.116:0/328066376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c80b4a70 con 0x7f61c80cd620 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cd000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c7e00640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c8148cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61c80b6b20 tx=0x7f61b8019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:39.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80b2070 con 0x7f61c80cd620 2024-05-14T11:25:39.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b80b3040 con 0x7f61c80cd620 2024-05-14T11:25:39.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a5d90 con 0x7f61c80cd620 2024-05-14T11:25:39.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c8160770 con 0x7f61c80cd620 2024-05-14T11:25:39.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c8160d30 con 0x7f61c80cd620 2024-05-14T11:25:39.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c806aa30 con 0x7f61c80cd620 2024-05-14T11:25:39.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.326+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61b8004040 con 0x7f61c80cd620 2024-05-14T11:25:39.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.330+0000 7f61c4c00640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 0x7f61a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:39.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.330+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f61b8128cd0 con 0x7f61c80cd620 2024-05-14T11:25:39.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.330+0000 7f61bf400640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 0x7f61a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:39.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.330+0000 7f61bf400640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 0x7f61a4071910 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f61c81fce40 tx=0x7f61b0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:39.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.334+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f61b8016560 con 0x7f61c80cd620 2024-05-14T11:25:39.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:39.494+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f61c80cfb40 con 0x7f61c80cd620 2024-05-14T11:25:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.426+0000 7f61c4c00640 1 -- 172.21.15.116:0/328066376 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) ==== 138+0+0 (secure 0 0 0) 0x7f61b80f57a0 con 0x7f61c80cd620 2024-05-14T11:25:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.426+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 msgr2=0x7f61a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.426+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 0x7f61a4071910 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f61c81fce40 tx=0x7f61b0009190 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.426+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 msgr2=0x7f61c8148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.426+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c8148cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61c80b6b20 tx=0x7f61b8019710 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 shutdown_connections 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61a406f400 0x7f61a4071910 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80b5f30 0x7f61c815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f61c8161710 0x7f61c80b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 --2- 172.21.15.116:0/328066376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61c80cd620 0x7f61c8148cd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 >> 172.21.15.116:0/328066376 conn(0x7f61c80cde80 msgr2=0x7f61c8072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:40.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 shutdown_connections 2024-05-14T11:25:40.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.430+0000 7f61cf000640 1 -- 172.21.15.116:0/328066376 wait complete. 2024-05-14T11:25:40.450 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T11:25:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.610+0000 7f374d000640 1 Processor -- start 2024-05-14T11:25:40.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f374d000640 1 -- start start 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f374d000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37480ce450 0x7f3748167ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f374d000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374806cc10 con 0x7f37480d0c50 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f374d000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374806cef0 con 0x7f374815c1d0 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f374d000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f37480d1e40 con 0x7f37480ce450 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f3746800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37480ce450 0x7f3748167ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f373d400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f37480d0c50 0x7f37480cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37310/0 (socket says 172.21.15.116:37310) 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f373d400640 1 -- 172.21.15.116:0/1909670528 learned_addr learned my addr 172.21.15.116:0/1909670528 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f3746800640 1 -- 172.21.15.116:0/1909670528 >> v1:172.21.15.149:6789/0 conn(0x7f374815c1d0 legacy=0x7f37480d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f3746800640 1 -- 172.21.15.116:0/1909670528 >> v1:172.21.15.116:6789/0 conn(0x7f37480d0c50 legacy=0x7f37480cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f3746800640 1 -- 172.21.15.116:0/1909670528 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374806d1d0 con 0x7f37480ce450 2024-05-14T11:25:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.614+0000 7f3745400640 1 -- 172.21.15.116:0/1909670528 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820634868 0 0) 0x7f374806cc10 con 0x7f37480d0c50 2024-05-14T11:25:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f3746800640 1 --2- 172.21.15.116:0/1909670528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480ce450 0x7f3748167ff0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d90 tx=0x7f374001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61de9c7b346dd930 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f3745400640 1 -- 172.21.15.116:0/1909670528 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37400b2070 con 0x7f37480ce450 2024-05-14T11:25:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f3745400640 1 -- 172.21.15.116:0/1909670528 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37400a5710 con 0x7f37480ce450 2024-05-14T11:25:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f3745400640 1 -- 172.21.15.116:0/1909670528 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37400a5ae0 con 0x7f37480ce450 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 -- 172.21.15.116:0/1909670528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480ce450 msgr2=0x7f3748167ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 --2- 172.21.15.116:0/1909670528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480ce450 0x7f3748167ff0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d90 tx=0x7f374001c850 comp rx=0 tx=0).stop 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 -- 172.21.15.116:0/1909670528 shutdown_connections 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 --2- 172.21.15.116:0/1909670528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480ce450 0x7f3748167ff0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 -- 172.21.15.116:0/1909670528 >> 172.21.15.116:0/1909670528 conn(0x7f37480b4720 msgr2=0x7f37480b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 -- 172.21.15.116:0/1909670528 shutdown_connections 2024-05-14T11:25:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.618+0000 7f374d000640 1 -- 172.21.15.116:0/1909670528 wait complete. 2024-05-14T11:25:40.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 Processor -- start 2024-05-14T11:25:40.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- start start 2024-05-14T11:25:40.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:40.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374815c1d0 0x7f3748156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:40.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37480c78a0 0x7f37481574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:40.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37480ccff0 con 0x7f37480d0c50 2024-05-14T11:25:40.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37480cce70 con 0x7f374815c1d0 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f374816ad50 con 0x7f37480c78a0 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43260/0 (socket says 172.21.15.116:43260) 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 -- 172.21.15.116:0/133503241 learned_addr learned my addr 172.21.15.116:0/133503241 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f373d400640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374815c1d0 0x7f3748156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3746800640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37480c78a0 0x7f37481574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 -- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37480c78a0 msgr2=0x7f37481574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37480c78a0 0x7f37481574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 -- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374815c1d0 msgr2=0x7f3748156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374815c1d0 0x7f3748156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 -- 172.21.15.116:0/133503241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374815d440 con 0x7f37480d0c50 2024-05-14T11:25:40.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f3745e00640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f373800c450 tx=0x7f373800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380052d0 con 0x7f37480d0c50 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3738005470 con 0x7f37480d0c50 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374815d720 con 0x7f37480d0c50 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738012850 con 0x7f37480d0c50 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.622+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374815dce0 con 0x7f37480d0c50 2024-05-14T11:25:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374806a9c0 con 0x7f37480d0c50 2024-05-14T11:25:40.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f37380129f0 con 0x7f37480d0c50 2024-05-14T11:25:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f373ea00640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 0x7f3724071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5619+0+0 (secure 0 0 0) 0x7f373808f920 con 0x7f37480d0c50 2024-05-14T11:25:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f373d400640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 0x7f3724071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.626+0000 7f373d400640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 0x7f3724071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f37480c4790 tx=0x7f3730007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.630+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f373805c8d0 con 0x7f37480d0c50 2024-05-14T11:25:40.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:40.790+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3748069de0 con 0x7f37480d0c50 2024-05-14T11:25:41.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f373ea00640 1 -- 172.21.15.116:0/133503241 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) ==== 170+0+0 (secure 0 0 0) 0x7f373805c1d0 con 0x7f37480d0c50 2024-05-14T11:25:41.419 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 msgr2=0x7f3724071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 0x7f3724071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f37480c4790 tx=0x7f3730007d30 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 msgr2=0x7f37480c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f373800c450 tx=0x7f373800c910 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 shutdown_connections 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f372406f400 0x7f3724071910 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374815c1d0 0x7f3748156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f37480c78a0 0x7f37481574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 --2- 172.21.15.116:0/133503241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37480d0c50 0x7f37480c7360 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 >> 172.21.15.116:0/133503241 conn(0x7f37480b4720 msgr2=0x7f3748156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 shutdown_connections 2024-05-14T11:25:41.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.422+0000 7f374d000640 1 -- 172.21.15.116:0/133503241 wait complete. 2024-05-14T11:25:41.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T11:25:41.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 Processor -- start 2024-05-14T11:25:41.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 -- start start 2024-05-14T11:25:41.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcfa40d1730 0x7fcfa40d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcfa4160670 0x7fcfa4160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa406f0a0 con 0x7fcfa40d1730 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa40cbb60 con 0x7fcfa4160670 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfaa200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa40d2080 con 0x7fcfa40cd1c0 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcfa40d1730 0x7fcfa40d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcfa40d1730 0x7fcfa40d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43272/0 (socket says 172.21.15.116:43272) 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 -- 172.21.15.116:0/176772118 learned_addr learned my addr 172.21.15.116:0/176772118 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa3400640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa3e00640 1 --2- 172.21.15.116:0/176772118 >> v2:172.21.15.149:3300/0 conn(0x7fcfa4160670 0x7fcfa4160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 -- 172.21.15.116:0/176772118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 msgr2=0x7fcfa40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 -- 172.21.15.116:0/176772118 >> v2:172.21.15.149:3300/0 conn(0x7fcfa4160670 msgr2=0x7fcfa4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:41.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/176772118 >> v2:172.21.15.149:3300/0 conn(0x7fcfa4160670 0x7fcfa4160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.618+0000 7fcfa2a00640 1 -- 172.21.15.116:0/176772118 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa406f420 con 0x7fcfa40d1730 2024-05-14T11:25:41.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa40d1b40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009870 tx=0x7fcf9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4c2d2a9d2324b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:41.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfa2000640 1 -- 172.21.15.116:0/176772118 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900a6040 con 0x7fcfa40d1730 2024-05-14T11:25:41.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfa2000640 1 -- 172.21.15.116:0/176772118 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf900af070 con 0x7fcfa40d1730 2024-05-14T11:25:41.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfa2000640 1 -- 172.21.15.116:0/176772118 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900aa860 con 0x7fcfa40d1730 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 -- 172.21.15.116:0/176772118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 msgr2=0x7fcfa40d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa40d1b40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009870 tx=0x7fcf9001c520 comp rx=0 tx=0).stop 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 -- 172.21.15.116:0/176772118 shutdown_connections 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa40d1b40 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 --2- 172.21.15.116:0/176772118 >> v2:172.21.15.149:3300/0 conn(0x7fcfa4160670 0x7fcfa4160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 --2- 172.21.15.116:0/176772118 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 -- 172.21.15.116:0/176772118 >> 172.21.15.116:0/176772118 conn(0x7fcfa40b4720 msgr2=0x7fcfa40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 -- 172.21.15.116:0/176772118 shutdown_connections 2024-05-14T11:25:41.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.622+0000 7fcfaa200640 1 -- 172.21.15.116:0/176772118 wait complete. 2024-05-14T11:25:41.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 Processor -- start 2024-05-14T11:25:41.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 -- start start 2024-05-14T11:25:41.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfa4160670 0x7fcfa4157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa415cf10 con 0x7fcfa40d1730 2024-05-14T11:25:41.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa406ffa0 con 0x7fcfa4160670 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfaa200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa41613d0 con 0x7fcfa40cd1c0 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43282/0 (socket says 172.21.15.116:43282) 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 -- 172.21.15.116:0/3501870712 learned_addr learned my addr 172.21.15.116:0/3501870712 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa3400640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa3e00640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfa4160670 0x7fcfa4157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 -- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 msgr2=0x7fcfa40c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 -- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfa4160670 msgr2=0x7fcfa4157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfa4160670 0x7fcfa4157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.626+0000 7fcfa2a00640 1 -- 172.21.15.116:0/3501870712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa4159ee0 con 0x7fcfa40d1730 2024-05-14T11:25:41.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcfa2a00640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fcf9001abd0 tx=0x7fcf9001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900bd3f0 con 0x7fcfa40d1730 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf900bd5b0 con 0x7fcfa40d1730 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900bd9a0 con 0x7fcfa40d1730 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa415a1c0 con 0x7fcfa40d1730 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41659f0 con 0x7fcfa40d1730 2024-05-14T11:25:41.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa406a9c0 con 0x7fcfa40d1730 2024-05-14T11:25:41.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf900b33f0 con 0x7fcfa40d1730 2024-05-14T11:25:41.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 0x7fcf84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:41.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6001+0+0 (secure 0 0 0) 0x7fcf90129270 con 0x7fcfa40d1730 2024-05-14T11:25:41.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.630+0000 7fcfa3400640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 0x7fcf84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:41.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.634+0000 7fcfa3400640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 0x7fcf84071940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf940045f0 tx=0x7fcf94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:41.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.638+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcf900c3060 con 0x7fcfa40d1730 2024-05-14T11:25:41.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:41.826+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcfa4069de0 con 0x7fcfa40d1730 2024-05-14T11:25:42.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.434+0000 7fcf9b400640 1 -- 172.21.15.116:0/3501870712 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v96) ==== 174+0+0 (secure 0 0 0) 0x7fcf90016110 con 0x7fcfa40d1730 2024-05-14T11:25:42.438 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 17 and data pool 18 2024-05-14T11:25:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 msgr2=0x7fcf84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 0x7fcf84071940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf940045f0 tx=0x7fcf94008040 comp rx=0 tx=0).stop 2024-05-14T11:25:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 msgr2=0x7fcfa41577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fcf9001abd0 tx=0x7fcf9001ad60 comp rx=0 tx=0).stop 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 shutdown_connections 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf8406f430 0x7fcf84071940 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfa4160670 0x7fcfa4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa40d1730 0x7fcfa41577e0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 --2- 172.21.15.116:0/3501870712 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcfa40cd1c0 0x7fcfa40c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 >> 172.21.15.116:0/3501870712 conn(0x7fcfa40b4720 msgr2=0x7fcfa40ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:42.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 shutdown_connections 2024-05-14T11:25:42.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.446+0000 7fcfaa200640 1 -- 172.21.15.116:0/3501870712 wait complete. 2024-05-14T11:25:42.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.630+0000 7f666f800640 1 Processor -- start 2024-05-14T11:25:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.630+0000 7f666f800640 1 -- start start 2024-05-14T11:25:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666f800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6668164f80 0x7f66681673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666f800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666806cf10 con 0x7f66680b0770 2024-05-14T11:25:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666f800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666806d1f0 con 0x7f666806f730 2024-05-14T11:25:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666f800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f666806db20 con 0x7f6668164f80 2024-05-14T11:25:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666ce00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f66680b0770 0x7f666806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37326/0 (socket says 172.21.15.116:37326) 2024-05-14T11:25:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666ce00640 1 -- 172.21.15.116:0/1889411980 learned_addr learned my addr 172.21.15.116:0/1889411980 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666d800640 1 --2- 172.21.15.116:0/1889411980 >> v2:172.21.15.149:3300/0 conn(0x7f6668164f80 0x7f66681673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222030756 0 0) 0x7f666806cf10 con 0x7f66680b0770 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f664c003530 con 0x7f66680b0770 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898786260 0 0) 0x7f666806d1f0 con 0x7f666806f730 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f666806cf10 con 0x7f666806f730 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666d800640 1 -- 172.21.15.116:0/1889411980 >> v1:172.21.15.149:6789/0 conn(0x7f666806f730 legacy=0x7f666806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666d800640 1 -- 172.21.15.116:0/1889411980 >> v1:172.21.15.116:6789/0 conn(0x7f66680b0770 legacy=0x7f666806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4232188805 0 0) 0x7f664c003530 con 0x7f66680b0770 2024-05-14T11:25:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.634+0000 7f666d800640 1 -- 172.21.15.116:0/1889411980 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666806d4d0 con 0x7f6668164f80 2024-05-14T11:25:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666d800640 1 --2- 172.21.15.116:0/1889411980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6668164f80 0x7f66681673a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45dda48a9d7ff40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600b0070 con 0x7f6668164f80 2024-05-14T11:25:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600ab400 con 0x7f6668164f80 2024-05-14T11:25:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f6667400640 1 -- 172.21.15.116:0/1889411980 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658001aa0 con 0x7f6668164f80 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 -- 172.21.15.116:0/1889411980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6668164f80 msgr2=0x7f66681673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 --2- 172.21.15.116:0/1889411980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6668164f80 0x7f66681673a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001c300 comp rx=0 tx=0).stop 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 -- 172.21.15.116:0/1889411980 shutdown_connections 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 --2- 172.21.15.116:0/1889411980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6668164f80 0x7f66681673a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 -- 172.21.15.116:0/1889411980 >> 172.21.15.116:0/1889411980 conn(0x7f66680b4620 msgr2=0x7f66680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 -- 172.21.15.116:0/1889411980 shutdown_connections 2024-05-14T11:25:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.638+0000 7f666f800640 1 -- 172.21.15.116:0/1889411980 wait complete. 2024-05-14T11:25:42.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 Processor -- start 2024-05-14T11:25:42.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 -- start start 2024-05-14T11:25:42.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f666806f730 0x7f666815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66680b0770 0x7f666814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f666816ab90 con 0x7f6668164f80 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f666816aa10 con 0x7f66680b0770 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f666816ad10 con 0x7f666806f730 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43284/0 (socket says 172.21.15.116:43284) 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 -- 172.21.15.116:0/3474238606 learned_addr learned my addr 172.21.15.116:0/3474238606 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666ce00640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f666806f730 0x7f666815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f6667e00640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66680b0770 0x7f666814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 -- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f666806f730 msgr2=0x7f666815f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f666806f730 0x7f666815f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 -- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66680b0770 msgr2=0x7f666814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66680b0770 0x7f666814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 -- 172.21.15.116:0/3474238606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666814d9b0 con 0x7f6668164f80 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.642+0000 7f666d800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600b0050 con 0x7f6668164f80 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600bc070 con 0x7f6668164f80 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66680ca4a0 con 0x7f6668164f80 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660006e20 con 0x7f6668164f80 2024-05-14T11:25:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6668156850 con 0x7f6668164f80 2024-05-14T11:25:42.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66681562e0 con 0x7f6668164f80 2024-05-14T11:25:42.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66600b7850 con 0x7f6668164f80 2024-05-14T11:25:42.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 0x7f6648071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f6660129250 con 0x7f6668164f80 2024-05-14T11:25:42.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.646+0000 7f666ce00640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 0x7f6648071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.650+0000 7f666ce00640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 0x7f6648071940 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6658001730 tx=0x7f6658009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.650+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f66600c5060 con 0x7f6668164f80 2024-05-14T11:25:42.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.810+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f666815f7f0 con 0x7f6668164f80 2024-05-14T11:25:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f6664c00640 1 -- 172.21.15.116:0/3474238606 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17013 (secure 0 0 0) 0x7f6660015fb0 con 0x7f6668164f80 2024-05-14T11:25:42.812 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:42.812 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":112,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:41.831237+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:25:39.509510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-14T11:25:40.801825+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 msgr2=0x7f6648071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 0x7f6648071940 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6658001730 tx=0x7f6658009040 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 msgr2=0x7f6668157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001ab40 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 shutdown_connections 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f664806f430 0x7f6648071940 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66680b0770 0x7f666814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6668164f80 0x7f6668157730 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 --2- 172.21.15.116:0/3474238606 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f666806f730 0x7f666815f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 >> 172.21.15.116:0/3474238606 conn(0x7f66680b4620 msgr2=0x7f666806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 shutdown_connections 2024-05-14T11:25:42.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.814+0000 7f666f800640 1 -- 172.21.15.116:0/3474238606 wait complete. 2024-05-14T11:25:42.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:42.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 Processor -- start 2024-05-14T11:25:42.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 -- start start 2024-05-14T11:25:42.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6de40cdde0 0x7f6de415de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de415e370 con 0x7f6de40ce330 2024-05-14T11:25:42.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6de4162630 con 0x7f6de40cdde0 2024-05-14T11:25:42.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6debc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6de4162910 con 0x7f6de40b3950 2024-05-14T11:25:42.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de3e00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f6de40ce330 0x7f6de40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:55256/0 (socket says 172.21.15.116:55256) 2024-05-14T11:25:42.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:42968/0 (socket says 172.21.15.116:42968) 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de3e00640 1 -- 172.21.15.116:0/1712459576 learned_addr learned my addr 172.21.15.116:0/1712459576 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9200640 1 --2- 172.21.15.116:0/1712459576 >> v2:172.21.15.149:3300/0 conn(0x7f6de40cdde0 0x7f6de415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 -- 172.21.15.116:0/1712459576 >> v2:172.21.15.149:3300/0 conn(0x7f6de40cdde0 msgr2=0x7f6de415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 --2- 172.21.15.116:0/1712459576 >> v2:172.21.15.149:3300/0 conn(0x7f6de40cdde0 0x7f6de415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de3400640 1 -- 172.21.15.116:0/1712459576 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1580744006 0 0) 0x7f6de415e370 con 0x7f6de40ce330 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 -- 172.21.15.116:0/1712459576 >> v1:172.21.15.149:6789/0 conn(0x7f6de40ce330 legacy=0x7f6de40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.982+0000 7f6de9c00640 1 -- 172.21.15.116:0/1712459576 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40b2e70 con 0x7f6de40b3950 2024-05-14T11:25:42.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6de9c00640 1 --2- 172.21.15.116:0/1712459576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009d90 tx=0x7f6ddc01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1722e33d8a5e81a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6de3400640 1 -- 172.21.15.116:0/1712459576 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0b2070 con 0x7f6de40b3950 2024-05-14T11:25:42.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6de3400640 1 -- 172.21.15.116:0/1712459576 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0a5970 con 0x7f6de40b3950 2024-05-14T11:25:42.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6de3400640 1 -- 172.21.15.116:0/1712459576 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0a5d40 con 0x7f6de40b3950 2024-05-14T11:25:42.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 -- 172.21.15.116:0/1712459576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 msgr2=0x7f6de40b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 --2- 172.21.15.116:0/1712459576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009d90 tx=0x7f6ddc01c8b0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 -- 172.21.15.116:0/1712459576 shutdown_connections 2024-05-14T11:25:42.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 --2- 172.21.15.116:0/1712459576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40b3950 0x7f6de40b3d60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 --2- 172.21.15.116:0/1712459576 >> v2:172.21.15.149:3300/0 conn(0x7f6de40cdde0 0x7f6de415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 -- 172.21.15.116:0/1712459576 >> 172.21.15.116:0/1712459576 conn(0x7f6de40d3670 msgr2=0x7f6de40d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:42.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 -- 172.21.15.116:0/1712459576 shutdown_connections 2024-05-14T11:25:42.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.986+0000 7f6debc00640 1 -- 172.21.15.116:0/1712459576 wait complete. 2024-05-14T11:25:42.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 Processor -- start 2024-05-14T11:25:42.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 -- start start 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 0x7f6de4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 0x7f6de415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40cf070 con 0x7f6de40b3950 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6de40ceef0 con 0x7f6de40cdde0 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6debc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6de416ae90 con 0x7f6de40ce330 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.990+0000 7f6de9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43286/0 (socket says 172.21.15.116:43286) 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 -- 172.21.15.116:0/864649990 learned_addr learned my addr 172.21.15.116:0/864649990 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de3e00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 0x7f6de4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9c00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 0x7f6de415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 -- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 msgr2=0x7f6de415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 0x7f6de415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 -- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 msgr2=0x7f6de4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 0x7f6de4154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 -- 172.21.15.116:0/864649990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de4151dd0 con 0x7f6de40b3950 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9c00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 0x7f6de415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de3e00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 0x7f6de4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:42.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de9200640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400e870 tx=0x7f6dd400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4017070 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd400c940 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de41520b0 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400cd10 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de4152670 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de406a9c0 con 0x7f6de40b3950 2024-05-14T11:25:42.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.994+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6dd401a070 con 0x7f6de40b3950 2024-05-14T11:25:42.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.998+0000 7f6de0c00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 0x7f6dc0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:42.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.998+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f6dd408f770 con 0x7f6de40b3950 2024-05-14T11:25:42.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.998+0000 7f6de3e00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 0x7f6dc0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:42.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:42.998+0000 7f6de3e00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 0x7f6dc0071910 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:42.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.002+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6dd405c430 con 0x7f6de40b3950 2024-05-14T11:25:43.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.158+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6de4151890 con 0x7f6de40b3950 2024-05-14T11:25:43.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6de0c00640 1 -- 172.21.15.116:0/864649990 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17013 (secure 0 0 0) 0x7f6dd405bd30 con 0x7f6de40b3950 2024-05-14T11:25:43.159 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:43.159 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":112,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:41.831237+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:25:39.509510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-14T11:25:40.801825+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 msgr2=0x7f6dc0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 0x7f6dc0071910 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 msgr2=0x7f6de4151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400e870 tx=0x7f6dd400ec70 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 shutdown_connections 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6dc006f400 0x7f6dc0071910 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de40cdde0 0x7f6de4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de40ce330 0x7f6de415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 --2- 172.21.15.116:0/864649990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6de40b3950 0x7f6de4151350 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 >> 172.21.15.116:0/864649990 conn(0x7f6de40d3670 msgr2=0x7f6de40b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 shutdown_connections 2024-05-14T11:25:43.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.162+0000 7f6debc00640 1 -- 172.21.15.116:0/864649990 wait complete. 2024-05-14T11:25:43.181 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-14T11:25:43.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.330+0000 7f3a13e00640 1 Processor -- start 2024-05-14T11:25:43.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.330+0000 7f3a13e00640 1 -- start start 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f3a14160960 0x7f3a140718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a140b5b50 0x7f3a140d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a14072210 con 0x7f3a14071e00 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3a140d0140 con 0x7f3a140b5b50 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f3a140b6170 con 0x7f3a14160960 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a140b5b50 0x7f3a140d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a0a000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3a14071e00 0x7f3a140b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37328/0 (socket says 172.21.15.116:37328) 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a0a000640 1 -- 172.21.15.116:0/2566803301 learned_addr learned my addr 172.21.15.116:0/2566803301 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a12a00640 1 --2- 172.21.15.116:0/2566803301 >> v2:172.21.15.149:3300/0 conn(0x7f3a14160960 0x7f3a140718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13400640 1 -- 172.21.15.116:0/2566803301 >> v2:172.21.15.149:3300/0 conn(0x7f3a14160960 msgr2=0x7f3a140718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13400640 1 --2- 172.21.15.116:0/2566803301 >> v2:172.21.15.149:3300/0 conn(0x7f3a14160960 0x7f3a140718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13400640 1 -- 172.21.15.116:0/2566803301 >> v1:172.21.15.116:6789/0 conn(0x7f3a14071e00 legacy=0x7f3a140b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a13400640 1 -- 172.21.15.116:0/2566803301 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a140cf240 con 0x7f3a140b5b50 2024-05-14T11:25:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.334+0000 7f3a12000640 1 -- 172.21.15.116:0/2566803301 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920996457 0 0) 0x7f39fc001120 con 0x7f3a14071e00 2024-05-14T11:25:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13400640 1 --2- 172.21.15.116:0/2566803301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a140b5b50 0x7f3a140d0870 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3a140b73a0 tx=0x7f3a0c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f6b31b98f823bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a12000640 1 -- 172.21.15.116:0/2566803301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0b2070 con 0x7f3a140b5b50 2024-05-14T11:25:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a12000640 1 -- 172.21.15.116:0/2566803301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0a5760 con 0x7f3a140b5b50 2024-05-14T11:25:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a12000640 1 -- 172.21.15.116:0/2566803301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0a5b30 con 0x7f3a140b5b50 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- 172.21.15.116:0/2566803301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a140b5b50 msgr2=0x7f3a140d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 --2- 172.21.15.116:0/2566803301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a140b5b50 0x7f3a140d0870 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3a140b73a0 tx=0x7f3a0c01c8a0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- 172.21.15.116:0/2566803301 shutdown_connections 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 --2- 172.21.15.116:0/2566803301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a140b5b50 0x7f3a140d0870 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 --2- 172.21.15.116:0/2566803301 >> v2:172.21.15.149:3300/0 conn(0x7f3a14160960 0x7f3a140718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- 172.21.15.116:0/2566803301 >> 172.21.15.116:0/2566803301 conn(0x7f3a140d2d70 msgr2=0x7f3a140711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- 172.21.15.116:0/2566803301 shutdown_connections 2024-05-14T11:25:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- 172.21.15.116:0/2566803301 wait complete. 2024-05-14T11:25:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 Processor -- start 2024-05-14T11:25:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.338+0000 7f3a13e00640 1 -- start start 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a14071e00 0x7f3a140ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 0x7f3a1415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1416c7a0 con 0x7f3a14160960 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14160f80 con 0x7f3a140b5b50 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1416c920 con 0x7f3a14071e00 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43302/0 (socket says 172.21.15.116:43302) 2024-05-14T11:25:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 -- 172.21.15.116:0/4191289598 learned_addr learned my addr 172.21.15.116:0/4191289598 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a12a00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a14071e00 0x7f3a140ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a0a000640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 0x7f3a1415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 -- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a14071e00 msgr2=0x7f3a140ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a14071e00 0x7f3a140ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 -- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 msgr2=0x7f3a1415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 0x7f3a1415bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 -- 172.21.15.116:0/4191289598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1406d930 con 0x7f3a14160960 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a0a000640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 0x7f3a1415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13400640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b0040 tx=0x7f3a0c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0b2050 con 0x7f3a14160960 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0b3040 con 0x7f3a14160960 2024-05-14T11:25:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0ad650 con 0x7f3a14160960 2024-05-14T11:25:43.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a14158230 con 0x7f3a14160960 2024-05-14T11:25:43.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141587f0 con 0x7f3a14160960 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.342+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1406a9c0 con 0x7f3a14160960 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.346+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3a0c0c0020 con 0x7f3a14160960 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.346+0000 7f3a0b400640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 0x7f39f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.346+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f3a0c129c60 con 0x7f3a14160960 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.346+0000 7f3a12a00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 0x7f39f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.346+0000 7f3a12a00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 0x7f39f0071940 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3a140706a0 tx=0x7f3a04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.350+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3a0c016520 con 0x7f3a14160960 2024-05-14T11:25:43.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:43.546+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3a14070480 con 0x7f3a14160960 2024-05-14T11:25:43.745 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:25:43.749 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:25:44.399 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:25:44.402+0000 7f4c11200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:25:44.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.450+0000 7f3a0b400640 1 -- 172.21.15.116:0/4191289598 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7f3a0c0f6920 con 0x7f3a14160960 2024-05-14T11:25:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 msgr2=0x7f39f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 0x7f39f0071940 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3a140706a0 tx=0x7f3a04008040 comp rx=0 tx=0).stop 2024-05-14T11:25:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 msgr2=0x7f3a14145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b0040 tx=0x7f3a0c019640 comp rx=0 tx=0).stop 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 shutdown_connections 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f39f006f430 0x7f39f0071940 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a140b5b50 0x7f3a1415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a14160960 0x7f3a14145970 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 --2- 172.21.15.116:0/4191289598 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a14071e00 0x7f3a140ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 >> 172.21.15.116:0/4191289598 conn(0x7f3a140d2d70 msgr2=0x7f3a140711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 shutdown_connections 2024-05-14T11:25:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.454+0000 7f3a13e00640 1 -- 172.21.15.116:0/4191289598 wait complete. 2024-05-14T11:25:44.475 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-14T11:25:44.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-14T11:25:44.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.630+0000 7fa337800640 1 Processor -- start 2024-05-14T11:25:44.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.630+0000 7fa337800640 1 -- start start 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa337800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33015c1d0 0x7fa3300d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa337800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 0x7fa3300ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa337800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3300ce690 con 0x7fa33015fc50 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa337800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa33006d790 con 0x7fa33015c1d0 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa337800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa33006cc10 con 0x7fa3300cdd40 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33015c1d0 0x7fa3300d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33015c1d0 0x7fa3300d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43310/0 (socket says 172.21.15.116:43310) 2024-05-14T11:25:44.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 -- 172.21.15.116:0/956459597 learned_addr learned my addr 172.21.15.116:0/956459597 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa327e00640 1 --2- 172.21.15.116:0/956459597 >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 0x7fa3300ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 -- 172.21.15.116:0/956459597 >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 msgr2=0x7fa3300ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 --2- 172.21.15.116:0/956459597 >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 0x7fa3300ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 -- 172.21.15.116:0/956459597 >> v1:172.21.15.149:6789/0 conn(0x7fa33015fc50 legacy=0x7fa330160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa334e00640 1 -- 172.21.15.116:0/956459597 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33006cef0 con 0x7fa33015c1d0 2024-05-14T11:25:44.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.634+0000 7fa327e00640 1 --2- 172.21.15.116:0/956459597 >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 0x7fa3300ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa334e00640 1 --2- 172.21.15.116:0/956459597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015c1d0 0x7fa3300d0c50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa3300b0e60 tx=0x7fa32001c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d54772fd369bc44 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa32fe00640 1 -- 172.21.15.116:0/956459597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b0070 con 0x7fa33015c1d0 2024-05-14T11:25:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa32fe00640 1 -- 172.21.15.116:0/956459597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3200ab600 con 0x7fa33015c1d0 2024-05-14T11:25:44.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa32fe00640 1 -- 172.21.15.116:0/956459597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200ab9d0 con 0x7fa33015c1d0 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 -- 172.21.15.116:0/956459597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015c1d0 msgr2=0x7fa3300d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 --2- 172.21.15.116:0/956459597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015c1d0 0x7fa3300d0c50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa3300b0e60 tx=0x7fa32001c220 comp rx=0 tx=0).stop 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 -- 172.21.15.116:0/956459597 shutdown_connections 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 --2- 172.21.15.116:0/956459597 >> v2:172.21.15.149:3300/0 conn(0x7fa3300cdd40 0x7fa3300ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 --2- 172.21.15.116:0/956459597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015c1d0 0x7fa3300d0c50 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 -- 172.21.15.116:0/956459597 >> 172.21.15.116:0/956459597 conn(0x7fa3300b4720 msgr2=0x7fa3300b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 -- 172.21.15.116:0/956459597 shutdown_connections 2024-05-14T11:25:44.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 -- 172.21.15.116:0/956459597 wait complete. 2024-05-14T11:25:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.638+0000 7fa337800640 1 Processor -- start 2024-05-14T11:25:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- start start 2024-05-14T11:25:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3300cdd40 0x7fa330158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa33015c1d0 0x7fa330158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa330167160 con 0x7fa33015fc50 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa330166fe0 con 0x7fa3300cdd40 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa3301672e0 con 0x7fa33015c1d0 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43318/0 (socket says 172.21.15.116:43318) 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 -- 172.21.15.116:0/337732044 learned_addr learned my addr 172.21.15.116:0/337732044 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa334e00640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3300cdd40 0x7fa330158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa327e00640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa33015c1d0 0x7fa330158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 -- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa33015c1d0 msgr2=0x7fa330158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa33015c1d0 0x7fa330158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 -- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3300cdd40 msgr2=0x7fa330158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3300cdd40 0x7fa330158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 -- 172.21.15.116:0/337732044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3301524c0 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa335800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b970 tx=0x7fa32800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328019070 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa32800cae0 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3301527a0 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328014440 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa330152d60 con 0x7fa33015fc50 2024-05-14T11:25:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.642+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33006a9c0 con 0x7fa33015fc50 2024-05-14T11:25:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.646+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa32801c070 con 0x7fa33015fc50 2024-05-14T11:25:44.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.646+0000 7fa32d600640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 0x7fa30c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:44.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.646+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fa32808f2b0 con 0x7fa33015fc50 2024-05-14T11:25:44.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.646+0000 7fa334e00640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 0x7fa30c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:44.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.646+0000 7fa334e00640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 0x7fa30c071940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ad000 tx=0x7fa320008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:44.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.650+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa328023080 con 0x7fa33015fc50 2024-05-14T11:25:44.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:44.838+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa330069de0 con 0x7fa33015fc50 2024-05-14T11:25:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.458+0000 7fa32d600640 1 -- 172.21.15.116:0/337732044 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7fa32805b9a0 con 0x7fa33015fc50 2024-05-14T11:25:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 msgr2=0x7fa30c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 0x7fa30c071940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ad000 tx=0x7fa320008040 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 msgr2=0x7fa330153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b970 tx=0x7fa32800be30 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 shutdown_connections 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa30c06f430 0x7fa30c071940 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33015fc50 0x7fa330153890 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa33015c1d0 0x7fa330158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 --2- 172.21.15.116:0/337732044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3300cdd40 0x7fa330158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 >> 172.21.15.116:0/337732044 conn(0x7fa3300b4720 msgr2=0x7fa3301676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 shutdown_connections 2024-05-14T11:25:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.462+0000 7fa337800640 1 -- 172.21.15.116:0/337732044 wait complete. 2024-05-14T11:25:45.484 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-14T11:25:45.484 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:45.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 Processor -- start 2024-05-14T11:25:45.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 -- start start 2024-05-14T11:25:45.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 0x7f7b980cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b980d0570 con 0x7f7b9815e980 2024-05-14T11:25:45.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7b98161e00 con 0x7f7b9806de20 2024-05-14T11:25:45.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7b981614d0 con 0x7f7b9815bb20 2024-05-14T11:25:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b97e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b97e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:42994/0 (socket says 172.21.15.116:42994) 2024-05-14T11:25:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b97e00640 1 -- 172.21.15.116:0/1370853583 learned_addr learned my addr 172.21.15.116:0/1370853583 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b97400640 1 --1- 172.21.15.116:0/1370853583 >> v1:172.21.15.149:6789/0 conn(0x7f7b9815e980 0x7f7b9806d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:55268/0 (socket says 172.21.15.116:55268) 2024-05-14T11:25:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b8ea00640 1 --2- 172.21.15.116:0/1370853583 >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 0x7f7b980cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.634+0000 7f7b96a00640 1 -- 172.21.15.116:0/1370853583 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020763157 0 0) 0x7f7b980d0570 con 0x7f7b9815e980 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b96a00640 1 -- 172.21.15.116:0/1370853583 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b7c003530 con 0x7f7b9815e980 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b97e00640 1 -- 172.21.15.116:0/1370853583 >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 msgr2=0x7f7b980cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b97e00640 1 --2- 172.21.15.116:0/1370853583 >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 0x7f7b980cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b97e00640 1 -- 172.21.15.116:0/1370853583 >> v1:172.21.15.149:6789/0 conn(0x7f7b9815e980 legacy=0x7f7b9806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b97e00640 1 -- 172.21.15.116:0/1370853583 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b981617b0 con 0x7f7b9815bb20 2024-05-14T11:25:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b8ea00640 1 --2- 172.21.15.116:0/1370853583 >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 0x7f7b980cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:45.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b97e00640 1 --2- 172.21.15.116:0/1370853583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009e50 tx=0x7f7b9001d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d6ca61d1b67f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b96a00640 1 -- 172.21.15.116:0/1370853583 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900a5810 con 0x7f7b9815bb20 2024-05-14T11:25:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b96a00640 1 -- 172.21.15.116:0/1370853583 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b900a59b0 con 0x7f7b9815bb20 2024-05-14T11:25:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.638+0000 7f7b96a00640 1 -- 172.21.15.116:0/1370853583 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900a5d60 con 0x7f7b9815bb20 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- 172.21.15.116:0/1370853583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 msgr2=0x7f7b9815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/1370853583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009e50 tx=0x7f7b9001d490 comp rx=0 tx=0).stop 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- 172.21.15.116:0/1370853583 shutdown_connections 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/1370853583 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9815bb20 0x7f7b9815bf30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/1370853583 >> v2:172.21.15.149:3300/0 conn(0x7f7b9806de20 0x7f7b980cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- 172.21.15.116:0/1370853583 >> 172.21.15.116:0/1370853583 conn(0x7f7b980735f0 msgr2=0x7f7b98073a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- 172.21.15.116:0/1370853583 shutdown_connections 2024-05-14T11:25:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- 172.21.15.116:0/1370853583 wait complete. 2024-05-14T11:25:45.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 Processor -- start 2024-05-14T11:25:45.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.642+0000 7f7b9ca00640 1 -- start start 2024-05-14T11:25:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9815bb20 0x7f7b98129f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9813c700 0x7f7b9815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9816a9f0 con 0x7f7b9815e980 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9816a870 con 0x7f7b9815bb20 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9816ab70 con 0x7f7b9813c700 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43326/0 (socket says 172.21.15.116:43326) 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 -- 172.21.15.116:0/2095285772 learned_addr learned my addr 172.21.15.116:0/2095285772 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b97400640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9815bb20 0x7f7b98129f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b97e00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9813c700 0x7f7b9815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 -- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9813c700 msgr2=0x7f7b9815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9813c700 0x7f7b9815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 -- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9815bb20 msgr2=0x7f7b98129f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9815bb20 0x7f7b98129f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 -- 172.21.15.116:0/2095285772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9815b8e0 con 0x7f7b9815e980 2024-05-14T11:25:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8ea00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e980 tx=0x7f7b8000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017070 con 0x7f7b9815e980 2024-05-14T11:25:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8000ca00 con 0x7f7b9815e980 2024-05-14T11:25:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9812a4c0 con 0x7f7b9815e980 2024-05-14T11:25:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000cdd0 con 0x7f7b9815e980 2024-05-14T11:25:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b981fab70 con 0x7f7b9815e980 2024-05-14T11:25:45.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.646+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b981fa600 con 0x7f7b9815e980 2024-05-14T11:25:45.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.650+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7b8001a070 con 0x7f7b9815e980 2024-05-14T11:25:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.650+0000 7f7b8fe00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 0x7f7b74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.650+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f7b8008f350 con 0x7f7b9815e980 2024-05-14T11:25:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.650+0000 7f7b97400640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 0x7f7b74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.650+0000 7f7b97400640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 0x7f7b74071910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001230 tx=0x7f7b88007b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:45.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.654+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7b8005c140 con 0x7f7b9815e980 2024-05-14T11:25:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b98050880 con 0x7f7b9815e980 2024-05-14T11:25:45.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b8fe00640 1 -- 172.21.15.116:0/2095285772 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5197 (secure 0 0 0) 0x7f7b8005ba40 con 0x7f7b9815e980 2024-05-14T11:25:45.843 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:45.843 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:25:44.842592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8208":{"gid":8208,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/1742968683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1742968683},{"type":"v1","addr":"172.21.15.116:6837","nonce":1742968683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8208,"qdb_cluster":[8208]},"id":8}]} 2024-05-14T11:25:45.844 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 100 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 msgr2=0x7f7b74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 0x7f7b74071910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001230 tx=0x7f7b88007b80 comp rx=0 tx=0).stop 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 msgr2=0x7f7b9813c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e980 tx=0x7f7b8000ee40 comp rx=0 tx=0).stop 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 shutdown_connections 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7b7406f400 0x7f7b74071910 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7b9813c700 0x7f7b9815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b9815e980 0x7f7b9813c1c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 --2- 172.21.15.116:0/2095285772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9815bb20 0x7f7b98129f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 >> 172.21.15.116:0/2095285772 conn(0x7f7b980735f0 msgr2=0x7f7b98140210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 shutdown_connections 2024-05-14T11:25:45.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:45.846+0000 7f7b9ca00640 1 -- 172.21.15.116:0/2095285772 wait complete. 2024-05-14T11:25:45.865 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:25:46.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 Processor -- start 2024-05-14T11:25:46.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 -- start start 2024-05-14T11:25:46.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd7900b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd790161580 0x7fd790161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7900cea10 con 0x7fd79015daf0 2024-05-14T11:25:46.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7900b2f80 con 0x7fd790161580 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd796c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7900b31d0 con 0x7fd7900b66f0 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd790161580 0x7fd790161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd790161580 0x7fd790161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43334/0 (socket says 172.21.15.116:43334) 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 -- 172.21.15.116:0/1391603728 learned_addr learned my addr 172.21.15.116:0/1391603728 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd787400640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd7900b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 -- 172.21.15.116:0/1391603728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 msgr2=0x7fd7900b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:46.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd7900b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 -- 172.21.15.116:0/1391603728 >> v1:172.21.15.149:6789/0 conn(0x7fd79015daf0 legacy=0x7fd7900b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.014+0000 7fd794c00640 1 -- 172.21.15.116:0/1391603728 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900b34b0 con 0x7fd790161580 2024-05-14T11:25:46.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd794c00640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd790161580 0x7fd790161990 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd788009e50 tx=0x7fd78801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b254ccb0f37e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd78f400640 1 -- 172.21.15.116:0/1391603728 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a5810 con 0x7fd790161580 2024-05-14T11:25:46.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd78f400640 1 -- 172.21.15.116:0/1391603728 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7880a59b0 con 0x7fd790161580 2024-05-14T11:25:46.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd78f400640 1 -- 172.21.15.116:0/1391603728 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a5d80 con 0x7fd790161580 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 -- 172.21.15.116:0/1391603728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd790161580 msgr2=0x7fd790161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd790161580 0x7fd790161990 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd788009e50 tx=0x7fd78801ca30 comp rx=0 tx=0).stop 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 -- 172.21.15.116:0/1391603728 shutdown_connections 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd790161580 0x7fd790161990 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 --2- 172.21.15.116:0/1391603728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd7900b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 -- 172.21.15.116:0/1391603728 >> 172.21.15.116:0/1391603728 conn(0x7fd7900b5710 msgr2=0x7fd7900d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 -- 172.21.15.116:0/1391603728 shutdown_connections 2024-05-14T11:25:46.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.018+0000 7fd796c00640 1 -- 172.21.15.116:0/1391603728 wait complete. 2024-05-14T11:25:46.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 Processor -- start 2024-05-14T11:25:46.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 -- start start 2024-05-14T11:25:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd790151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd790161580 0x7fd790167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900b94c0 con 0x7fd79015daf0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900b9340 con 0x7fd790161580 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd796c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7901622e0 con 0x7fd7900b66f0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43336/0 (socket says 172.21.15.116:43336) 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 -- 172.21.15.116:0/1566841543 learned_addr learned my addr 172.21.15.116:0/1566841543 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd78fe00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd790151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd794c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd790161580 0x7fd790167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 -- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 msgr2=0x7fd790151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd790151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 -- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd790161580 msgr2=0x7fd790167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.022+0000 7fd787400640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd790161580 0x7fd790167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd787400640 1 -- 172.21.15.116:0/1566841543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7901669e0 con 0x7fd79015daf0 2024-05-14T11:25:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd787400640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd77800e8a0 tx=0x7fd77800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778017070 con 0x7fd79015daf0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77800c940 con 0x7fd79015daf0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800ccf0 con 0x7fd79015daf0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd790166ee0 con 0x7fd79015daf0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901674a0 con 0x7fd79015daf0 2024-05-14T11:25:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79006a9c0 con 0x7fd79015daf0 2024-05-14T11:25:46.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd77801a070 con 0x7fd79015daf0 2024-05-14T11:25:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 0x7fd76c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.026+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fd77808f330 con 0x7fd79015daf0 2024-05-14T11:25:46.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.030+0000 7fd78fe00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 0x7fd76c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.030+0000 7fd78fe00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 0x7fd76c071910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd780007fd0 tx=0x7fd780009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.034+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd77805b9a0 con 0x7fd79015daf0 2024-05-14T11:25:46.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd790166c70 con 0x7fd79015daf0 2024-05-14T11:25:46.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd78cc00640 1 -- 172.21.15.116:0/1566841543 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17000 (secure 0 0 0) 0x7fd77805fca0 con 0x7fd79015daf0 2024-05-14T11:25:46.188 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:46.189 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:25:43.437284+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:25:39.509510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-14T11:25:40.801825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:25:46.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 msgr2=0x7fd76c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 0x7fd76c071910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd780007fd0 tx=0x7fd780009040 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 msgr2=0x7fd790151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd77800e8a0 tx=0x7fd77800eca0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 shutdown_connections 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd76c06f400 0x7fd76c071910 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd790161580 0x7fd790167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd79015daf0 0x7fd790151640 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 --2- 172.21.15.116:0/1566841543 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7900b66f0 0x7fd790151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 >> 172.21.15.116:0/1566841543 conn(0x7fd7900b5710 msgr2=0x7fd7900b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.190+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 shutdown_connections 2024-05-14T11:25:46.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.194+0000 7fd796c00640 1 -- 172.21.15.116:0/1566841543 wait complete. 2024-05-14T11:25:46.211 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:25:46.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:25:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.358+0000 7fd71ee00640 1 Processor -- start 2024-05-14T11:25:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.358+0000 7fd71ee00640 1 -- start start 2024-05-14T11:25:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ee00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7180d1670 0x7fd7180d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ee00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd71806c700 0x7fd71806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ee00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71806ff20 con 0x7fd71815db30 2024-05-14T11:25:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ee00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7180ae100 con 0x7fd7180d1670 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ee00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7180ae870 con 0x7fd71806c700 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7180d1670 0x7fd7180d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7180d1670 0x7fd7180d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43344/0 (socket says 172.21.15.116:43344) 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 -- 172.21.15.116:0/4103583456 learned_addr learned my addr 172.21.15.116:0/4103583456 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd71ce00640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd71806c700 0x7fd71806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 -- 172.21.15.116:0/4103583456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd71806c700 msgr2=0x7fd71806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd71806c700 0x7fd71806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 -- 172.21.15.116:0/4103583456 >> v1:172.21.15.149:6789/0 conn(0x7fd71815db30 legacy=0x7fd7180aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.362+0000 7fd70f400640 1 -- 172.21.15.116:0/4103583456 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7180d1fc0 con 0x7fd7180d1670 2024-05-14T11:25:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd70f400640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7180d1670 0x7fd7180d1a80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd700009870 tx=0x7fd70001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3313b04056e2abd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd717400640 1 -- 172.21.15.116:0/4103583456 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000b0070 con 0x7fd7180d1670 2024-05-14T11:25:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd717400640 1 -- 172.21.15.116:0/4103583456 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7000ab600 con 0x7fd7180d1670 2024-05-14T11:25:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd717400640 1 -- 172.21.15.116:0/4103583456 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000ab9d0 con 0x7fd7180d1670 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 -- 172.21.15.116:0/4103583456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7180d1670 msgr2=0x7fd7180d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7180d1670 0x7fd7180d1a80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd700009870 tx=0x7fd70001c400 comp rx=0 tx=0).stop 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 -- 172.21.15.116:0/4103583456 shutdown_connections 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd71806c700 0x7fd71806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 --2- 172.21.15.116:0/4103583456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7180d1670 0x7fd7180d1a80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 -- 172.21.15.116:0/4103583456 >> 172.21.15.116:0/4103583456 conn(0x7fd7180b4620 msgr2=0x7fd7180b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 -- 172.21.15.116:0/4103583456 shutdown_connections 2024-05-14T11:25:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.366+0000 7fd71ee00640 1 -- 172.21.15.116:0/4103583456 wait complete. 2024-05-14T11:25:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 Processor -- start 2024-05-14T11:25:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- start start 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7180d1670 0x7fd7180cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 0x7fd71815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd71816a770 con 0x7fd71806c700 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd71815df40 con 0x7fd71815db30 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd71816a8f0 con 0x7fd7180d1670 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43358/0 (socket says 172.21.15.116:43358) 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 -- 172.21.15.116:0/3012203745 learned_addr learned my addr 172.21.15.116:0/3012203745 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd70f400640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7180d1670 0x7fd7180cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ce00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 0x7fd71815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 -- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7180d1670 msgr2=0x7fd7180cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7180d1670 0x7fd7180cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 -- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 msgr2=0x7fd71815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 0x7fd71815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 -- 172.21.15.116:0/3012203745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7180d0320 con 0x7fd71806c700 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ce00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 0x7fd71815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd717e00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c3f0 tx=0x7fd70800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708017070 con 0x7fd71806c700 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7080052d0 con 0x7fd71806c700 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708003820 con 0x7fd71806c700 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7180d0600 con 0x7fd71806c700 2024-05-14T11:25:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.370+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7180d0bc0 con 0x7fd71806c700 2024-05-14T11:25:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71806a9c0 con 0x7fd71806c700 2024-05-14T11:25:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd70801a070 con 0x7fd71806c700 2024-05-14T11:25:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd714c00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 0x7fd6f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fd70808e630 con 0x7fd71806c700 2024-05-14T11:25:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd70f400640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 0x7fd6f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.374+0000 7fd70f400640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 0x7fd6f4071940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7000ad000 tx=0x7fd700008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.378+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd708005480 con 0x7fd71806c700 2024-05-14T11:25:46.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.538+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd718168540 con 0x7fd6f406f430 2024-05-14T11:25:46.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.542+0000 7fd714c00640 1 -- 172.21.15.116:0/3012203745 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7fd718168540 con 0x7fd6f406f430 2024-05-14T11:25:46.544 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:46.546 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 msgr2=0x7fd6f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 0x7fd6f4071940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7000ad000 tx=0x7fd700008040 comp rx=0 tx=0).stop 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 msgr2=0x7fd71809a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fd70800c3f0 tx=0x7fd70800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 shutdown_connections 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd6f406f430 0x7fd6f4071940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd71815db30 0x7fd71815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7180d1670 0x7fd7180cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 --2- 172.21.15.116:0/3012203745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd71806c700 0x7fd71809a8b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 >> 172.21.15.116:0/3012203745 conn(0x7fd7180b4620 msgr2=0x7fd7180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 shutdown_connections 2024-05-14T11:25:46.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.546+0000 7fd71ee00640 1 -- 172.21.15.116:0/3012203745 wait complete. 2024-05-14T11:25:46.565 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-05-14T11:25:46.507208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261720,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713000,"statfs":{"total":773094113280,"available":772826112000,"internally_reserved":0,"allocated":12525568,"data_stored":6434884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001241"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448833+0000","last_change":"2024-05-14T11:25:42.447556+0000","last_active":"2024-05-14T11:25:43.448833+0000","last_peered":"2024-05-14T11:25:43.448833+0000","last_clean":"2024-05-14T11:25:43.448833+0000","last_became_active":"2024-05-14T11:25:42.447253+0000","last_became_peered":"2024-05-14T11:25:42.447253+0000","last_unstale":"2024-05-14T11:25:43.448833+0000","last_undegraded":"2024-05-14T11:25:43.448833+0000","last_fullsized":"2024-05-14T11:25:43.448833+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640228+0000","last_change":"2024-05-14T11:25:42.456318+0000","last_active":"2024-05-14T11:25:43.640228+0000","last_peered":"2024-05-14T11:25:43.640228+0000","last_clean":"2024-05-14T11:25:43.640228+0000","last_became_active":"2024-05-14T11:25:42.455767+0000","last_became_peered":"2024-05-14T11:25:42.455767+0000","last_unstale":"2024-05-14T11:25:43.640228+0000","last_undegraded":"2024-05-14T11:25:43.640228+0000","last_fullsized":"2024-05-14T11:25:43.640228+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:35.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640181+0000","last_change":"2024-05-14T11:25:42.443177+0000","last_active":"2024-05-14T11:25:43.640181+0000","last_peered":"2024-05-14T11:25:43.640181+0000","last_clean":"2024-05-14T11:25:43.640181+0000","last_became_active":"2024-05-14T11:25:42.442819+0000","last_became_peered":"2024-05-14T11:25:42.442819+0000","last_unstale":"2024-05-14T11:25:43.640181+0000","last_undegraded":"2024-05-14T11:25:43.640181+0000","last_fullsized":"2024-05-14T11:25:43.640181+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:16.135038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640422+0000","last_change":"2024-05-14T11:25:42.457086+0000","last_active":"2024-05-14T11:25:43.640422+0000","last_peered":"2024-05-14T11:25:43.640422+0000","last_clean":"2024-05-14T11:25:43.640422+0000","last_became_active":"2024-05-14T11:25:42.453585+0000","last_became_peered":"2024-05-14T11:25:42.453585+0000","last_unstale":"2024-05-14T11:25:43.640422+0000","last_undegraded":"2024-05-14T11:25:43.640422+0000","last_fullsized":"2024-05-14T11:25:43.640422+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:16.135038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448420+0000","last_change":"2024-05-14T11:25:42.439427+0000","last_active":"2024-05-14T11:25:43.448420+0000","last_peered":"2024-05-14T11:25:43.448420+0000","last_clean":"2024-05-14T11:25:43.448420+0000","last_became_active":"2024-05-14T11:25:42.438976+0000","last_became_peered":"2024-05-14T11:25:42.438976+0000","last_unstale":"2024-05-14T11:25:43.448420+0000","last_undegraded":"2024-05-14T11:25:43.448420+0000","last_fullsized":"2024-05-14T11:25:43.448420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452029+0000","last_change":"2024-05-14T11:25:42.440982+0000","last_active":"2024-05-14T11:25:43.452029+0000","last_peered":"2024-05-14T11:25:43.452029+0000","last_clean":"2024-05-14T11:25:43.452029+0000","last_became_active":"2024-05-14T11:25:42.440628+0000","last_became_peered":"2024-05-14T11:25:42.440628+0000","last_unstale":"2024-05-14T11:25:43.452029+0000","last_undegraded":"2024-05-14T11:25:43.452029+0000","last_fullsized":"2024-05-14T11:25:43.452029+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:46:54.833574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451909+0000","last_change":"2024-05-14T11:25:42.458739+0000","last_active":"2024-05-14T11:25:43.451909+0000","last_peered":"2024-05-14T11:25:43.451909+0000","last_clean":"2024-05-14T11:25:43.451909+0000","last_became_active":"2024-05-14T11:25:42.458431+0000","last_became_peered":"2024-05-14T11:25:42.458431+0000","last_unstale":"2024-05-14T11:25:43.451909+0000","last_undegraded":"2024-05-14T11:25:43.451909+0000","last_fullsized":"2024-05-14T11:25:43.451909+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:29:02.860814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639338+0000","last_change":"2024-05-14T11:25:42.447950+0000","last_active":"2024-05-14T11:25:43.639338+0000","last_peered":"2024-05-14T11:25:43.639338+0000","last_clean":"2024-05-14T11:25:43.639338+0000","last_became_active":"2024-05-14T11:25:42.439231+0000","last_became_peered":"2024-05-14T11:25:42.439231+0000","last_unstale":"2024-05-14T11:25:43.639338+0000","last_undegraded":"2024-05-14T11:25:43.639338+0000","last_fullsized":"2024-05-14T11:25:43.639338+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:52.918452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641443+0000","last_change":"2024-05-14T11:25:42.454508+0000","last_active":"2024-05-14T11:25:43.641443+0000","last_peered":"2024-05-14T11:25:43.641443+0000","last_clean":"2024-05-14T11:25:43.641443+0000","last_became_active":"2024-05-14T11:25:42.454124+0000","last_became_peered":"2024-05-14T11:25:42.454124+0000","last_unstale":"2024-05-14T11:25:43.641443+0000","last_undegraded":"2024-05-14T11:25:43.641443+0000","last_fullsized":"2024-05-14T11:25:43.641443+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:22:13.710478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639860+0000","last_change":"2024-05-14T11:25:42.450093+0000","last_active":"2024-05-14T11:25:43.639860+0000","last_peered":"2024-05-14T11:25:43.639860+0000","last_clean":"2024-05-14T11:25:43.639860+0000","last_became_active":"2024-05-14T11:25:42.449773+0000","last_became_peered":"2024-05-14T11:25:42.449773+0000","last_unstale":"2024-05-14T11:25:43.639860+0000","last_undegraded":"2024-05-14T11:25:43.639860+0000","last_fullsized":"2024-05-14T11:25:43.639860+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448490+0000","last_change":"2024-05-14T11:25:42.463006+0000","last_active":"2024-05-14T11:25:43.448490+0000","last_peered":"2024-05-14T11:25:43.448490+0000","last_clean":"2024-05-14T11:25:43.448490+0000","last_became_active":"2024-05-14T11:25:42.462703+0000","last_became_peered":"2024-05-14T11:25:42.462703+0000","last_unstale":"2024-05-14T11:25:43.448490+0000","last_undegraded":"2024-05-14T11:25:43.448490+0000","last_fullsized":"2024-05-14T11:25:43.448490+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452524+0000","last_change":"2024-05-14T11:25:42.446705+0000","last_active":"2024-05-14T11:25:43.452524+0000","last_peered":"2024-05-14T11:25:43.452524+0000","last_clean":"2024-05-14T11:25:43.452524+0000","last_became_active":"2024-05-14T11:25:42.446406+0000","last_became_peered":"2024-05-14T11:25:42.446406+0000","last_unstale":"2024-05-14T11:25:43.452524+0000","last_undegraded":"2024-05-14T11:25:43.452524+0000","last_fullsized":"2024-05-14T11:25:43.452524+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:12:51.273783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639385+0000","last_change":"2024-05-14T11:25:42.458936+0000","last_active":"2024-05-14T11:25:43.639385+0000","last_peered":"2024-05-14T11:25:43.639385+0000","last_clean":"2024-05-14T11:25:43.639385+0000","last_became_active":"2024-05-14T11:25:42.458629+0000","last_became_peered":"2024-05-14T11:25:42.458629+0000","last_unstale":"2024-05-14T11:25:43.639385+0000","last_undegraded":"2024-05-14T11:25:43.639385+0000","last_fullsized":"2024-05-14T11:25:43.639385+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:13.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639717+0000","last_change":"2024-05-14T11:25:42.438206+0000","last_active":"2024-05-14T11:25:43.639717+0000","last_peered":"2024-05-14T11:25:43.639717+0000","last_clean":"2024-05-14T11:25:43.639717+0000","last_became_active":"2024-05-14T11:25:42.437473+0000","last_became_peered":"2024-05-14T11:25:42.437473+0000","last_unstale":"2024-05-14T11:25:43.639717+0000","last_undegraded":"2024-05-14T11:25:43.639717+0000","last_fullsized":"2024-05-14T11:25:43.639717+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:16.391415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448314+0000","last_change":"2024-05-14T11:25:42.453451+0000","last_active":"2024-05-14T11:25:43.448314+0000","last_peered":"2024-05-14T11:25:43.448314+0000","last_clean":"2024-05-14T11:25:43.448314+0000","last_became_active":"2024-05-14T11:25:42.451879+0000","last_became_peered":"2024-05-14T11:25:42.451879+0000","last_unstale":"2024-05-14T11:25:43.448314+0000","last_undegraded":"2024-05-14T11:25:43.448314+0000","last_fullsized":"2024-05-14T11:25:43.448314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:24.831507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451852+0000","last_change":"2024-05-14T11:25:42.442433+0000","last_active":"2024-05-14T11:25:43.451852+0000","last_peered":"2024-05-14T11:25:43.451852+0000","last_clean":"2024-05-14T11:25:43.451852+0000","last_became_active":"2024-05-14T11:25:42.441863+0000","last_became_peered":"2024-05-14T11:25:42.441863+0000","last_unstale":"2024-05-14T11:25:43.451852+0000","last_undegraded":"2024-05-14T11:25:43.451852+0000","last_fullsized":"2024-05-14T11:25:43.451852+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:01.383403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640038+0000","last_change":"2024-05-14T11:25:42.464244+0000","last_active":"2024-05-14T11:25:43.640038+0000","last_peered":"2024-05-14T11:25:43.640038+0000","last_clean":"2024-05-14T11:25:43.640038+0000","last_became_active":"2024-05-14T11:25:42.463888+0000","last_became_peered":"2024-05-14T11:25:42.463888+0000","last_unstale":"2024-05-14T11:25:43.640038+0000","last_undegraded":"2024-05-14T11:25:43.640038+0000","last_fullsized":"2024-05-14T11:25:43.640038+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640291+0000","last_change":"2024-05-14T11:25:42.456055+0000","last_active":"2024-05-14T11:25:43.640291+0000","last_peered":"2024-05-14T11:25:43.640291+0000","last_clean":"2024-05-14T11:25:43.640291+0000","last_became_active":"2024-05-14T11:25:42.455319+0000","last_became_peered":"2024-05-14T11:25:42.455319+0000","last_unstale":"2024-05-14T11:25:43.640291+0000","last_undegraded":"2024-05-14T11:25:43.640291+0000","last_fullsized":"2024-05-14T11:25:43.640291+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:58.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.446968+0000","last_change":"2024-05-14T11:25:42.458336+0000","last_active":"2024-05-14T11:25:43.446968+0000","last_peered":"2024-05-14T11:25:43.446968+0000","last_clean":"2024-05-14T11:25:43.446968+0000","last_became_active":"2024-05-14T11:25:42.458037+0000","last_became_peered":"2024-05-14T11:25:42.458037+0000","last_unstale":"2024-05-14T11:25:43.446968+0000","last_undegraded":"2024-05-14T11:25:43.446968+0000","last_fullsized":"2024-05-14T11:25:43.446968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444825+0000","last_change":"2024-05-14T11:25:42.440552+0000","last_active":"2024-05-14T11:25:43.444825+0000","last_peered":"2024-05-14T11:25:43.444825+0000","last_clean":"2024-05-14T11:25:43.444825+0000","last_became_active":"2024-05-14T11:25:42.440229+0000","last_became_peered":"2024-05-14T11:25:42.440229+0000","last_unstale":"2024-05-14T11:25:43.444825+0000","last_undegraded":"2024-05-14T11:25:43.444825+0000","last_fullsized":"2024-05-14T11:25:43.444825+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:18.387079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640778+0000","last_change":"2024-05-14T11:25:42.454795+0000","last_active":"2024-05-14T11:25:43.640778+0000","last_peered":"2024-05-14T11:25:43.640778+0000","last_clean":"2024-05-14T11:25:43.640778+0000","last_became_active":"2024-05-14T11:25:42.454401+0000","last_became_peered":"2024-05-14T11:25:42.454401+0000","last_unstale":"2024-05-14T11:25:43.640778+0000","last_undegraded":"2024-05-14T11:25:43.640778+0000","last_fullsized":"2024-05-14T11:25:43.640778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:35.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448516+0000","last_change":"2024-05-14T11:25:42.445590+0000","last_active":"2024-05-14T11:25:43.448516+0000","last_peered":"2024-05-14T11:25:43.448516+0000","last_clean":"2024-05-14T11:25:43.448516+0000","last_became_active":"2024-05-14T11:25:42.445285+0000","last_became_peered":"2024-05-14T11:25:42.445285+0000","last_unstale":"2024-05-14T11:25:43.448516+0000","last_undegraded":"2024-05-14T11:25:43.448516+0000","last_fullsized":"2024-05-14T11:25:43.448516+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640133+0000","last_change":"2024-05-14T11:25:42.458371+0000","last_active":"2024-05-14T11:25:43.640133+0000","last_peered":"2024-05-14T11:25:43.640133+0000","last_clean":"2024-05-14T11:25:43.640133+0000","last_became_active":"2024-05-14T11:25:42.458044+0000","last_became_peered":"2024-05-14T11:25:42.458044+0000","last_unstale":"2024-05-14T11:25:43.640133+0000","last_undegraded":"2024-05-14T11:25:43.640133+0000","last_fullsized":"2024-05-14T11:25:43.640133+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:56.412006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640062+0000","last_change":"2024-05-14T11:25:42.452231+0000","last_active":"2024-05-14T11:25:43.640062+0000","last_peered":"2024-05-14T11:25:43.640062+0000","last_clean":"2024-05-14T11:25:43.640062+0000","last_became_active":"2024-05-14T11:25:42.451867+0000","last_became_peered":"2024-05-14T11:25:42.451867+0000","last_unstale":"2024-05-14T11:25:43.640062+0000","last_undegraded":"2024-05-14T11:25:43.640062+0000","last_fullsized":"2024-05-14T11:25:43.640062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640662+0000","last_change":"2024-05-14T11:25:42.460533+0000","last_active":"2024-05-14T11:25:43.640662+0000","last_peered":"2024-05-14T11:25:43.640662+0000","last_clean":"2024-05-14T11:25:43.640662+0000","last_became_active":"2024-05-14T11:25:42.460097+0000","last_became_peered":"2024-05-14T11:25:42.460097+0000","last_unstale":"2024-05-14T11:25:43.640662+0000","last_undegraded":"2024-05-14T11:25:43.640662+0000","last_fullsized":"2024-05-14T11:25:43.640662+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:56.550588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445040+0000","last_change":"2024-05-14T11:25:42.456247+0000","last_active":"2024-05-14T11:25:43.445040+0000","last_peered":"2024-05-14T11:25:43.445040+0000","last_clean":"2024-05-14T11:25:43.445040+0000","last_became_active":"2024-05-14T11:25:42.455636+0000","last_became_peered":"2024-05-14T11:25:42.455636+0000","last_unstale":"2024-05-14T11:25:43.445040+0000","last_undegraded":"2024-05-14T11:25:43.445040+0000","last_fullsized":"2024-05-14T11:25:43.445040+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:09:22.975248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447227+0000","last_change":"2024-05-14T11:25:42.449933+0000","last_active":"2024-05-14T11:25:43.447227+0000","last_peered":"2024-05-14T11:25:43.447227+0000","last_clean":"2024-05-14T11:25:43.447227+0000","last_became_active":"2024-05-14T11:25:42.449626+0000","last_became_peered":"2024-05-14T11:25:42.449626+0000","last_unstale":"2024-05-14T11:25:43.447227+0000","last_undegraded":"2024-05-14T11:25:43.447227+0000","last_fullsized":"2024-05-14T11:25:43.447227+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452508+0000","last_change":"2024-05-14T11:25:42.467028+0000","last_active":"2024-05-14T11:25:43.452508+0000","last_peered":"2024-05-14T11:25:43.452508+0000","last_clean":"2024-05-14T11:25:43.452508+0000","last_became_active":"2024-05-14T11:25:42.464463+0000","last_became_peered":"2024-05-14T11:25:42.464463+0000","last_unstale":"2024-05-14T11:25:43.452508+0000","last_undegraded":"2024-05-14T11:25:43.452508+0000","last_fullsized":"2024-05-14T11:25:43.452508+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:27:38.876560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444848+0000","last_change":"2024-05-14T11:25:42.456410+0000","last_active":"2024-05-14T11:25:43.444848+0000","last_peered":"2024-05-14T11:25:43.444848+0000","last_clean":"2024-05-14T11:25:43.444848+0000","last_became_active":"2024-05-14T11:25:42.456090+0000","last_became_peered":"2024-05-14T11:25:42.456090+0000","last_unstale":"2024-05-14T11:25:43.444848+0000","last_undegraded":"2024-05-14T11:25:43.444848+0000","last_fullsized":"2024-05-14T11:25:43.444848+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:32.551235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640144+0000","last_change":"2024-05-14T11:25:42.438603+0000","last_active":"2024-05-14T11:25:43.640144+0000","last_peered":"2024-05-14T11:25:43.640144+0000","last_clean":"2024-05-14T11:25:43.640144+0000","last_became_active":"2024-05-14T11:25:42.437916+0000","last_became_peered":"2024-05-14T11:25:42.437916+0000","last_unstale":"2024-05-14T11:25:43.640144+0000","last_undegraded":"2024-05-14T11:25:43.640144+0000","last_fullsized":"2024-05-14T11:25:43.640144+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:56.412006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640817+0000","last_change":"2024-05-14T11:25:42.463056+0000","last_active":"2024-05-14T11:25:43.640817+0000","last_peered":"2024-05-14T11:25:43.640817+0000","last_clean":"2024-05-14T11:25:43.640817+0000","last_became_active":"2024-05-14T11:25:42.462760+0000","last_became_peered":"2024-05-14T11:25:42.462760+0000","last_unstale":"2024-05-14T11:25:43.640817+0000","last_undegraded":"2024-05-14T11:25:43.640817+0000","last_fullsized":"2024-05-14T11:25:43.640817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:27.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640256+0000","last_change":"2024-05-14T11:25:42.451960+0000","last_active":"2024-05-14T11:25:43.640256+0000","last_peered":"2024-05-14T11:25:43.640256+0000","last_clean":"2024-05-14T11:25:43.640256+0000","last_became_active":"2024-05-14T11:25:42.451595+0000","last_became_peered":"2024-05-14T11:25:42.451595+0000","last_unstale":"2024-05-14T11:25:43.640256+0000","last_undegraded":"2024-05-14T11:25:43.640256+0000","last_fullsized":"2024-05-14T11:25:43.640256+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:15:48.415733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639643+0000","last_change":"2024-05-14T11:25:42.448325+0000","last_active":"2024-05-14T11:25:43.639643+0000","last_peered":"2024-05-14T11:25:43.639643+0000","last_clean":"2024-05-14T11:25:43.639643+0000","last_became_active":"2024-05-14T11:25:42.446632+0000","last_became_peered":"2024-05-14T11:25:42.446632+0000","last_unstale":"2024-05-14T11:25:43.639643+0000","last_undegraded":"2024-05-14T11:25:43.639643+0000","last_fullsized":"2024-05-14T11:25:43.639643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447206+0000","last_change":"2024-05-14T11:25:41.447134+0000","last_active":"2024-05-14T11:25:43.447206+0000","last_peered":"2024-05-14T11:25:43.447206+0000","last_clean":"2024-05-14T11:25:43.447206+0000","last_became_active":"2024-05-14T11:25:41.446765+0000","last_became_peered":"2024-05-14T11:25:41.446765+0000","last_unstale":"2024-05-14T11:25:43.447206+0000","last_undegraded":"2024-05-14T11:25:43.447206+0000","last_fullsized":"2024-05-14T11:25:43.447206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:54:03.068472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452554+0000","last_change":"2024-05-14T11:25:42.453611+0000","last_active":"2024-05-14T11:25:43.452554+0000","last_peered":"2024-05-14T11:25:43.452554+0000","last_clean":"2024-05-14T11:25:43.452554+0000","last_became_active":"2024-05-14T11:25:42.453269+0000","last_became_peered":"2024-05-14T11:25:42.453269+0000","last_unstale":"2024-05-14T11:25:43.452554+0000","last_undegraded":"2024-05-14T11:25:43.452554+0000","last_fullsized":"2024-05-14T11:25:43.452554+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:37:46.535205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639790+0000","last_change":"2024-05-14T11:25:41.451612+0000","last_active":"2024-05-14T11:25:43.639790+0000","last_peered":"2024-05-14T11:25:43.639790+0000","last_clean":"2024-05-14T11:25:43.639790+0000","last_became_active":"2024-05-14T11:25:41.450327+0000","last_became_peered":"2024-05-14T11:25:41.450327+0000","last_unstale":"2024-05-14T11:25:43.639790+0000","last_undegraded":"2024-05-14T11:25:43.639790+0000","last_fullsized":"2024-05-14T11:25:43.639790+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:33.984835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640765+0000","last_change":"2024-05-14T11:25:42.463967+0000","last_active":"2024-05-14T11:25:43.640765+0000","last_peered":"2024-05-14T11:25:43.640765+0000","last_clean":"2024-05-14T11:25:43.640765+0000","last_became_active":"2024-05-14T11:25:42.463475+0000","last_became_peered":"2024-05-14T11:25:42.463475+0000","last_unstale":"2024-05-14T11:25:43.640765+0000","last_undegraded":"2024-05-14T11:25:43.640765+0000","last_fullsized":"2024-05-14T11:25:43.640765+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:36.318016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641304+0000","last_change":"2024-05-14T11:25:41.451594+0000","last_active":"2024-05-14T11:25:43.641304+0000","last_peered":"2024-05-14T11:25:43.641304+0000","last_clean":"2024-05-14T11:25:43.641304+0000","last_became_active":"2024-05-14T11:25:41.451120+0000","last_became_peered":"2024-05-14T11:25:41.451120+0000","last_unstale":"2024-05-14T11:25:43.641304+0000","last_undegraded":"2024-05-14T11:25:43.641304+0000","last_fullsized":"2024-05-14T11:25:43.641304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:55.802811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641502+0000","last_change":"2024-05-14T11:25:42.459423+0000","last_active":"2024-05-14T11:25:43.641502+0000","last_peered":"2024-05-14T11:25:43.641502+0000","last_clean":"2024-05-14T11:25:43.641502+0000","last_became_active":"2024-05-14T11:25:42.459090+0000","last_became_peered":"2024-05-14T11:25:42.459090+0000","last_unstale":"2024-05-14T11:25:43.641502+0000","last_undegraded":"2024-05-14T11:25:43.641502+0000","last_fullsized":"2024-05-14T11:25:43.641502+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:17.791776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639663+0000","last_change":"2024-05-14T11:25:41.447678+0000","last_active":"2024-05-14T11:25:43.639663+0000","last_peered":"2024-05-14T11:25:43.639663+0000","last_clean":"2024-05-14T11:25:43.639663+0000","last_became_active":"2024-05-14T11:25:41.447302+0000","last_became_peered":"2024-05-14T11:25:41.447302+0000","last_unstale":"2024-05-14T11:25:43.639663+0000","last_undegraded":"2024-05-14T11:25:43.639663+0000","last_fullsized":"2024-05-14T11:25:43.639663+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:20.434168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639255+0000","last_change":"2024-05-14T11:25:42.453356+0000","last_active":"2024-05-14T11:25:43.639255+0000","last_peered":"2024-05-14T11:25:43.639255+0000","last_clean":"2024-05-14T11:25:43.639255+0000","last_became_active":"2024-05-14T11:25:42.448013+0000","last_became_peered":"2024-05-14T11:25:42.448013+0000","last_unstale":"2024-05-14T11:25:43.639255+0000","last_undegraded":"2024-05-14T11:25:43.639255+0000","last_fullsized":"2024-05-14T11:25:43.639255+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:27.029938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448514+0000","last_change":"2024-05-14T11:25:41.442352+0000","last_active":"2024-05-14T11:25:43.448514+0000","last_peered":"2024-05-14T11:25:43.448514+0000","last_clean":"2024-05-14T11:25:43.448514+0000","last_became_active":"2024-05-14T11:25:41.442036+0000","last_became_peered":"2024-05-14T11:25:41.442036+0000","last_unstale":"2024-05-14T11:25:43.448514+0000","last_undegraded":"2024-05-14T11:25:43.448514+0000","last_fullsized":"2024-05-14T11:25:43.448514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:43:53.519056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448302+0000","last_change":"2024-05-14T11:25:42.455349+0000","last_active":"2024-05-14T11:25:43.448302+0000","last_peered":"2024-05-14T11:25:43.448302+0000","last_clean":"2024-05-14T11:25:43.448302+0000","last_became_active":"2024-05-14T11:25:42.454970+0000","last_became_peered":"2024-05-14T11:25:42.454970+0000","last_unstale":"2024-05-14T11:25:43.448302+0000","last_undegraded":"2024-05-14T11:25:43.448302+0000","last_fullsized":"2024-05-14T11:25:43.448302+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:13.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451996+0000","last_change":"2024-05-14T11:25:41.438326+0000","last_active":"2024-05-14T11:25:43.451996+0000","last_peered":"2024-05-14T11:25:43.451996+0000","last_clean":"2024-05-14T11:25:43.451996+0000","last_became_active":"2024-05-14T11:25:41.436383+0000","last_became_peered":"2024-05-14T11:25:41.436383+0000","last_unstale":"2024-05-14T11:25:43.451996+0000","last_undegraded":"2024-05-14T11:25:43.451996+0000","last_fullsized":"2024-05-14T11:25:43.451996+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:23:12.047661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640367+0000","last_change":"2024-05-14T11:25:42.451946+0000","last_active":"2024-05-14T11:25:43.640367+0000","last_peered":"2024-05-14T11:25:43.640367+0000","last_clean":"2024-05-14T11:25:43.640367+0000","last_became_active":"2024-05-14T11:25:42.450540+0000","last_became_peered":"2024-05-14T11:25:42.450540+0000","last_unstale":"2024-05-14T11:25:43.640367+0000","last_undegraded":"2024-05-14T11:25:43.640367+0000","last_fullsized":"2024-05-14T11:25:43.640367+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:13:48.977220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640248+0000","last_change":"2024-05-14T11:25:41.446237+0000","last_active":"2024-05-14T11:25:43.640248+0000","last_peered":"2024-05-14T11:25:43.640248+0000","last_clean":"2024-05-14T11:25:43.640248+0000","last_became_active":"2024-05-14T11:25:41.445944+0000","last_became_peered":"2024-05-14T11:25:41.445944+0000","last_unstale":"2024-05-14T11:25:43.640248+0000","last_undegraded":"2024-05-14T11:25:43.640248+0000","last_fullsized":"2024-05-14T11:25:43.640248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:54:03.068472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639425+0000","last_change":"2024-05-14T11:25:42.451009+0000","last_active":"2024-05-14T11:25:43.639425+0000","last_peered":"2024-05-14T11:25:43.639425+0000","last_clean":"2024-05-14T11:25:43.639425+0000","last_became_active":"2024-05-14T11:25:42.450662+0000","last_became_peered":"2024-05-14T11:25:42.450662+0000","last_unstale":"2024-05-14T11:25:43.639425+0000","last_undegraded":"2024-05-14T11:25:43.639425+0000","last_fullsized":"2024-05-14T11:25:43.639425+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:24.831507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452102+0000","last_change":"2024-05-14T11:25:41.441419+0000","last_active":"2024-05-14T11:25:43.452102+0000","last_peered":"2024-05-14T11:25:43.452102+0000","last_clean":"2024-05-14T11:25:43.452102+0000","last_became_active":"2024-05-14T11:25:41.441127+0000","last_became_peered":"2024-05-14T11:25:41.441127+0000","last_unstale":"2024-05-14T11:25:43.452102+0000","last_undegraded":"2024-05-14T11:25:43.452102+0000","last_fullsized":"2024-05-14T11:25:43.452102+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:49.973678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447108+0000","last_change":"2024-05-14T11:25:42.449597+0000","last_active":"2024-05-14T11:25:43.447108+0000","last_peered":"2024-05-14T11:25:43.447108+0000","last_clean":"2024-05-14T11:25:43.447108+0000","last_became_active":"2024-05-14T11:25:42.449079+0000","last_became_peered":"2024-05-14T11:25:42.449079+0000","last_unstale":"2024-05-14T11:25:43.447108+0000","last_undegraded":"2024-05-14T11:25:43.447108+0000","last_fullsized":"2024-05-14T11:25:43.447108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447207+0000","last_change":"2024-05-14T11:25:41.445403+0000","last_active":"2024-05-14T11:25:43.447207+0000","last_peered":"2024-05-14T11:25:43.447207+0000","last_clean":"2024-05-14T11:25:43.447207+0000","last_became_active":"2024-05-14T11:25:41.444690+0000","last_became_peered":"2024-05-14T11:25:41.444690+0000","last_unstale":"2024-05-14T11:25:43.447207+0000","last_undegraded":"2024-05-14T11:25:43.447207+0000","last_fullsized":"2024-05-14T11:25:43.447207+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:05:31.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640566+0000","last_change":"2024-05-14T11:25:42.461154+0000","last_active":"2024-05-14T11:25:43.640566+0000","last_peered":"2024-05-14T11:25:43.640566+0000","last_clean":"2024-05-14T11:25:43.640566+0000","last_became_active":"2024-05-14T11:25:42.460800+0000","last_became_peered":"2024-05-14T11:25:42.460800+0000","last_unstale":"2024-05-14T11:25:43.640566+0000","last_undegraded":"2024-05-14T11:25:43.640566+0000","last_fullsized":"2024-05-14T11:25:43.640566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:40.533349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640316+0000","last_change":"2024-05-14T11:25:41.451689+0000","last_active":"2024-05-14T11:25:43.640316+0000","last_peered":"2024-05-14T11:25:43.640316+0000","last_clean":"2024-05-14T11:25:43.640316+0000","last_became_active":"2024-05-14T11:25:41.451397+0000","last_became_peered":"2024-05-14T11:25:41.451397+0000","last_unstale":"2024-05-14T11:25:43.640316+0000","last_undegraded":"2024-05-14T11:25:43.640316+0000","last_fullsized":"2024-05-14T11:25:43.640316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:30:01.569546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640412+0000","last_change":"2024-05-14T11:25:42.438561+0000","last_active":"2024-05-14T11:25:43.640412+0000","last_peered":"2024-05-14T11:25:43.640412+0000","last_clean":"2024-05-14T11:25:43.640412+0000","last_became_active":"2024-05-14T11:25:42.438224+0000","last_became_peered":"2024-05-14T11:25:42.438224+0000","last_unstale":"2024-05-14T11:25:43.640412+0000","last_undegraded":"2024-05-14T11:25:43.640412+0000","last_fullsized":"2024-05-14T11:25:43.640412+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:17.791776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640458+0000","last_change":"2024-05-14T11:25:41.451008+0000","last_active":"2024-05-14T11:25:43.640458+0000","last_peered":"2024-05-14T11:25:43.640458+0000","last_clean":"2024-05-14T11:25:43.640458+0000","last_became_active":"2024-05-14T11:25:41.450664+0000","last_became_peered":"2024-05-14T11:25:41.450664+0000","last_unstale":"2024-05-14T11:25:43.640458+0000","last_undegraded":"2024-05-14T11:25:43.640458+0000","last_fullsized":"2024-05-14T11:25:43.640458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:34:06.193622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640855+0000","last_change":"2024-05-14T11:25:42.460450+0000","last_active":"2024-05-14T11:25:43.640855+0000","last_peered":"2024-05-14T11:25:43.640855+0000","last_clean":"2024-05-14T11:25:43.640855+0000","last_became_active":"2024-05-14T11:25:42.459143+0000","last_became_peered":"2024-05-14T11:25:42.459143+0000","last_unstale":"2024-05-14T11:25:43.640855+0000","last_undegraded":"2024-05-14T11:25:43.640855+0000","last_fullsized":"2024-05-14T11:25:43.640855+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:41.153552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639708+0000","last_change":"2024-05-14T11:25:41.440845+0000","last_active":"2024-05-14T11:25:43.639708+0000","last_peered":"2024-05-14T11:25:43.639708+0000","last_clean":"2024-05-14T11:25:43.639708+0000","last_became_active":"2024-05-14T11:25:41.440569+0000","last_became_peered":"2024-05-14T11:25:41.440569+0000","last_unstale":"2024-05-14T11:25:43.639708+0000","last_undegraded":"2024-05-14T11:25:43.639708+0000","last_fullsized":"2024-05-14T11:25:43.639708+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:15.436967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.446968+0000","last_change":"2024-05-14T11:25:42.456472+0000","last_active":"2024-05-14T11:25:43.446968+0000","last_peered":"2024-05-14T11:25:43.446968+0000","last_clean":"2024-05-14T11:25:43.446968+0000","last_became_active":"2024-05-14T11:25:42.454755+0000","last_became_peered":"2024-05-14T11:25:42.454755+0000","last_unstale":"2024-05-14T11:25:43.446968+0000","last_undegraded":"2024-05-14T11:25:43.446968+0000","last_fullsized":"2024-05-14T11:25:43.446968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640062+0000","last_change":"2024-05-14T11:25:41.442029+0000","last_active":"2024-05-14T11:25:43.640062+0000","last_peered":"2024-05-14T11:25:43.640062+0000","last_clean":"2024-05-14T11:25:43.640062+0000","last_became_active":"2024-05-14T11:25:41.441709+0000","last_became_peered":"2024-05-14T11:25:41.441709+0000","last_unstale":"2024-05-14T11:25:43.640062+0000","last_undegraded":"2024-05-14T11:25:43.640062+0000","last_fullsized":"2024-05-14T11:25:43.640062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:12.727764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639313+0000","last_change":"2024-05-14T11:25:42.449228+0000","last_active":"2024-05-14T11:25:43.639313+0000","last_peered":"2024-05-14T11:25:43.639313+0000","last_clean":"2024-05-14T11:25:43.639313+0000","last_became_active":"2024-05-14T11:25:42.448920+0000","last_became_peered":"2024-05-14T11:25:42.448920+0000","last_unstale":"2024-05-14T11:25:43.639313+0000","last_undegraded":"2024-05-14T11:25:43.639313+0000","last_fullsized":"2024-05-14T11:25:43.639313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:28.042844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640435+0000","last_change":"2024-05-14T11:25:41.446609+0000","last_active":"2024-05-14T11:25:43.640435+0000","last_peered":"2024-05-14T11:25:43.640435+0000","last_clean":"2024-05-14T11:25:43.640435+0000","last_became_active":"2024-05-14T11:25:41.446318+0000","last_became_peered":"2024-05-14T11:25:41.446318+0000","last_unstale":"2024-05-14T11:25:43.640435+0000","last_undegraded":"2024-05-14T11:25:43.640435+0000","last_fullsized":"2024-05-14T11:25:43.640435+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:23.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444577+0000","last_change":"2024-05-14T11:25:42.444498+0000","last_active":"2024-05-14T11:25:43.444577+0000","last_peered":"2024-05-14T11:25:43.444577+0000","last_clean":"2024-05-14T11:25:43.444577+0000","last_became_active":"2024-05-14T11:25:42.444139+0000","last_became_peered":"2024-05-14T11:25:42.444139+0000","last_unstale":"2024-05-14T11:25:43.444577+0000","last_undegraded":"2024-05-14T11:25:43.444577+0000","last_fullsized":"2024-05-14T11:25:43.444577+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:35:23.524571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639848+0000","last_change":"2024-05-14T11:25:41.440411+0000","last_active":"2024-05-14T11:25:43.639848+0000","last_peered":"2024-05-14T11:25:43.639848+0000","last_clean":"2024-05-14T11:25:43.639848+0000","last_became_active":"2024-05-14T11:25:41.436144+0000","last_became_peered":"2024-05-14T11:25:41.436144+0000","last_unstale":"2024-05-14T11:25:43.639848+0000","last_undegraded":"2024-05-14T11:25:43.639848+0000","last_fullsized":"2024-05-14T11:25:43.639848+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:26.027238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640863+0000","last_change":"2024-05-14T11:25:42.461740+0000","last_active":"2024-05-14T11:25:43.640863+0000","last_peered":"2024-05-14T11:25:43.640863+0000","last_clean":"2024-05-14T11:25:43.640863+0000","last_became_active":"2024-05-14T11:25:42.461303+0000","last_became_peered":"2024-05-14T11:25:42.461303+0000","last_unstale":"2024-05-14T11:25:43.640863+0000","last_undegraded":"2024-05-14T11:25:43.640863+0000","last_fullsized":"2024-05-14T11:25:43.640863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:52.918452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640082+0000","last_change":"2024-05-14T11:25:41.451489+0000","last_active":"2024-05-14T11:25:43.640082+0000","last_peered":"2024-05-14T11:25:43.640082+0000","last_clean":"2024-05-14T11:25:43.640082+0000","last_became_active":"2024-05-14T11:25:41.449899+0000","last_became_peered":"2024-05-14T11:25:41.449899+0000","last_unstale":"2024-05-14T11:25:43.640082+0000","last_undegraded":"2024-05-14T11:25:43.640082+0000","last_fullsized":"2024-05-14T11:25:43.640082+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:53:48.683805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448822+0000","last_change":"2024-05-14T11:25:42.444839+0000","last_active":"2024-05-14T11:25:43.448822+0000","last_peered":"2024-05-14T11:25:43.448822+0000","last_clean":"2024-05-14T11:25:43.448822+0000","last_became_active":"2024-05-14T11:25:42.444473+0000","last_became_peered":"2024-05-14T11:25:42.444473+0000","last_unstale":"2024-05-14T11:25:43.448822+0000","last_undegraded":"2024-05-14T11:25:43.448822+0000","last_fullsized":"2024-05-14T11:25:43.448822+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445012+0000","last_change":"2024-05-14T11:25:41.450018+0000","last_active":"2024-05-14T11:25:43.445012+0000","last_peered":"2024-05-14T11:25:43.445012+0000","last_clean":"2024-05-14T11:25:43.445012+0000","last_became_active":"2024-05-14T11:25:41.449676+0000","last_became_peered":"2024-05-14T11:25:41.449676+0000","last_unstale":"2024-05-14T11:25:43.445012+0000","last_undegraded":"2024-05-14T11:25:43.445012+0000","last_fullsized":"2024-05-14T11:25:43.445012+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:43.613522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452377+0000","last_change":"2024-05-14T11:25:42.445131+0000","last_active":"2024-05-14T11:25:43.452377+0000","last_peered":"2024-05-14T11:25:43.452377+0000","last_clean":"2024-05-14T11:25:43.452377+0000","last_became_active":"2024-05-14T11:25:42.444777+0000","last_became_peered":"2024-05-14T11:25:42.444777+0000","last_unstale":"2024-05-14T11:25:43.452377+0000","last_undegraded":"2024-05-14T11:25:43.452377+0000","last_fullsized":"2024-05-14T11:25:43.452377+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:11:01.891209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641446+0000","last_change":"2024-05-14T11:25:41.453270+0000","last_active":"2024-05-14T11:25:43.641446+0000","last_peered":"2024-05-14T11:25:43.641446+0000","last_clean":"2024-05-14T11:25:43.641446+0000","last_became_active":"2024-05-14T11:25:41.452958+0000","last_became_peered":"2024-05-14T11:25:41.452958+0000","last_unstale":"2024-05-14T11:25:43.641446+0000","last_undegraded":"2024-05-14T11:25:43.641446+0000","last_fullsized":"2024-05-14T11:25:43.641446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:20.013256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640986+0000","last_change":"2024-05-14T11:25:42.459804+0000","last_active":"2024-05-14T11:25:43.640986+0000","last_peered":"2024-05-14T11:25:43.640986+0000","last_clean":"2024-05-14T11:25:43.640986+0000","last_became_active":"2024-05-14T11:25:42.459475+0000","last_became_peered":"2024-05-14T11:25:42.459475+0000","last_unstale":"2024-05-14T11:25:43.640986+0000","last_undegraded":"2024-05-14T11:25:43.640986+0000","last_fullsized":"2024-05-14T11:25:43.640986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:07.864810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641046+0000","last_change":"2024-05-14T11:25:41.450378+0000","last_active":"2024-05-14T11:25:43.641046+0000","last_peered":"2024-05-14T11:25:43.641046+0000","last_clean":"2024-05-14T11:25:43.641046+0000","last_became_active":"2024-05-14T11:25:41.450018+0000","last_became_peered":"2024-05-14T11:25:41.450018+0000","last_unstale":"2024-05-14T11:25:43.641046+0000","last_undegraded":"2024-05-14T11:25:43.641046+0000","last_fullsized":"2024-05-14T11:25:43.641046+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:30.405786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451856+0000","last_change":"2024-05-14T11:25:42.446288+0000","last_active":"2024-05-14T11:25:43.451856+0000","last_peered":"2024-05-14T11:25:43.451856+0000","last_clean":"2024-05-14T11:25:43.451856+0000","last_became_active":"2024-05-14T11:25:42.445982+0000","last_became_peered":"2024-05-14T11:25:42.445982+0000","last_unstale":"2024-05-14T11:25:43.451856+0000","last_undegraded":"2024-05-14T11:25:43.451856+0000","last_fullsized":"2024-05-14T11:25:43.451856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:09.568395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451948+0000","last_change":"2024-05-14T11:25:41.446589+0000","last_active":"2024-05-14T11:25:43.451948+0000","last_peered":"2024-05-14T11:25:43.451948+0000","last_clean":"2024-05-14T11:25:43.451948+0000","last_became_active":"2024-05-14T11:25:41.446305+0000","last_became_peered":"2024-05-14T11:25:41.446305+0000","last_unstale":"2024-05-14T11:25:43.451948+0000","last_undegraded":"2024-05-14T11:25:43.451948+0000","last_fullsized":"2024-05-14T11:25:43.451948+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:22:33.461147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639831+0000","last_change":"2024-05-14T11:25:42.460267+0000","last_active":"2024-05-14T11:25:43.639831+0000","last_peered":"2024-05-14T11:25:43.639831+0000","last_clean":"2024-05-14T11:25:43.639831+0000","last_became_active":"2024-05-14T11:25:42.459964+0000","last_became_peered":"2024-05-14T11:25:42.459964+0000","last_unstale":"2024-05-14T11:25:43.639831+0000","last_undegraded":"2024-05-14T11:25:43.639831+0000","last_fullsized":"2024-05-14T11:25:43.639831+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:49:46.489691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640870+0000","last_change":"2024-05-14T11:25:41.449734+0000","last_active":"2024-05-14T11:25:43.640870+0000","last_peered":"2024-05-14T11:25:43.640870+0000","last_clean":"2024-05-14T11:25:43.640870+0000","last_became_active":"2024-05-14T11:25:41.449320+0000","last_became_peered":"2024-05-14T11:25:41.449320+0000","last_unstale":"2024-05-14T11:25:43.640870+0000","last_undegraded":"2024-05-14T11:25:43.640870+0000","last_fullsized":"2024-05-14T11:25:43.640870+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:41.376864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640354+0000","last_change":"2024-05-14T11:25:42.436596+0000","last_active":"2024-05-14T11:25:43.640354+0000","last_peered":"2024-05-14T11:25:43.640354+0000","last_clean":"2024-05-14T11:25:43.640354+0000","last_became_active":"2024-05-14T11:25:42.434944+0000","last_became_peered":"2024-05-14T11:25:42.434944+0000","last_unstale":"2024-05-14T11:25:43.640354+0000","last_undegraded":"2024-05-14T11:25:43.640354+0000","last_fullsized":"2024-05-14T11:25:43.640354+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:22:13.710478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639918+0000","last_change":"2024-05-14T11:25:41.439181+0000","last_active":"2024-05-14T11:25:43.639918+0000","last_peered":"2024-05-14T11:25:43.639918+0000","last_clean":"2024-05-14T11:25:43.639918+0000","last_became_active":"2024-05-14T11:25:41.438850+0000","last_became_peered":"2024-05-14T11:25:41.438850+0000","last_unstale":"2024-05-14T11:25:43.639918+0000","last_undegraded":"2024-05-14T11:25:43.639918+0000","last_fullsized":"2024-05-14T11:25:43.639918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:27.040144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640955+0000","last_change":"2024-05-14T11:25:42.458286+0000","last_active":"2024-05-14T11:25:43.640955+0000","last_peered":"2024-05-14T11:25:43.640955+0000","last_clean":"2024-05-14T11:25:43.640955+0000","last_became_active":"2024-05-14T11:25:42.457881+0000","last_became_peered":"2024-05-14T11:25:42.457881+0000","last_unstale":"2024-05-14T11:25:43.640955+0000","last_undegraded":"2024-05-14T11:25:43.640955+0000","last_fullsized":"2024-05-14T11:25:43.640955+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:16.391415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639788+0000","last_change":"2024-05-14T11:25:41.450647+0000","last_active":"2024-05-14T11:25:43.639788+0000","last_peered":"2024-05-14T11:25:43.639788+0000","last_clean":"2024-05-14T11:25:43.639788+0000","last_became_active":"2024-05-14T11:25:41.450309+0000","last_became_peered":"2024-05-14T11:25:41.450309+0000","last_unstale":"2024-05-14T11:25:43.639788+0000","last_undegraded":"2024-05-14T11:25:43.639788+0000","last_fullsized":"2024-05-14T11:25:43.639788+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:23.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452634+0000","last_change":"2024-05-14T11:25:42.452679+0000","last_active":"2024-05-14T11:25:43.452634+0000","last_peered":"2024-05-14T11:25:43.452634+0000","last_clean":"2024-05-14T11:25:43.452634+0000","last_became_active":"2024-05-14T11:25:42.452361+0000","last_became_peered":"2024-05-14T11:25:42.452361+0000","last_unstale":"2024-05-14T11:25:43.452634+0000","last_undegraded":"2024-05-14T11:25:43.452634+0000","last_fullsized":"2024-05-14T11:25:43.452634+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:09.546113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452165+0000","last_change":"2024-05-14T11:25:41.442252+0000","last_active":"2024-05-14T11:25:43.452165+0000","last_peered":"2024-05-14T11:25:43.452165+0000","last_clean":"2024-05-14T11:25:43.452165+0000","last_became_active":"2024-05-14T11:25:41.441969+0000","last_became_peered":"2024-05-14T11:25:41.441969+0000","last_unstale":"2024-05-14T11:25:43.452165+0000","last_undegraded":"2024-05-14T11:25:43.452165+0000","last_fullsized":"2024-05-14T11:25:43.452165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:55.071938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639810+0000","last_change":"2024-05-14T11:25:42.445337+0000","last_active":"2024-05-14T11:25:43.639810+0000","last_peered":"2024-05-14T11:25:43.639810+0000","last_clean":"2024-05-14T11:25:43.639810+0000","last_became_active":"2024-05-14T11:25:42.444949+0000","last_became_peered":"2024-05-14T11:25:42.444949+0000","last_unstale":"2024-05-14T11:25:43.639810+0000","last_undegraded":"2024-05-14T11:25:43.639810+0000","last_fullsized":"2024-05-14T11:25:43.639810+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447047+0000","last_change":"2024-05-14T11:25:41.438532+0000","last_active":"2024-05-14T11:25:43.447047+0000","last_peered":"2024-05-14T11:25:43.447047+0000","last_clean":"2024-05-14T11:25:43.447047+0000","last_became_active":"2024-05-14T11:25:41.438203+0000","last_became_peered":"2024-05-14T11:25:41.438203+0000","last_unstale":"2024-05-14T11:25:43.447047+0000","last_undegraded":"2024-05-14T11:25:43.447047+0000","last_fullsized":"2024-05-14T11:25:43.447047+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:34:06.193622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641461+0000","last_change":"2024-05-14T11:25:42.457522+0000","last_active":"2024-05-14T11:25:43.641461+0000","last_peered":"2024-05-14T11:25:43.641461+0000","last_clean":"2024-05-14T11:25:43.641461+0000","last_became_active":"2024-05-14T11:25:42.457088+0000","last_became_peered":"2024-05-14T11:25:42.457088+0000","last_unstale":"2024-05-14T11:25:43.641461+0000","last_undegraded":"2024-05-14T11:25:43.641461+0000","last_fullsized":"2024-05-14T11:25:43.641461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:13:48.977220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447147+0000","last_change":"2024-05-14T11:25:41.440129+0000","last_active":"2024-05-14T11:25:43.447147+0000","last_peered":"2024-05-14T11:25:43.447147+0000","last_clean":"2024-05-14T11:25:43.447147+0000","last_became_active":"2024-05-14T11:25:41.439752+0000","last_became_peered":"2024-05-14T11:25:41.439752+0000","last_unstale":"2024-05-14T11:25:43.447147+0000","last_undegraded":"2024-05-14T11:25:43.447147+0000","last_fullsized":"2024-05-14T11:25:43.447147+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:15.436967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445186+0000","last_change":"2024-05-14T11:25:42.462505+0000","last_active":"2024-05-14T11:25:43.445186+0000","last_peered":"2024-05-14T11:25:43.445186+0000","last_clean":"2024-05-14T11:25:43.445186+0000","last_became_active":"2024-05-14T11:25:42.462134+0000","last_became_peered":"2024-05-14T11:25:42.462134+0000","last_unstale":"2024-05-14T11:25:43.445186+0000","last_undegraded":"2024-05-14T11:25:43.445186+0000","last_fullsized":"2024-05-14T11:25:43.445186+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:34:37.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452591+0000","last_change":"2024-05-14T11:25:41.445773+0000","last_active":"2024-05-14T11:25:43.452591+0000","last_peered":"2024-05-14T11:25:43.452591+0000","last_clean":"2024-05-14T11:25:43.452591+0000","last_became_active":"2024-05-14T11:25:41.445486+0000","last_became_peered":"2024-05-14T11:25:41.445486+0000","last_unstale":"2024-05-14T11:25:43.452591+0000","last_undegraded":"2024-05-14T11:25:43.452591+0000","last_fullsized":"2024-05-14T11:25:43.452591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:19:27.181952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452262+0000","last_change":"2024-05-14T11:25:42.442154+0000","last_active":"2024-05-14T11:25:43.452262+0000","last_peered":"2024-05-14T11:25:43.452262+0000","last_clean":"2024-05-14T11:25:43.452262+0000","last_became_active":"2024-05-14T11:25:42.441491+0000","last_became_peered":"2024-05-14T11:25:42.441491+0000","last_unstale":"2024-05-14T11:25:43.452262+0000","last_undegraded":"2024-05-14T11:25:43.452262+0000","last_fullsized":"2024-05-14T11:25:43.452262+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:34:42.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448655+0000","last_change":"2024-05-14T11:25:41.443751+0000","last_active":"2024-05-14T11:25:43.448655+0000","last_peered":"2024-05-14T11:25:43.448655+0000","last_clean":"2024-05-14T11:25:43.448655+0000","last_became_active":"2024-05-14T11:25:41.443434+0000","last_became_peered":"2024-05-14T11:25:41.443434+0000","last_unstale":"2024-05-14T11:25:43.448655+0000","last_undegraded":"2024-05-14T11:25:43.448655+0000","last_fullsized":"2024-05-14T11:25:43.448655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:16:59.944993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639790+0000","last_change":"2024-05-14T11:25:42.437081+0000","last_active":"2024-05-14T11:25:43.639790+0000","last_peered":"2024-05-14T11:25:43.639790+0000","last_clean":"2024-05-14T11:25:43.639790+0000","last_became_active":"2024-05-14T11:25:42.436710+0000","last_became_peered":"2024-05-14T11:25:42.436710+0000","last_unstale":"2024-05-14T11:25:43.639790+0000","last_undegraded":"2024-05-14T11:25:43.639790+0000","last_fullsized":"2024-05-14T11:25:43.639790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:07.864810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448590+0000","last_change":"2024-05-14T11:25:41.446643+0000","last_active":"2024-05-14T11:25:43.448590+0000","last_peered":"2024-05-14T11:25:43.448590+0000","last_clean":"2024-05-14T11:25:43.448590+0000","last_became_active":"2024-05-14T11:25:41.444544+0000","last_became_peered":"2024-05-14T11:25:41.444544+0000","last_unstale":"2024-05-14T11:25:43.448590+0000","last_undegraded":"2024-05-14T11:25:43.448590+0000","last_fullsized":"2024-05-14T11:25:43.448590+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:37.989036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640925+0000","last_change":"2024-05-14T11:25:42.462308+0000","last_active":"2024-05-14T11:25:43.640925+0000","last_peered":"2024-05-14T11:25:43.640925+0000","last_clean":"2024-05-14T11:25:43.640925+0000","last_became_active":"2024-05-14T11:25:42.461911+0000","last_became_peered":"2024-05-14T11:25:42.461911+0000","last_unstale":"2024-05-14T11:25:43.640925+0000","last_undegraded":"2024-05-14T11:25:43.640925+0000","last_fullsized":"2024-05-14T11:25:43.640925+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:49:46.489691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452039+0000","last_change":"2024-05-14T11:25:41.440667+0000","last_active":"2024-05-14T11:25:43.452039+0000","last_peered":"2024-05-14T11:25:43.452039+0000","last_clean":"2024-05-14T11:25:43.452039+0000","last_became_active":"2024-05-14T11:25:41.440174+0000","last_became_peered":"2024-05-14T11:25:41.440174+0000","last_unstale":"2024-05-14T11:25:43.452039+0000","last_undegraded":"2024-05-14T11:25:43.452039+0000","last_fullsized":"2024-05-14T11:25:43.452039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:32.578199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639567+0000","last_change":"2024-05-14T11:25:42.457633+0000","last_active":"2024-05-14T11:25:43.639567+0000","last_peered":"2024-05-14T11:25:43.639567+0000","last_clean":"2024-05-14T11:25:43.639567+0000","last_became_active":"2024-05-14T11:25:42.457350+0000","last_became_peered":"2024-05-14T11:25:42.457350+0000","last_unstale":"2024-05-14T11:25:43.639567+0000","last_undegraded":"2024-05-14T11:25:43.639567+0000","last_fullsized":"2024-05-14T11:25:43.639567+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639813+0000","last_change":"2024-05-14T11:25:41.450892+0000","last_active":"2024-05-14T11:25:43.639813+0000","last_peered":"2024-05-14T11:25:43.639813+0000","last_clean":"2024-05-14T11:25:43.639813+0000","last_became_active":"2024-05-14T11:25:41.450584+0000","last_became_peered":"2024-05-14T11:25:41.450584+0000","last_unstale":"2024-05-14T11:25:43.639813+0000","last_undegraded":"2024-05-14T11:25:43.639813+0000","last_fullsized":"2024-05-14T11:25:43.639813+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:47.402359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640791+0000","last_change":"2024-05-14T11:25:42.455949+0000","last_active":"2024-05-14T11:25:43.640791+0000","last_peered":"2024-05-14T11:25:43.640791+0000","last_clean":"2024-05-14T11:25:43.640791+0000","last_became_active":"2024-05-14T11:25:42.455604+0000","last_became_peered":"2024-05-14T11:25:42.455604+0000","last_unstale":"2024-05-14T11:25:43.640791+0000","last_undegraded":"2024-05-14T11:25:43.640791+0000","last_fullsized":"2024-05-14T11:25:43.640791+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:58.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639275+0000","last_change":"2024-05-14T11:25:41.442618+0000","last_active":"2024-05-14T11:25:43.639275+0000","last_peered":"2024-05-14T11:25:43.639275+0000","last_clean":"2024-05-14T11:25:43.639275+0000","last_became_active":"2024-05-14T11:25:41.439550+0000","last_became_peered":"2024-05-14T11:25:41.439550+0000","last_unstale":"2024-05-14T11:25:43.639275+0000","last_undegraded":"2024-05-14T11:25:43.639275+0000","last_fullsized":"2024-05-14T11:25:43.639275+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:05:31.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":279,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640515+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:25:43.640515+0000","last_peered":"2024-05-14T11:25:43.640515+0000","last_clean":"2024-05-14T11:25:43.640515+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:25:43.640515+0000","last_undegraded":"2024-05-14T11:25:43.640515+0000","last_fullsized":"2024-05-14T11:25:43.640515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1247310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":656748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1417216,"data_stored":656326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":657778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":657812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":656166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1413120,"data_stored":656188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1246556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:25:46.570 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:25:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 Processor -- start 2024-05-14T11:25:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 -- start start 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb85415db30 0x7fb8540aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 0x7fb85406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85406ff20 con 0x7fb8540d1670 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb8540ae100 con 0x7fb85415db30 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb85c400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb8540ae870 con 0x7fb85406c700 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb859a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb85415db30 0x7fb8540aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb859000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb8540d1670 0x7fb8540d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:37340/0 (socket says 172.21.15.116:37340) 2024-05-14T11:25:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.754+0000 7fb859000640 1 -- 172.21.15.116:0/1958862669 learned_addr learned my addr 172.21.15.116:0/1958862669 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb859a00640 1 -- 172.21.15.116:0/1958862669 >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 msgr2=0x7fb85406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb85a400640 1 --2- 172.21.15.116:0/1958862669 >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 0x7fb85406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb859a00640 1 --2- 172.21.15.116:0/1958862669 >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 0x7fb85406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb853e00640 1 -- 172.21.15.116:0/1958862669 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050328249 0 0) 0x7fb85406ff20 con 0x7fb8540d1670 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb859a00640 1 -- 172.21.15.116:0/1958862669 >> v1:172.21.15.116:6789/0 conn(0x7fb8540d1670 legacy=0x7fb8540d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb859a00640 1 -- 172.21.15.116:0/1958862669 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854070470 con 0x7fb85415db30 2024-05-14T11:25:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb85a400640 1 --2- 172.21.15.116:0/1958862669 >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 0x7fb85406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb859a00640 1 --2- 172.21.15.116:0/1958862669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb8540aeaa0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb8541626a0 tx=0x7fb84801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf96e2d497b36249 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb853e00640 1 -- 172.21.15.116:0/1958862669 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b0070 con 0x7fb85415db30 2024-05-14T11:25:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb853e00640 1 -- 172.21.15.116:0/1958862669 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8480ab600 con 0x7fb85415db30 2024-05-14T11:25:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb853e00640 1 -- 172.21.15.116:0/1958862669 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8480ab9d0 con 0x7fb85415db30 2024-05-14T11:25:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb85c400640 1 -- 172.21.15.116:0/1958862669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 msgr2=0x7fb8540aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.758+0000 7fb85c400640 1 --2- 172.21.15.116:0/1958862669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb8540aeaa0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb8541626a0 tx=0x7fb84801c3b0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- 172.21.15.116:0/1958862669 shutdown_connections 2024-05-14T11:25:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 --2- 172.21.15.116:0/1958862669 >> v2:172.21.15.149:3300/0 conn(0x7fb85406c700 0x7fb85406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 --2- 172.21.15.116:0/1958862669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb8540aeaa0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- 172.21.15.116:0/1958862669 >> 172.21.15.116:0/1958862669 conn(0x7fb8540b4620 msgr2=0x7fb8540b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- 172.21.15.116:0/1958862669 shutdown_connections 2024-05-14T11:25:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- 172.21.15.116:0/1958862669 wait complete. 2024-05-14T11:25:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 Processor -- start 2024-05-14T11:25:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- start start 2024-05-14T11:25:46.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 0x7fb8540cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 0x7fb85409a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb85416a770 con 0x7fb85415db30 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb85415df40 con 0x7fb85406c700 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.762+0000 7fb85c400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb85416a8f0 con 0x7fb8540d1670 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43374/0 (socket says 172.21.15.116:43374) 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 -- 172.21.15.116:0/3835941047 learned_addr learned my addr 172.21.15.116:0/3835941047 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb859000640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 0x7fb85409a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb859a00640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 0x7fb8540cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 -- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 msgr2=0x7fb85409a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 0x7fb85409a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 -- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 msgr2=0x7fb8540cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 0x7fb8540cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 -- 172.21.15.116:0/3835941047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8540ce030 con 0x7fb85415db30 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb859000640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 0x7fb85409a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:46.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb859a00640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 0x7fb8540cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85a400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00cdb0 tx=0x7fb84c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c003ba0 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c003d60 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004260 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8540ce310 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8540ce8d0 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8540cdda0 con 0x7fb85415db30 2024-05-14T11:25:46.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.766+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb84c004400 con 0x7fb85415db30 2024-05-14T11:25:46.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.770+0000 7fb851600640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 0x7fb834071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:46.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.770+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fb84c08fdc0 con 0x7fb85415db30 2024-05-14T11:25:46.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.770+0000 7fb859a00640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 0x7fb834071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:46.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.770+0000 7fb859a00640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 0x7fb834071940 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb8480ad000 tx=0x7fb848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:46.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.774+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb84c014030 con 0x7fb85415db30 2024-05-14T11:25:46.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.930+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb85406a9c0 con 0x7fb83406f430 2024-05-14T11:25:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb851600640 1 -- 172.21.15.116:0/3835941047 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7fb85406a9c0 con 0x7fb83406f430 2024-05-14T11:25:46.937 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 msgr2=0x7fb834071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 0x7fb834071940 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb8480ad000 tx=0x7fb848008040 comp rx=0 tx=0).stop 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 msgr2=0x7fb854150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00cdb0 tx=0x7fb84c00a650 comp rx=0 tx=0).stop 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 shutdown_connections 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb83406f430 0x7fb834071940 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb85415db30 0x7fb854150cf0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8540d1670 0x7fb85409a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 --2- 172.21.15.116:0/3835941047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85406c700 0x7fb8540cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 >> 172.21.15.116:0/3835941047 conn(0x7fb8540b4620 msgr2=0x7fb85415cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 shutdown_connections 2024-05-14T11:25:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:46.938+0000 7fb85c400640 1 -- 172.21.15.116:0/3835941047 wait complete. 2024-05-14T11:25:46.958 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-05-14T11:25:46.507208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261720,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713000,"statfs":{"total":773094113280,"available":772826112000,"internally_reserved":0,"allocated":12525568,"data_stored":6434884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001241"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448833+0000","last_change":"2024-05-14T11:25:42.447556+0000","last_active":"2024-05-14T11:25:43.448833+0000","last_peered":"2024-05-14T11:25:43.448833+0000","last_clean":"2024-05-14T11:25:43.448833+0000","last_became_active":"2024-05-14T11:25:42.447253+0000","last_became_peered":"2024-05-14T11:25:42.447253+0000","last_unstale":"2024-05-14T11:25:43.448833+0000","last_undegraded":"2024-05-14T11:25:43.448833+0000","last_fullsized":"2024-05-14T11:25:43.448833+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640228+0000","last_change":"2024-05-14T11:25:42.456318+0000","last_active":"2024-05-14T11:25:43.640228+0000","last_peered":"2024-05-14T11:25:43.640228+0000","last_clean":"2024-05-14T11:25:43.640228+0000","last_became_active":"2024-05-14T11:25:42.455767+0000","last_became_peered":"2024-05-14T11:25:42.455767+0000","last_unstale":"2024-05-14T11:25:43.640228+0000","last_undegraded":"2024-05-14T11:25:43.640228+0000","last_fullsized":"2024-05-14T11:25:43.640228+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:35.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640181+0000","last_change":"2024-05-14T11:25:42.443177+0000","last_active":"2024-05-14T11:25:43.640181+0000","last_peered":"2024-05-14T11:25:43.640181+0000","last_clean":"2024-05-14T11:25:43.640181+0000","last_became_active":"2024-05-14T11:25:42.442819+0000","last_became_peered":"2024-05-14T11:25:42.442819+0000","last_unstale":"2024-05-14T11:25:43.640181+0000","last_undegraded":"2024-05-14T11:25:43.640181+0000","last_fullsized":"2024-05-14T11:25:43.640181+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:16.135038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640422+0000","last_change":"2024-05-14T11:25:42.457086+0000","last_active":"2024-05-14T11:25:43.640422+0000","last_peered":"2024-05-14T11:25:43.640422+0000","last_clean":"2024-05-14T11:25:43.640422+0000","last_became_active":"2024-05-14T11:25:42.453585+0000","last_became_peered":"2024-05-14T11:25:42.453585+0000","last_unstale":"2024-05-14T11:25:43.640422+0000","last_undegraded":"2024-05-14T11:25:43.640422+0000","last_fullsized":"2024-05-14T11:25:43.640422+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:09:16.135038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448420+0000","last_change":"2024-05-14T11:25:42.439427+0000","last_active":"2024-05-14T11:25:43.448420+0000","last_peered":"2024-05-14T11:25:43.448420+0000","last_clean":"2024-05-14T11:25:43.448420+0000","last_became_active":"2024-05-14T11:25:42.438976+0000","last_became_peered":"2024-05-14T11:25:42.438976+0000","last_unstale":"2024-05-14T11:25:43.448420+0000","last_undegraded":"2024-05-14T11:25:43.448420+0000","last_fullsized":"2024-05-14T11:25:43.448420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452029+0000","last_change":"2024-05-14T11:25:42.440982+0000","last_active":"2024-05-14T11:25:43.452029+0000","last_peered":"2024-05-14T11:25:43.452029+0000","last_clean":"2024-05-14T11:25:43.452029+0000","last_became_active":"2024-05-14T11:25:42.440628+0000","last_became_peered":"2024-05-14T11:25:42.440628+0000","last_unstale":"2024-05-14T11:25:43.452029+0000","last_undegraded":"2024-05-14T11:25:43.452029+0000","last_fullsized":"2024-05-14T11:25:43.452029+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:46:54.833574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451909+0000","last_change":"2024-05-14T11:25:42.458739+0000","last_active":"2024-05-14T11:25:43.451909+0000","last_peered":"2024-05-14T11:25:43.451909+0000","last_clean":"2024-05-14T11:25:43.451909+0000","last_became_active":"2024-05-14T11:25:42.458431+0000","last_became_peered":"2024-05-14T11:25:42.458431+0000","last_unstale":"2024-05-14T11:25:43.451909+0000","last_undegraded":"2024-05-14T11:25:43.451909+0000","last_fullsized":"2024-05-14T11:25:43.451909+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:29:02.860814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639338+0000","last_change":"2024-05-14T11:25:42.447950+0000","last_active":"2024-05-14T11:25:43.639338+0000","last_peered":"2024-05-14T11:25:43.639338+0000","last_clean":"2024-05-14T11:25:43.639338+0000","last_became_active":"2024-05-14T11:25:42.439231+0000","last_became_peered":"2024-05-14T11:25:42.439231+0000","last_unstale":"2024-05-14T11:25:43.639338+0000","last_undegraded":"2024-05-14T11:25:43.639338+0000","last_fullsized":"2024-05-14T11:25:43.639338+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:52.918452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641443+0000","last_change":"2024-05-14T11:25:42.454508+0000","last_active":"2024-05-14T11:25:43.641443+0000","last_peered":"2024-05-14T11:25:43.641443+0000","last_clean":"2024-05-14T11:25:43.641443+0000","last_became_active":"2024-05-14T11:25:42.454124+0000","last_became_peered":"2024-05-14T11:25:42.454124+0000","last_unstale":"2024-05-14T11:25:43.641443+0000","last_undegraded":"2024-05-14T11:25:43.641443+0000","last_fullsized":"2024-05-14T11:25:43.641443+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:22:13.710478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639860+0000","last_change":"2024-05-14T11:25:42.450093+0000","last_active":"2024-05-14T11:25:43.639860+0000","last_peered":"2024-05-14T11:25:43.639860+0000","last_clean":"2024-05-14T11:25:43.639860+0000","last_became_active":"2024-05-14T11:25:42.449773+0000","last_became_peered":"2024-05-14T11:25:42.449773+0000","last_unstale":"2024-05-14T11:25:43.639860+0000","last_undegraded":"2024-05-14T11:25:43.639860+0000","last_fullsized":"2024-05-14T11:25:43.639860+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448490+0000","last_change":"2024-05-14T11:25:42.463006+0000","last_active":"2024-05-14T11:25:43.448490+0000","last_peered":"2024-05-14T11:25:43.448490+0000","last_clean":"2024-05-14T11:25:43.448490+0000","last_became_active":"2024-05-14T11:25:42.462703+0000","last_became_peered":"2024-05-14T11:25:42.462703+0000","last_unstale":"2024-05-14T11:25:43.448490+0000","last_undegraded":"2024-05-14T11:25:43.448490+0000","last_fullsized":"2024-05-14T11:25:43.448490+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452524+0000","last_change":"2024-05-14T11:25:42.446705+0000","last_active":"2024-05-14T11:25:43.452524+0000","last_peered":"2024-05-14T11:25:43.452524+0000","last_clean":"2024-05-14T11:25:43.452524+0000","last_became_active":"2024-05-14T11:25:42.446406+0000","last_became_peered":"2024-05-14T11:25:42.446406+0000","last_unstale":"2024-05-14T11:25:43.452524+0000","last_undegraded":"2024-05-14T11:25:43.452524+0000","last_fullsized":"2024-05-14T11:25:43.452524+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:12:51.273783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639385+0000","last_change":"2024-05-14T11:25:42.458936+0000","last_active":"2024-05-14T11:25:43.639385+0000","last_peered":"2024-05-14T11:25:43.639385+0000","last_clean":"2024-05-14T11:25:43.639385+0000","last_became_active":"2024-05-14T11:25:42.458629+0000","last_became_peered":"2024-05-14T11:25:42.458629+0000","last_unstale":"2024-05-14T11:25:43.639385+0000","last_undegraded":"2024-05-14T11:25:43.639385+0000","last_fullsized":"2024-05-14T11:25:43.639385+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:13.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639717+0000","last_change":"2024-05-14T11:25:42.438206+0000","last_active":"2024-05-14T11:25:43.639717+0000","last_peered":"2024-05-14T11:25:43.639717+0000","last_clean":"2024-05-14T11:25:43.639717+0000","last_became_active":"2024-05-14T11:25:42.437473+0000","last_became_peered":"2024-05-14T11:25:42.437473+0000","last_unstale":"2024-05-14T11:25:43.639717+0000","last_undegraded":"2024-05-14T11:25:43.639717+0000","last_fullsized":"2024-05-14T11:25:43.639717+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:16.391415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448314+0000","last_change":"2024-05-14T11:25:42.453451+0000","last_active":"2024-05-14T11:25:43.448314+0000","last_peered":"2024-05-14T11:25:43.448314+0000","last_clean":"2024-05-14T11:25:43.448314+0000","last_became_active":"2024-05-14T11:25:42.451879+0000","last_became_peered":"2024-05-14T11:25:42.451879+0000","last_unstale":"2024-05-14T11:25:43.448314+0000","last_undegraded":"2024-05-14T11:25:43.448314+0000","last_fullsized":"2024-05-14T11:25:43.448314+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:24.831507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451852+0000","last_change":"2024-05-14T11:25:42.442433+0000","last_active":"2024-05-14T11:25:43.451852+0000","last_peered":"2024-05-14T11:25:43.451852+0000","last_clean":"2024-05-14T11:25:43.451852+0000","last_became_active":"2024-05-14T11:25:42.441863+0000","last_became_peered":"2024-05-14T11:25:42.441863+0000","last_unstale":"2024-05-14T11:25:43.451852+0000","last_undegraded":"2024-05-14T11:25:43.451852+0000","last_fullsized":"2024-05-14T11:25:43.451852+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:01.383403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640038+0000","last_change":"2024-05-14T11:25:42.464244+0000","last_active":"2024-05-14T11:25:43.640038+0000","last_peered":"2024-05-14T11:25:43.640038+0000","last_clean":"2024-05-14T11:25:43.640038+0000","last_became_active":"2024-05-14T11:25:42.463888+0000","last_became_peered":"2024-05-14T11:25:42.463888+0000","last_unstale":"2024-05-14T11:25:43.640038+0000","last_undegraded":"2024-05-14T11:25:43.640038+0000","last_fullsized":"2024-05-14T11:25:43.640038+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640291+0000","last_change":"2024-05-14T11:25:42.456055+0000","last_active":"2024-05-14T11:25:43.640291+0000","last_peered":"2024-05-14T11:25:43.640291+0000","last_clean":"2024-05-14T11:25:43.640291+0000","last_became_active":"2024-05-14T11:25:42.455319+0000","last_became_peered":"2024-05-14T11:25:42.455319+0000","last_unstale":"2024-05-14T11:25:43.640291+0000","last_undegraded":"2024-05-14T11:25:43.640291+0000","last_fullsized":"2024-05-14T11:25:43.640291+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:58.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.446968+0000","last_change":"2024-05-14T11:25:42.458336+0000","last_active":"2024-05-14T11:25:43.446968+0000","last_peered":"2024-05-14T11:25:43.446968+0000","last_clean":"2024-05-14T11:25:43.446968+0000","last_became_active":"2024-05-14T11:25:42.458037+0000","last_became_peered":"2024-05-14T11:25:42.458037+0000","last_unstale":"2024-05-14T11:25:43.446968+0000","last_undegraded":"2024-05-14T11:25:43.446968+0000","last_fullsized":"2024-05-14T11:25:43.446968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444825+0000","last_change":"2024-05-14T11:25:42.440552+0000","last_active":"2024-05-14T11:25:43.444825+0000","last_peered":"2024-05-14T11:25:43.444825+0000","last_clean":"2024-05-14T11:25:43.444825+0000","last_became_active":"2024-05-14T11:25:42.440229+0000","last_became_peered":"2024-05-14T11:25:42.440229+0000","last_unstale":"2024-05-14T11:25:43.444825+0000","last_undegraded":"2024-05-14T11:25:43.444825+0000","last_fullsized":"2024-05-14T11:25:43.444825+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:18.387079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640778+0000","last_change":"2024-05-14T11:25:42.454795+0000","last_active":"2024-05-14T11:25:43.640778+0000","last_peered":"2024-05-14T11:25:43.640778+0000","last_clean":"2024-05-14T11:25:43.640778+0000","last_became_active":"2024-05-14T11:25:42.454401+0000","last_became_peered":"2024-05-14T11:25:42.454401+0000","last_unstale":"2024-05-14T11:25:43.640778+0000","last_undegraded":"2024-05-14T11:25:43.640778+0000","last_fullsized":"2024-05-14T11:25:43.640778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:55:35.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448516+0000","last_change":"2024-05-14T11:25:42.445590+0000","last_active":"2024-05-14T11:25:43.448516+0000","last_peered":"2024-05-14T11:25:43.448516+0000","last_clean":"2024-05-14T11:25:43.448516+0000","last_became_active":"2024-05-14T11:25:42.445285+0000","last_became_peered":"2024-05-14T11:25:42.445285+0000","last_unstale":"2024-05-14T11:25:43.448516+0000","last_undegraded":"2024-05-14T11:25:43.448516+0000","last_fullsized":"2024-05-14T11:25:43.448516+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640133+0000","last_change":"2024-05-14T11:25:42.458371+0000","last_active":"2024-05-14T11:25:43.640133+0000","last_peered":"2024-05-14T11:25:43.640133+0000","last_clean":"2024-05-14T11:25:43.640133+0000","last_became_active":"2024-05-14T11:25:42.458044+0000","last_became_peered":"2024-05-14T11:25:42.458044+0000","last_unstale":"2024-05-14T11:25:43.640133+0000","last_undegraded":"2024-05-14T11:25:43.640133+0000","last_fullsized":"2024-05-14T11:25:43.640133+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:56.412006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640062+0000","last_change":"2024-05-14T11:25:42.452231+0000","last_active":"2024-05-14T11:25:43.640062+0000","last_peered":"2024-05-14T11:25:43.640062+0000","last_clean":"2024-05-14T11:25:43.640062+0000","last_became_active":"2024-05-14T11:25:42.451867+0000","last_became_peered":"2024-05-14T11:25:42.451867+0000","last_unstale":"2024-05-14T11:25:43.640062+0000","last_undegraded":"2024-05-14T11:25:43.640062+0000","last_fullsized":"2024-05-14T11:25:43.640062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:42.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640662+0000","last_change":"2024-05-14T11:25:42.460533+0000","last_active":"2024-05-14T11:25:43.640662+0000","last_peered":"2024-05-14T11:25:43.640662+0000","last_clean":"2024-05-14T11:25:43.640662+0000","last_became_active":"2024-05-14T11:25:42.460097+0000","last_became_peered":"2024-05-14T11:25:42.460097+0000","last_unstale":"2024-05-14T11:25:43.640662+0000","last_undegraded":"2024-05-14T11:25:43.640662+0000","last_fullsized":"2024-05-14T11:25:43.640662+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:56.550588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445040+0000","last_change":"2024-05-14T11:25:42.456247+0000","last_active":"2024-05-14T11:25:43.445040+0000","last_peered":"2024-05-14T11:25:43.445040+0000","last_clean":"2024-05-14T11:25:43.445040+0000","last_became_active":"2024-05-14T11:25:42.455636+0000","last_became_peered":"2024-05-14T11:25:42.455636+0000","last_unstale":"2024-05-14T11:25:43.445040+0000","last_undegraded":"2024-05-14T11:25:43.445040+0000","last_fullsized":"2024-05-14T11:25:43.445040+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:09:22.975248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447227+0000","last_change":"2024-05-14T11:25:42.449933+0000","last_active":"2024-05-14T11:25:43.447227+0000","last_peered":"2024-05-14T11:25:43.447227+0000","last_clean":"2024-05-14T11:25:43.447227+0000","last_became_active":"2024-05-14T11:25:42.449626+0000","last_became_peered":"2024-05-14T11:25:42.449626+0000","last_unstale":"2024-05-14T11:25:43.447227+0000","last_undegraded":"2024-05-14T11:25:43.447227+0000","last_fullsized":"2024-05-14T11:25:43.447227+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452508+0000","last_change":"2024-05-14T11:25:42.467028+0000","last_active":"2024-05-14T11:25:43.452508+0000","last_peered":"2024-05-14T11:25:43.452508+0000","last_clean":"2024-05-14T11:25:43.452508+0000","last_became_active":"2024-05-14T11:25:42.464463+0000","last_became_peered":"2024-05-14T11:25:42.464463+0000","last_unstale":"2024-05-14T11:25:43.452508+0000","last_undegraded":"2024-05-14T11:25:43.452508+0000","last_fullsized":"2024-05-14T11:25:43.452508+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:27:38.876560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444848+0000","last_change":"2024-05-14T11:25:42.456410+0000","last_active":"2024-05-14T11:25:43.444848+0000","last_peered":"2024-05-14T11:25:43.444848+0000","last_clean":"2024-05-14T11:25:43.444848+0000","last_became_active":"2024-05-14T11:25:42.456090+0000","last_became_peered":"2024-05-14T11:25:42.456090+0000","last_unstale":"2024-05-14T11:25:43.444848+0000","last_undegraded":"2024-05-14T11:25:43.444848+0000","last_fullsized":"2024-05-14T11:25:43.444848+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:32.551235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640144+0000","last_change":"2024-05-14T11:25:42.438603+0000","last_active":"2024-05-14T11:25:43.640144+0000","last_peered":"2024-05-14T11:25:43.640144+0000","last_clean":"2024-05-14T11:25:43.640144+0000","last_became_active":"2024-05-14T11:25:42.437916+0000","last_became_peered":"2024-05-14T11:25:42.437916+0000","last_unstale":"2024-05-14T11:25:43.640144+0000","last_undegraded":"2024-05-14T11:25:43.640144+0000","last_fullsized":"2024-05-14T11:25:43.640144+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:33:56.412006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640817+0000","last_change":"2024-05-14T11:25:42.463056+0000","last_active":"2024-05-14T11:25:43.640817+0000","last_peered":"2024-05-14T11:25:43.640817+0000","last_clean":"2024-05-14T11:25:43.640817+0000","last_became_active":"2024-05-14T11:25:42.462760+0000","last_became_peered":"2024-05-14T11:25:42.462760+0000","last_unstale":"2024-05-14T11:25:43.640817+0000","last_undegraded":"2024-05-14T11:25:43.640817+0000","last_fullsized":"2024-05-14T11:25:43.640817+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:27.505782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640256+0000","last_change":"2024-05-14T11:25:42.451960+0000","last_active":"2024-05-14T11:25:43.640256+0000","last_peered":"2024-05-14T11:25:43.640256+0000","last_clean":"2024-05-14T11:25:43.640256+0000","last_became_active":"2024-05-14T11:25:42.451595+0000","last_became_peered":"2024-05-14T11:25:42.451595+0000","last_unstale":"2024-05-14T11:25:43.640256+0000","last_undegraded":"2024-05-14T11:25:43.640256+0000","last_fullsized":"2024-05-14T11:25:43.640256+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:15:48.415733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639643+0000","last_change":"2024-05-14T11:25:42.448325+0000","last_active":"2024-05-14T11:25:43.639643+0000","last_peered":"2024-05-14T11:25:43.639643+0000","last_clean":"2024-05-14T11:25:43.639643+0000","last_became_active":"2024-05-14T11:25:42.446632+0000","last_became_peered":"2024-05-14T11:25:42.446632+0000","last_unstale":"2024-05-14T11:25:43.639643+0000","last_undegraded":"2024-05-14T11:25:43.639643+0000","last_fullsized":"2024-05-14T11:25:43.639643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447206+0000","last_change":"2024-05-14T11:25:41.447134+0000","last_active":"2024-05-14T11:25:43.447206+0000","last_peered":"2024-05-14T11:25:43.447206+0000","last_clean":"2024-05-14T11:25:43.447206+0000","last_became_active":"2024-05-14T11:25:41.446765+0000","last_became_peered":"2024-05-14T11:25:41.446765+0000","last_unstale":"2024-05-14T11:25:43.447206+0000","last_undegraded":"2024-05-14T11:25:43.447206+0000","last_fullsized":"2024-05-14T11:25:43.447206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:54:03.068472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452554+0000","last_change":"2024-05-14T11:25:42.453611+0000","last_active":"2024-05-14T11:25:43.452554+0000","last_peered":"2024-05-14T11:25:43.452554+0000","last_clean":"2024-05-14T11:25:43.452554+0000","last_became_active":"2024-05-14T11:25:42.453269+0000","last_became_peered":"2024-05-14T11:25:42.453269+0000","last_unstale":"2024-05-14T11:25:43.452554+0000","last_undegraded":"2024-05-14T11:25:43.452554+0000","last_fullsized":"2024-05-14T11:25:43.452554+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:37:46.535205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639790+0000","last_change":"2024-05-14T11:25:41.451612+0000","last_active":"2024-05-14T11:25:43.639790+0000","last_peered":"2024-05-14T11:25:43.639790+0000","last_clean":"2024-05-14T11:25:43.639790+0000","last_became_active":"2024-05-14T11:25:41.450327+0000","last_became_peered":"2024-05-14T11:25:41.450327+0000","last_unstale":"2024-05-14T11:25:43.639790+0000","last_undegraded":"2024-05-14T11:25:43.639790+0000","last_fullsized":"2024-05-14T11:25:43.639790+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:33.984835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640765+0000","last_change":"2024-05-14T11:25:42.463967+0000","last_active":"2024-05-14T11:25:43.640765+0000","last_peered":"2024-05-14T11:25:43.640765+0000","last_clean":"2024-05-14T11:25:43.640765+0000","last_became_active":"2024-05-14T11:25:42.463475+0000","last_became_peered":"2024-05-14T11:25:42.463475+0000","last_unstale":"2024-05-14T11:25:43.640765+0000","last_undegraded":"2024-05-14T11:25:43.640765+0000","last_fullsized":"2024-05-14T11:25:43.640765+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:36.318016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641304+0000","last_change":"2024-05-14T11:25:41.451594+0000","last_active":"2024-05-14T11:25:43.641304+0000","last_peered":"2024-05-14T11:25:43.641304+0000","last_clean":"2024-05-14T11:25:43.641304+0000","last_became_active":"2024-05-14T11:25:41.451120+0000","last_became_peered":"2024-05-14T11:25:41.451120+0000","last_unstale":"2024-05-14T11:25:43.641304+0000","last_undegraded":"2024-05-14T11:25:43.641304+0000","last_fullsized":"2024-05-14T11:25:43.641304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:55.802811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641502+0000","last_change":"2024-05-14T11:25:42.459423+0000","last_active":"2024-05-14T11:25:43.641502+0000","last_peered":"2024-05-14T11:25:43.641502+0000","last_clean":"2024-05-14T11:25:43.641502+0000","last_became_active":"2024-05-14T11:25:42.459090+0000","last_became_peered":"2024-05-14T11:25:42.459090+0000","last_unstale":"2024-05-14T11:25:43.641502+0000","last_undegraded":"2024-05-14T11:25:43.641502+0000","last_fullsized":"2024-05-14T11:25:43.641502+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:17.791776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639663+0000","last_change":"2024-05-14T11:25:41.447678+0000","last_active":"2024-05-14T11:25:43.639663+0000","last_peered":"2024-05-14T11:25:43.639663+0000","last_clean":"2024-05-14T11:25:43.639663+0000","last_became_active":"2024-05-14T11:25:41.447302+0000","last_became_peered":"2024-05-14T11:25:41.447302+0000","last_unstale":"2024-05-14T11:25:43.639663+0000","last_undegraded":"2024-05-14T11:25:43.639663+0000","last_fullsized":"2024-05-14T11:25:43.639663+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:47:20.434168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639255+0000","last_change":"2024-05-14T11:25:42.453356+0000","last_active":"2024-05-14T11:25:43.639255+0000","last_peered":"2024-05-14T11:25:43.639255+0000","last_clean":"2024-05-14T11:25:43.639255+0000","last_became_active":"2024-05-14T11:25:42.448013+0000","last_became_peered":"2024-05-14T11:25:42.448013+0000","last_unstale":"2024-05-14T11:25:43.639255+0000","last_undegraded":"2024-05-14T11:25:43.639255+0000","last_fullsized":"2024-05-14T11:25:43.639255+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:27.029938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448514+0000","last_change":"2024-05-14T11:25:41.442352+0000","last_active":"2024-05-14T11:25:43.448514+0000","last_peered":"2024-05-14T11:25:43.448514+0000","last_clean":"2024-05-14T11:25:43.448514+0000","last_became_active":"2024-05-14T11:25:41.442036+0000","last_became_peered":"2024-05-14T11:25:41.442036+0000","last_unstale":"2024-05-14T11:25:43.448514+0000","last_undegraded":"2024-05-14T11:25:43.448514+0000","last_fullsized":"2024-05-14T11:25:43.448514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:43:53.519056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448302+0000","last_change":"2024-05-14T11:25:42.455349+0000","last_active":"2024-05-14T11:25:43.448302+0000","last_peered":"2024-05-14T11:25:43.448302+0000","last_clean":"2024-05-14T11:25:43.448302+0000","last_became_active":"2024-05-14T11:25:42.454970+0000","last_became_peered":"2024-05-14T11:25:42.454970+0000","last_unstale":"2024-05-14T11:25:43.448302+0000","last_undegraded":"2024-05-14T11:25:43.448302+0000","last_fullsized":"2024-05-14T11:25:43.448302+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:13.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451996+0000","last_change":"2024-05-14T11:25:41.438326+0000","last_active":"2024-05-14T11:25:43.451996+0000","last_peered":"2024-05-14T11:25:43.451996+0000","last_clean":"2024-05-14T11:25:43.451996+0000","last_became_active":"2024-05-14T11:25:41.436383+0000","last_became_peered":"2024-05-14T11:25:41.436383+0000","last_unstale":"2024-05-14T11:25:43.451996+0000","last_undegraded":"2024-05-14T11:25:43.451996+0000","last_fullsized":"2024-05-14T11:25:43.451996+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:23:12.047661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640367+0000","last_change":"2024-05-14T11:25:42.451946+0000","last_active":"2024-05-14T11:25:43.640367+0000","last_peered":"2024-05-14T11:25:43.640367+0000","last_clean":"2024-05-14T11:25:43.640367+0000","last_became_active":"2024-05-14T11:25:42.450540+0000","last_became_peered":"2024-05-14T11:25:42.450540+0000","last_unstale":"2024-05-14T11:25:43.640367+0000","last_undegraded":"2024-05-14T11:25:43.640367+0000","last_fullsized":"2024-05-14T11:25:43.640367+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:13:48.977220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640248+0000","last_change":"2024-05-14T11:25:41.446237+0000","last_active":"2024-05-14T11:25:43.640248+0000","last_peered":"2024-05-14T11:25:43.640248+0000","last_clean":"2024-05-14T11:25:43.640248+0000","last_became_active":"2024-05-14T11:25:41.445944+0000","last_became_peered":"2024-05-14T11:25:41.445944+0000","last_unstale":"2024-05-14T11:25:43.640248+0000","last_undegraded":"2024-05-14T11:25:43.640248+0000","last_fullsized":"2024-05-14T11:25:43.640248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:54:03.068472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639425+0000","last_change":"2024-05-14T11:25:42.451009+0000","last_active":"2024-05-14T11:25:43.639425+0000","last_peered":"2024-05-14T11:25:43.639425+0000","last_clean":"2024-05-14T11:25:43.639425+0000","last_became_active":"2024-05-14T11:25:42.450662+0000","last_became_peered":"2024-05-14T11:25:42.450662+0000","last_unstale":"2024-05-14T11:25:43.639425+0000","last_undegraded":"2024-05-14T11:25:43.639425+0000","last_fullsized":"2024-05-14T11:25:43.639425+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:24.831507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452102+0000","last_change":"2024-05-14T11:25:41.441419+0000","last_active":"2024-05-14T11:25:43.452102+0000","last_peered":"2024-05-14T11:25:43.452102+0000","last_clean":"2024-05-14T11:25:43.452102+0000","last_became_active":"2024-05-14T11:25:41.441127+0000","last_became_peered":"2024-05-14T11:25:41.441127+0000","last_unstale":"2024-05-14T11:25:43.452102+0000","last_undegraded":"2024-05-14T11:25:43.452102+0000","last_fullsized":"2024-05-14T11:25:43.452102+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:49.973678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447108+0000","last_change":"2024-05-14T11:25:42.449597+0000","last_active":"2024-05-14T11:25:43.447108+0000","last_peered":"2024-05-14T11:25:43.447108+0000","last_clean":"2024-05-14T11:25:43.447108+0000","last_became_active":"2024-05-14T11:25:42.449079+0000","last_became_peered":"2024-05-14T11:25:42.449079+0000","last_unstale":"2024-05-14T11:25:43.447108+0000","last_undegraded":"2024-05-14T11:25:43.447108+0000","last_fullsized":"2024-05-14T11:25:43.447108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447207+0000","last_change":"2024-05-14T11:25:41.445403+0000","last_active":"2024-05-14T11:25:43.447207+0000","last_peered":"2024-05-14T11:25:43.447207+0000","last_clean":"2024-05-14T11:25:43.447207+0000","last_became_active":"2024-05-14T11:25:41.444690+0000","last_became_peered":"2024-05-14T11:25:41.444690+0000","last_unstale":"2024-05-14T11:25:43.447207+0000","last_undegraded":"2024-05-14T11:25:43.447207+0000","last_fullsized":"2024-05-14T11:25:43.447207+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:05:31.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640566+0000","last_change":"2024-05-14T11:25:42.461154+0000","last_active":"2024-05-14T11:25:43.640566+0000","last_peered":"2024-05-14T11:25:43.640566+0000","last_clean":"2024-05-14T11:25:43.640566+0000","last_became_active":"2024-05-14T11:25:42.460800+0000","last_became_peered":"2024-05-14T11:25:42.460800+0000","last_unstale":"2024-05-14T11:25:43.640566+0000","last_undegraded":"2024-05-14T11:25:43.640566+0000","last_fullsized":"2024-05-14T11:25:43.640566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:40.533349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640316+0000","last_change":"2024-05-14T11:25:41.451689+0000","last_active":"2024-05-14T11:25:43.640316+0000","last_peered":"2024-05-14T11:25:43.640316+0000","last_clean":"2024-05-14T11:25:43.640316+0000","last_became_active":"2024-05-14T11:25:41.451397+0000","last_became_peered":"2024-05-14T11:25:41.451397+0000","last_unstale":"2024-05-14T11:25:43.640316+0000","last_undegraded":"2024-05-14T11:25:43.640316+0000","last_fullsized":"2024-05-14T11:25:43.640316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:30:01.569546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640412+0000","last_change":"2024-05-14T11:25:42.438561+0000","last_active":"2024-05-14T11:25:43.640412+0000","last_peered":"2024-05-14T11:25:43.640412+0000","last_clean":"2024-05-14T11:25:43.640412+0000","last_became_active":"2024-05-14T11:25:42.438224+0000","last_became_peered":"2024-05-14T11:25:42.438224+0000","last_unstale":"2024-05-14T11:25:43.640412+0000","last_undegraded":"2024-05-14T11:25:43.640412+0000","last_fullsized":"2024-05-14T11:25:43.640412+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:39:17.791776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640458+0000","last_change":"2024-05-14T11:25:41.451008+0000","last_active":"2024-05-14T11:25:43.640458+0000","last_peered":"2024-05-14T11:25:43.640458+0000","last_clean":"2024-05-14T11:25:43.640458+0000","last_became_active":"2024-05-14T11:25:41.450664+0000","last_became_peered":"2024-05-14T11:25:41.450664+0000","last_unstale":"2024-05-14T11:25:43.640458+0000","last_undegraded":"2024-05-14T11:25:43.640458+0000","last_fullsized":"2024-05-14T11:25:43.640458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:34:06.193622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640855+0000","last_change":"2024-05-14T11:25:42.460450+0000","last_active":"2024-05-14T11:25:43.640855+0000","last_peered":"2024-05-14T11:25:43.640855+0000","last_clean":"2024-05-14T11:25:43.640855+0000","last_became_active":"2024-05-14T11:25:42.459143+0000","last_became_peered":"2024-05-14T11:25:42.459143+0000","last_unstale":"2024-05-14T11:25:43.640855+0000","last_undegraded":"2024-05-14T11:25:43.640855+0000","last_fullsized":"2024-05-14T11:25:43.640855+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:41.153552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639708+0000","last_change":"2024-05-14T11:25:41.440845+0000","last_active":"2024-05-14T11:25:43.639708+0000","last_peered":"2024-05-14T11:25:43.639708+0000","last_clean":"2024-05-14T11:25:43.639708+0000","last_became_active":"2024-05-14T11:25:41.440569+0000","last_became_peered":"2024-05-14T11:25:41.440569+0000","last_unstale":"2024-05-14T11:25:43.639708+0000","last_undegraded":"2024-05-14T11:25:43.639708+0000","last_fullsized":"2024-05-14T11:25:43.639708+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:15.436967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.446968+0000","last_change":"2024-05-14T11:25:42.456472+0000","last_active":"2024-05-14T11:25:43.446968+0000","last_peered":"2024-05-14T11:25:43.446968+0000","last_clean":"2024-05-14T11:25:43.446968+0000","last_became_active":"2024-05-14T11:25:42.454755+0000","last_became_peered":"2024-05-14T11:25:42.454755+0000","last_unstale":"2024-05-14T11:25:43.446968+0000","last_undegraded":"2024-05-14T11:25:43.446968+0000","last_fullsized":"2024-05-14T11:25:43.446968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:31.408487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640062+0000","last_change":"2024-05-14T11:25:41.442029+0000","last_active":"2024-05-14T11:25:43.640062+0000","last_peered":"2024-05-14T11:25:43.640062+0000","last_clean":"2024-05-14T11:25:43.640062+0000","last_became_active":"2024-05-14T11:25:41.441709+0000","last_became_peered":"2024-05-14T11:25:41.441709+0000","last_unstale":"2024-05-14T11:25:43.640062+0000","last_undegraded":"2024-05-14T11:25:43.640062+0000","last_fullsized":"2024-05-14T11:25:43.640062+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:05:12.727764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639313+0000","last_change":"2024-05-14T11:25:42.449228+0000","last_active":"2024-05-14T11:25:43.639313+0000","last_peered":"2024-05-14T11:25:43.639313+0000","last_clean":"2024-05-14T11:25:43.639313+0000","last_became_active":"2024-05-14T11:25:42.448920+0000","last_became_peered":"2024-05-14T11:25:42.448920+0000","last_unstale":"2024-05-14T11:25:43.639313+0000","last_undegraded":"2024-05-14T11:25:43.639313+0000","last_fullsized":"2024-05-14T11:25:43.639313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:28.042844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640435+0000","last_change":"2024-05-14T11:25:41.446609+0000","last_active":"2024-05-14T11:25:43.640435+0000","last_peered":"2024-05-14T11:25:43.640435+0000","last_clean":"2024-05-14T11:25:43.640435+0000","last_became_active":"2024-05-14T11:25:41.446318+0000","last_became_peered":"2024-05-14T11:25:41.446318+0000","last_unstale":"2024-05-14T11:25:43.640435+0000","last_undegraded":"2024-05-14T11:25:43.640435+0000","last_fullsized":"2024-05-14T11:25:43.640435+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:23.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.444577+0000","last_change":"2024-05-14T11:25:42.444498+0000","last_active":"2024-05-14T11:25:43.444577+0000","last_peered":"2024-05-14T11:25:43.444577+0000","last_clean":"2024-05-14T11:25:43.444577+0000","last_became_active":"2024-05-14T11:25:42.444139+0000","last_became_peered":"2024-05-14T11:25:42.444139+0000","last_unstale":"2024-05-14T11:25:43.444577+0000","last_undegraded":"2024-05-14T11:25:43.444577+0000","last_fullsized":"2024-05-14T11:25:43.444577+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:35:23.524571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639848+0000","last_change":"2024-05-14T11:25:41.440411+0000","last_active":"2024-05-14T11:25:43.639848+0000","last_peered":"2024-05-14T11:25:43.639848+0000","last_clean":"2024-05-14T11:25:43.639848+0000","last_became_active":"2024-05-14T11:25:41.436144+0000","last_became_peered":"2024-05-14T11:25:41.436144+0000","last_unstale":"2024-05-14T11:25:43.639848+0000","last_undegraded":"2024-05-14T11:25:43.639848+0000","last_fullsized":"2024-05-14T11:25:43.639848+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:10:26.027238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640863+0000","last_change":"2024-05-14T11:25:42.461740+0000","last_active":"2024-05-14T11:25:43.640863+0000","last_peered":"2024-05-14T11:25:43.640863+0000","last_clean":"2024-05-14T11:25:43.640863+0000","last_became_active":"2024-05-14T11:25:42.461303+0000","last_became_peered":"2024-05-14T11:25:42.461303+0000","last_unstale":"2024-05-14T11:25:43.640863+0000","last_undegraded":"2024-05-14T11:25:43.640863+0000","last_fullsized":"2024-05-14T11:25:43.640863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:52:52.918452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640082+0000","last_change":"2024-05-14T11:25:41.451489+0000","last_active":"2024-05-14T11:25:43.640082+0000","last_peered":"2024-05-14T11:25:43.640082+0000","last_clean":"2024-05-14T11:25:43.640082+0000","last_became_active":"2024-05-14T11:25:41.449899+0000","last_became_peered":"2024-05-14T11:25:41.449899+0000","last_unstale":"2024-05-14T11:25:43.640082+0000","last_undegraded":"2024-05-14T11:25:43.640082+0000","last_fullsized":"2024-05-14T11:25:43.640082+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T14:53:48.683805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448822+0000","last_change":"2024-05-14T11:25:42.444839+0000","last_active":"2024-05-14T11:25:43.448822+0000","last_peered":"2024-05-14T11:25:43.448822+0000","last_clean":"2024-05-14T11:25:43.448822+0000","last_became_active":"2024-05-14T11:25:42.444473+0000","last_became_peered":"2024-05-14T11:25:42.444473+0000","last_unstale":"2024-05-14T11:25:43.448822+0000","last_undegraded":"2024-05-14T11:25:43.448822+0000","last_fullsized":"2024-05-14T11:25:43.448822+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:56.805511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445012+0000","last_change":"2024-05-14T11:25:41.450018+0000","last_active":"2024-05-14T11:25:43.445012+0000","last_peered":"2024-05-14T11:25:43.445012+0000","last_clean":"2024-05-14T11:25:43.445012+0000","last_became_active":"2024-05-14T11:25:41.449676+0000","last_became_peered":"2024-05-14T11:25:41.449676+0000","last_unstale":"2024-05-14T11:25:43.445012+0000","last_undegraded":"2024-05-14T11:25:43.445012+0000","last_fullsized":"2024-05-14T11:25:43.445012+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:43.613522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452377+0000","last_change":"2024-05-14T11:25:42.445131+0000","last_active":"2024-05-14T11:25:43.452377+0000","last_peered":"2024-05-14T11:25:43.452377+0000","last_clean":"2024-05-14T11:25:43.452377+0000","last_became_active":"2024-05-14T11:25:42.444777+0000","last_became_peered":"2024-05-14T11:25:42.444777+0000","last_unstale":"2024-05-14T11:25:43.452377+0000","last_undegraded":"2024-05-14T11:25:43.452377+0000","last_fullsized":"2024-05-14T11:25:43.452377+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:11:01.891209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641446+0000","last_change":"2024-05-14T11:25:41.453270+0000","last_active":"2024-05-14T11:25:43.641446+0000","last_peered":"2024-05-14T11:25:43.641446+0000","last_clean":"2024-05-14T11:25:43.641446+0000","last_became_active":"2024-05-14T11:25:41.452958+0000","last_became_peered":"2024-05-14T11:25:41.452958+0000","last_unstale":"2024-05-14T11:25:43.641446+0000","last_undegraded":"2024-05-14T11:25:43.641446+0000","last_fullsized":"2024-05-14T11:25:43.641446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:20.013256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640986+0000","last_change":"2024-05-14T11:25:42.459804+0000","last_active":"2024-05-14T11:25:43.640986+0000","last_peered":"2024-05-14T11:25:43.640986+0000","last_clean":"2024-05-14T11:25:43.640986+0000","last_became_active":"2024-05-14T11:25:42.459475+0000","last_became_peered":"2024-05-14T11:25:42.459475+0000","last_unstale":"2024-05-14T11:25:43.640986+0000","last_undegraded":"2024-05-14T11:25:43.640986+0000","last_fullsized":"2024-05-14T11:25:43.640986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:07.864810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641046+0000","last_change":"2024-05-14T11:25:41.450378+0000","last_active":"2024-05-14T11:25:43.641046+0000","last_peered":"2024-05-14T11:25:43.641046+0000","last_clean":"2024-05-14T11:25:43.641046+0000","last_became_active":"2024-05-14T11:25:41.450018+0000","last_became_peered":"2024-05-14T11:25:41.450018+0000","last_unstale":"2024-05-14T11:25:43.641046+0000","last_undegraded":"2024-05-14T11:25:43.641046+0000","last_fullsized":"2024-05-14T11:25:43.641046+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:27:30.405786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451856+0000","last_change":"2024-05-14T11:25:42.446288+0000","last_active":"2024-05-14T11:25:43.451856+0000","last_peered":"2024-05-14T11:25:43.451856+0000","last_clean":"2024-05-14T11:25:43.451856+0000","last_became_active":"2024-05-14T11:25:42.445982+0000","last_became_peered":"2024-05-14T11:25:42.445982+0000","last_unstale":"2024-05-14T11:25:43.451856+0000","last_undegraded":"2024-05-14T11:25:43.451856+0000","last_fullsized":"2024-05-14T11:25:43.451856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:38:09.568395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.451948+0000","last_change":"2024-05-14T11:25:41.446589+0000","last_active":"2024-05-14T11:25:43.451948+0000","last_peered":"2024-05-14T11:25:43.451948+0000","last_clean":"2024-05-14T11:25:43.451948+0000","last_became_active":"2024-05-14T11:25:41.446305+0000","last_became_peered":"2024-05-14T11:25:41.446305+0000","last_unstale":"2024-05-14T11:25:43.451948+0000","last_undegraded":"2024-05-14T11:25:43.451948+0000","last_fullsized":"2024-05-14T11:25:43.451948+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:22:33.461147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639831+0000","last_change":"2024-05-14T11:25:42.460267+0000","last_active":"2024-05-14T11:25:43.639831+0000","last_peered":"2024-05-14T11:25:43.639831+0000","last_clean":"2024-05-14T11:25:43.639831+0000","last_became_active":"2024-05-14T11:25:42.459964+0000","last_became_peered":"2024-05-14T11:25:42.459964+0000","last_unstale":"2024-05-14T11:25:43.639831+0000","last_undegraded":"2024-05-14T11:25:43.639831+0000","last_fullsized":"2024-05-14T11:25:43.639831+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:49:46.489691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640870+0000","last_change":"2024-05-14T11:25:41.449734+0000","last_active":"2024-05-14T11:25:43.640870+0000","last_peered":"2024-05-14T11:25:43.640870+0000","last_clean":"2024-05-14T11:25:43.640870+0000","last_became_active":"2024-05-14T11:25:41.449320+0000","last_became_peered":"2024-05-14T11:25:41.449320+0000","last_unstale":"2024-05-14T11:25:43.640870+0000","last_undegraded":"2024-05-14T11:25:43.640870+0000","last_fullsized":"2024-05-14T11:25:43.640870+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:03:41.376864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640354+0000","last_change":"2024-05-14T11:25:42.436596+0000","last_active":"2024-05-14T11:25:43.640354+0000","last_peered":"2024-05-14T11:25:43.640354+0000","last_clean":"2024-05-14T11:25:43.640354+0000","last_became_active":"2024-05-14T11:25:42.434944+0000","last_became_peered":"2024-05-14T11:25:42.434944+0000","last_unstale":"2024-05-14T11:25:43.640354+0000","last_undegraded":"2024-05-14T11:25:43.640354+0000","last_fullsized":"2024-05-14T11:25:43.640354+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:22:13.710478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639918+0000","last_change":"2024-05-14T11:25:41.439181+0000","last_active":"2024-05-14T11:25:43.639918+0000","last_peered":"2024-05-14T11:25:43.639918+0000","last_clean":"2024-05-14T11:25:43.639918+0000","last_became_active":"2024-05-14T11:25:41.438850+0000","last_became_peered":"2024-05-14T11:25:41.438850+0000","last_unstale":"2024-05-14T11:25:43.639918+0000","last_undegraded":"2024-05-14T11:25:43.639918+0000","last_fullsized":"2024-05-14T11:25:43.639918+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:44:27.040144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640955+0000","last_change":"2024-05-14T11:25:42.458286+0000","last_active":"2024-05-14T11:25:43.640955+0000","last_peered":"2024-05-14T11:25:43.640955+0000","last_clean":"2024-05-14T11:25:43.640955+0000","last_became_active":"2024-05-14T11:25:42.457881+0000","last_became_peered":"2024-05-14T11:25:42.457881+0000","last_unstale":"2024-05-14T11:25:43.640955+0000","last_undegraded":"2024-05-14T11:25:43.640955+0000","last_fullsized":"2024-05-14T11:25:43.640955+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:12:16.391415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639788+0000","last_change":"2024-05-14T11:25:41.450647+0000","last_active":"2024-05-14T11:25:43.639788+0000","last_peered":"2024-05-14T11:25:43.639788+0000","last_clean":"2024-05-14T11:25:43.639788+0000","last_became_active":"2024-05-14T11:25:41.450309+0000","last_became_peered":"2024-05-14T11:25:41.450309+0000","last_unstale":"2024-05-14T11:25:43.639788+0000","last_undegraded":"2024-05-14T11:25:43.639788+0000","last_fullsized":"2024-05-14T11:25:43.639788+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:12:23.828807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452634+0000","last_change":"2024-05-14T11:25:42.452679+0000","last_active":"2024-05-14T11:25:43.452634+0000","last_peered":"2024-05-14T11:25:43.452634+0000","last_clean":"2024-05-14T11:25:43.452634+0000","last_became_active":"2024-05-14T11:25:42.452361+0000","last_became_peered":"2024-05-14T11:25:42.452361+0000","last_unstale":"2024-05-14T11:25:43.452634+0000","last_undegraded":"2024-05-14T11:25:43.452634+0000","last_fullsized":"2024-05-14T11:25:43.452634+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:27:09.546113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452165+0000","last_change":"2024-05-14T11:25:41.442252+0000","last_active":"2024-05-14T11:25:43.452165+0000","last_peered":"2024-05-14T11:25:43.452165+0000","last_clean":"2024-05-14T11:25:43.452165+0000","last_became_active":"2024-05-14T11:25:41.441969+0000","last_became_peered":"2024-05-14T11:25:41.441969+0000","last_unstale":"2024-05-14T11:25:43.452165+0000","last_undegraded":"2024-05-14T11:25:43.452165+0000","last_fullsized":"2024-05-14T11:25:43.452165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:12:55.071938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639810+0000","last_change":"2024-05-14T11:25:42.445337+0000","last_active":"2024-05-14T11:25:43.639810+0000","last_peered":"2024-05-14T11:25:43.639810+0000","last_clean":"2024-05-14T11:25:43.639810+0000","last_became_active":"2024-05-14T11:25:42.444949+0000","last_became_peered":"2024-05-14T11:25:42.444949+0000","last_unstale":"2024-05-14T11:25:43.639810+0000","last_undegraded":"2024-05-14T11:25:43.639810+0000","last_fullsized":"2024-05-14T11:25:43.639810+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:09:21.015956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447047+0000","last_change":"2024-05-14T11:25:41.438532+0000","last_active":"2024-05-14T11:25:43.447047+0000","last_peered":"2024-05-14T11:25:43.447047+0000","last_clean":"2024-05-14T11:25:43.447047+0000","last_became_active":"2024-05-14T11:25:41.438203+0000","last_became_peered":"2024-05-14T11:25:41.438203+0000","last_unstale":"2024-05-14T11:25:43.447047+0000","last_undegraded":"2024-05-14T11:25:43.447047+0000","last_fullsized":"2024-05-14T11:25:43.447047+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:34:06.193622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.641461+0000","last_change":"2024-05-14T11:25:42.457522+0000","last_active":"2024-05-14T11:25:43.641461+0000","last_peered":"2024-05-14T11:25:43.641461+0000","last_clean":"2024-05-14T11:25:43.641461+0000","last_became_active":"2024-05-14T11:25:42.457088+0000","last_became_peered":"2024-05-14T11:25:42.457088+0000","last_unstale":"2024-05-14T11:25:43.641461+0000","last_undegraded":"2024-05-14T11:25:43.641461+0000","last_fullsized":"2024-05-14T11:25:43.641461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:13:48.977220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.447147+0000","last_change":"2024-05-14T11:25:41.440129+0000","last_active":"2024-05-14T11:25:43.447147+0000","last_peered":"2024-05-14T11:25:43.447147+0000","last_clean":"2024-05-14T11:25:43.447147+0000","last_became_active":"2024-05-14T11:25:41.439752+0000","last_became_peered":"2024-05-14T11:25:41.439752+0000","last_unstale":"2024-05-14T11:25:43.447147+0000","last_undegraded":"2024-05-14T11:25:43.447147+0000","last_fullsized":"2024-05-14T11:25:43.447147+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T13:07:15.436967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.445186+0000","last_change":"2024-05-14T11:25:42.462505+0000","last_active":"2024-05-14T11:25:43.445186+0000","last_peered":"2024-05-14T11:25:43.445186+0000","last_clean":"2024-05-14T11:25:43.445186+0000","last_became_active":"2024-05-14T11:25:42.462134+0000","last_became_peered":"2024-05-14T11:25:42.462134+0000","last_unstale":"2024-05-14T11:25:43.445186+0000","last_undegraded":"2024-05-14T11:25:43.445186+0000","last_fullsized":"2024-05-14T11:25:43.445186+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:34:37.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452591+0000","last_change":"2024-05-14T11:25:41.445773+0000","last_active":"2024-05-14T11:25:43.452591+0000","last_peered":"2024-05-14T11:25:43.452591+0000","last_clean":"2024-05-14T11:25:43.452591+0000","last_became_active":"2024-05-14T11:25:41.445486+0000","last_became_peered":"2024-05-14T11:25:41.445486+0000","last_unstale":"2024-05-14T11:25:43.452591+0000","last_undegraded":"2024-05-14T11:25:43.452591+0000","last_fullsized":"2024-05-14T11:25:43.452591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:19:27.181952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452262+0000","last_change":"2024-05-14T11:25:42.442154+0000","last_active":"2024-05-14T11:25:43.452262+0000","last_peered":"2024-05-14T11:25:43.452262+0000","last_clean":"2024-05-14T11:25:43.452262+0000","last_became_active":"2024-05-14T11:25:42.441491+0000","last_became_peered":"2024-05-14T11:25:42.441491+0000","last_unstale":"2024-05-14T11:25:43.452262+0000","last_undegraded":"2024-05-14T11:25:43.452262+0000","last_fullsized":"2024-05-14T11:25:43.452262+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:34:42.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448655+0000","last_change":"2024-05-14T11:25:41.443751+0000","last_active":"2024-05-14T11:25:43.448655+0000","last_peered":"2024-05-14T11:25:43.448655+0000","last_clean":"2024-05-14T11:25:43.448655+0000","last_became_active":"2024-05-14T11:25:41.443434+0000","last_became_peered":"2024-05-14T11:25:41.443434+0000","last_unstale":"2024-05-14T11:25:43.448655+0000","last_undegraded":"2024-05-14T11:25:43.448655+0000","last_fullsized":"2024-05-14T11:25:43.448655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T12:16:59.944993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639790+0000","last_change":"2024-05-14T11:25:42.437081+0000","last_active":"2024-05-14T11:25:43.639790+0000","last_peered":"2024-05-14T11:25:43.639790+0000","last_clean":"2024-05-14T11:25:43.639790+0000","last_became_active":"2024-05-14T11:25:42.436710+0000","last_became_peered":"2024-05-14T11:25:42.436710+0000","last_unstale":"2024-05-14T11:25:43.639790+0000","last_undegraded":"2024-05-14T11:25:43.639790+0000","last_fullsized":"2024-05-14T11:25:43.639790+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:07.864810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.448590+0000","last_change":"2024-05-14T11:25:41.446643+0000","last_active":"2024-05-14T11:25:43.448590+0000","last_peered":"2024-05-14T11:25:43.448590+0000","last_clean":"2024-05-14T11:25:43.448590+0000","last_became_active":"2024-05-14T11:25:41.444544+0000","last_became_peered":"2024-05-14T11:25:41.444544+0000","last_unstale":"2024-05-14T11:25:43.448590+0000","last_undegraded":"2024-05-14T11:25:43.448590+0000","last_fullsized":"2024-05-14T11:25:43.448590+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:37.989036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640925+0000","last_change":"2024-05-14T11:25:42.462308+0000","last_active":"2024-05-14T11:25:43.640925+0000","last_peered":"2024-05-14T11:25:43.640925+0000","last_clean":"2024-05-14T11:25:43.640925+0000","last_became_active":"2024-05-14T11:25:42.461911+0000","last_became_peered":"2024-05-14T11:25:42.461911+0000","last_unstale":"2024-05-14T11:25:43.640925+0000","last_undegraded":"2024-05-14T11:25:43.640925+0000","last_fullsized":"2024-05-14T11:25:43.640925+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:49:46.489691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.452039+0000","last_change":"2024-05-14T11:25:41.440667+0000","last_active":"2024-05-14T11:25:43.452039+0000","last_peered":"2024-05-14T11:25:43.452039+0000","last_clean":"2024-05-14T11:25:43.452039+0000","last_became_active":"2024-05-14T11:25:41.440174+0000","last_became_peered":"2024-05-14T11:25:41.440174+0000","last_unstale":"2024-05-14T11:25:43.452039+0000","last_undegraded":"2024-05-14T11:25:43.452039+0000","last_fullsized":"2024-05-14T11:25:43.452039+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T11:46:32.578199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639567+0000","last_change":"2024-05-14T11:25:42.457633+0000","last_active":"2024-05-14T11:25:43.639567+0000","last_peered":"2024-05-14T11:25:43.639567+0000","last_clean":"2024-05-14T11:25:43.639567+0000","last_became_active":"2024-05-14T11:25:42.457350+0000","last_became_peered":"2024-05-14T11:25:42.457350+0000","last_unstale":"2024-05-14T11:25:43.639567+0000","last_undegraded":"2024-05-14T11:25:43.639567+0000","last_fullsized":"2024-05-14T11:25:43.639567+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:48.405060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639813+0000","last_change":"2024-05-14T11:25:41.450892+0000","last_active":"2024-05-14T11:25:43.639813+0000","last_peered":"2024-05-14T11:25:43.639813+0000","last_clean":"2024-05-14T11:25:43.639813+0000","last_became_active":"2024-05-14T11:25:41.450584+0000","last_became_peered":"2024-05-14T11:25:41.450584+0000","last_unstale":"2024-05-14T11:25:43.639813+0000","last_undegraded":"2024-05-14T11:25:43.639813+0000","last_fullsized":"2024-05-14T11:25:43.639813+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:47.402359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640791+0000","last_change":"2024-05-14T11:25:42.455949+0000","last_active":"2024-05-14T11:25:43.640791+0000","last_peered":"2024-05-14T11:25:43.640791+0000","last_clean":"2024-05-14T11:25:43.640791+0000","last_became_active":"2024-05-14T11:25:42.455604+0000","last_became_peered":"2024-05-14T11:25:42.455604+0000","last_unstale":"2024-05-14T11:25:43.640791+0000","last_undegraded":"2024-05-14T11:25:43.640791+0000","last_fullsized":"2024-05-14T11:25:43.640791+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:41.416277+0000","last_clean_scrub_stamp":"2024-05-14T11:25:41.416277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:47:58.260359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.639275+0000","last_change":"2024-05-14T11:25:41.442618+0000","last_active":"2024-05-14T11:25:43.639275+0000","last_peered":"2024-05-14T11:25:43.639275+0000","last_clean":"2024-05-14T11:25:43.639275+0000","last_became_active":"2024-05-14T11:25:41.439550+0000","last_became_peered":"2024-05-14T11:25:41.439550+0000","last_unstale":"2024-05-14T11:25:43.639275+0000","last_undegraded":"2024-05-14T11:25:43.639275+0000","last_fullsized":"2024-05-14T11:25:43.639275+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:25:40.413576+0000","last_clean_scrub_stamp":"2024-05-14T11:25:40.413576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:05:31.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":279,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-14T11:25:43.640515+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:25:43.640515+0000","last_peered":"2024-05-14T11:25:43.640515+0000","last_clean":"2024-05-14T11:25:43.640515+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:25:43.640515+0000","last_undegraded":"2024-05-14T11:25:43.640515+0000","last_fullsized":"2024-05-14T11:25:43.640515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2019328,"data_stored":1247310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":656748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1417216,"data_stored":656326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":657778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":657812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":656166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1413120,"data_stored":656188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1246556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:25:46.961 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:25:46.961 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-14T11:25:47.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.158+0000 7f4e3fe00640 1 Processor -- start 2024-05-14T11:25:47.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.158+0000 7f4e3fe00640 1 -- start start 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4e4015fc30 0x7f4e400c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e400cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e4006eb20 0x7f4e4006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4e400cec10 con 0x7f4e4006eb20 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4e400cdda0 con 0x7f4e4015fc30 2024-05-14T11:25:47.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400ce040 con 0x7f4e400cf0a0 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e4006eb20 0x7f4e4006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e4006eb20 0x7f4e4006ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43388/0 (socket says 172.21.15.116:43388) 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 -- 172.21.15.116:0/953074645 learned_addr learned my addr 172.21.15.116:0/953074645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3e000640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e400cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3ea00640 1 --2- 172.21.15.116:0/953074645 >> v2:172.21.15.149:3300/0 conn(0x7f4e4015fc30 0x7f4e400c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 -- 172.21.15.116:0/953074645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 msgr2=0x7f4e400cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e400cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 -- 172.21.15.116:0/953074645 >> v2:172.21.15.149:3300/0 conn(0x7f4e4015fc30 msgr2=0x7f4e400c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 --2- 172.21.15.116:0/953074645 >> v2:172.21.15.149:3300/0 conn(0x7f4e4015fc30 0x7f4e400c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.162+0000 7f4e3f400640 1 -- 172.21.15.116:0/953074645 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e40160040 con 0x7f4e4006eb20 2024-05-14T11:25:47.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3f400640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4006eb20 0x7f4e4006ef30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4e40074610 tx=0x7f4e3801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca7fd569f4e7f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3d600640 1 -- 172.21.15.116:0/953074645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380a57b0 con 0x7f4e4006eb20 2024-05-14T11:25:47.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3d600640 1 -- 172.21.15.116:0/953074645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e380a5950 con 0x7f4e4006eb20 2024-05-14T11:25:47.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3d600640 1 -- 172.21.15.116:0/953074645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380a5d20 con 0x7f4e4006eb20 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 -- 172.21.15.116:0/953074645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4006eb20 msgr2=0x7f4e4006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4006eb20 0x7f4e4006ef30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4e40074610 tx=0x7f4e3801c930 comp rx=0 tx=0).stop 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 -- 172.21.15.116:0/953074645 shutdown_connections 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4006eb20 0x7f4e4006ef30 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/953074645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e400cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/953074645 >> v2:172.21.15.149:3300/0 conn(0x7f4e4015fc30 0x7f4e400c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 -- 172.21.15.116:0/953074645 >> 172.21.15.116:0/953074645 conn(0x7f4e400d34e0 msgr2=0x7f4e400d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 -- 172.21.15.116:0/953074645 shutdown_connections 2024-05-14T11:25:47.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.166+0000 7f4e3fe00640 1 -- 172.21.15.116:0/953074645 wait complete. 2024-05-14T11:25:47.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 Processor -- start 2024-05-14T11:25:47.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- start start 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4006eb20 0x7f4e4014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e40160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4006f880 con 0x7f4e4015fc30 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4006e800 con 0x7f4e4006eb20 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400cf9f0 con 0x7f4e400cf0a0 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43404/0 (socket says 172.21.15.116:43404) 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 -- 172.21.15.116:0/1931625837 learned_addr learned my addr 172.21.15.116:0/1931625837 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:47.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3ea00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4006eb20 0x7f4e4014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3e000640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e40160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 -- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 msgr2=0x7f4e40160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e40160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 -- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4006eb20 msgr2=0x7f4e4014f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4006eb20 0x7f4e4014f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 -- 172.21.15.116:0/1931625837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e40161190 con 0x7f4e4015fc30 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3f400640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4e380af040 tx=0x7f4e380196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380bc020 con 0x7f4e4015fc30 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e380b86b0 con 0x7f4e4015fc30 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38009510 con 0x7f4e4015fc30 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e40153450 con 0x7f4e4015fc30 2024-05-14T11:25:47.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.170+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e40153a10 con 0x7f4e4015fc30 2024-05-14T11:25:47.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e400508f0 con 0x7f4e4015fc30 2024-05-14T11:25:47.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e380bf020 con 0x7f4e4015fc30 2024-05-14T11:25:47.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e37400640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f4e38129b50 con 0x7f4e4015fc30 2024-05-14T11:25:47.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e3ea00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.174+0000 7f4e3ea00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 0x7f4e20071940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009ac0 tx=0x7f4e30009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.178+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4e380c7020 con 0x7f4e4015fc30 2024-05-14T11:25:47.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.394+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4e401531b0 con 0x7f4e4015fc30 2024-05-14T11:25:47.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.394+0000 7f4e37400640 1 -- 172.21.15.116:0/1931625837 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f4e40161190 con 0x7f4e4015fc30 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDBR0NmHeMhEBAAgPkXnKYkWgMN7b548nNGBw== 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-14T11:25:47.394 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 msgr2=0x7f4e20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 0x7f4e20071940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009ac0 tx=0x7f4e30009190 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 msgr2=0x7f4e40152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4e380af040 tx=0x7f4e380196d0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 shutdown_connections 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4015fc30 0x7f4e40152310 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e400cf0a0 0x7f4e40160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 --2- 172.21.15.116:0/1931625837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4006eb20 0x7f4e4014f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 >> 172.21.15.116:0/1931625837 conn(0x7f4e400d34e0 msgr2=0x7f4e40155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 shutdown_connections 2024-05-14T11:25:47.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.398+0000 7f4e3fe00640 1 -- 172.21.15.116:0/1931625837 wait complete. 2024-05-14T11:25:47.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:25:47.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 Processor -- start 2024-05-14T11:25:47.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 -- start start 2024-05-14T11:25:47.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f740d15e0 0x7f1f740d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f7406ead0 0x7f1f7406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f740d1f30 con 0x7f1f7406c700 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1f7415ce20 con 0x7f1f7406ead0 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f78c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7406f850 con 0x7f1f740d15e0 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f7406ead0 0x7f1f7406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1f7406ead0 0x7f1f7406eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38570/0 (socket says 172.21.15.116:38570) 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 -- 172.21.15.116:0/3992604463 learned_addr learned my addr 172.21.15.116:0/3992604463 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:47.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f73e00640 1 --1- 172.21.15.116:0/3992604463 >> v1:172.21.15.116:6789/0 conn(0x7f1f7406c700 0x7f1f7406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50706/0 (socket says 172.21.15.116:50706) 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f73400640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f740d15e0 0x7f1f740d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 -- 172.21.15.116:0/3992604463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f740d15e0 msgr2=0x7f1f740d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f740d15e0 0x7f1f740d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 -- 172.21.15.116:0/3992604463 >> v1:172.21.15.116:6789/0 conn(0x7f1f7406c700 legacy=0x7f1f7406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f6aa00640 1 -- 172.21.15.116:0/3992604463 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7406fb30 con 0x7f1f7406ead0 2024-05-14T11:25:47.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.570+0000 7f1f72a00640 1 -- 172.21.15.116:0/3992604463 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155814620 0 0) 0x7f1f6c0014a0 con 0x7f1f7406c700 2024-05-14T11:25:47.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f6aa00640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f7406ead0 0x7f1f7406eee0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400bb10 tx=0x7f1f6401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320cafb2f5941961 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f72a00640 1 -- 172.21.15.116:0/3992604463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640b4070 con 0x7f1f7406ead0 2024-05-14T11:25:47.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f72a00640 1 -- 172.21.15.116:0/3992604463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f640a7c00 con 0x7f1f7406ead0 2024-05-14T11:25:47.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f72a00640 1 -- 172.21.15.116:0/3992604463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640af440 con 0x7f1f7406ead0 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 -- 172.21.15.116:0/3992604463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f7406ead0 msgr2=0x7f1f7406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f7406ead0 0x7f1f7406eee0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400bb10 tx=0x7f1f6401ec30 comp rx=0 tx=0).stop 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 -- 172.21.15.116:0/3992604463 shutdown_connections 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f7406ead0 0x7f1f7406eee0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 --2- 172.21.15.116:0/3992604463 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f740d15e0 0x7f1f740d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 -- 172.21.15.116:0/3992604463 >> 172.21.15.116:0/3992604463 conn(0x7f1f740b4530 msgr2=0x7f1f740b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:47.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 -- 172.21.15.116:0/3992604463 shutdown_connections 2024-05-14T11:25:47.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.574+0000 7f1f78c00640 1 -- 172.21.15.116:0/3992604463 wait complete. 2024-05-14T11:25:47.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 Processor -- start 2024-05-14T11:25:47.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 -- start start 2024-05-14T11:25:47.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f7406c700 0x7f1f740adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f7406ead0 0x7f1f74099450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7406f5a0 con 0x7f1f740d15e0 2024-05-14T11:25:47.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7406f420 con 0x7f1f7406c700 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f78c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1f740ce470 con 0x7f1f7406ead0 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38582/0 (socket says 172.21.15.116:38582) 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 -- 172.21.15.116:0/2948936073 learned_addr learned my addr 172.21.15.116:0/2948936073 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f6aa00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f7406ead0 0x7f1f74099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73400640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f7406c700 0x7f1f740adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 -- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f7406ead0 msgr2=0x7f1f74099450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f7406ead0 0x7f1f74099450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 -- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f7406c700 msgr2=0x7f1f740adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.578+0000 7f1f73e00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f7406c700 0x7f1f740adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f73e00640 1 -- 172.21.15.116:0/2948936073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f74099990 con 0x7f1f740d15e0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f73e00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00c6b0 tx=0x7f1f6c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c018070 con 0x7f1f740d15e0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f6c004db0 con 0x7f1f740d15e0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00a880 con 0x7f1f740d15e0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f740ae840 con 0x7f1f740d15e0 2024-05-14T11:25:47.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f740aee00 con 0x7f1f740d15e0 2024-05-14T11:25:47.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f740ae5b0 con 0x7f1f740d15e0 2024-05-14T11:25:47.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f6c00aa20 con 0x7f1f740d15e0 2024-05-14T11:25:47.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.582+0000 7f1f6be00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 0x7f1f50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:47.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.586+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f1f6c08ea30 con 0x7f1f740d15e0 2024-05-14T11:25:47.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.586+0000 7f1f73400640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 0x7f1f50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:47.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.586+0000 7f1f73400640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 0x7f1f50071940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b930 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.590+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1f740d1f30 con 0x7f1f740d15e0 2024-05-14T11:25:47.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.778+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f740743c0 con 0x7f1f740d15e0 2024-05-14T11:25:47.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.778+0000 7f1f6be00640 1 -- 172.21.15.116:0/2948936073 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5197 (secure 0 0 0) 0x7f1f6c05b6f0 con 0x7f1f740d15e0 2024-05-14T11:25:47.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:25:47.778 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:25:44.842592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8208":{"gid":8208,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6837/1742968683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":1742968683},{"type":"v1","addr":"172.21.15.116:6837","nonce":1742968683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8208,"qdb_cluster":[8208]},"id":8}]} 2024-05-14T11:25:47.779 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 100 2024-05-14T11:25:47.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 msgr2=0x7f1f50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 0x7f1f50071940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00b930 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-05-14T11:25:47.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 msgr2=0x7f1f74158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00c6b0 tx=0x7f1f6c00cb70 comp rx=0 tx=0).stop 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 shutdown_connections 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1f5006f430 0x7f1f50071940 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f740d15e0 0x7f1f74158d70 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1f7406ead0 0x7f1f74099450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 --2- 172.21.15.116:0/2948936073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f7406c700 0x7f1f740adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 >> 172.21.15.116:0/2948936073 conn(0x7f1f740b4530 msgr2=0x7f1f740686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 shutdown_connections 2024-05-14T11:25:47.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:47.782+0000 7f1f78c00640 1 -- 172.21.15.116:0/2948936073 wait complete. 2024-05-14T11:25:48.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:25:44.842592+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8208': {'gid': 8208, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6837/1742968683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 1742968683}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 1742968683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8208, 'qdb_cluster': [8208]} 2024-05-14T11:25:48.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:25:48.202 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:25:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.350+0000 7f3e77000640 1 Processor -- start 2024-05-14T11:25:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.350+0000 7f3e77000640 1 -- start start 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e77000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3e70164f80 0x7f3e701673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e77000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7006cf10 con 0x7f3e7006f730 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e77000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7006d1f0 con 0x7f3e700b0770 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e77000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7006db20 con 0x7f3e70164f80 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e75000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3e70164f80 0x7f3e701673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e6f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3e7006f730 0x7f3e7006fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50718/0 (socket says 172.21.15.116:50718) 2024-05-14T11:25:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e6f400640 1 -- 172.21.15.116:0/1397155036 learned_addr learned my addr 172.21.15.116:0/1397155036 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e75000640 1 -- 172.21.15.116:0/1397155036 >> v1:172.21.15.149:6789/0 conn(0x7f3e700b0770 legacy=0x7f3e7006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e75000640 1 -- 172.21.15.116:0/1397155036 >> v1:172.21.15.116:6789/0 conn(0x7f3e7006f730 legacy=0x7f3e7006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e75000640 1 -- 172.21.15.116:0/1397155036 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7006d4d0 con 0x7f3e70164f80 2024-05-14T11:25:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.354+0000 7f3e6ea00640 1 -- 172.21.15.116:0/1397155036 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642380306 0 0) 0x7f3e7006cf10 con 0x7f3e7006f730 2024-05-14T11:25:48.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e75000640 1 --2- 172.21.15.116:0/1397155036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e701673a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009d90 tx=0x7f3e6801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f64f944bec0cbfd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e6ea00640 1 -- 172.21.15.116:0/1397155036 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b2070 con 0x7f3e70164f80 2024-05-14T11:25:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e6ea00640 1 -- 172.21.15.116:0/1397155036 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680a5890 con 0x7f3e70164f80 2024-05-14T11:25:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e6ea00640 1 -- 172.21.15.116:0/1397155036 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680a5c60 con 0x7f3e70164f80 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 -- 172.21.15.116:0/1397155036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 msgr2=0x7f3e701673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 --2- 172.21.15.116:0/1397155036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e701673a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009d90 tx=0x7f3e6801c8b0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 -- 172.21.15.116:0/1397155036 shutdown_connections 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 --2- 172.21.15.116:0/1397155036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e701673a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 -- 172.21.15.116:0/1397155036 >> 172.21.15.116:0/1397155036 conn(0x7f3e700b4620 msgr2=0x7f3e700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 -- 172.21.15.116:0/1397155036 shutdown_connections 2024-05-14T11:25:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.358+0000 7f3e77000640 1 -- 172.21.15.116:0/1397155036 wait complete. 2024-05-14T11:25:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 Processor -- start 2024-05-14T11:25:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- start start 2024-05-14T11:25:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 0x7f3e70152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e700b0770 0x7f3e70166c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7016ab90 con 0x7f3e70164f80 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7016aa10 con 0x7f3e7006f730 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7016ad10 con 0x7f3e700b0770 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38592/0 (socket says 172.21.15.116:38592) 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 -- 172.21.15.116:0/3435031773 learned_addr learned my addr 172.21.15.116:0/3435031773 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e6f400640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e700b0770 0x7f3e70166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 0x7f3e70152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 -- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e700b0770 msgr2=0x7f3e70166c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e700b0770 0x7f3e70166c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 -- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 msgr2=0x7f3e70152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 0x7f3e70152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 -- 172.21.15.116:0/3435031773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e701671b0 con 0x7f3e70164f80 2024-05-14T11:25:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 0x7f3e70152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e75000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b0040 tx=0x7f3e68019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e67e00640 1 -- 172.21.15.116:0/3435031773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b2050 con 0x7f3e70164f80 2024-05-14T11:25:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e67e00640 1 -- 172.21.15.116:0/3435031773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680be070 con 0x7f3e70164f80 2024-05-14T11:25:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e67e00640 1 -- 172.21.15.116:0/3435031773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680ada50 con 0x7f3e70164f80 2024-05-14T11:25:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7015b940 con 0x7f3e70164f80 2024-05-14T11:25:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.362+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7015bf00 con 0x7f3e70164f80 2024-05-14T11:25:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e67e00640 1 -- 172.21.15.116:0/3435031773 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e680adbf0 con 0x7f3e70164f80 2024-05-14T11:25:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e67e00640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 0x7f3e50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e67e00640 1 -- 172.21.15.116:0/3435031773 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3e6812a650 con 0x7f3e70164f80 2024-05-14T11:25:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 0x7f3e50071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e7e753000 1 Processor -- start 2024-05-14T11:25:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e7e753000 1 -- start start 2024-05-14T11:25:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.366+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 0x7f3e50071910 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000f70 tx=0x7f3e60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d7e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5621c0c1b620 con 0x5621c08447c0 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5621c0b36740 con 0x5621c0c166b0 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5621c0b84090 con 0x5621c0c2cf80 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38604/0 (socket says 172.21.15.116:38604) 2024-05-14T11:25:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 -- 172.21.15.116:0/4072451 learned_addr learned my addr 172.21.15.116:0/4072451 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 -- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 msgr2=0x5621c0d7e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 -- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 msgr2=0x5621c0d7eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 -- 172.21.15.116:0/4072451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c0d180a0 con 0x5621c08447c0 2024-05-14T11:25:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3e60001ec0 tx=0x7f3e60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d6aa76c94e79d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e64c00640 1 -- 172.21.15.116:0/4072451 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003600 con 0x5621c08447c0 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e64c00640 1 -- 172.21.15.116:0/4072451 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e600037a0 con 0x5621c08447c0 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e64c00640 1 -- 172.21.15.116:0/4072451 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003b50 con 0x5621c08447c0 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- 172.21.15.116:0/4072451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 msgr2=0x5621c0d7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3e60001ec0 tx=0x7f3e60011000 comp rx=0 tx=0).stop 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- 172.21.15.116:0/4072451 shutdown_connections 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d7eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d7e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 --2- 172.21.15.116:0/4072451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c08447c0 0x5621c0d7e320 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- 172.21.15.116:0/4072451 >> 172.21.15.116:0/4072451 conn(0x5621c0d03340 msgr2=0x5621c0d32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- 172.21.15.116:0/4072451 shutdown_connections 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.370+0000 7f3e7e753000 1 -- 172.21.15.116:0/4072451 wait complete. 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- start start 2024-05-14T11:25:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 Processor -- start 2024-05-14T11:25:48.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d84980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5621c0b80050 con 0x5621c0c166b0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5621c0b3cec0 con 0x5621c08447c0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5621c0b23c90 con 0x5621c0c2cf80 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 10 client.0 _ll_register_callbacks cb 0x5621c0d1fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38612/0 (socket says 172.21.15.116:38612) 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 -- 172.21.15.116:0/2301037265 learned_addr learned my addr 172.21.15.116:0/2301037265 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e75000640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d84980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 -- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 msgr2=0x5621c0d62720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d62720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 -- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 msgr2=0x5621c0d84980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d84980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 -- 172.21.15.116:0/2301037265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c0c2e6f0 con 0x5621c0c166b0 2024-05-14T11:25:48.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d84980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e6f400640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c007650 tx=0x7f3e5c0015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cf5b888bcab7af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c01a060 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3e5c015c90 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e600037a0 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3e5c005860 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 10 client.8334 fetch_fsmap learned FSMap version 100 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5621c0c2a6a0 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e600037a0 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f3e5c005a00 con 0x5621c0c166b0 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 10 client.8334 fetch_fsmap finished waiting for FSMap version 100 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 10 client.8334 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 20 client.8334 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c08297d0 0x5621c0d63490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 4 client.8334 mds_command: new command op to 8208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:25:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.374+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 --> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c0b1cbe0 con 0x5621c08297d0 2024-05-14T11:25:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c08297d0 0x5621c0d63490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e59600640 10 client.8334 ms_handle_connect on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c08297d0 0x5621c0d63490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedc2d87b6f546d0 server_cookie=a70092de9155084f in_seq=0 out_seq=0 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e59600640 1 -- 172.21.15.116:0/2301037265 <== mds.0 v2:172.21.15.116:6836/1742968683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621c0b1cbe0 con 0x5621c08297d0 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e59600640 10 client.8334 handle_command_reply: tid=0 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 client.8334 shutdown 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 msgr2=0x5621c0d84ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c007650 tx=0x7f3e5c0015e0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 shutdown_connections 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c08297d0 0x5621c0d63490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c2cf80 0x5621c0d62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c166b0 0x5621c0d84ec0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e59600640 0 client.8334 ms_handle_reset on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 --2- 172.21.15.116:0/2301037265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d84980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 >> 172.21.15.116:0/2301037265 conn(0x5621c0d03340 msgr2=0x5621c0d33490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 shutdown_connections 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 1 -- 172.21.15.116:0/2301037265 wait complete. 2024-05-14T11:25:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.378+0000 7f3e7e753000 20 client.8334 trim_cache size 0 max 16384 2024-05-14T11:25:48.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 Processor -- start 2024-05-14T11:25:48.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- start start 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d93b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c166b0 0x5621c0d94050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70204f00 con 0x5621c0c2cf80 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70166150 con 0x5621c08447c0 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7015ec00 con 0x5621c0c166b0 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38622/0 (socket says 172.21.15.116:38622) 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 -- 172.21.15.116:0/1056330919 learned_addr learned my addr 172.21.15.116:0/1056330919 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d93b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e6f400640 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c166b0 0x5621c0d94050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 -- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c166b0 msgr2=0x5621c0d94050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c166b0 0x5621c0d94050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 -- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 msgr2=0x5621c0d93b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d93b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 -- 172.21.15.116:0/1056330919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c0d37e10 con 0x5621c0c2cf80 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e75000640 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3e6801cb70 tx=0x7f3e680adf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7dce48a8ef7687 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e5be00640 1 -- 172.21.15.116:0/1056330919 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68126070 con 0x5621c0c2cf80 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e5be00640 1 -- 172.21.15.116:0/1056330919 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680b3040 con 0x5621c0c2cf80 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e5be00640 1 -- 172.21.15.116:0/1056330919 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680f67b0 con 0x5621c0c2cf80 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- 172.21.15.116:0/1056330919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 msgr2=0x5621c0d94590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3e6801cb70 tx=0x7f3e680adf90 comp rx=0 tx=0).stop 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- 172.21.15.116:0/1056330919 shutdown_connections 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0d94590 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c0c166b0 0x5621c0d94050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1056330919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c08447c0 0x5621c0d93b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.398+0000 7f3e7e753000 1 -- 172.21.15.116:0/1056330919 >> 172.21.15.116:0/1056330919 conn(0x5621c0d03340 msgr2=0x5621c0d8ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- 172.21.15.116:0/1056330919 shutdown_connections 2024-05-14T11:25:48.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- 172.21.15.116:0/1056330919 wait complete. 2024-05-14T11:25:48.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- start start 2024-05-14T11:25:48.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 Processor -- start 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c08447c0 0x5621c0d34900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5621c0b82f80 con 0x5621c0c2cf80 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5621c0b886c0 con 0x5621c0c166b0 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5621c0b8dc30 con 0x5621c08447c0 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 10 client.0 _ll_register_callbacks cb 0x7f3e5c0068f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38638/0 (socket says 172.21.15.116:38638) 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 -- 172.21.15.116:0/1754700380 learned_addr learned my addr 172.21.15.116:0/1754700380 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c08447c0 0x5621c0d34900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e6f400640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d34e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 -- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c08447c0 msgr2=0x5621c0d34900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c08447c0 0x5621c0d34900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 -- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 msgr2=0x5621c0d34e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d34e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 -- 172.21.15.116:0/1754700380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c0b5daf0 con 0x5621c0c2cf80 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e6f400640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e75000640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3e680c8040 tx=0x7f3e68009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed15f72c45d8bc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e59600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68126040 con 0x5621c0c2cf80 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3e68111c40 con 0x5621c0c2cf80 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e680b3040 con 0x5621c0c2cf80 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3e68111e40 con 0x5621c0c2cf80 2024-05-14T11:25:48.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 10 client.8340 fetch_fsmap learned FSMap version 100 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.402+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5621c0b5ce40 con 0x5621c0c2cf80 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680b3040 con 0x5621c0c2cf80 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f3e680ff5e0 con 0x5621c0c2cf80 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 10 client.8340 fetch_fsmap finished waiting for FSMap version 100 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 10 client.8340 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 20 client.8340 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c0817830 0x5621c0dade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 4 client.8340 mds_command: new command op to 8208 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 --> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] -- command(tid 0: {"prefix": "status"}) -- 0x5621c0b91ca0 con 0x5621c0817830 2024-05-14T11:25:48.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c0817830 0x5621c0dade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e6fe00640 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c0817830 0x5621c0dade40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c689cb11e6e39bb server_cookie=a046427a485656db in_seq=0 out_seq=0 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 10 client.8340 ms_handle_connect on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 1 -- 172.21.15.116:0/1754700380 <== mds.0 v2:172.21.15.116:6836/1742968683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5621c0b91ca0 con 0x5621c0817830 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 10 client.8340 handle_command_reply: tid=0 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 client.8340 shutdown 2024-05-14T11:25:48.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 msgr2=0x5621c0dad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3e680c8040 tx=0x7f3e68009750 comp rx=0 tx=0).stop 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 shutdown_connections 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x5621c0817830 0x5621c0dade40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621c0c166b0 0x5621c0d34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5621c0c2cf80 0x5621c0dad480 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 --2- 172.21.15.116:0/1754700380 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5621c08447c0 0x5621c0d34900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e59600640 0 client.8340 ms_handle_reset on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 >> 172.21.15.116:0/1754700380 conn(0x5621c0d03340 msgr2=0x5621c0d388c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 shutdown_connections 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 1 -- 172.21.15.116:0/1754700380 wait complete. 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e7e753000 20 client.8340 trim_cache size 0 max 16384 2024-05-14T11:25:48.407 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8208, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 5.9704343069999997, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 100, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 113, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 113, 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 10.854557416 2024-05-14T11:25:48.408 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 msgr2=0x7f3e50071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 0x7f3e50071910 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000f70 tx=0x7f3e60009040 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 msgr2=0x7f3e700b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b0040 tx=0x7f3e68019650 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 shutdown_connections 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f3e5006f400 0x7f3e50071910 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e70164f80 0x7f3e700b1ce0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e700b0770 0x7f3e70166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 --2- 172.21.15.116:0/3435031773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e7006f730 0x7f3e70152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 >> 172.21.15.116:0/3435031773 conn(0x7f3e700b4620 msgr2=0x7f3e7015fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 shutdown_connections 2024-05-14T11:25:48.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.406+0000 7f3e77000640 1 -- 172.21.15.116:0/3435031773 wait complete. 2024-05-14T11:25:48.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:25:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 Processor -- start 2024-05-14T11:25:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 -- start start 2024-05-14T11:25:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4c0160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff4c00b54b0 0x7ff4c00b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c006f5c0 con 0x7ff4c0070130 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c0161200 con 0x7ff4c00b54b0 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4c6600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c0161450 con 0x7ff4c0070610 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff4c0070130 0x7ff4c01605b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50734/0 (socket says 172.21.15.116:50734) 2024-05-14T11:25:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 -- 172.21.15.116:0/3474115013 learned_addr learned my addr 172.21.15.116:0/3474115013 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4c0160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1043422690 0 0) 0x7ff4c006f5c0 con 0x7ff4c0070130 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bfe00640 1 --2- 172.21.15.116:0/3474115013 >> v2:172.21.15.149:3300/0 conn(0x7ff4c00b54b0 0x7ff4c00b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4a4005740 con 0x7ff4c0070130 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2718760713 0 0) 0x7ff4a4005740 con 0x7ff4c0070130 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 msgr2=0x7ff4c0160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4c0160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 >> v2:172.21.15.149:3300/0 conn(0x7ff4c00b54b0 msgr2=0x7ff4c00b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 --2- 172.21.15.116:0/3474115013 >> v2:172.21.15.149:3300/0 conn(0x7ff4c00b54b0 0x7ff4c00b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c00d1fb0 con 0x7ff4c0070130 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7ff4b0001700 con 0x7ff4c0070130 2024-05-14T11:25:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 >> v1:172.21.15.116:6789/0 conn(0x7ff4c0070130 legacy=0x7ff4c01605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4a4006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4a4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4a4006f60 0x7ff4a4007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0161200 con 0x7ff4a4005c20 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0161450 con 0x7ff4a4006f60 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a400dcc0 con 0x7ff4c0070610 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4a4006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4a4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bfe00640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4a4006f60 0x7ff4a4007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 -- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 msgr2=0x7ff4a4006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4a4006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 -- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4a4006f60 msgr2=0x7ff4a4007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4a4006f60 0x7ff4a4007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bf400640 1 -- 172.21.15.116:0/3474115013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a4005740 con 0x7ff4a4005c20 2024-05-14T11:25:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.578+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c0070610 0x7ff4a4006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4a4006090 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c210 tx=0x7ff4b001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82d7bcf6384b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0007d60 con 0x7ff4a4005c20 2024-05-14T11:25:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b00a9070 con 0x7ff4a4005c20 2024-05-14T11:25:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4be000640 1 -- 172.21.15.116:0/3474115013 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0005d20 con 0x7ff4a4005c20 2024-05-14T11:25:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 -- 172.21.15.116:0/3474115013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 msgr2=0x7ff4a4006090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3474115013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4a4006090 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c210 tx=0x7ff4b001cc00 comp rx=0 tx=0).stop 2024-05-14T11:25:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4bfe00640 1 -- 172.21.15.116:0/3474115013 reap_dead start 2024-05-14T11:25:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 -- 172.21.15.116:0/3474115013 shutdown_connections 2024-05-14T11:25:48.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 -- 172.21.15.116:0/3474115013 >> 172.21.15.116:0/3474115013 conn(0x7ff4c0071370 msgr2=0x7ff4c0071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 -- 172.21.15.116:0/3474115013 shutdown_connections 2024-05-14T11:25:48.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.582+0000 7ff4c6600640 1 -- 172.21.15.116:0/3474115013 wait complete. 2024-05-14T11:25:48.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 Processor -- start 2024-05-14T11:25:48.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 -- start start 2024-05-14T11:25:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 0x7ff4c015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 0x7ff4c00ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c006d110 con 0x7ff4a4005c20 2024-05-14T11:25:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c006cf90 con 0x7ff4c015d570 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4c6600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c006d290 con 0x7ff4c00b0e10 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4bf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4bf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38652/0 (socket says 172.21.15.116:38652) 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4bf400640 1 -- 172.21.15.116:0/3020193554 learned_addr learned my addr 172.21.15.116:0/3020193554 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 0x7ff4c015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.586+0000 7ff4bfe00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 0x7ff4c00ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 -- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 msgr2=0x7ff4c015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 0x7ff4c015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 -- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 msgr2=0x7ff4c00ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 0x7ff4c00ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 -- 172.21.15.116:0/3020193554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c015d980 con 0x7ff4a4005c20 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bfe00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 0x7ff4c00ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 0x7ff4c015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4bf400640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c210 tx=0x7ff4a4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00047f0 con 0x7ff4a4005c20 2024-05-14T11:25:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b0004990 con 0x7ff4a4005c20 2024-05-14T11:25:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c01f95d0 con 0x7ff4a4005c20 2024-05-14T11:25:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0004d80 con 0x7ff4a4005c20 2024-05-14T11:25:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c01f9b90 con 0x7ff4a4005c20 2024-05-14T11:25:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c006a9c0 con 0x7ff4a4005c20 2024-05-14T11:25:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.590+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4b00b7030 con 0x7ff4a4005c20 2024-05-14T11:25:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.594+0000 7ff4b7400640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 0x7ff4a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.594+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7ff4b0128580 con 0x7ff4a4005c20 2024-05-14T11:25:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.594+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 0x7ff4a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.594+0000 7ff4bea00640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 0x7ff4a0071910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0071e50 tx=0x7ff4ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.598+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff4c015d980 con 0x7ff4a4005c20 2024-05-14T11:25:48.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.786+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4c00b23c0 con 0x7ff4a4005c20 2024-05-14T11:25:48.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.786+0000 7ff4b7400640 1 -- 172.21.15.116:0/3020193554 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7ff4b00169b0 con 0x7ff4a4005c20 2024-05-14T11:25:48.786 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 msgr2=0x7ff4a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 0x7ff4a0071910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0071e50 tx=0x7ff4ac008040 comp rx=0 tx=0).stop 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 msgr2=0x7ff4c00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c210 tx=0x7ff4a4007a50 comp rx=0 tx=0).stop 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 shutdown_connections 2024-05-14T11:25:48.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff4a006f400 0x7ff4a0071910 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4c015d570 0x7ff4c00ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff4c00b0e10 0x7ff4c015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 --2- 172.21.15.116:0/3020193554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4a4005c20 0x7ff4c00b05f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 >> 172.21.15.116:0/3020193554 conn(0x7ff4c0071370 msgr2=0x7ff4c006dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 shutdown_connections 2024-05-14T11:25:48.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:48.790+0000 7ff4c6600640 1 -- 172.21.15.116:0/3020193554 wait complete. 2024-05-14T11:25:48.807 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:25:48.808 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:48.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:48.808 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:48.817 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41854sec preferred_lft 41854sec 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.818 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:25:48.819 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:48.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:48.820 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:25:48.879 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:25:48.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:48.881 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:25:48.932 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:48.933 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:25:48.934 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:25:49.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:25:49.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:49.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.068 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:25:49.077 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:25:49.077 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:25:49.077 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:49.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.077 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:25:49.137 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:25:49.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:25:49.137 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:49.189 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:49.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:25:49.190 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:49.241 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:25:49.302 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:25:49.359 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:25:49.360 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:49.361 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:49.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.362 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:25:49.417 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:25:49.418 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:25:49.461 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:25:49.481 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:25:49.531 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:25:49.531 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:25:49.531 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:25:49.532 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:49.533 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.534 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:25:49.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:25:49.570+0000 7fbdb42bd580 -1 init, newargv = 0x560139d510f0 newargc=15 2024-05-14T11:25:49.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49036]: starting ceph client 2024-05-14T11:25:49.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49036]: starting fuse 2024-05-14T11:25:49.576 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:25:49.577 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:25:49.577 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:25:49.578 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:49.697 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.49036.asok 2024-05-14T11:25:49.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:49.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49036.asok 2024-05-14T11:25:49.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.708 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49036.asok status 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "49036", 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:25:49.885 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8352, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 8352 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 4202537462 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 4202537462 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.8352 192.168.0.1:0/4202537462", 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/4202537462", 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 100, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 113, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:25:49.887 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:49.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.901 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:25:49.901 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:49.910 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:25:49.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:49.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:49.910 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:49.969 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> parent = os.path.join(mount_point, "parent") 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> os.mkdir(parent) 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> flushed = os.path.join(parent, "flushed") 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> os.mkdir(flushed) 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> for f in ["alpha", "bravo", "charlie"]: 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-14T11:25:49.970 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:50.103 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-14T11:25:50.167 INFO:teuthology.orchestra.run.smithi116.stdout:./ 2024-05-14T11:25:50.168 INFO:teuthology.orchestra.run.smithi116.stdout:./parent 2024-05-14T11:25:50.168 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed 2024-05-14T11:25:50.168 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/charlie 2024-05-14T11:25:50.168 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/alpha 2024-05-14T11:25:50.168 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/bravo 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-14T11:25:50.169 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.274 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:25:50.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-14T11:25:50.283 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.390 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:25:50.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.398 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.399 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.399 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.399 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.399 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-14T11:25:50.399 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.513 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:25:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.523 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.523 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.524 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.524 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.524 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-14T11:25:50.524 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.630 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628280 2024-05-14T11:25:50.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-14T11:25:50.640 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.750 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627778 2024-05-14T11:25:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-14T11:25:50.760 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:50.866 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628279 2024-05-14T11:25:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:50.875 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:50.875 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:50.926 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:25:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 Processor -- start 2024-05-14T11:25:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 -- start start 2024-05-14T11:25:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6fac0d2780 0x7f6fac0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac0d18b0 con 0x7f6fac161e80 2024-05-14T11:25:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac071510 con 0x7f6fac0b8fc0 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb3400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6fac0d1be0 con 0x7f6fac0d2780 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fb1400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6fac0d2780 0x7f6fac0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fabe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6fac161e80 0x7f6fac0d2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50740/0 (socket says 172.21.15.116:50740) 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.118+0000 7f6fabe00640 1 -- 172.21.15.116:0/1244829129 learned_addr learned my addr 172.21.15.116:0/1244829129 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb1400640 1 -- 172.21.15.116:0/1244829129 >> v1:172.21.15.149:6789/0 conn(0x7f6fac0b8fc0 legacy=0x7f6fac161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb1400640 1 -- 172.21.15.116:0/1244829129 >> v1:172.21.15.116:6789/0 conn(0x7f6fac161e80 legacy=0x7f6fac0d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb1400640 1 -- 172.21.15.116:0/1244829129 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac0717f0 con 0x7f6fac0d2780 2024-05-14T11:25:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb1400640 1 --2- 172.21.15.116:0/1244829129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac0d2780 0x7f6fac0d2b90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40097c0 tx=0x7f6fa401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a219212cb7400fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fab400640 1 -- 172.21.15.116:0/1244829129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b0070 con 0x7f6fac0d2780 2024-05-14T11:25:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fab400640 1 -- 172.21.15.116:0/1244829129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa40ab600 con 0x7f6fac0d2780 2024-05-14T11:25:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fab400640 1 -- 172.21.15.116:0/1244829129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40ab9d0 con 0x7f6fac0d2780 2024-05-14T11:25:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb3400640 1 -- 172.21.15.116:0/1244829129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac0d2780 msgr2=0x7f6fac0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.122+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1244829129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac0d2780 0x7f6fac0d2b90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40097c0 tx=0x7f6fa401c4a0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- 172.21.15.116:0/1244829129 shutdown_connections 2024-05-14T11:25:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1244829129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac0d2780 0x7f6fac0d2b90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- 172.21.15.116:0/1244829129 >> 172.21.15.116:0/1244829129 conn(0x7f6fac0d3740 msgr2=0x7f6fac0d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- 172.21.15.116:0/1244829129 shutdown_connections 2024-05-14T11:25:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- 172.21.15.116:0/1244829129 wait complete. 2024-05-14T11:25:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 Processor -- start 2024-05-14T11:25:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- start start 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fac0b8fc0 0x7f6fac14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fac0d2780 0x7f6fac14c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac16c6e0 con 0x7f6fac1500f0 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac0b95e0 con 0x7f6fac0d2780 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.126+0000 7f6fb3400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac16c860 con 0x7f6fac0b8fc0 2024-05-14T11:25:51.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38670/0 (socket says 172.21.15.116:38670) 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 -- 172.21.15.116:0/1207263014 learned_addr learned my addr 172.21.15.116:0/1207263014 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb0a00640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fac0b8fc0 0x7f6fac14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fabe00640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fac0d2780 0x7f6fac14c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 -- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fac0b8fc0 msgr2=0x7f6fac14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fac0b8fc0 0x7f6fac14f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 -- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fac0d2780 msgr2=0x7f6fac14c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fac0d2780 0x7f6fac14c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 -- 172.21.15.116:0/1207263014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac14fa70 con 0x7f6fac1500f0 2024-05-14T11:25:51.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb1400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6fa401a0d0 tx=0x7f6fa401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b0050 con 0x7f6fac1500f0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa40bc070 con 0x7f6fac1500f0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac14e2b0 con 0x7f6fac1500f0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b5830 con 0x7f6fac1500f0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac14e870 con 0x7f6fac1500f0 2024-05-14T11:25:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac06a9c0 con 0x7f6fac1500f0 2024-05-14T11:25:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.130+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6fa40b59d0 con 0x7f6fac1500f0 2024-05-14T11:25:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.134+0000 7f6fa8c00640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 0x7f6f8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.134+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f6fa412a870 con 0x7f6fac1500f0 2024-05-14T11:25:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.134+0000 7f6fb0a00640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 0x7f6f8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.134+0000 7f6fb0a00640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 0x7f6f8c071910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00b450 tx=0x7f6f9c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.138+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6fa4016090 con 0x7f6fac1500f0 2024-05-14T11:25:51.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.302+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6fac0d1690 con 0x7f6fac1500f0 2024-05-14T11:25:51.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fa8c00640 1 -- 172.21.15.116:0/1207263014 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f6fa40f7640 con 0x7f6fac1500f0 2024-05-14T11:25:51.303 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 msgr2=0x7f6f8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 0x7f6f8c071910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00b450 tx=0x7f6f9c00d040 comp rx=0 tx=0).stop 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 msgr2=0x7f6fac0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6fa401a0d0 tx=0x7f6fa401ace0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 shutdown_connections 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6f8c06f400 0x7f6f8c071910 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fac0d2780 0x7f6fac14c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fac1500f0 0x7f6fac0ca7d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 --2- 172.21.15.116:0/1207263014 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fac0b8fc0 0x7f6fac14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 >> 172.21.15.116:0/1207263014 conn(0x7f6fac0d3740 msgr2=0x7f6fac1551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:51.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 shutdown_connections 2024-05-14T11:25:51.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.306+0000 7f6fb3400640 1 -- 172.21.15.116:0/1207263014 wait complete. 2024-05-14T11:25:51.324 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:25:51.325 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:25:51.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:25:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.526+0000 7fbe12800640 1 Processor -- start 2024-05-14T11:25:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.526+0000 7fbe12800640 1 -- start start 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.526+0000 7fbe12800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.526+0000 7fbe12800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbe0c06f520 0x7fbe0c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.526+0000 7fbe12800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe0c15dbb0 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe12800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c074530 con 0x7fbe0c06f520 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe12800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c1610c0 con 0x7fbe0c0b6100 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbe0c06f520 0x7fbe0c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe10a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbe0c06dfa0 0x7fbe0c06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50744/0 (socket says 172.21.15.116:50744) 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe10a00640 1 -- 172.21.15.116:0/1673693206 learned_addr learned my addr 172.21.15.116:0/1673693206 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11400640 1 --2- 172.21.15.116:0/1673693206 >> v2:172.21.15.149:3300/0 conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 -- 172.21.15.116:0/1673693206 >> v2:172.21.15.149:3300/0 conn(0x7fbe0c0b6100 msgr2=0x7fbe0c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 --2- 172.21.15.116:0/1673693206 >> v2:172.21.15.149:3300/0 conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 -- 172.21.15.116:0/1673693206 >> v1:172.21.15.116:6789/0 conn(0x7fbe0c06dfa0 legacy=0x7fbe0c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 -- 172.21.15.116:0/1673693206 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c1604b0 con 0x7fbe0c06f520 2024-05-14T11:25:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe0be00640 1 -- 172.21.15.116:0/1673693206 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007734974 0 0) 0x7fbe0c15dbb0 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.530+0000 7fbe11e00640 1 --2- 172.21.15.116:0/1673693206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06f520 0x7fbe0c06f930 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c06c9b0 tx=0x7fbe0401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6f2941503fd5d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe0be00640 1 -- 172.21.15.116:0/1673693206 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040b2070 con 0x7fbe0c06f520 2024-05-14T11:25:51.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe0be00640 1 -- 172.21.15.116:0/1673693206 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe040a5930 con 0x7fbe0c06f520 2024-05-14T11:25:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe0be00640 1 -- 172.21.15.116:0/1673693206 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040a5d00 con 0x7fbe0c06f520 2024-05-14T11:25:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- 172.21.15.116:0/1673693206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06f520 msgr2=0x7fbe0c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 --2- 172.21.15.116:0/1673693206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06f520 0x7fbe0c06f930 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c06c9b0 tx=0x7fbe0401c870 comp rx=0 tx=0).stop 2024-05-14T11:25:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- 172.21.15.116:0/1673693206 shutdown_connections 2024-05-14T11:25:51.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 --2- 172.21.15.116:0/1673693206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06f520 0x7fbe0c06f930 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 --2- 172.21.15.116:0/1673693206 >> v2:172.21.15.149:3300/0 conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- 172.21.15.116:0/1673693206 >> 172.21.15.116:0/1673693206 conn(0x7fbe0c0731e0 msgr2=0x7fbe0c0735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:51.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- 172.21.15.116:0/1673693206 shutdown_connections 2024-05-14T11:25:51.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- 172.21.15.116:0/1673693206 wait complete. 2024-05-14T11:25:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 Processor -- start 2024-05-14T11:25:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.534+0000 7fbe12800640 1 -- start start 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 0x7fbe0c158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c16caf0 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c160160 con 0x7fbe0c06f520 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c16cc70 con 0x7fbe0c0b6100 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38696/0 (socket says 172.21.15.116:38696) 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 -- 172.21.15.116:0/1377903227 learned_addr learned my addr 172.21.15.116:0/1377903227 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe10a00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 0x7fbe0c158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11e00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 -- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 msgr2=0x7fbe0c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 -- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 msgr2=0x7fbe0c158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 0x7fbe0c158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 -- 172.21.15.116:0/1377903227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c15ad70 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe10a00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 0x7fbe0c158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11e00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe11400640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e870 tx=0x7fbe0000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00017070 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0000c940 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c15b050 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cd10 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c167170 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.538+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c1673d0 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.542+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe0001a070 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.542+0000 7fbe09600640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 0x7fbdec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:51.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.542+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fbe0008f6b0 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.542+0000 7fbe10a00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 0x7fbdec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:51.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.542+0000 7fbe10a00640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 0x7fbdec071910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b1d50 tx=0x7fbdf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:51.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.546+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbe0005c400 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.714+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbe0c050880 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.714+0000 7fbe09600640 1 -- 172.21.15.116:0/1377903227 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7fbe0005bd00 con 0x7fbe0c06dfa0 2024-05-14T11:25:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:25:51.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 msgr2=0x7fbdec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 0x7fbdec071910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b1d50 tx=0x7fbdf8009040 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 msgr2=0x7fbe0c15cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e870 tx=0x7fbe0000ec70 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 shutdown_connections 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbdec06f400 0x7fbdec071910 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe0c06f520 0x7fbe0c158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe0c0b6100 0x7fbe0c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 --2- 172.21.15.116:0/1377903227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe0c06dfa0 0x7fbe0c15cac0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 >> 172.21.15.116:0/1377903227 conn(0x7fbe0c0731e0 msgr2=0x7fbe0c0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 shutdown_connections 2024-05-14T11:25:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:51.718+0000 7fbe12800640 1 -- 172.21.15.116:0/1377903227 wait complete. 2024-05-14T11:25:51.735 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:25:51.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:51.735 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:51.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49036]: fuse finished with error 0 and tester_r 0 2024-05-14T11:25:51.756 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:25:57.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:25:57.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:57.859 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:25:57.869 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.074+0000 7f099fe00640 1 Processor -- start 2024-05-14T11:25:58.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.074+0000 7f099fe00640 1 -- start start 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a0163870 0x7f09a0163c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f09a00ce340 0x7f09a00ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099fe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a01641c0 con 0x7f09a00b5d70 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f09a00708a0 con 0x7f09a00ce340 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09a00b6550 con 0x7f09a0163870 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f09a00b5d70 0x7f09a0070190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41904/0 (socket says 172.21.15.116:41904) 2024-05-14T11:25:58.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099ea00640 1 -- 172.21.15.116:0/1379448637 learned_addr learned my addr 172.21.15.116:0/1379448637 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099e000640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a0163870 0x7f09a0163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> v2:172.21.15.149:3300/0 conn(0x7f09a00ce340 0x7f09a00ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360123883 0 0) 0x7f098c001120 con 0x7f09a00b5d70 2024-05-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0984003530 con 0x7f09a00b5d70 2024-05-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3329844923 0 0) 0x7f0984003530 con 0x7f09a00b5d70 2024-05-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a0163870 msgr2=0x7f09a0163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a0163870 0x7f09a0163c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 >> v2:172.21.15.149:3300/0 conn(0x7f09a00ce340 msgr2=0x7f09a00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 --2- 172.21.15.116:0/1379448637 >> v2:172.21.15.149:3300/0 conn(0x7f09a00ce340 0x7f09a00ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a00b6830 con 0x7f09a00b5d70 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f098c0018d0 con 0x7f09a00b5d70 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 >> v1:172.21.15.116:6789/0 conn(0x7f09a00b5d70 legacy=0x7f09a0070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0984003a10 0x7f0984003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0984004810 0x7f0984004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f098400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00708a0 con 0x7f0984005160 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00b6550 con 0x7f0984004810 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f098400c070 con 0x7f0984003a10 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099e000640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a0163870 0x7f09a0163c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> v2:172.21.15.149:3300/0 conn(0x7f09a00ce340 0x7f09a00ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:25:58.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099ea00640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0984003a10 0x7f0984003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f098400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099e000640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0984004810 0x7f0984004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 -- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0984003a10 msgr2=0x7f0984003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0984003a10 0x7f0984003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 -- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0984004810 msgr2=0x7f0984004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0984004810 0x7f0984004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099f400640 1 -- 172.21.15.116:0/1379448637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0984003530 con 0x7f0984005160 2024-05-14T11:25:58.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.078+0000 7f099e000640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0984004810 0x7f0984004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099f400640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f098400b9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f099800e320 tx=0x7f099801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb2e14374fd6659 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998009dc0 con 0x7f0984005160 2024-05-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09980a5e30 con 0x7f0984005160 2024-05-14T11:25:58.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099d600640 1 -- 172.21.15.116:0/1379448637 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998003ba0 con 0x7f0984005160 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 -- 172.21.15.116:0/1379448637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 msgr2=0x7f098400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 --2- 172.21.15.116:0/1379448637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f098400b9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f099800e320 tx=0x7f099801ce20 comp rx=0 tx=0).stop 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099f400640 1 -- 172.21.15.116:0/1379448637 reap_dead start 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 -- 172.21.15.116:0/1379448637 shutdown_connections 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 -- 172.21.15.116:0/1379448637 >> 172.21.15.116:0/1379448637 conn(0x7f09a00711b0 msgr2=0x7f09a00715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 -- 172.21.15.116:0/1379448637 shutdown_connections 2024-05-14T11:25:58.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.082+0000 7f099fe00640 1 -- 172.21.15.116:0/1379448637 wait complete. 2024-05-14T11:25:58.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 Processor -- start 2024-05-14T11:25:58.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- start start 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09a015e310 0x7f09a0147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a01486e0 0x7f09a0159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cec90 con 0x7f0984005160 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00b6df0 con 0x7f09a015e310 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09a015ee00 con 0x7f09a01486e0 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44374/0 (socket says 172.21.15.116:44374) 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 -- 172.21.15.116:0/2793175695 learned_addr learned my addr 172.21.15.116:0/2793175695 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099f400640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a01486e0 0x7f09a0159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099e000640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09a015e310 0x7f09a0147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 -- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a01486e0 msgr2=0x7f09a0159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a01486e0 0x7f09a0159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 -- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09a015e310 msgr2=0x7f09a0147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09a015e310 0x7f09a0147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 -- 172.21.15.116:0/2793175695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a01468f0 con 0x7f0984005160 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099ea00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f098c001330 tx=0x7f098c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f0996a00640 1 -- 172.21.15.116:0/2793175695 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c007740 con 0x7f0984005160 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f0996a00640 1 -- 172.21.15.116:0/2793175695 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c018070 con 0x7f0984005160 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a0146bd0 con 0x7f0984005160 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f0996a00640 1 -- 172.21.15.116:0/2793175695 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00fc00 con 0x7f0984005160 2024-05-14T11:25:58.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.086+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a0147190 con 0x7f0984005160 2024-05-14T11:25:58.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f0996a00640 1 -- 172.21.15.116:0/2793175695 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f098c01d460 con 0x7f0984005160 2024-05-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f0996a00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 0x7f0980071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f0996a00640 1 -- 172.21.15.116:0/2793175695 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f098c08f530 con 0x7f0984005160 2024-05-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f099e000640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 0x7f0980071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f09acff2000 1 Processor -- start 2024-05-14T11:25:58.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f09acff2000 1 -- start start 2024-05-14T11:25:58.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.090+0000 7f099e000640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 0x7f0980071910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09a00b1b70 tx=0x7f0990008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b29472350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29472dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561b29277100 con 0x561b292f8f00 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b2922fca0 con 0x561b28ef83e0 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561b292730c0 con 0x561b2930b120 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44390/0 (socket says 172.21.15.116:44390) 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 -- 172.21.15.116:0/4058804732 learned_addr learned my addr 172.21.15.116:0/4058804732 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099ea00640 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b29472350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099f400640 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29472dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 -- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 msgr2=0x561b29472350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b29472350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 -- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 msgr2=0x561b29472dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29472dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 -- 172.21.15.116:0/4058804732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b29234800 con 0x561b292f8f00 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f099e000640 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0990013cf0 tx=0x7f0990016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52735e0c5e7af049 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f098b400640 1 -- 172.21.15.116:0/4058804732 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990023440 con 0x561b292f8f00 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f098b400640 1 -- 172.21.15.116:0/4058804732 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0990028070 con 0x561b292f8f00 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f098b400640 1 -- 172.21.15.116:0/4058804732 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09900237e0 con 0x561b292f8f00 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- 172.21.15.116:0/4058804732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 msgr2=0x561b29472890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0990013cf0 tx=0x7f0990016000 comp rx=0 tx=0).stop 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- 172.21.15.116:0/4058804732 shutdown_connections 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29472dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29472890 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 --2- 172.21.15.116:0/4058804732 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b29472350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- 172.21.15.116:0/4058804732 >> 172.21.15.116:0/4058804732 conn(0x561b28f38400 msgr2=0x561b29455310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- 172.21.15.116:0/4058804732 shutdown_connections 2024-05-14T11:25:58.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- 172.21.15.116:0/4058804732 wait complete. 2024-05-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.094+0000 7f09acff2000 1 -- start start 2024-05-14T11:25:58.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 Processor -- start 2024-05-14T11:25:58.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29478dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2942bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561b293422a0 con 0x561b292f8f00 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b29332f40 con 0x561b28ef83e0 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561b29233080 con 0x561b2930b120 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 10 client.0 _ll_register_callbacks cb 0x561b29413c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44394/0 (socket says 172.21.15.116:44394) 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 -- 172.21.15.116:0/283491299 learned_addr learned my addr 172.21.15.116:0/283491299 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099f400640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2942bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099ea00640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29478dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 -- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 msgr2=0x561b2942bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2942bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 -- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 msgr2=0x561b29478dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29478dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 -- 172.21.15.116:0/283491299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b29221860 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f099e000640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0990008010 tx=0x7f0990004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849e7b386edfdb69 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990013ea0 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0990028070 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f0990023850 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0990041050 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 10 client.8382 fetch_fsmap learned FSMap version 100 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561b2922fef0 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0990028070 con 0x561b292f8f00 2024-05-14T11:25:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f0990044020 con 0x561b292f8f00 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 10 client.8382 fetch_fsmap finished waiting for FSMap version 100 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 10 client.8382 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 20 client.8382 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f1c930 0x561b2942cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 4 client.8382 mds_command: new command op to 8208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.098+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 --> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2920fdf0 con 0x561b28f1c930 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f099ea00640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f1c930 0x561b2942cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f099ea00640 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f1c930 0x561b2942cd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e014aeb75b7e3a8 server_cookie=858f3234e217b537 in_seq=0 out_seq=0 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f097fe00640 10 client.8382 ms_handle_connect on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f097fe00640 1 -- 172.21.15.116:0/283491299 <== mds.0 v2:172.21.15.116:6836/1742968683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b2920fdf0 con 0x561b28f1c930 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f097fe00640 10 client.8382 handle_command_reply: tid=0 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 client.8382 shutdown 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 msgr2=0x561b29479310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0990008010 tx=0x7f0990004740 comp rx=0 tx=0).stop 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 shutdown_connections 2024-05-14T11:25:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f1c930 0x561b2942cd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2942bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b292f8f00 0x561b29479310 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 --2- 172.21.15.116:0/283491299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b28ef83e0 0x561b29478dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f097fe00640 0 client.8382 ms_handle_reset on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 >> 172.21.15.116:0/283491299 conn(0x561b28f38400 msgr2=0x561b29455200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 shutdown_connections 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 1 -- 172.21.15.116:0/283491299 wait complete. 2024-05-14T11:25:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.102+0000 7f09acff2000 20 client.8382 trim_cache size 0 max 16384 2024-05-14T11:25:58.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 Processor -- start 2024-05-14T11:25:58.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- start start 2024-05-14T11:25:58.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b2947ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09a02027e0 con 0x561b28ef83e0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09a015b7a0 con 0x561b292f8f00 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09a0202190 con 0x561b2930b120 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44410/0 (socket says 172.21.15.116:44410) 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 -- 172.21.15.116:0/3215887511 learned_addr learned my addr 172.21.15.116:0/3215887511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 -- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 msgr2=0x561b2947b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099e000640 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b2947ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 -- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 msgr2=0x561b2947ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b2947ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 -- 172.21.15.116:0/3215887511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2942a070 con 0x561b28ef83e0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099e000640 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b2947ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f099ea00640 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f098c057b70 tx=0x7f098c072040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7b62667f069ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f098aa00640 1 -- 172.21.15.116:0/3215887511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c064290 con 0x561b28ef83e0 2024-05-14T11:25:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f098aa00640 1 -- 172.21.15.116:0/3215887511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c05baf0 con 0x561b28ef83e0 2024-05-14T11:25:58.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f098aa00640 1 -- 172.21.15.116:0/3215887511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c05fea0 con 0x561b28ef83e0 2024-05-14T11:25:58.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- 172.21.15.116:0/3215887511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 msgr2=0x561b2947e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f098c057b70 tx=0x7f098c072040 comp rx=0 tx=0).stop 2024-05-14T11:25:58.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- 172.21.15.116:0/3215887511 shutdown_connections 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b2947ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 --2- 172.21.15.116:0/3215887511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b2947e260 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.118+0000 7f09acff2000 1 -- 172.21.15.116:0/3215887511 >> 172.21.15.116:0/3215887511 conn(0x561b28f38400 msgr2=0x561b29453d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- 172.21.15.116:0/3215887511 shutdown_connections 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- 172.21.15.116:0/3215887511 wait complete. 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- start start 2024-05-14T11:25:58.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 Processor -- start 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b294e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561b2927b730 con 0x561b28ef83e0 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b29276410 con 0x561b292f8f00 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x561b29284500 con 0x561b2930b120 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 10 client.0 _ll_register_callbacks cb 0x7f0984003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44422/0 (socket says 172.21.15.116:44422) 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 -- 172.21.15.116:0/3446766305 learned_addr learned my addr 172.21.15.116:0/3446766305 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099f400640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099e000640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b294e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 -- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 msgr2=0x561b2947c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 -- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 msgr2=0x561b294e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b294e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 -- 172.21.15.116:0/3446766305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b292f8340 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f099ea00640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f098c05ba80 tx=0x7f098c08f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073587e47a2afc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f097fe00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c064950 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f098c0644c0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.122+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c05baf0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f098c05fea0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 10 client.8388 fetch_fsmap learned FSMap version 100 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561b292138e0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c05baf0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f098c08d2b0 con 0x561b28ef83e0 2024-05-14T11:25:58.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 10 client.8388 fetch_fsmap finished waiting for FSMap version 100 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 10 client.8388 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 20 client.8388 populate_metadata read hostname 'smithi116' 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f0a990 0x561b2947d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 4 client.8388 mds_command: new command op to 8208 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 --> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b29284d10 con 0x561b28f0a990 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f099e000640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f0a990 0x561b2947d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f099e000640 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f0a990 0x561b2947d0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13323e9442c557c server_cookie=737d7b66986cddd6 in_seq=0 out_seq=0 2024-05-14T11:25:58.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.126+0000 7f097fe00640 10 client.8388 ms_handle_connect on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f097fe00640 1 -- 172.21.15.116:0/3446766305 <== mds.0 v2:172.21.15.116:6836/1742968683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b29284d10 con 0x561b28f0a990 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f097fe00640 10 client.8388 handle_command_reply: tid=0 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 client.8388 shutdown 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 msgr2=0x561b294e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f098c05ba80 tx=0x7f098c08f040 comp rx=0 tx=0).stop 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 shutdown_connections 2024-05-14T11:25:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:6836/1742968683,v1:172.21.15.116:6837/1742968683] conn(0x561b28f0a990 0x561b2947d0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b292f8f00 0x561b294e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x561b2930b120 0x561b2947c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 --2- 172.21.15.116:0/3446766305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561b28ef83e0 0x561b294e4d40 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f097fe00640 0 client.8388 ms_handle_reset on v2:172.21.15.116:6836/1742968683 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 >> 172.21.15.116:0/3446766305 conn(0x561b28f38400 msgr2=0x561b29453300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 shutdown_connections 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 1 -- 172.21.15.116:0/3446766305 wait complete. 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f09acff2000 20 client.8388 trim_cache size 0 max 16384 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "", 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0 2024-05-14T11:25:58.141 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 msgr2=0x7f0980071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 0x7f0980071910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09a00b1b70 tx=0x7f0990008040 comp rx=0 tx=0).stop 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 msgr2=0x7f09a0145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f098c001330 tx=0x7f098c0074f0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 shutdown_connections 2024-05-14T11:25:58.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f098006f400 0x7f0980071910 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09a015e310 0x7f09a0147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09a01486e0 0x7f09a0159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 --2- 172.21.15.116:0/2793175695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0984005160 0x7f09a0145420 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 >> 172.21.15.116:0/2793175695 conn(0x7f09a00711b0 msgr2=0x7f09a00b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 shutdown_connections 2024-05-14T11:25:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.142+0000 7f099fe00640 1 -- 172.21.15.116:0/2793175695 wait complete. 2024-05-14T11:25:58.160 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:25:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 Processor -- start 2024-05-14T11:25:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 -- start start 2024-05-14T11:25:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb31c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb31c06c700 0x7fb31c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb31c06ff20 con 0x7fb31c15db30 2024-05-14T11:25:58.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb31c0ae100 con 0x7fb31c06c700 2024-05-14T11:25:58.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb321200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c0ae870 con 0x7fb31c0d1670 2024-05-14T11:25:58.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb31c15db30 0x7fb31c0aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41914/0 (socket says 172.21.15.116:41914) 2024-05-14T11:25:58.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319e00640 1 -- 172.21.15.116:0/3249496622 learned_addr learned my addr 172.21.15.116:0/3249496622 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:25:58.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144481305 0 0) 0x7fb31c06ff20 con 0x7fb31c15db30 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb300003530 con 0x7fb31c15db30 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb313e00640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb31c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 --2- 172.21.15.116:0/3249496622 >> v2:172.21.15.149:3300/0 conn(0x7fb31c06c700 0x7fb31c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 485378332 0 0) 0x7fb300003530 con 0x7fb31c15db30 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 msgr2=0x7fb31c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb31c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 >> v2:172.21.15.149:3300/0 conn(0x7fb31c06c700 msgr2=0x7fb31c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 --2- 172.21.15.116:0/3249496622 >> v2:172.21.15.149:3300/0 conn(0x7fb31c06c700 0x7fb31c06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0d1fc0 con 0x7fb31c15db30 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fb30c001700 con 0x7fb31c15db30 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 >> v1:172.21.15.116:6789/0 conn(0x7fb31c15db30 legacy=0x7fb31c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb300003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004090 0x7fb300004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb300004da0 0x7fb300005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ae100 con 0x7fb300004da0 2024-05-14T11:25:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ae870 con 0x7fb300004090 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb30000bd50 con 0x7fb31c0d1670 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb300004da0 0x7fb300005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb313e00640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004090 0x7fb300004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb319e00640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb300003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 -- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 msgr2=0x7fb300003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c0d1670 0x7fb300003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 -- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004090 msgr2=0x7fb300004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004090 0x7fb300004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.310+0000 7fb31a800640 1 -- 172.21.15.116:0/3249496622 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb300003530 con 0x7fb300004da0 2024-05-14T11:25:58.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb31a800640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb300004da0 0x7fb300005270 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0b2dd0 tx=0x7fb31401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db2f78f8a6b5447 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140b2070 con 0x7fb300004da0 2024-05-14T11:25:58.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3140a5ac0 con 0x7fb300004da0 2024-05-14T11:25:58.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb319400640 1 -- 172.21.15.116:0/3249496622 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314009be0 con 0x7fb300004da0 2024-05-14T11:25:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 -- 172.21.15.116:0/3249496622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb300004da0 msgr2=0x7fb300005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 --2- 172.21.15.116:0/3249496622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb300004da0 0x7fb300005270 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0b2dd0 tx=0x7fb31401ce20 comp rx=0 tx=0).stop 2024-05-14T11:25:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb31a800640 1 -- 172.21.15.116:0/3249496622 reap_dead start 2024-05-14T11:25:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 -- 172.21.15.116:0/3249496622 shutdown_connections 2024-05-14T11:25:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 -- 172.21.15.116:0/3249496622 >> 172.21.15.116:0/3249496622 conn(0x7fb31c0b4620 msgr2=0x7fb31c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 -- 172.21.15.116:0/3249496622 shutdown_connections 2024-05-14T11:25:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.314+0000 7fb321200640 1 -- 172.21.15.116:0/3249496622 wait complete. 2024-05-14T11:25:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 Processor -- start 2024-05-14T11:25:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 -- start start 2024-05-14T11:25:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004da0 0x7fb31c1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c148d00 0x7fb31c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c168760 con 0x7fb31c1557b0 2024-05-14T11:25:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c15df40 con 0x7fb300004da0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb321200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c1688e0 con 0x7fb31c148d00 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb31a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb31a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44438/0 (socket says 172.21.15.116:44438) 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.318+0000 7fb31a800640 1 -- 172.21.15.116:0/3278716729 learned_addr learned my addr 172.21.15.116:0/3278716729 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb319e00640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004da0 0x7fb31c1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb313e00640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c148d00 0x7fb31c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 -- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c148d00 msgr2=0x7fb31c155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c148d00 0x7fb31c155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 -- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004da0 msgr2=0x7fb31c1487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004da0 0x7fb31c1487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 -- 172.21.15.116:0/3278716729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0b13c0 con 0x7fb31c1557b0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb31a800640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb31401c890 tx=0x7fb300005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140b2040 con 0x7fb31c1557b0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3140a58c0 con 0x7fb31c1557b0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c072a30 con 0x7fb31c1557b0 2024-05-14T11:25:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314009be0 con 0x7fb31c1557b0 2024-05-14T11:25:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c072ff0 con 0x7fb31c1557b0 2024-05-14T11:25:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c06a9c0 con 0x7fb31c1557b0 2024-05-14T11:25:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.322+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3140bd020 con 0x7fb31c1557b0 2024-05-14T11:25:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.326+0000 7fb312000640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 0x7fb2f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:25:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.326+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fb314128b00 con 0x7fb31c1557b0 2024-05-14T11:25:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.326+0000 7fb319e00640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 0x7fb2f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:25:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.326+0000 7fb319e00640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 0x7fb2f8071940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007ce0 tx=0x7fb30c000f60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:25:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.330+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb314016b30 con 0x7fb31c1557b0 2024-05-14T11:25:58.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb31c157070 con 0x7fb31c1557b0 2024-05-14T11:25:58.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb312000640 1 -- 172.21.15.116:0/3278716729 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7fb3140f5210 con 0x7fb31c1557b0 2024-05-14T11:25:58.528 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:25:58.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 msgr2=0x7fb2f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 0x7fb2f8071940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007ce0 tx=0x7fb30c000f60 comp rx=0 tx=0).stop 2024-05-14T11:25:58.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 msgr2=0x7fb31c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:25:58.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb31401c890 tx=0x7fb300005840 comp rx=0 tx=0).stop 2024-05-14T11:25:58.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 shutdown_connections 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb2f806f430 0x7fb2f8071940 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb31c1557b0 0x7fb31c155bc0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb31c148d00 0x7fb31c155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 --2- 172.21.15.116:0/3278716729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb300004da0 0x7fb31c1487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 >> 172.21.15.116:0/3278716729 conn(0x7fb31c0b4620 msgr2=0x7fb31c152f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 shutdown_connections 2024-05-14T11:25:58.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58.530+0000 7fb321200640 1 -- 172.21.15.116:0/3278716729 wait complete. 2024-05-14T11:25:58.549 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:25:58.549 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:25:58.549 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:25:58.549 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:25:58.550 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:58.550 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:25:58.550 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:25:58.550 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:25:58.550 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:58.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:58.550 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.559 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41844sec preferred_lft 41844sec 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:25:58.560 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:25:58.561 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:25:58.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:58.562 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:25:58.623 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:25:58.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:58.626 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:25:58.677 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:25:58.678 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:25:58.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:25:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:58.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:58.816 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:25:58.824 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:25:58.825 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:25:58.825 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:58.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:58.825 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:25:58.885 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:25:58.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:25:58.885 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:58.938 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:25:58.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:25:58.938 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:58.993 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:25:59.053 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:25:59.103 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:25:59.104 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:59.105 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:59.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.106 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:25:59.158 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:25:59.158 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:25:59.200 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:25:59.221 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:25:59.275 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:25:59.276 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:25:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:59.278 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:25:59.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.279 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:25:59.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:25:59.306+0000 7f451144a580 -1 init, newargv = 0x7f44f4001090 newargc=15 2024-05-14T11:25:59.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49266]: starting ceph client 2024-05-14T11:25:59.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49266]: starting fuse 2024-05-14T11:25:59.313 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:25:59.313 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:25:59.313 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:25:59.314 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:25:59.315 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.315 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:25:59.315 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:59.438 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.49266.asok 2024-05-14T11:25:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:59.450 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49266.asok 2024-05-14T11:25:59.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.450 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49266.asok status 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:25:59.632 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "49266", 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8400, 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 8400 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 525114772 2024-05-14T11:25:59.633 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 525114772 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.8400 192.168.0.1:0/525114772", 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/525114772", 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 100, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 113, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:25:59.634 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:25:59.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.648 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:25:59.648 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:59.657 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:25:59.657 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:59.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.657 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:25:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.717 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-14T11:25:59.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:25:59.777 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-14T11:25:59.894 INFO:teuthology.orchestra.run.smithi116.stderr:8+0 records in 2024-05-14T11:25:59.894 INFO:teuthology.orchestra.run.smithi116.stderr:8+0 records out 2024-05-14T11:25:59.894 INFO:teuthology.orchestra.run.smithi116.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0662172 s, 127 MB/s 2024-05-14T11:25:59.895 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:59.896 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:59.896 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:59.896 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.896 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-14T11:25:59.896 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:25:59.963 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627779 2024-05-14T11:25:59.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:25:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:25:59.971 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:25:59.972 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:25:59.972 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:25:59.972 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:25:59.972 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-14T11:25:59.972 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:00.082 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627780 2024-05-14T11:26:00.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:00.092 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:00.092 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:00.143 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:26:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.338+0000 7efe04a00640 1 Processor -- start 2024-05-14T11:26:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.338+0000 7efe04a00640 1 -- start start 2024-05-14T11:26:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efe04a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7efe000d1670 0x7efe000d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efe04a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe000af380 0x7efe000af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efe04a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe000d1fc0 con 0x7efe00156fa0 2024-05-14T11:26:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efe04a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efe000706f0 con 0x7efe000af380 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efe04a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7efe001573b0 con 0x7efe000d1670 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe000af380 0x7efe000af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe000af380 0x7efe000af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44448/0 (socket says 172.21.15.116:44448) 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 -- 172.21.15.116:0/93988127 learned_addr learned my addr 172.21.15.116:0/93988127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdf5400640 1 --2- 172.21.15.116:0/93988127 >> v2:172.21.15.149:3300/0 conn(0x7efe000d1670 0x7efe000d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 -- 172.21.15.116:0/93988127 >> v2:172.21.15.149:3300/0 conn(0x7efe000d1670 msgr2=0x7efe000d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 --2- 172.21.15.116:0/93988127 >> v2:172.21.15.149:3300/0 conn(0x7efe000d1670 0x7efe000d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 -- 172.21.15.116:0/93988127 >> v1:172.21.15.149:6789/0 conn(0x7efe00156fa0 legacy=0x7efe0006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 -- 172.21.15.116:0/93988127 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe00160510 con 0x7efe000af380 2024-05-14T11:26:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.342+0000 7efdfe800640 1 --2- 172.21.15.116:0/93988127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe000af790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7efe00070c70 tx=0x7efdf801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7272e965e2cf201 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efdfd400640 1 -- 172.21.15.116:0/93988127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf80ab610 con 0x7efe000af380 2024-05-14T11:26:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efdfd400640 1 -- 172.21.15.116:0/93988127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf80b0070 con 0x7efe000af380 2024-05-14T11:26:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efdfd400640 1 -- 172.21.15.116:0/93988127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf80ab9d0 con 0x7efe000af380 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- 172.21.15.116:0/93988127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 msgr2=0x7efe000af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 --2- 172.21.15.116:0/93988127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe000af790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7efe00070c70 tx=0x7efdf801c520 comp rx=0 tx=0).stop 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- 172.21.15.116:0/93988127 shutdown_connections 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 --2- 172.21.15.116:0/93988127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe000af790 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 --2- 172.21.15.116:0/93988127 >> v2:172.21.15.149:3300/0 conn(0x7efe000d1670 0x7efe000d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- 172.21.15.116:0/93988127 >> 172.21.15.116:0/93988127 conn(0x7efe000b4620 msgr2=0x7efe000b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- 172.21.15.116:0/93988127 shutdown_connections 2024-05-14T11:26:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- 172.21.15.116:0/93988127 wait complete. 2024-05-14T11:26:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 Processor -- start 2024-05-14T11:26:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.346+0000 7efe04a00640 1 -- start start 2024-05-14T11:26:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe000d1670 0x7efe0014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00156fa0 0x7efe0014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe000b0660 con 0x7efe000af380 2024-05-14T11:26:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe000b04e0 con 0x7efe00156fa0 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe00169810 con 0x7efe000d1670 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44456/0 (socket says 172.21.15.116:44456) 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 -- 172.21.15.116:0/2638693444 learned_addr learned my addr 172.21.15.116:0/2638693444 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfe800640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00156fa0 0x7efe0014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdf5400640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe000d1670 0x7efe0014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 -- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe000d1670 msgr2=0x7efe0014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe000d1670 0x7efe0014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 -- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00156fa0 msgr2=0x7efe0014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00156fa0 0x7efe0014eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 -- 172.21.15.116:0/2638693444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0014b040 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdfde00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c710 tx=0x7efdf000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003b80 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0003d20 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0009ae60 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000aa90 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0014b5c0 con 0x7efe000af380 2024-05-14T11:26:00.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.350+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0006a9c0 con 0x7efe000af380 2024-05-14T11:26:00.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.354+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efdf000ac30 con 0x7efe000af380 2024-05-14T11:26:00.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.354+0000 7efdf6a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 0x7efddc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.354+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7efdf008f2f0 con 0x7efe000af380 2024-05-14T11:26:00.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.354+0000 7efdf5400640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 0x7efddc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.354+0000 7efdf5400640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 0x7efddc071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efe00157eb0 tx=0x7efde8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.358+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efe000d1fc0 con 0x7efe000af380 2024-05-14T11:26:00.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efe000b3e80 con 0x7efe000af380 2024-05-14T11:26:00.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efdf6a00640 1 -- 172.21.15.116:0/2638693444 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7efdf000ce20 con 0x7efe000af380 2024-05-14T11:26:00.531 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:26:00.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 msgr2=0x7efddc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 0x7efddc071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efe00157eb0 tx=0x7efde8009190 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 msgr2=0x7efe0009a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c710 tx=0x7efdf000cbd0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 shutdown_connections 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7efddc06f400 0x7efddc071910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00156fa0 0x7efe0014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe000d1670 0x7efe0014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 --2- 172.21.15.116:0/2638693444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe000af380 0x7efe0009a920 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 >> 172.21.15.116:0/2638693444 conn(0x7efe000b4620 msgr2=0x7efe0006c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 shutdown_connections 2024-05-14T11:26:00.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.534+0000 7efe04a00640 1 -- 172.21.15.116:0/2638693444 wait complete. 2024-05-14T11:26:00.553 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:00.553 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:00.561 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:26:00.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.750+0000 7f134ea00640 1 Processor -- start 2024-05-14T11:26:00.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.750+0000 7f134ea00640 1 -- start start 2024-05-14T11:26:00.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f134815db30 0x7f13480aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f134806c700 0x7f134806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ea00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134806ff20 con 0x7f13480d1670 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f13480ae100 con 0x7f134806c700 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ea00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f13480ae870 con 0x7f134815db30 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f134806c700 0x7f134806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f134806c700 0x7f134806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44468/0 (socket says 172.21.15.116:44468) 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 -- 172.21.15.116:0/1331227525 learned_addr learned my addr 172.21.15.116:0/1331227525 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:00.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f1347e00640 1 --2- 172.21.15.116:0/1331227525 >> v2:172.21.15.149:3300/0 conn(0x7f134815db30 0x7f13480aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 -- 172.21.15.116:0/1331227525 >> v2:172.21.15.149:3300/0 conn(0x7f134815db30 msgr2=0x7f13480aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:00.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 --2- 172.21.15.116:0/1331227525 >> v2:172.21.15.149:3300/0 conn(0x7f134815db30 0x7f13480aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 -- 172.21.15.116:0/1331227525 >> v1:172.21.15.149:6789/0 conn(0x7f13480d1670 legacy=0x7f13480d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.754+0000 7f134ca00640 1 -- 172.21.15.116:0/1331227525 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348070470 con 0x7f134806c700 2024-05-14T11:26:00.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ca00640 1 --2- 172.21.15.116:0/1331227525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134806c700 0x7f134806cb10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1348071f20 tx=0x7f134001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6ec3f504516c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f1347400640 1 -- 172.21.15.116:0/1331227525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400b0070 con 0x7f134806c700 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f1347400640 1 -- 172.21.15.116:0/1331227525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13400ab600 con 0x7f134806c700 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f1347400640 1 -- 172.21.15.116:0/1331227525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400ab9d0 con 0x7f134806c700 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 -- 172.21.15.116:0/1331227525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134806c700 msgr2=0x7f134806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 --2- 172.21.15.116:0/1331227525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134806c700 0x7f134806cb10 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1348071f20 tx=0x7f134001c450 comp rx=0 tx=0).stop 2024-05-14T11:26:00.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 -- 172.21.15.116:0/1331227525 shutdown_connections 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 --2- 172.21.15.116:0/1331227525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134806c700 0x7f134806cb10 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 --2- 172.21.15.116:0/1331227525 >> v2:172.21.15.149:3300/0 conn(0x7f134815db30 0x7f13480aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 -- 172.21.15.116:0/1331227525 >> 172.21.15.116:0/1331227525 conn(0x7f13480b4620 msgr2=0x7f13480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 -- 172.21.15.116:0/1331227525 shutdown_connections 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.758+0000 7f134ea00640 1 -- 172.21.15.116:0/1331227525 wait complete. 2024-05-14T11:26:00.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 Processor -- start 2024-05-14T11:26:00.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- start start 2024-05-14T11:26:00.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13480d1670 0x7f1348160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134815db30 0x7f1348160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f134816a770 con 0x7f134815db30 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f134815df40 con 0x7f13480d1670 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f134816a8f0 con 0x7f134806c700 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:41184/0 (socket says 172.21.15.116:41184) 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 -- 172.21.15.116:0/401399771 learned_addr learned my addr 172.21.15.116:0/401399771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:00.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f133f400640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13480d1670 0x7f1348160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ca00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134815db30 0x7f1348160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 -- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13480d1670 msgr2=0x7f1348160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13480d1670 0x7f1348160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 -- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134815db30 msgr2=0x7f1348160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134815db30 0x7f1348160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 -- 172.21.15.116:0/401399771 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134815feb0 con 0x7f134806c700 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1347e00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f133800efc0 tx=0x7f133800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338018020 con 0x7f134806c700 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338014480 con 0x7f134806c700 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134815f610 con 0x7f134806c700 2024-05-14T11:26:00.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338014850 con 0x7f134806c700 2024-05-14T11:26:00.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.762+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f134815fbd0 con 0x7f134806c700 2024-05-14T11:26:00.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134806a9c0 con 0x7f134806c700 2024-05-14T11:26:00.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f133801b020 con 0x7f134806c700 2024-05-14T11:26:00.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f1344c00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 0x7f1324071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:00.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f133808fef0 con 0x7f134806c700 2024-05-14T11:26:00.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f133f400640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 0x7f1324071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:00.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.766+0000 7f133f400640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 0x7f1324071910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1348073060 tx=0x7f1330009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:00.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.770+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1338022070 con 0x7f134806c700 2024-05-14T11:26:00.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.938+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1348159c60 con 0x7f134806c700 2024-05-14T11:26:00.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.938+0000 7f1344c00640 1 -- 172.21.15.116:0/401399771 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f1338010040 con 0x7f134806c700 2024-05-14T11:26:00.938 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-05-14T11:26:00.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 msgr2=0x7f1324071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 0x7f1324071910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1348073060 tx=0x7f1330009040 comp rx=0 tx=0).stop 2024-05-14T11:26:00.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 msgr2=0x7f13480cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:00.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f133800efc0 tx=0x7f133800c490 comp rx=0 tx=0).stop 2024-05-14T11:26:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 shutdown_connections 2024-05-14T11:26:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f132406f400 0x7f1324071910 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13480d1670 0x7f1348160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f134815db30 0x7f1348160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 --2- 172.21.15.116:0/401399771 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f134806c700 0x7f13480cd7e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:00.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 >> 172.21.15.116:0/401399771 conn(0x7f13480b4620 msgr2=0x7f13480b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:00.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 shutdown_connections 2024-05-14T11:26:00.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:00.942+0000 7f134ea00640 1 -- 172.21.15.116:0/401399771 wait complete. 2024-05-14T11:26:00.959 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:26:00.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:00.959 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:00.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49266]: fuse finished with error 0 and tester_r 0 2024-05-14T11:26:00.980 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:26:01.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:26:01.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:01.077 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:26:01.086 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:01.086 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:01.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:26:01.138 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi116.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:01.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:26:01.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:01.138 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:26:01.185 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-14T11:26:01.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:26:01.186 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:26:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 Processor -- start 2024-05-14T11:26:01.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 -- start start 2024-05-14T11:26:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 0x7f4054167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4054160510 con 0x7f40540d1670 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40541607f0 con 0x7f4054156fa0 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f40540706f0 con 0x7f40540b0390 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4059400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f40540d1670 0x7f40540d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41924/0 (socket says 172.21.15.116:41924) 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4059400640 1 -- 172.21.15.116:0/3315448990 learned_addr learned my addr 172.21.15.116:0/3315448990 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f405a800640 1 --2- 172.21.15.116:0/3315448990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 0x7f4054167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467445967 0 0) 0x7f4054160510 con 0x7f40540d1670 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f403c003530 con 0x7f40540d1670 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560571134 0 0) 0x7f40541607f0 con 0x7f4054156fa0 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.378+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4054160510 con 0x7f4054156fa0 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f405a800640 1 -- 172.21.15.116:0/3315448990 >> v1:172.21.15.149:6789/0 conn(0x7f4054156fa0 legacy=0x7f405406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f405a800640 1 -- 172.21.15.116:0/3315448990 >> v1:172.21.15.116:6789/0 conn(0x7f40540d1670 legacy=0x7f40540d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f405a800640 1 -- 172.21.15.116:0/3315448990 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4054160ad0 con 0x7f40540b0390 2024-05-14T11:26:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f405a800640 1 --2- 172.21.15.116:0/3315448990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 0x7f4054167800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4050009dc0 tx=0x7f405001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c86e64546d58619 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500b2070 con 0x7f40540b0390 2024-05-14T11:26:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40500a5850 con 0x7f40540b0390 2024-05-14T11:26:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.382+0000 7f4058a00640 1 -- 172.21.15.116:0/3315448990 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500a5c20 con 0x7f40540b0390 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- 172.21.15.116:0/3315448990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 msgr2=0x7f4054167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 --2- 172.21.15.116:0/3315448990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 0x7f4054167800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4050009dc0 tx=0x7f405001c990 comp rx=0 tx=0).stop 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- 172.21.15.116:0/3315448990 shutdown_connections 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 --2- 172.21.15.116:0/3315448990 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f40540b0390 0x7f4054167800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- 172.21.15.116:0/3315448990 >> 172.21.15.116:0/3315448990 conn(0x7f40540b4620 msgr2=0x7f40540b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- 172.21.15.116:0/3315448990 shutdown_connections 2024-05-14T11:26:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- 172.21.15.116:0/3315448990 wait complete. 2024-05-14T11:26:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 Processor -- start 2024-05-14T11:26:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.386+0000 7f405c800640 1 -- start start 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40540b0390 0x7f4054155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 0x7f405415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f405416ad90 con 0x7f40540d1670 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f40540b07a0 con 0x7f40540b0390 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f405416af10 con 0x7f4054156fa0 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44482/0 (socket says 172.21.15.116:44482) 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 -- 172.21.15.116:0/2240537408 learned_addr learned my addr 172.21.15.116:0/2240537408 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059e00640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40540b0390 0x7f4054155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405a800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 0x7f405415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 -- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 msgr2=0x7f405415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 0x7f405415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 -- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40540b0390 msgr2=0x7f4054155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40540b0390 0x7f4054155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 -- 172.21.15.116:0/2240537408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40541fa4e0 con 0x7f40540d1670 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405a800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 0x7f405415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f4059400640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f404400c430 tx=0x7f404400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044017070 con 0x7f40540d1670 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044005cf0 con 0x7f40540d1670 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4054156850 con 0x7f40540d1670 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40440072f0 con 0x7f40540d1670 2024-05-14T11:26:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40541f9a00 con 0x7f40540d1670 2024-05-14T11:26:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405406a9c0 con 0x7f40540d1670 2024-05-14T11:26:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.390+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f404401a070 con 0x7f40540d1670 2024-05-14T11:26:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.394+0000 7f404e000640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 0x7f4038071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.394+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f404408e830 con 0x7f40540d1670 2024-05-14T11:26:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.394+0000 7f4059e00640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 0x7f4038071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.394+0000 7f4059e00640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 0x7f4038071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c10 tx=0x7f4048009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.398+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f404405aea0 con 0x7f40540d1670 2024-05-14T11:26:01.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.590+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f405406fdc0 con 0x7f40540d1670 2024-05-14T11:26:01.589 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:26:01.590+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:26:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.594+0000 7f404e000640 1 -- 172.21.15.116:0/2240537408 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f404405f1a0 con 0x7f40540d1670 2024-05-14T11:26:01.593 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 msgr2=0x7f4038071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 0x7f4038071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c10 tx=0x7f4048009040 comp rx=0 tx=0).stop 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 msgr2=0x7f4054150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f404400c430 tx=0x7f404400c8f0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 shutdown_connections 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f403806f430 0x7f4038071940 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4054156fa0 0x7f405415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40540d1670 0x7f4054150040 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 --2- 172.21.15.116:0/2240537408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40540b0390 0x7f4054155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 >> 172.21.15.116:0/2240537408 conn(0x7f40540b4620 msgr2=0x7f405415c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 shutdown_connections 2024-05-14T11:26:01.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.598+0000 7f405c800640 1 -- 172.21.15.116:0/2240537408 wait complete. 2024-05-14T11:26:01.608 INFO:tasks.ceph.mds.c.smithi116.stderr: -7654> 2024-05-14T11:25:44.402+0000 7f4c11200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:26:01.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:01.622 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:01.663 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-05-14T11:26:01.663 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-05-14T11:26:01.663 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:01.663 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:01.664 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:01.665 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.666 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-05-14T11:26:01.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:01.668 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:01.669 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:01.670 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-05-14T11:26:01.671 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-05-14T11:26:01.672 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = False 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.673 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-14T11:26:01.674 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-14T11:26:01.677 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-14T11:26:01.678 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 20 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 20 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 10 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-14T11:26:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-14T11:26:01.680 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: mon op complaint time = 120 2024-05-14T11:26:01.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: client mount timeout = 600 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-14T11:26:01.682 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout:[mds] 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds = 20 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds balancer = 20 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug frag = True 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug scatterstat = True 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: mds op complaint time = 180 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: mds verify scatter = False 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.683 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:01.730 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:26:01.732 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:01.772 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:01.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.775 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:01.776 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:01.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-05-14T11:26:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-05-14T11:26:01.779 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.780 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = False 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = False 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:01.781 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.782 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:01.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:01.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.785 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-14T11:26:01.786 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 20 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 20 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 10 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:01.787 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:01.788 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:01.789 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: mon op complaint time = 120 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: client mount timeout = 600 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.790 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout:[mds] 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds = 20 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds balancer = 20 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug frag = True 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug scatterstat = False 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: mds op complaint time = 180 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: mds verify scatter = False 2024-05-14T11:26:01.791 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:01.792 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:01.792 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:01.793 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-14T11:26:01.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.894+0000 7f2ca4b93080 1 Processor -- start 2024-05-14T11:26:01.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.894+0000 7f2ca4b93080 1 -- start start 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c826d100 0x55c2c82afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c8324560 0x55c2c8298620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c82b3f60 con 0x55c2c82b3570 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8210aa0 con 0x55c2c826d100 2024-05-14T11:26:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55c2c82b4200 con 0x55c2c8324560 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44536/0 (socket says 172.21.15.116:44536) 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 -- 172.21.15.116:0/1079713802 learned_addr learned my addr 172.21.15.116:0/1079713802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9e800640 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c8324560 0x55c2c8298620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9de00640 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c826d100 0x55c2c82afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 -- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c8324560 msgr2=0x55c2c8298620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c8324560 0x55c2c8298620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 -- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c826d100 msgr2=0x55c2c82afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c826d100 0x55c2c82afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 -- 172.21.15.116:0/1079713802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c826ca40 con 0x55c2c82b3570 2024-05-14T11:26:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c9d400640 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c001e80 tx=0x7f2c8c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf3ec2ca65f1366 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c97e00640 1 -- 172.21.15.116:0/1079713802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c012020 con 0x55c2c82b3570 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c97e00640 1 -- 172.21.15.116:0/1079713802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00e690 con 0x55c2c82b3570 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2c97e00640 1 -- 172.21.15.116:0/1079713802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00ea40 con 0x55c2c82b3570 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- 172.21.15.116:0/1079713802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 msgr2=0x55c2c82b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c001e80 tx=0x7f2c8c00b790 comp rx=0 tx=0).stop 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- 172.21.15.116:0/1079713802 shutdown_connections 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c8324560 0x55c2c8298620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c82b3570 0x55c2c82b0120 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/1079713802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c826d100 0x55c2c82afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- 172.21.15.116:0/1079713802 >> 172.21.15.116:0/1079713802 conn(0x55c2c8255e10 msgr2=0x55c2c82adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- 172.21.15.116:0/1079713802 shutdown_connections 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.898+0000 7f2ca4b93080 1 -- 172.21.15.116:0/1079713802 wait complete. 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 Processor -- start 2024-05-14T11:26:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- start start 2024-05-14T11:26:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 0x55c2c8343560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c82b3570 0x55c2c8293180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c8324560 0x55c2c83daf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8295c30 con 0x55c2c826d100 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8295990 con 0x55c2c8324560 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55c2c83cf990 con 0x55c2c82b3570 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 0x55c2c8343560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 0x55c2c8343560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44538/0 (socket says 172.21.15.116:44538) 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 -- 172.21.15.116:0/989147517 learned_addr learned my addr 172.21.15.116:0/989147517 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9d400640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c82b3570 0x55c2c8293180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9e800640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c8324560 0x55c2c83daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 -- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c82b3570 msgr2=0x55c2c8293180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c82b3570 0x55c2c8293180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 -- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c8324560 msgr2=0x55c2c83daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2c8324560 0x55c2c83daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 -- 172.21.15.116:0/989147517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c828e5e0 con 0x55c2c826d100 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9d400640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55c2c82b3570 0x55c2c8293180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c9de00640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 0x55c2c8343560 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2c9001b7d0 tx=0x7f2c9001a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c96a00640 1 -- 172.21.15.116:0/989147517 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900be030 con 0x55c2c826d100 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c96a00640 1 -- 172.21.15.116:0/989147517 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c900b5040 con 0x55c2c826d100 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2c96a00640 1 -- 172.21.15.116:0/989147517 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900b4070 con 0x55c2c826d100 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c2c828e880 con 0x55c2c826d100 2024-05-14T11:26:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.902+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c828edc0 con 0x55c2c826d100 2024-05-14T11:26:01.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c96a00640 1 -- 172.21.15.116:0/989147517 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2c900c1030 con 0x55c2c826d100 2024-05-14T11:26:01.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c96a00640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c8006de10 0x7f2c80070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c96a00640 1 -- 172.21.15.116:0/989147517 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f2c9012bfb0 con 0x55c2c826d100 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c9d400640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c8006de10 0x7f2c80070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55c2c82936c0 0x55c2c828f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55c2c828fa40 con 0x55c2c82936c0 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c9e800640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55c2c82936c0 0x55c2c828f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c9d400640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c8006de10 0x7f2c80070320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b7d0 tx=0x7f2c8c01b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.906+0000 7f2c9e800640 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55c2c82936c0 0x55c2c828f500 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2c9e800640 1 -- 172.21.15.116:0/989147517 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v114'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c9800da30 con 0x55c2c82936c0 2024-05-14T11:26:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55c2c82936c0 msgr2=0x55c2c828f500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55c2c82936c0 0x55c2c828f500 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c8006de10 msgr2=0x7f2c80070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c8006de10 0x7f2c80070320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b7d0 tx=0x7f2c8c01b040 comp rx=0 tx=0).stop 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 msgr2=0x55c2c8343560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 --2- 172.21.15.116:0/989147517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c826d100 0x55c2c8343560 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2c9001b7d0 tx=0x7f2c9001a940 comp rx=0 tx=0).stop 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2c9e800640 1 -- 172.21.15.116:0/989147517 reap_dead start 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 shutdown_connections 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 >> 172.21.15.116:0/989147517 conn(0x55c2c8255e10 msgr2=0x55c2c82aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 shutdown_connections 2024-05-14T11:26:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:01.910+0000 7f2ca4b93080 1 -- 172.21.15.116:0/989147517 wait complete. 2024-05-14T11:26:01.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:26:02.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.062+0000 7f7094800640 1 Processor -- start 2024-05-14T11:26:02.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.062+0000 7f7094800640 1 -- start start 2024-05-14T11:26:02.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7094800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f708c0cd1c0 0x7f708c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7094800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f708c06c700 0x7f708c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7094800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708c06f0a0 con 0x7f708c0d1730 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7094800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f708c0cbb60 con 0x7f708c0cd1c0 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7094800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f708c06f420 con 0x7f708c06c700 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f708c0cd1c0 0x7f708c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f708c0cd1c0 0x7f708c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44544/0 (socket says 172.21.15.116:44544) 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 -- 172.21.15.116:0/2135294243 learned_addr learned my addr 172.21.15.116:0/2135294243 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091400640 1 --1- 172.21.15.116:0/2135294243 >> v1:172.21.15.116:6789/0 conn(0x7f708c0d1730 0x7f708c0d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41950/0 (socket says 172.21.15.116:41950) 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7092800640 1 --2- 172.21.15.116:0/2135294243 >> v2:172.21.15.149:3300/0 conn(0x7f708c06c700 0x7f708c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 -- 172.21.15.116:0/2135294243 >> v2:172.21.15.149:3300/0 conn(0x7f708c06c700 msgr2=0x7f708c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 --2- 172.21.15.116:0/2135294243 >> v2:172.21.15.149:3300/0 conn(0x7f708c06c700 0x7f708c06cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 -- 172.21.15.116:0/2135294243 >> v1:172.21.15.116:6789/0 conn(0x7f708c0d1730 legacy=0x7f708c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.066+0000 7f7091e00640 1 -- 172.21.15.116:0/2135294243 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c06f700 con 0x7f708c0cd1c0 2024-05-14T11:26:02.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7091e00640 1 --2- 172.21.15.116:0/2135294243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0cd5d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f708c06e2c0 tx=0x7f708001ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba3d732b47e6d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7090a00640 1 -- 172.21.15.116:0/2135294243 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70800b4070 con 0x7f708c0cd1c0 2024-05-14T11:26:02.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7090a00640 1 -- 172.21.15.116:0/2135294243 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70800a7ae0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7090a00640 1 -- 172.21.15.116:0/2135294243 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70800af440 con 0x7f708c0cd1c0 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 -- 172.21.15.116:0/2135294243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 msgr2=0x7f708c0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 --2- 172.21.15.116:0/2135294243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0cd5d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f708c06e2c0 tx=0x7f708001ea10 comp rx=0 tx=0).stop 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 -- 172.21.15.116:0/2135294243 shutdown_connections 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 --2- 172.21.15.116:0/2135294243 >> v2:172.21.15.149:3300/0 conn(0x7f708c06c700 0x7f708c06cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 --2- 172.21.15.116:0/2135294243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0cd5d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 -- 172.21.15.116:0/2135294243 >> 172.21.15.116:0/2135294243 conn(0x7f708c0b4720 msgr2=0x7f708c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 -- 172.21.15.116:0/2135294243 shutdown_connections 2024-05-14T11:26:02.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.070+0000 7f7094800640 1 -- 172.21.15.116:0/2135294243 wait complete. 2024-05-14T11:26:02.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 Processor -- start 2024-05-14T11:26:02.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- start start 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f708c06c700 0x7f708c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f708c0d1730 0x7f708c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f708c0cddb0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f708c15cf10 con 0x7f708c06c700 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f708c0cdf30 con 0x7f708c0d1730 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44560/0 (socket says 172.21.15.116:44560) 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 -- 172.21.15.116:0/421762810 learned_addr learned my addr 172.21.15.116:0/421762810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091e00640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f708c06c700 0x7f708c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7092800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f708c0d1730 0x7f708c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 -- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f708c0d1730 msgr2=0x7f708c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f708c0d1730 0x7f708c0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 -- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f708c06c700 msgr2=0x7f708c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f708c06c700 0x7f708c15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 -- 172.21.15.116:0/421762810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c0caf20 con 0x7f708c0cd1c0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7091400640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c430 tx=0x7f707c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0074e0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c007680 con 0x7f708c0cd1c0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c0c9090 con 0x7f708c0cd1c0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c004a10 con 0x7f708c0cd1c0 2024-05-14T11:26:02.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.074+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c0c9650 con 0x7f708c0cd1c0 2024-05-14T11:26:02.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c06a9c0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f707c01a020 con 0x7f708c0cd1c0 2024-05-14T11:26:02.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7086000640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 0x7f7070071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f707c08eaf0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7091e00640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 0x7f7070071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.078+0000 7f7091e00640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 0x7f7070071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f70800b1000 tx=0x7f7080008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.082+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f707c05b920 con 0x7f708c0cd1c0 2024-05-14T11:26:02.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.274+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f708c069de0 con 0x7f708c0cd1c0 2024-05-14T11:26:02.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.602+0000 7f7086000640 1 -- 172.21.15.116:0/421762810 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) ==== 162+0+0 (secure 0 0 0) 0x7f707c05b220 con 0x7f708c0cd1c0 2024-05-14T11:26:02.603 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:26:02.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 msgr2=0x7f7070071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 0x7f7070071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f70800b1000 tx=0x7f7080008040 comp rx=0 tx=0).stop 2024-05-14T11:26:02.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 msgr2=0x7f708c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c430 tx=0x7f707c00c8f0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 shutdown_connections 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f707006f430 0x7f7070071940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f708c0d1730 0x7f708c0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f708c0cd1c0 0x7f708c0b2f00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 --2- 172.21.15.116:0/421762810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f708c06c700 0x7f708c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 >> 172.21.15.116:0/421762810 conn(0x7f708c0b4720 msgr2=0x7f708c072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 shutdown_connections 2024-05-14T11:26:02.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.606+0000 7f7094800640 1 -- 172.21.15.116:0/421762810 wait complete. 2024-05-14T11:26:02.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:02.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 Processor -- start 2024-05-14T11:26:02.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 -- start start 2024-05-14T11:26:02.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81c00cdf80 0x7f81c00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f81c00b40c0 0x7f81c00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c00b4a10 con 0x7f81c00d1510 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b3060 con 0x7f81c00cdf80 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c7800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b3300 con 0x7f81c00b40c0 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c4e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81c00cdf80 0x7f81c00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c4e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81c00cdf80 0x7f81c00ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44576/0 (socket says 172.21.15.116:44576) 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c4e00640 1 -- 172.21.15.116:0/2801505054 learned_addr learned my addr 172.21.15.116:0/2801505054 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.778+0000 7f81c5800640 1 --2- 172.21.15.116:0/2801505054 >> v2:172.21.15.149:3300/0 conn(0x7f81c00b40c0 0x7f81c00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81c4e00640 1 -- 172.21.15.116:0/2801505054 >> v2:172.21.15.149:3300/0 conn(0x7f81c00b40c0 msgr2=0x7f81c00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81c4e00640 1 --2- 172.21.15.116:0/2801505054 >> v2:172.21.15.149:3300/0 conn(0x7f81c00b40c0 0x7f81c00b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81c4e00640 1 -- 172.21.15.116:0/2801505054 >> v1:172.21.15.149:6789/0 conn(0x7f81c00d1510 legacy=0x7f81c00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81c4e00640 1 -- 172.21.15.116:0/2801505054 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00b35e0 con 0x7f81c00cdf80 2024-05-14T11:26:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81c4e00640 1 --2- 172.21.15.116:0/2801505054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c00ce390 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f81c00d0030 tx=0x7f81b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7804e00d304ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81bf400640 1 -- 172.21.15.116:0/2801505054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00aa650 con 0x7f81c00cdf80 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81bf400640 1 -- 172.21.15.116:0/2801505054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b00af070 con 0x7f81c00cdf80 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.782+0000 7f81bf400640 1 -- 172.21.15.116:0/2801505054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00aaa10 con 0x7f81c00cdf80 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- 172.21.15.116:0/2801505054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 msgr2=0x7f81c00ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 --2- 172.21.15.116:0/2801505054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c00ce390 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f81c00d0030 tx=0x7f81b001c4f0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- 172.21.15.116:0/2801505054 shutdown_connections 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 --2- 172.21.15.116:0/2801505054 >> v2:172.21.15.149:3300/0 conn(0x7f81c00b40c0 0x7f81c00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 --2- 172.21.15.116:0/2801505054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c00ce390 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- 172.21.15.116:0/2801505054 >> 172.21.15.116:0/2801505054 conn(0x7f81c00d0770 msgr2=0x7f81c00d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- 172.21.15.116:0/2801505054 shutdown_connections 2024-05-14T11:26:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- 172.21.15.116:0/2801505054 wait complete. 2024-05-14T11:26:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 Processor -- start 2024-05-14T11:26:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.786+0000 7f81c7800640 1 -- start start 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81c00b40c0 0x7f81c00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81c00d1510 0x7f81c0153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81c006ff40 con 0x7f81c00cdf80 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81c006fdc0 con 0x7f81c00d1510 2024-05-14T11:26:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f81c016cd60 con 0x7f81c00b40c0 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c4e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81c00b40c0 0x7f81c00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44586/0 (socket says 172.21.15.116:44586) 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 -- 172.21.15.116:0/1326988284 learned_addr learned my addr 172.21.15.116:0/1326988284 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c5800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81c00d1510 0x7f81c0153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 -- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81c00b40c0 msgr2=0x7f81c00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81c00b40c0 0x7f81c00cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 -- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81c00d1510 msgr2=0x7f81c0153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81c00d1510 0x7f81c0153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 -- 172.21.15.116:0/1326988284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c0161580 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bfe00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c3f0 tx=0x7f81ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac017070 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac0052d0 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01604e0 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac003820 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c0160aa0 con 0x7f81c00cdf80 2024-05-14T11:26:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c0159690 con 0x7f81c00cdf80 2024-05-14T11:26:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.790+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81ac01a070 con 0x7f81c00cdf80 2024-05-14T11:26:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.794+0000 7f81bcc00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 0x7f81a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.794+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f81ac08e7f0 con 0x7f81c00cdf80 2024-05-14T11:26:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.794+0000 7f81c4e00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 0x7f81a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.794+0000 7f81c4e00640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 0x7f81a0071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ae000 tx=0x7f81b00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.798+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f81ac021080 con 0x7f81c00cdf80 2024-05-14T11:26:02.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c006a9c0 con 0x7f81c00cdf80 2024-05-14T11:26:02.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81bcc00640 1 -- 172.21.15.116:0/1326988284 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5193 (secure 0 0 0) 0x7f81ac005480 con 0x7f81c00cdf80 2024-05-14T11:26:02.984 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:02.984 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:02.609650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"c","rank":0,"incarnation":103,"state":"up:replay","state_seq":1,"addr":"172.21.15.116:6837/3756089525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3756089525},{"type":"v1","addr":"172.21.15.116:6837","nonce":3756089525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-14T11:26:02.984 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 103 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 msgr2=0x7f81a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 0x7f81a0071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ae000 tx=0x7f81b00a7000 comp rx=0 tx=0).stop 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 msgr2=0x7f81c0153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00c3f0 tx=0x7f81ac00c8b0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 shutdown_connections 2024-05-14T11:26:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f81a006f400 0x7f81a0071910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81c00d1510 0x7f81c0153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c00cdf80 0x7f81c0153530 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 --2- 172.21.15.116:0/1326988284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f81c00b40c0 0x7f81c00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 >> 172.21.15.116:0/1326988284 conn(0x7f81c00d0770 msgr2=0x7f81c0155250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 shutdown_connections 2024-05-14T11:26:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:02.986+0000 7f81c7800640 1 -- 172.21.15.116:0/1326988284 wait complete. 2024-05-14T11:26:03.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:02.609650+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.116:6837/3756089525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 3756089525}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 3756089525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:26:03.408 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8427:up:replay 2024-05-14T11:26:04.409 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:04.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.570+0000 7fa865800640 1 Processor -- start 2024-05-14T11:26:04.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.570+0000 7fa865800640 1 -- start start 2024-05-14T11:26:04.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa865800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa8600d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa865800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fa8600af380 0x7fa8600af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa865800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8600d1fc0 con 0x7fa860156fa0 2024-05-14T11:26:04.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa865800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fa8600706f0 con 0x7fa8600af380 2024-05-14T11:26:04.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa865800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa8601573b0 con 0x7fa8600d1670 2024-05-14T11:26:04.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85ea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa860156fa0 0x7fa86006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41958/0 (socket says 172.21.15.116:41958) 2024-05-14T11:26:04.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85ea00640 1 -- 172.21.15.116:0/2608337631 learned_addr learned my addr 172.21.15.116:0/2608337631 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:04.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 --2- 172.21.15.116:0/2608337631 >> v2:172.21.15.149:3300/0 conn(0x7fa8600af380 0x7fa8600af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa856000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa8600d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524218913 0 0) 0x7fa850001120 con 0x7fa860156fa0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844003530 con 0x7fa860156fa0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242684161 0 0) 0x7fa844003530 con 0x7fa860156fa0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 msgr2=0x7fa8600d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa8600d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 >> v2:172.21.15.149:3300/0 conn(0x7fa8600af380 msgr2=0x7fa8600af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 --2- 172.21.15.116:0/2608337631 >> v2:172.21.15.149:3300/0 conn(0x7fa8600af380 0x7fa8600af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa860160510 con 0x7fa860156fa0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fa8500018d0 con 0x7fa860156fa0 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 >> v1:172.21.15.116:6789/0 conn(0x7fa860156fa0 legacy=0x7fa86006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa844003a10 0x7fa844003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa844004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa844004d50 0x7fa844005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600706f0 con 0x7fa844004d50 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa84400ba80 con 0x7fa844003a10 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa84400bc00 con 0x7fa8600d1670 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa844004d50 0x7fa844005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85ea00640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa844003a10 0x7fa844003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa856000640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa844004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 -- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 msgr2=0x7fa844004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8600d1670 0x7fa844004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 -- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa844003a10 msgr2=0x7fa844003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa844003a10 0x7fa844003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.574+0000 7fa85f400640 1 -- 172.21.15.116:0/2608337631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844003530 con 0x7fa844004d50 2024-05-14T11:26:04.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa85f400640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa844004d50 0x7fa844005160 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa858009f90 tx=0x7fa85801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cf8e897bf2436a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:04.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580b2070 con 0x7fa844004d50 2024-05-14T11:26:04.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8580a5c90 con 0x7fa844004d50 2024-05-14T11:26:04.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa85e000640 1 -- 172.21.15.116:0/2608337631 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858009c10 con 0x7fa844004d50 2024-05-14T11:26:04.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 -- 172.21.15.116:0/2608337631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa844004d50 msgr2=0x7fa844005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 --2- 172.21.15.116:0/2608337631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa844004d50 0x7fa844005160 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa858009f90 tx=0x7fa85801cc80 comp rx=0 tx=0).stop 2024-05-14T11:26:04.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa85f400640 1 -- 172.21.15.116:0/2608337631 reap_dead start 2024-05-14T11:26:04.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 -- 172.21.15.116:0/2608337631 shutdown_connections 2024-05-14T11:26:04.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 -- 172.21.15.116:0/2608337631 >> 172.21.15.116:0/2608337631 conn(0x7fa8600b4620 msgr2=0x7fa8600b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:04.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 -- 172.21.15.116:0/2608337631 shutdown_connections 2024-05-14T11:26:04.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.578+0000 7fa865800640 1 -- 172.21.15.116:0/2608337631 wait complete. 2024-05-14T11:26:04.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 Processor -- start 2024-05-14T11:26:04.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- start start 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa860151b30 0x7fa8601f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8601f3dc0 0x7fa860068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa860160ad0 con 0x7fa8601f39b0 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa860161120 con 0x7fa860151b30 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa860160db0 con 0x7fa8601f3dc0 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44614/0 (socket says 172.21.15.116:44614) 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 -- 172.21.15.116:0/3813818081 learned_addr learned my addr 172.21.15.116:0/3813818081 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa85ea00640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa860151b30 0x7fa8601f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa85f400640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8601f3dc0 0x7fa860068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 -- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8601f3dc0 msgr2=0x7fa860068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8601f3dc0 0x7fa860068c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 -- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa860151b30 msgr2=0x7fa8601f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa860151b30 0x7fa8601f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 -- 172.21.15.116:0/3813818081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601f7860 con 0x7fa8601f39b0 2024-05-14T11:26:04.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa856000640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa8480115e0 tx=0x7fa8480119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:04.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848015870 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa848015a10 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848015dc0 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601f7b40 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.582+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8601f8100 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86006a9c0 con 0x7fa8601f39b0 2024-05-14T11:26:04.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa848013400 con 0x7fa8601f39b0 2024-05-14T11:26:04.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa857400640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:04.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fa84808f8a0 con 0x7fa8601f39b0 2024-05-14T11:26:04.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa85ea00640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:04.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.586+0000 7fa85ea00640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 0x7fa83c071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa850004850 tx=0x7fa850007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:04.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.590+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa848019030 con 0x7fa8601f39b0 2024-05-14T11:26:04.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.790+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa86006fdc0 con 0x7fa8601f39b0 2024-05-14T11:26:04.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.790+0000 7fa857400640 1 -- 172.21.15.116:0/3813818081 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5193 (secure 0 0 0) 0x7fa84805c630 con 0x7fa8601f39b0 2024-05-14T11:26:04.789 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:04.790 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:03.618918+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"c","rank":0,"incarnation":103,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.116:6837/3756089525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3756089525},{"type":"v1","addr":"172.21.15.116:6837","nonce":3756089525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-14T11:26:04.790 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 105 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 msgr2=0x7fa83c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 0x7fa83c071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa850004850 tx=0x7fa850007df0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 msgr2=0x7fa8600686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa8480115e0 tx=0x7fa8480119e0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 shutdown_connections 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa8601f3dc0 0x7fa860068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8601f39b0 0x7fa8600686c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 --2- 172.21.15.116:0/3813818081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa860151b30 0x7fa8601f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 >> 172.21.15.116:0/3813818081 conn(0x7fa8600b4620 msgr2=0x7fa86014dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 shutdown_connections 2024-05-14T11:26:04.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:04.794+0000 7fa865800640 1 -- 172.21.15.116:0/3813818081 wait complete. 2024-05-14T11:26:05.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:03.618918+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.116:6837/3756089525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 3756089525}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 3756089525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:26:05.215 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8427:up:rejoin 2024-05-14T11:26:06.215 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:06.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 Processor -- start 2024-05-14T11:26:06.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 -- start start 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcd700ce330 0x7fcd700ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd700cec80 con 0x7fcd700d20c0 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7015e370 con 0x7fcd700ce330 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd76200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcd70162630 con 0x7fcd700cdde0 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:41312/0 (socket says 172.21.15.116:41312) 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd6f400640 1 -- 172.21.15.116:0/1368651139 learned_addr learned my addr 172.21.15.116:0/1368651139 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd66a00640 1 --2- 172.21.15.116:0/1368651139 >> v2:172.21.15.149:3300/0 conn(0x7fcd700ce330 0x7fcd700ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd66a00640 1 -- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 msgr2=0x7fcd7015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd66a00640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd66a00640 1 -- 172.21.15.116:0/1368651139 >> v1:172.21.15.149:6789/0 conn(0x7fcd700d20c0 legacy=0x7fcd700d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd66a00640 1 -- 172.21.15.116:0/1368651139 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70162910 con 0x7fcd700ce330 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd6f400640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:06.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.378+0000 7fcd6ea00640 1 -- 172.21.15.116:0/1368651139 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425295370 0 0) 0x7fcd680014a0 con 0x7fcd700d20c0 2024-05-14T11:26:06.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd66a00640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd700ce740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097c0 tx=0x7fcd5801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3821a001ee9a0374 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:06.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd6ea00640 1 -- 172.21.15.116:0/1368651139 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd580b1070 con 0x7fcd700ce330 2024-05-14T11:26:06.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd6ea00640 1 -- 172.21.15.116:0/1368651139 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd580a8050 con 0x7fcd700ce330 2024-05-14T11:26:06.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd6ea00640 1 -- 172.21.15.116:0/1368651139 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd580ac860 con 0x7fcd700ce330 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 -- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 msgr2=0x7fcd700ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd700ce740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097c0 tx=0x7fcd5801c3e0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 -- 172.21.15.116:0/1368651139 shutdown_connections 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd700ce740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 --2- 172.21.15.116:0/1368651139 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd7015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 -- 172.21.15.116:0/1368651139 >> 172.21.15.116:0/1368651139 conn(0x7fcd700d3670 msgr2=0x7fcd700d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 -- 172.21.15.116:0/1368651139 shutdown_connections 2024-05-14T11:26:06.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.382+0000 7fcd76200640 1 -- 172.21.15.116:0/1368651139 wait complete. 2024-05-14T11:26:06.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 Processor -- start 2024-05-14T11:26:06.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 -- start start 2024-05-14T11:26:06.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd70156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd70159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b6ad0 con 0x7fcd700d20c0 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b4960 con 0x7fcd700ce330 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd76200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700b6c50 con 0x7fcd700cdde0 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44620/0 (socket says 172.21.15.116:44620) 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 -- 172.21.15.116:0/3317938834 learned_addr learned my addr 172.21.15.116:0/3317938834 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd66a00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd70159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6f400640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd70156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 -- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 msgr2=0x7fcd70156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd70156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 -- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 msgr2=0x7fcd70159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd70159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 -- 172.21.15.116:0/3317938834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7015f340 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd66a00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd70159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.386+0000 7fcd6fe00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c730 tx=0x7fcd6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68003b80 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68003d40 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680052b0 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7015f620 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7015fbe0 con 0x7fcd700d20c0 2024-05-14T11:26:06.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7015f0b0 con 0x7fcd700d20c0 2024-05-14T11:26:06.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.390+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcd68005450 con 0x7fcd700d20c0 2024-05-14T11:26:06.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.394+0000 7fcd67e00640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 0x7fcd4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:06.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.394+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fcd6808f150 con 0x7fcd700d20c0 2024-05-14T11:26:06.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.394+0000 7fcd6f400640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 0x7fcd4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:06.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.394+0000 7fcd6f400640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 0x7fcd4c071940 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006cd0 tx=0x7fcd60009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.398+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcd700cec80 con 0x7fcd700d20c0 2024-05-14T11:26:06.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.594+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd7006a9c0 con 0x7fcd700d20c0 2024-05-14T11:26:06.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.594+0000 7fcd67e00640 1 -- 172.21.15.116:0/3317938834 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5200 (secure 0 0 0) 0x7fcd6805b880 con 0x7fcd700d20c0 2024-05-14T11:26:06.593 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:06.593 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:05.625205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/3756089525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3756089525},{"type":"v1","addr":"172.21.15.116:6837","nonce":3756089525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":8}]} 2024-05-14T11:26:06.593 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 106 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 msgr2=0x7fcd4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 0x7fcd4c071940 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd60006cd0 tx=0x7fcd60009240 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 msgr2=0x7fcd701606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c730 tx=0x7fcd6800cbf0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 shutdown_connections 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcd4c06f430 0x7fcd4c071940 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd700ce330 0x7fcd70159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd700d20c0 0x7fcd701606f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 --2- 172.21.15.116:0/3317938834 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd700cdde0 0x7fcd70156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 >> 172.21.15.116:0/3317938834 conn(0x7fcd700d3670 msgr2=0x7fcd70070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:06.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 shutdown_connections 2024-05-14T11:26:06.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:06.598+0000 7fcd76200640 1 -- 172.21.15.116:0/3317938834 wait complete. 2024-05-14T11:26:07.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:05.625205+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8427': {'gid': 8427, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6837/3756089525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 3756089525}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 3756089525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8427, 'qdb_cluster': [8427]} 2024-05-14T11:26:07.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:26:07.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:26:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.166+0000 7fcaca000640 1 Processor -- start 2024-05-14T11:26:07.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.166+0000 7fcaca000640 1 -- start start 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcaca000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcac40af420 0x7fcac4074630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcaca000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 0x7fcac406edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcaca000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac415d340 con 0x7fcac4074b70 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcaca000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcac415d620 con 0x7fcac40af420 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcaca000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcac415d900 con 0x7fcac40d1d80 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcac40af420 0x7fcac4074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcac40af420 0x7fcac4074630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44636/0 (socket says 172.21.15.116:44636) 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 -- 172.21.15.116:0/1914021705 learned_addr learned my addr 172.21.15.116:0/1914021705 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac2a00640 1 --1- 172.21.15.116:0/1914021705 >> v1:172.21.15.116:6789/0 conn(0x7fcac4074b70 0x7fcac40d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41968/0 (socket says 172.21.15.116:41968) 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3e00640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 0x7fcac406edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 -- 172.21.15.116:0/1914021705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 msgr2=0x7fcac406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 0x7fcac406edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 -- 172.21.15.116:0/1914021705 >> v1:172.21.15.116:6789/0 conn(0x7fcac4074b70 legacy=0x7fcac40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3400640 1 -- 172.21.15.116:0/1914021705 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac415dbe0 con 0x7fcac40af420 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac2000640 1 -- 172.21.15.116:0/1914021705 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379828583 0 0) 0x7fcac415d340 con 0x7fcac4074b70 2024-05-14T11:26:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.170+0000 7fcac3e00640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 0x7fcac406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcac3400640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac40af420 0x7fcac4074630 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009e50 tx=0x7fcab401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd43af28d852e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcac2000640 1 -- 172.21.15.116:0/1914021705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40a58f0 con 0x7fcac40af420 2024-05-14T11:26:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcac2000640 1 -- 172.21.15.116:0/1914021705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab40a5a90 con 0x7fcac40af420 2024-05-14T11:26:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcac2000640 1 -- 172.21.15.116:0/1914021705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40a5e60 con 0x7fcac40af420 2024-05-14T11:26:07.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 -- 172.21.15.116:0/1914021705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac40af420 msgr2=0x7fcac4074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac40af420 0x7fcac4074630 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009e50 tx=0x7fcab401ca30 comp rx=0 tx=0).stop 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 -- 172.21.15.116:0/1914021705 shutdown_connections 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40d1d80 0x7fcac406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 --2- 172.21.15.116:0/1914021705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac40af420 0x7fcac4074630 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 -- 172.21.15.116:0/1914021705 >> 172.21.15.116:0/1914021705 conn(0x7fcac40b4620 msgr2=0x7fcac40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 -- 172.21.15.116:0/1914021705 shutdown_connections 2024-05-14T11:26:07.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.174+0000 7fcaca000640 1 -- 172.21.15.116:0/1914021705 wait complete. 2024-05-14T11:26:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 Processor -- start 2024-05-14T11:26:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- start start 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4074b70 0x7fcac40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40af420 0x7fcac409a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4169c10 con 0x7fcac4150cd0 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4169a90 con 0x7fcac4074b70 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcac4169d90 con 0x7fcac40af420 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44640/0 (socket says 172.21.15.116:44640) 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 -- 172.21.15.116:0/549012117 learned_addr learned my addr 172.21.15.116:0/549012117 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac2a00640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40af420 0x7fcac409a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 -- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40af420 msgr2=0x7fcac409a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40af420 0x7fcac409a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 -- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4074b70 msgr2=0x7fcac40cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4074b70 0x7fcac40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 -- 172.21.15.116:0/549012117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac40b0ce0 con 0x7fcac4150cd0 2024-05-14T11:26:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcac3e00640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c7c0 tx=0x7fcabc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcabb400640 1 -- 172.21.15.116:0/549012117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc015070 con 0x7fcac4150cd0 2024-05-14T11:26:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcabb400640 1 -- 172.21.15.116:0/549012117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc004220 con 0x7fcac4150cd0 2024-05-14T11:26:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcabb400640 1 -- 172.21.15.116:0/549012117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003620 con 0x7fcac4150cd0 2024-05-14T11:26:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac40cf6b0 con 0x7fcac4150cd0 2024-05-14T11:26:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.178+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac40b0f80 con 0x7fcac4150cd0 2024-05-14T11:26:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcabb400640 1 -- 172.21.15.116:0/549012117 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcabc0037c0 con 0x7fcac4150cd0 2024-05-14T11:26:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcabb400640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 0x7fcaa4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcabb400640 1 -- 172.21.15.116:0/549012117 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fcabc08f340 con 0x7fcac4150cd0 2024-05-14T11:26:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcac3400640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 0x7fcaa4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcad174a000 1 Processor -- start 2024-05-14T11:26:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcad174a000 1 -- start start 2024-05-14T11:26:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.182+0000 7fcac3400640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 0x7fcaa4071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ae000 tx=0x7fcab4004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d655760320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d655760da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d6555fd620 con 0x55d65560ef80 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d655518740 con 0x55d6552267c0 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d655566090 con 0x55d6555f86b0 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44650/0 (socket says 172.21.15.116:44650) 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 -- 172.21.15.116:0/429239418 learned_addr learned my addr 172.21.15.116:0/429239418 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 -- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 msgr2=0x55d655760da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac3400640 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d655760320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d655760da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 -- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 msgr2=0x55d655760320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d655760320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 -- 172.21.15.116:0/429239418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6556fa0a0 con 0x55d65560ef80 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac3400640 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d655760320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcac2a00640 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007500 tx=0x7fcab0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6636cddf976d0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcaaf400640 1 -- 172.21.15.116:0/429239418 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0019070 con 0x55d65560ef80 2024-05-14T11:26:07.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcaaf400640 1 -- 172.21.15.116:0/429239418 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab0001740 con 0x55d65560ef80 2024-05-14T11:26:07.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcaaf400640 1 -- 172.21.15.116:0/429239418 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0001b10 con 0x55d65560ef80 2024-05-14T11:26:07.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- 172.21.15.116:0/429239418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 msgr2=0x55d655760860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007500 tx=0x7fcab0005bd0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- 172.21.15.116:0/429239418 shutdown_connections 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d655760da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655760860 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 --2- 172.21.15.116:0/429239418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d655760320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- 172.21.15.116:0/429239418 >> 172.21.15.116:0/429239418 conn(0x55d6556e5340 msgr2=0x55d655714d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- 172.21.15.116:0/429239418 shutdown_connections 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.186+0000 7fcad174a000 1 -- 172.21.15.116:0/429239418 wait complete. 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- start start 2024-05-14T11:26:07.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 Processor -- start 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d65571b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d65571b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d655562050 con 0x55d65560ef80 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d65551eec0 con 0x55d6555f86b0 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d655505c90 con 0x55d6552267c0 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 10 client.0 _ll_register_callbacks cb 0x55d655701e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44664/0 (socket says 172.21.15.116:44664) 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 -- 172.21.15.116:0/4109917249 learned_addr learned my addr 172.21.15.116:0/4109917249 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3400640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d65571b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac2a00640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d65571b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 -- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 msgr2=0x55d65571b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d65571b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 -- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 msgr2=0x55d65571b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d65571b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 -- 172.21.15.116:0/4109917249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6556106f0 con 0x55d65560ef80 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3e00640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fcabc057f60 tx=0x7fcabc05bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9c3ea42539ac98 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc08b060 con 0x55d65560ef80 2024-05-14T11:26:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fcabc0574e0 con 0x55d65560ef80 2024-05-14T11:26:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0001740 con 0x55d65560ef80 2024-05-14T11:26:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7fcabc01e070 con 0x55d65560ef80 2024-05-14T11:26:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 10 client.8478 fetch_fsmap learned FSMap version 106 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55d65560c6a0 con 0x55d65560ef80 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab0001740 con 0x55d65560ef80 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7fcabc05a640 con 0x55d65560ef80 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 10 client.8478 fetch_fsmap finished waiting for FSMap version 106 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 10 client.8478 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 20 client.8478 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d65520b7d0 0x55d6557718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 4 client.8478 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 --> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6554febe0 con 0x55d65520b7d0 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.190+0000 7fcac3400640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d65520b7d0 0x55d6557718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcaa3e00640 10 client.8478 ms_handle_connect on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcac3400640 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d65520b7d0 0x55d6557718e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc1641c44244736 server_cookie=82e27fbf3cd9fc02 in_seq=0 out_seq=0 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcaa3e00640 1 -- 172.21.15.116:0/4109917249 <== mds.0 v2:172.21.15.116:6836/3756089525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6554febe0 con 0x55d65520b7d0 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcaa3e00640 10 client.8478 handle_command_reply: tid=0 2024-05-14T11:26:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 client.8478 shutdown 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 msgr2=0x55d655745770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fcabc057f60 tx=0x7fcabc05bf40 comp rx=0 tx=0).stop 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 shutdown_connections 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d65520b7d0 0x55d6557718e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d65571b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d655745770 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 --2- 172.21.15.116:0/4109917249 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d65571b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 >> 172.21.15.116:0/4109917249 conn(0x55d6556e5340 msgr2=0x55d655715490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcaa3e00640 0 client.8478 ms_handle_reset on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 shutdown_connections 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 1 -- 172.21.15.116:0/4109917249 wait complete. 2024-05-14T11:26:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.194+0000 7fcad174a000 20 client.8478 trim_cache size 0 max 16384 2024-05-14T11:26:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.210+0000 7fcad174a000 1 Processor -- start 2024-05-14T11:26:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- start start 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d655744e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d655745370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4204690 con 0x55d65560ef80 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4204be0 con 0x55d6555f86b0 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d6555097d0 con 0x55d6552267c0 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44676/0 (socket says 172.21.15.116:44676) 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 -- 172.21.15.116:0/2456830158 learned_addr learned my addr 172.21.15.116:0/2456830158 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3400640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d655744e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac2a00640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d655745370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 -- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 msgr2=0x55d655744e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d655744e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 -- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 msgr2=0x55d655745370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d655745370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 -- 172.21.15.116:0/2456830158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65553faf0 con 0x55d65560ef80 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac2a00640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d655745370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcac3e00640 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcabc05b620 tx=0x7fcabc05d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928c2f6218ee767 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcaaea00640 1 -- 172.21.15.116:0/2456830158 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc05b0e0 con 0x55d65560ef80 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcaaea00640 1 -- 172.21.15.116:0/2456830158 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc0647d0 con 0x55d65560ef80 2024-05-14T11:26:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcaaea00640 1 -- 172.21.15.116:0/2456830158 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc05b970 con 0x55d65560ef80 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- 172.21.15.116:0/2456830158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 msgr2=0x55d6557152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcabc05b620 tx=0x7fcabc05d000 comp rx=0 tx=0).stop 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- 172.21.15.116:0/2456830158 shutdown_connections 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6555f86b0 0x55d655745370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d6557152e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 --2- 172.21.15.116:0/2456830158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6552267c0 0x55d655744e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- 172.21.15.116:0/2456830158 >> 172.21.15.116:0/2456830158 conn(0x55d6556e5340 msgr2=0x55d655744720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- 172.21.15.116:0/2456830158 shutdown_connections 2024-05-14T11:26:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.214+0000 7fcad174a000 1 -- 172.21.15.116:0/2456830158 wait complete. 2024-05-14T11:26:07.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- start start 2024-05-14T11:26:07.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 Processor -- start 2024-05-14T11:26:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d6557d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d6557d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d655564f80 con 0x55d65560ef80 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d65556fc30 con 0x55d6552267c0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d655620d80 con 0x55d6555f86b0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 10 client.0 _ll_register_callbacks cb 0x7fcabc057fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44688/0 (socket says 172.21.15.116:44688) 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 -- 172.21.15.116:0/1198319866 learned_addr learned my addr 172.21.15.116:0/1198319866 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d6557d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac2a00640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d6557d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 -- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 msgr2=0x55d6557d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d6557d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 -- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 msgr2=0x55d6557d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d6557d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 -- 172.21.15.116:0/1198319866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65553ee40 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3400640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d6557d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcac3e00640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcabc05b670 tx=0x7fcabc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e7afecc5ffd906 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc08b040 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fcabc0574e0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc0647d0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7fcabc05b0e0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 10 client.8484 fetch_fsmap learned FSMap version 106 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55d6555e64e0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc0647d0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.218+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7fcabc0863d0 con 0x55d65560ef80 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 10 client.8484 fetch_fsmap finished waiting for FSMap version 106 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 10 client.8484 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 20 client.8484 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d6551f9830 0x55d65578e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 4 client.8484 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:26:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 --> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] -- command(tid 0: {"prefix": "status"}) -- 0x55d655573ca0 con 0x55d6551f9830 2024-05-14T11:26:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcac3400640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d6551f9830 0x55d65578e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaa3e00640 10 client.8484 ms_handle_connect on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcac3400640 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d6551f9830 0x55d65578e100 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3c854bd5b72be2 server_cookie=46c42a3fd654040a in_seq=0 out_seq=0 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaa3e00640 1 -- 172.21.15.116:0/1198319866 <== mds.0 v2:172.21.15.116:6836/3756089525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55d655573ca0 con 0x55d6551f9830 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaa3e00640 10 client.8484 handle_command_reply: tid=0 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 client.8484 shutdown 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 msgr2=0x55d65578d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcabc05b670 tx=0x7fcabc004740 comp rx=0 tx=0).stop 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 shutdown_connections 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55d6551f9830 0x55d65578e100 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d65560ef80 0x55d65578d7a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6555f86b0 0x55d6557d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 --2- 172.21.15.116:0/1198319866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6552267c0 0x55d6557d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 >> 172.21.15.116:0/1198319866 conn(0x55d6556e5340 msgr2=0x55d65576cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaa3e00640 0 client.8484 ms_handle_reset on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 shutdown_connections 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 1 -- 172.21.15.116:0/1198319866 wait complete. 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcad174a000 20 client.8484 trim_cache size 0 max 16384 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8427, 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.6080551139999999, 2024-05-14T11:26:07.223 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 106, 2024-05-14T11:26:07.224 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 114, 2024-05-14T11:26:07.224 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 114, 2024-05-14T11:26:07.224 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 5.4863337369999998 2024-05-14T11:26:07.224 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:07.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 msgr2=0x7fcaa4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 0x7fcaa4071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ae000 tx=0x7fcab4004ac0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 msgr2=0x7fcac40b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c7c0 tx=0x7fcabc00cc80 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 shutdown_connections 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcaa406f400 0x7fcaa4071910 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcac4150cd0 0x7fcac40b1780 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcac40af420 0x7fcac409a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 --2- 172.21.15.116:0/549012117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4074b70 0x7fcac40cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 >> 172.21.15.116:0/549012117 conn(0x7fcac40b4620 msgr2=0x7fcac40d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 shutdown_connections 2024-05-14T11:26:07.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.222+0000 7fcaca000640 1 -- 172.21.15.116:0/549012117 wait complete. 2024-05-14T11:26:07.240 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:26:07.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.386+0000 7f70c6000640 1 Processor -- start 2024-05-14T11:26:07.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.386+0000 7f70c6000640 1 -- start start 2024-05-14T11:26:07.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70c6000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70c6000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 0x7f70c00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70c6000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c0160510 con 0x7f70c00d1670 2024-05-14T11:26:07.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70c6000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f70c01573b0 con 0x7f70c00af380 2024-05-14T11:26:07.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70c6000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f70c0161120 con 0x7f70c0156fa0 2024-05-14T11:26:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bea00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f70c00d1670 0x7f70c00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41984/0 (socket says 172.21.15.116:41984) 2024-05-14T11:26:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bea00640 1 -- 172.21.15.116:0/935790816 learned_addr learned my addr 172.21.15.116:0/935790816 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bfe00640 1 --2- 172.21.15.116:0/935790816 >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 0x7f70c00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bf400640 1 --2- 172.21.15.116:0/935790816 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70be000640 1 -- 172.21.15.116:0/935790816 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173487721 0 0) 0x7f70c0160510 con 0x7f70c00d1670 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70be000640 1 -- 172.21.15.116:0/935790816 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70a4003530 con 0x7f70c00d1670 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bf400640 1 -- 172.21.15.116:0/935790816 >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 msgr2=0x7f70c00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bf400640 1 --2- 172.21.15.116:0/935790816 >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 0x7f70c00af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bf400640 1 -- 172.21.15.116:0/935790816 >> v1:172.21.15.116:6789/0 conn(0x7f70c00d1670 legacy=0x7f70c00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bf400640 1 -- 172.21.15.116:0/935790816 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c01607f0 con 0x7f70c0156fa0 2024-05-14T11:26:07.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.390+0000 7f70bfe00640 1 --2- 172.21.15.116:0/935790816 >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 0x7f70c00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:07.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70bf400640 1 --2- 172.21.15.116:0/935790816 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c006ffe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70b00097c0 tx=0x7f70b001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6c42572741c9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70be000640 1 -- 172.21.15.116:0/935790816 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00a7070 con 0x7f70c0156fa0 2024-05-14T11:26:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70be000640 1 -- 172.21.15.116:0/935790816 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b00b0070 con 0x7f70c0156fa0 2024-05-14T11:26:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70be000640 1 -- 172.21.15.116:0/935790816 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00ab890 con 0x7f70c0156fa0 2024-05-14T11:26:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 -- 172.21.15.116:0/935790816 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 msgr2=0x7f70c006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 --2- 172.21.15.116:0/935790816 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c006ffe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70b00097c0 tx=0x7f70b001c470 comp rx=0 tx=0).stop 2024-05-14T11:26:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 -- 172.21.15.116:0/935790816 shutdown_connections 2024-05-14T11:26:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 --2- 172.21.15.116:0/935790816 >> v2:172.21.15.149:3300/0 conn(0x7f70c00af380 0x7f70c00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 --2- 172.21.15.116:0/935790816 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c006ffe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 -- 172.21.15.116:0/935790816 >> 172.21.15.116:0/935790816 conn(0x7f70c00b4620 msgr2=0x7f70c00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 -- 172.21.15.116:0/935790816 shutdown_connections 2024-05-14T11:26:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.394+0000 7f70c6000640 1 -- 172.21.15.116:0/935790816 wait complete. 2024-05-14T11:26:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 Processor -- start 2024-05-14T11:26:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- start start 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70c00af380 0x7f70c0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00b0660 con 0x7f70c00d1670 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00b04e0 con 0x7f70c00af380 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f70c0169810 con 0x7f70c0156fa0 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44696/0 (socket says 172.21.15.116:44696) 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 -- 172.21.15.116:0/3167065578 learned_addr learned my addr 172.21.15.116:0/3167065578 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bf400640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70c00af380 0x7f70c0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bfe00640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 -- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 msgr2=0x7f70c015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 -- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70c00af380 msgr2=0x7f70c0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70c00af380 0x7f70c0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 -- 172.21.15.116:0/3167065578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c00d02b0 con 0x7f70c00d1670 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bfe00640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70bea00640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c430 tx=0x7f70a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8017070 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a8005cf0 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c00d0590 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80072f0 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c0154630 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.398+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c006a9c0 con 0x7f70c00d1670 2024-05-14T11:26:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.402+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f70a801a070 con 0x7f70c00d1670 2024-05-14T11:26:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.402+0000 7f70b7400640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 0x7f709c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.402+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f70a808e860 con 0x7f70c00d1670 2024-05-14T11:26:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.402+0000 7f70bf400640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 0x7f709c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:07.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.402+0000 7f70bf400640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 0x7f709c071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70b00af000 tx=0x7f70b00b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.406+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f70a805b620 con 0x7f70c00d1670 2024-05-14T11:26:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.602+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70c0153a70 con 0x7f70c00d1670 2024-05-14T11:26:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.602+0000 7f70b7400640 1 -- 172.21.15.116:0/3167065578 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) ==== 53+0+73 (secure 0 0 0) 0x7f70a805af20 con 0x7f70c00d1670 2024-05-14T11:26:07.602 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 msgr2=0x7f709c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 0x7f709c071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f70b00af000 tx=0x7f70b00b6040 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 msgr2=0x7f70c014f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c430 tx=0x7f70a800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 shutdown_connections 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f709c06f400 0x7f709c071910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70c0156fa0 0x7f70c015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c00d1670 0x7f70c014f630 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 --2- 172.21.15.116:0/3167065578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70c00af380 0x7f70c0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 >> 172.21.15.116:0/3167065578 conn(0x7f70c00b4620 msgr2=0x7f70c0152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 shutdown_connections 2024-05-14T11:26:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07.606+0000 7f70c6000640 1 -- 172.21.15.116:0/3167065578 wait complete. 2024-05-14T11:26:07.613 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:26:07.614+0000 7f7748e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:26:07.624 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:07.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:07.624 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:26:07.634 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41835sec preferred_lft 41835sec 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:26:07.636 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:07.637 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:26:07.637 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:07.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:07.637 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:26:07.696 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:26:07.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:07.698 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:26:07.749 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:26:07.749 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:26:07.749 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:07.750 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:26:07.750 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:26:07.750 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:26:07.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:26:07.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:26:07.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:26:07.751 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:26:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:26:07.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:07.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:07.884 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:26:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:26:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:26:07.893 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:07.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:07.894 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:26:07.957 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:26:07.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:26:07.958 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:08.006 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:08.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:26:08.006 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:08.057 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:26:08.118 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:26:08.171 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:26:08.171 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:26:08.171 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:26:08.171 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:26:08.171 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:26:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:08.173 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:08.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.174 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:26:08.226 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:26:08.226 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:26:08.273 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:26:08.293 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:26:08.344 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:26:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:26:08.346 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:26:08.346 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:08.346 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:08.346 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:08.346 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:08.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.347 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:26:08.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:26:08.378+0000 7fcd99d19580 -1 init, newargv = 0x7fcd80001090 newargc=15 2024-05-14T11:26:08.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49693]: starting ceph client 2024-05-14T11:26:08.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49693]: starting fuse 2024-05-14T11:26:08.386 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:26:08.387 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:26:08.387 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:26:08.388 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:08.515 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.49693.asok 2024-05-14T11:26:08.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:08.527 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49693.asok 2024-05-14T11:26:08.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.527 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49693.asok status 2024-05-14T11:26:08.707 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:08.707 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "49693", 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8499, 2024-05-14T11:26:08.708 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 8499 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2950520264 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2950520264 2024-05-14T11:26:08.709 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.8499 192.168.0.1:0/2950520264", 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/2950520264", 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 106, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 114, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:26:08.710 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:08.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.723 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:26:08.723 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:08.732 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:26:08.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:08.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.732 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:08.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:08.793 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-14T11:26:08.854 INFO:teuthology.orchestra.run.smithi116.stdout:./ 2024-05-14T11:26:08.854 INFO:teuthology.orchestra.run.smithi116.stdout:./parent 2024-05-14T11:26:08.854 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed 2024-05-14T11:26:08.854 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/charlie 2024-05-14T11:26:08.854 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/alpha 2024-05-14T11:26:08.855 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/unflushed 2024-05-14T11:26:08.855 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/unflushed/jfile 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-14T11:26:08.856 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:08.965 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:26:08.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-14T11:26:08.974 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.077 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:26:09.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.086 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:09.086 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:09.087 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:09.087 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:09.087 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-14T11:26:09.087 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.194 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:26:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-14T11:26:09.203 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.309 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628280 2024-05-14T11:26:09.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-14T11:26:09.319 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.430 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627778 2024-05-14T11:26:09.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-14T11:26:09.439 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.545 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627779 2024-05-14T11:26:09.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.554 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:09.554 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:09.555 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:09.555 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:09.555 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-14T11:26:09.555 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:09.664 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627780 2024-05-14T11:26:09.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:09.673 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:09.673 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:09.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:26:09.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 Processor -- start 2024-05-14T11:26:09.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 -- start start 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d00d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f90d00ce580 0x7f90d00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d006e860 con 0x7f90d00d0200 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f90d00ce0f0 con 0x7f90d00ce580 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90d006eb40 con 0x7f90d00d23f0 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90cfe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f90d00d0200 0x7f90d00d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40412/0 (socket says 172.21.15.116:40412) 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90cfe00640 1 -- 172.21.15.116:0/4060447946 learned_addr learned my addr 172.21.15.116:0/4060447946 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d5a00640 1 --2- 172.21.15.116:0/4060447946 >> v2:172.21.15.149:3300/0 conn(0x7f90d00ce580 0x7f90d00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.918+0000 7f90d5000640 1 --2- 172.21.15.116:0/4060447946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d00d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90cf400640 1 -- 172.21.15.116:0/4060447946 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867490445 0 0) 0x7f90d006e860 con 0x7f90d00d0200 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90cf400640 1 -- 172.21.15.116:0/4060447946 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90b4003530 con 0x7f90d00d0200 2024-05-14T11:26:09.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90d5000640 1 -- 172.21.15.116:0/4060447946 >> v2:172.21.15.149:3300/0 conn(0x7f90d00ce580 msgr2=0x7f90d00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90d5000640 1 --2- 172.21.15.116:0/4060447946 >> v2:172.21.15.149:3300/0 conn(0x7f90d00ce580 0x7f90d00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90d5000640 1 -- 172.21.15.116:0/4060447946 >> v1:172.21.15.116:6789/0 conn(0x7f90d00d0200 legacy=0x7f90d00d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:09.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90d5000640 1 -- 172.21.15.116:0/4060447946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d006ee20 con 0x7f90d00d23f0 2024-05-14T11:26:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90d5000640 1 --2- 172.21.15.116:0/4060447946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d00d2800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009d90 tx=0x7f90c001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5a380c72c5fa31 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90cf400640 1 -- 172.21.15.116:0/4060447946 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00b2070 con 0x7f90d00d23f0 2024-05-14T11:26:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90cf400640 1 -- 172.21.15.116:0/4060447946 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c00a5880 con 0x7f90d00d23f0 2024-05-14T11:26:09.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.922+0000 7f90cf400640 1 -- 172.21.15.116:0/4060447946 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00a5c30 con 0x7f90d00d23f0 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- 172.21.15.116:0/4060447946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 msgr2=0x7f90d00d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 --2- 172.21.15.116:0/4060447946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d00d2800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009d90 tx=0x7f90c001c970 comp rx=0 tx=0).stop 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- 172.21.15.116:0/4060447946 shutdown_connections 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 --2- 172.21.15.116:0/4060447946 >> v2:172.21.15.149:3300/0 conn(0x7f90d00ce580 0x7f90d00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 --2- 172.21.15.116:0/4060447946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d00d2800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- 172.21.15.116:0/4060447946 >> 172.21.15.116:0/4060447946 conn(0x7f90d00b51f0 msgr2=0x7f90d00b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:09.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- 172.21.15.116:0/4060447946 shutdown_connections 2024-05-14T11:26:09.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- 172.21.15.116:0/4060447946 wait complete. 2024-05-14T11:26:09.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 Processor -- start 2024-05-14T11:26:09.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.926+0000 7f90d6400640 1 -- start start 2024-05-14T11:26:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 0x7f90d01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90d016cbb0 con 0x7f90d00ce580 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90d006e5e0 con 0x7f90d00d0200 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90d016cd30 con 0x7f90d00d23f0 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52024/0 (socket says 172.21.15.116:52024) 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 -- 172.21.15.116:0/2634264329 learned_addr learned my addr 172.21.15.116:0/2634264329 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5a00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90cfe00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 0x7f90d01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 -- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 msgr2=0x7f90d01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d01607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 -- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 msgr2=0x7f90d01573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 0x7f90d01573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 -- 172.21.15.116:0/2634264329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d01602f0 con 0x7f90d00ce580 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90cfe00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 0x7f90d01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d5000640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009d90 tx=0x7f90c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:09.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00b2050 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c00b3040 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d01515a0 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00ad650 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d0151b60 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d0050880 con 0x7f90d00ce580 2024-05-14T11:26:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.930+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f90c00c0020 con 0x7f90d00ce580 2024-05-14T11:26:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.934+0000 7f90ccc00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 0x7f90ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.934+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f90c0129df0 con 0x7f90d00ce580 2024-05-14T11:26:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.934+0000 7f90cfe00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 0x7f90ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.934+0000 7f90cfe00640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 0x7f90ac071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005890 tx=0x7f90b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:09.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:09.938+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f90c0016560 con 0x7f90d00ce580 2024-05-14T11:26:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.106+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90d00d1390 con 0x7f90d00ce580 2024-05-14T11:26:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.106+0000 7f90ccc00640 1 -- 172.21.15.116:0/2634264329 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v114) ==== 80+0+124 (secure 0 0 0) 0x7f90c00f6a60 con 0x7f90d00ce580 2024-05-14T11:26:10.105 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6837/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:10.106 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:26:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 msgr2=0x7f90ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 0x7f90ac071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005890 tx=0x7f90b8009040 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 msgr2=0x7f90d0150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009d90 tx=0x7f90c0019710 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 shutdown_connections 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90ac06f400 0x7f90ac071910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d00d0200 0x7f90d01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d00d23f0 0x7f90d01607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 --2- 172.21.15.116:0/2634264329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d00ce580 0x7f90d0150c70 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 >> 172.21.15.116:0/2634264329 conn(0x7f90d00b51f0 msgr2=0x7f90d0154f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 shutdown_connections 2024-05-14T11:26:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.110+0000 7f90d6400640 1 -- 172.21.15.116:0/2634264329 wait complete. 2024-05-14T11:26:10.126 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:10.127 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:10.135 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:26:10.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 Processor -- start 2024-05-14T11:26:10.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 -- start start 2024-05-14T11:26:10.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240d1d80 0x7f2c2415db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c2415e070 0x7f2c2415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c2406ee40 con 0x7f2c240702e0 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2c2406fe70 con 0x7f2c2415e070 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c29600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2406f0b0 con 0x7f2c240d1d80 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c2415e070 0x7f2c2415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c2415e070 0x7f2c2415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52034/0 (socket says 172.21.15.116:52034) 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 -- 172.21.15.116:0/2090683474 learned_addr learned my addr 172.21.15.116:0/2090683474 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:10.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c1a000640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240d1d80 0x7f2c2415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 -- 172.21.15.116:0/2090683474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240d1d80 msgr2=0x7f2c2415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240d1d80 0x7f2c2415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 -- 172.21.15.116:0/2090683474 >> v1:172.21.15.149:6789/0 conn(0x7f2c240702e0 legacy=0x7f2c240d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.326+0000 7f2c23400640 1 -- 172.21.15.116:0/2090683474 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2406f390 con 0x7f2c2415e070 2024-05-14T11:26:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c23400640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c2415e070 0x7f2c2415e480 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009e50 tx=0x7f2c1c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a038d477652900ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c22000640 1 -- 172.21.15.116:0/2090683474 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0a57c0 con 0x7f2c2415e070 2024-05-14T11:26:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c22000640 1 -- 172.21.15.116:0/2090683474 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c0a5960 con 0x7f2c2415e070 2024-05-14T11:26:10.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c22000640 1 -- 172.21.15.116:0/2090683474 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0a5d30 con 0x7f2c2415e070 2024-05-14T11:26:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 -- 172.21.15.116:0/2090683474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c2415e070 msgr2=0x7f2c2415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c2415e070 0x7f2c2415e480 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009e50 tx=0x7f2c1c01ca00 comp rx=0 tx=0).stop 2024-05-14T11:26:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 -- 172.21.15.116:0/2090683474 shutdown_connections 2024-05-14T11:26:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c2415e070 0x7f2c2415e480 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 --2- 172.21.15.116:0/2090683474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240d1d80 0x7f2c2415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 -- 172.21.15.116:0/2090683474 >> 172.21.15.116:0/2090683474 conn(0x7f2c240b4620 msgr2=0x7f2c240b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:10.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 -- 172.21.15.116:0/2090683474 shutdown_connections 2024-05-14T11:26:10.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.330+0000 7f2c29600640 1 -- 172.21.15.116:0/2090683474 wait complete. 2024-05-14T11:26:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 Processor -- start 2024-05-14T11:26:10.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 -- start start 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c2415e070 0x7f2c24072d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240732d0 0x7f2c24156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2416a610 con 0x7f2c240702e0 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24070900 con 0x7f2c2415e070 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c29600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2416a790 con 0x7f2c240732d0 2024-05-14T11:26:10.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c22a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c22a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52050/0 (socket says 172.21.15.116:52050) 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c22a00640 1 -- 172.21.15.116:0/909562733 learned_addr learned my addr 172.21.15.116:0/909562733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c1a000640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c2415e070 0x7f2c24072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.334+0000 7f2c23400640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240732d0 0x7f2c24156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 -- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240732d0 msgr2=0x7f2c24156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240732d0 0x7f2c24156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 -- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c2415e070 msgr2=0x7f2c24072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c2415e070 0x7f2c24072d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 -- 172.21.15.116:0/909562733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c240749f0 con 0x7f2c240702e0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c22a00640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c3f0 tx=0x7f2c1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14017070 con 0x7f2c240702e0 2024-05-14T11:26:10.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c140052d0 con 0x7f2c240702e0 2024-05-14T11:26:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c24073950 con 0x7f2c240702e0 2024-05-14T11:26:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14003840 con 0x7f2c240702e0 2024-05-14T11:26:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c24073f10 con 0x7f2c240702e0 2024-05-14T11:26:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2406a9c0 con 0x7f2c240702e0 2024-05-14T11:26:10.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.338+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2c1401a070 con 0x7f2c240702e0 2024-05-14T11:26:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.342+0000 7f2c1b400640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 0x7f2c00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.342+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f2c1408e800 con 0x7f2c240702e0 2024-05-14T11:26:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.342+0000 7f2c1a000640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 0x7f2c00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:10.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.342+0000 7f2c1a000640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 0x7f2c00071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2c2414e620 tx=0x7f2c0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:10.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.346+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2c14005480 con 0x7f2c240702e0 2024-05-14T11:26:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.514+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c240cd6c0 con 0x7f2c240702e0 2024-05-14T11:26:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.514+0000 7f2c1b400640 1 -- 172.21.15.116:0/909562733 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v114) ==== 80+0+124 (secure 0 0 0) 0x7f2c14001090 con 0x7f2c240702e0 2024-05-14T11:26:10.515 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6837/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:10.515 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:listed 2 entries 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 msgr2=0x7f2c00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 0x7f2c00071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2c2414e620 tx=0x7f2c0c008040 comp rx=0 tx=0).stop 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 msgr2=0x7f2c2414a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c3f0 tx=0x7f2c1400c8b0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 shutdown_connections 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f2c0006f400 0x7f2c00071910 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c2415e070 0x7f2c24072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2c240732d0 0x7f2c24156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 --2- 172.21.15.116:0/909562733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c240702e0 0x7f2c2414a590 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 >> 172.21.15.116:0/909562733 conn(0x7f2c240b4620 msgr2=0x7f2c2406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 shutdown_connections 2024-05-14T11:26:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:10.518+0000 7f2c29600640 1 -- 172.21.15.116:0/909562733 wait complete. 2024-05-14T11:26:10.536 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:26:10.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:10.536 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:10.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[49693]: fuse finished with error 0 and tester_r 0 2024-05-14T11:26:10.556 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:26:13.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:26:13.806 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:26:16.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:26:16.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:16.659 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:26:16.669 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:16.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.874+0000 7f4b76600640 1 Processor -- start 2024-05-14T11:26:16.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.874+0000 7f4b76600640 1 -- start start 2024-05-14T11:26:16.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b76600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 0x7f4b701606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b76600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4b700c7b00 0x7f4b700c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b76600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b70160d50 con 0x7f4b701602c0 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b76600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4b70161310 con 0x7f4b700c7b00 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b76600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4b70161030 con 0x7f4b700cd020 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4b701602c0 0x7f4b700cc910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:40414/0 (socket says 172.21.15.116:40414) 2024-05-14T11:26:16.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75200640 1 -- 172.21.15.116:0/1447649994 learned_addr learned my addr 172.21.15.116:0/1447649994 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b67e00640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 0x7f4b701606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75c00640 1 --2- 172.21.15.116:0/1447649994 >> v2:172.21.15.149:3300/0 conn(0x7f4b700c7b00 0x7f4b700c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b6fe00640 1 -- 172.21.15.116:0/1447649994 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719083648 0 0) 0x7f4b70160d50 con 0x7f4b701602c0 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b6fe00640 1 -- 172.21.15.116:0/1447649994 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b54003530 con 0x7f4b701602c0 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75c00640 1 -- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 msgr2=0x7f4b701606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75c00640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 0x7f4b701606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75c00640 1 -- 172.21.15.116:0/1447649994 >> v1:172.21.15.116:6789/0 conn(0x7f4b701602c0 legacy=0x7f4b700cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b75c00640 1 -- 172.21.15.116:0/1447649994 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b700c7300 con 0x7f4b700c7b00 2024-05-14T11:26:16.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.878+0000 7f4b67e00640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 0x7f4b701606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:16.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b75c00640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b700c7b00 0x7f4b700c7f10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009d90 tx=0x7f4b6801c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebb85da77b747ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:16.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b6fe00640 1 -- 172.21.15.116:0/1447649994 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680b2070 con 0x7f4b700c7b00 2024-05-14T11:26:16.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b6fe00640 1 -- 172.21.15.116:0/1447649994 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b680a5920 con 0x7f4b700c7b00 2024-05-14T11:26:16.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b6fe00640 1 -- 172.21.15.116:0/1447649994 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680a5cf0 con 0x7f4b700c7b00 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b700c7b00 msgr2=0x7f4b700c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b700c7b00 0x7f4b700c7f10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009d90 tx=0x7f4b6801c900 comp rx=0 tx=0).stop 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- 172.21.15.116:0/1447649994 shutdown_connections 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b700c7b00 0x7f4b700c7f10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 --2- 172.21.15.116:0/1447649994 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700cd020 0x7f4b701606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- 172.21.15.116:0/1447649994 >> 172.21.15.116:0/1447649994 conn(0x7f4b700d28c0 msgr2=0x7f4b700ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- 172.21.15.116:0/1447649994 shutdown_connections 2024-05-14T11:26:16.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- 172.21.15.116:0/1447649994 wait complete. 2024-05-14T11:26:16.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 Processor -- start 2024-05-14T11:26:16.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.882+0000 7f4b76600640 1 -- start start 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 0x7f4b70142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 0x7f4b7015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70163c30 con 0x7f4b700b6ae0 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70163ab0 con 0x7f4b701602c0 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7016ca80 con 0x7f4b700c7b00 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52064/0 (socket says 172.21.15.116:52064) 2024-05-14T11:26:16.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 -- 172.21.15.116:0/3502110492 learned_addr learned my addr 172.21.15.116:0/3502110492 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b67e00640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 0x7f4b7015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75200640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 0x7f4b70142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 -- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 msgr2=0x7f4b70142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 0x7f4b70142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 -- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 msgr2=0x7f4b7015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 0x7f4b7015f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 -- 172.21.15.116:0/3502110492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b701417e0 con 0x7f4b700b6ae0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b67e00640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 0x7f4b7015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75200640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 0x7f4b70142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b75c00640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4b6801afb0 tx=0x7f4b680196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680b2050 con 0x7f4b700b6ae0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b680be070 con 0x7f4b700b6ae0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680ad850 con 0x7f4b700b6ae0 2024-05-14T11:26:16.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b70141ac0 con 0x7f4b700b6ae0 2024-05-14T11:26:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b70142080 con 0x7f4b700b6ae0 2024-05-14T11:26:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.886+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b70050880 con 0x7f4b700b6ae0 2024-05-14T11:26:16.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.890+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b680ad9f0 con 0x7f4b700b6ae0 2024-05-14T11:26:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.890+0000 7f4b6d600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.890+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f4b68129ff0 con 0x7f4b700b6ae0 2024-05-14T11:26:16.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.890+0000 7f4b75200640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:16.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.890+0000 7f4b75200640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4b60001230 tx=0x7f4b60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:16.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:16.894+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4b680c7020 con 0x7f4b700b6ae0 2024-05-14T11:26:17.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.082+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b700b3bc0 con 0x7f4b700b6ae0 2024-05-14T11:26:17.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.082+0000 7f4b6d600640 1 -- 172.21.15.116:0/3502110492 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5200 (secure 0 0 0) 0x7f4b68016520 con 0x7f4b700b6ae0 2024-05-14T11:26:17.083 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:17.083 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:05.625205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8427":{"gid":8427,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/3756089525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3756089525},{"type":"v1","addr":"172.21.15.116:6837","nonce":3756089525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8427,"qdb_cluster":[8427]},"id":8}]} 2024-05-14T11:26:17.083 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 106 2024-05-14T11:26:17.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 msgr2=0x7f4b4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4b60001230 tx=0x7f4b60009040 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 msgr2=0x7f4b700b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4b6801afb0 tx=0x7f4b680196a0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 shutdown_connections 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4b4c06f430 0x7f4b4c071940 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b701602c0 0x7f4b7015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b700b6ae0 0x7f4b700b6ef0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 --2- 172.21.15.116:0/3502110492 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b700c7b00 0x7f4b70142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 >> 172.21.15.116:0/3502110492 conn(0x7f4b700d28c0 msgr2=0x7f4b700b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 shutdown_connections 2024-05-14T11:26:17.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.086+0000 7f4b76600640 1 -- 172.21.15.116:0/3502110492 wait complete. 2024-05-14T11:26:17.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-05-14T11:26:17.275 INFO:teuthology.orchestra.run.smithi116.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-14T11:26:17.275 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-14T11:26:17.276 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-14T11:26:17.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 Processor -- start 2024-05-14T11:26:17.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- start start 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1fc8003100 0x556a7bd4e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7bd5b280 0x556a7bd5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556a7bcd2aa0 con 0x556a7be05590 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556a7bcd6ae0 con 0x556a7bd5b280 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556a7bd75f60 con 0x7f1fc8003100 2024-05-14T11:26:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52090/0 (socket says 172.21.15.116:52090) 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 -- 172.21.15.116:0/861713057 learned_addr learned my addr 172.21.15.116:0/861713057 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 -- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1fc8003100 msgr2=0x556a7bd4e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1fc8003100 0x556a7bd4e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 -- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7bd5b280 msgr2=0x556a7bd5b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7bd5b280 0x556a7bd5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 -- 172.21.15.116:0/861713057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7be02ab0 con 0x556a7be05590 2024-05-14T11:26:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe3800640 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006510 tx=0x7f1fdc0069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b0c23ffa8093fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe1000640 1 -- 172.21.15.116:0/861713057 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc011590 con 0x556a7be05590 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe1000640 1 -- 172.21.15.116:0/861713057 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc011730 con 0x556a7be05590 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe1000640 1 -- 172.21.15.116:0/861713057 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc011900 con 0x556a7be05590 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/861713057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 msgr2=0x556a7be059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006510 tx=0x7f1fdc0069d0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/861713057 shutdown_connections 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7bd5b280 0x556a7bd5b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556a7be05590 0x556a7be059a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/861713057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1fc8003100 0x556a7bd4e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.338+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/861713057 >> 172.21.15.116:0/861713057 conn(0x556a7bd17e10 msgr2=0x556a7be02da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/861713057 shutdown_connections 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/861713057 wait complete. 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 Processor -- start 2024-05-14T11:26:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- start start 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556a7bd5b280 0x556a7bdfb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7be05590 0x556a7bde4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 0x556a7be04520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556a7bd58040 con 0x7f1fc8003100 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556a7bd582e0 con 0x556a7be05590 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe9d4a080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556a7be02470 con 0x556a7bd5b280 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 0x556a7be04520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe2e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556a7bd5b280 0x556a7bdfb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 0x556a7be04520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52096/0 (socket says 172.21.15.116:52096) 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 -- 172.21.15.116:0/1589292952 learned_addr learned my addr 172.21.15.116:0/1589292952 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe2400640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7be05590 0x556a7bde4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 -- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556a7bd5b280 msgr2=0x556a7bdfb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556a7bd5b280 0x556a7bdfb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 -- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7be05590 msgr2=0x556a7bde4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7be05590 0x556a7bde4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 -- 172.21.15.116:0/1589292952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7be021d0 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe2400640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a7be05590 0x556a7bde4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fe3800640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 0x556a7be04520 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc011ee0 tx=0x7f1fdc001600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.342+0000 7f1fdb400640 1 -- 172.21.15.116:0/1589292952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0128b0 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fdb400640 1 -- 172.21.15.116:0/1589292952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc012a50 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fdb400640 1 -- 172.21.15.116:0/1589292952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc012e40 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556a7bd700e0 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7bd70620 con 0x7f1fc8003100 2024-05-14T11:26:17.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fdb400640 1 -- 172.21.15.116:0/1589292952 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1fdc019040 con 0x7f1fc8003100 2024-05-14T11:26:17.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fdb400640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1fc406dde0 0x7f1fc40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fdb400640 1 -- 172.21.15.116:0/1589292952 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f1fdc08ebe0 con 0x7f1fc8003100 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe2e00640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1fc406dde0 0x7f1fc40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x556a7bd70ac0 0x556a7bd70ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x556a7bd71410 con 0x556a7bd70ac0 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe2400640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x556a7bd70ac0 0x556a7bd70ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.346+0000 7f1fe2e00640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1fc406dde0 0x7f1fc40702f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40019c0 tx=0x7f1fd4012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe2400640 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x556a7bd70ac0 0x556a7bd70ed0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe2400640 1 -- 172.21.15.116:0/1589292952 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f1fd00ae070 con 0x556a7bd70ac0 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x556a7bd70ac0 msgr2=0x556a7bd70ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x556a7bd70ac0 0x556a7bd70ed0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1fc406dde0 msgr2=0x7f1fc40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1fc406dde0 0x7f1fc40702f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40019c0 tx=0x7f1fd4012040 comp rx=0 tx=0).stop 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 msgr2=0x556a7be04520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 --2- 172.21.15.116:0/1589292952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc8003100 0x556a7be04520 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc011ee0 tx=0x7f1fdc001600 comp rx=0 tx=0).stop 2024-05-14T11:26:17.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe3800640 1 -- 172.21.15.116:0/1589292952 reap_dead start 2024-05-14T11:26:17.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 shutdown_connections 2024-05-14T11:26:17.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 >> 172.21.15.116:0/1589292952 conn(0x556a7bd17e10 msgr2=0x556a7bd57910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 shutdown_connections 2024-05-14T11:26:17.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.350+0000 7f1fe9d4a080 1 -- 172.21.15.116:0/1589292952 wait complete. 2024-05-14T11:26:17.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:26:17.353 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-14T11:26:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.410+0000 7f888b00f080 1 Processor -- start 2024-05-14T11:26:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.410+0000 7f888b00f080 1 -- start start 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f37000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 0x5576a7f19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5576a7ea82d0 con 0x5576a7ea91f0 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5576a7e07aa0 con 0x5576a7e62c60 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5576a7ea8570 con 0x5576a7e91a20 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52106/0 (socket says 172.21.15.116:52106) 2024-05-14T11:26:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 -- 172.21.15.116:0/4088937669 learned_addr learned my addr 172.21.15.116:0/4088937669 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8884200640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f37000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 -- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 msgr2=0x5576a7f37000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8884c00640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 0x5576a7f19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f37000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 -- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 msgr2=0x5576a7f19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 0x5576a7f19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 -- 172.21.15.116:0/4088937669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a7f38000 con 0x5576a7ea91f0 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8884200640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f37000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8884c00640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 0x5576a7f19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8883800640 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8870001e60 tx=0x7f887000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b522cc772c1320 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8882400640 1 -- 172.21.15.116:0/4088937669 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870012020 con 0x5576a7ea91f0 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8882400640 1 -- 172.21.15.116:0/4088937669 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887000e690 con 0x5576a7ea91f0 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f8882400640 1 -- 172.21.15.116:0/4088937669 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887000ea60 con 0x5576a7ea91f0 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- 172.21.15.116:0/4088937669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 msgr2=0x5576a7e62720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8870001e60 tx=0x7f887000b790 comp rx=0 tx=0).stop 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- 172.21.15.116:0/4088937669 shutdown_connections 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7e62c60 0x5576a7f19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7ea91f0 0x5576a7e62720 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 --2- 172.21.15.116:0/4088937669 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f37000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- 172.21.15.116:0/4088937669 >> 172.21.15.116:0/4088937669 conn(0x5576a7e4ce10 msgr2=0x5576a7f32780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- 172.21.15.116:0/4088937669 shutdown_connections 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- 172.21.15.116:0/4088937669 wait complete. 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 Processor -- start 2024-05-14T11:26:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.414+0000 7f888b00f080 1 -- start start 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 0x5576a7f31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f068b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7ea91f0 0x5576a7f06df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5576a7f38540 con 0x5576a7e62c60 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5576a7f382a0 con 0x5576a7ea91f0 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5576a7f387e0 con 0x5576a7e91a20 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 0x5576a7f31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 0x5576a7f31ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52110/0 (socket says 172.21.15.116:52110) 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 -- 172.21.15.116:0/1616817378 learned_addr learned my addr 172.21.15.116:0/1616817378 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 -- 172.21.15.116:0/1616817378 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 msgr2=0x5576a7f068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5576a7e91a20 0x5576a7f068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 -- 172.21.15.116:0/1616817378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7ea91f0 msgr2=0x5576a7f06df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576a7ea91f0 0x5576a7f06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 -- 172.21.15.116:0/1616817378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a7e83600 con 0x5576a7e62c60 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8884200640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 0x5576a7f31ae0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f887401c250 tx=0x7f887401a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8881000640 1 -- 172.21.15.116:0/1616817378 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740c1070 con 0x5576a7e62c60 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8881000640 1 -- 172.21.15.116:0/1616817378 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88740af7a0 con 0x5576a7e62c60 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576a7e8f0c0 con 0x5576a7e62c60 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f8881000640 1 -- 172.21.15.116:0/1616817378 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740b4040 con 0x5576a7e62c60 2024-05-14T11:26:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.418+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a7e8f600 con 0x5576a7e62c60 2024-05-14T11:26:17.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8881000640 1 -- 172.21.15.116:0/1616817378 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f88740c4070 con 0x5576a7e62c60 2024-05-14T11:26:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8881000640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406dde0 0x7f88640702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8881000640 1 -- 172.21.15.116:0/1616817378 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f887412c0e0 con 0x5576a7e62c60 2024-05-14T11:26:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8883800640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406dde0 0x7f88640702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5576a7e8faa0 0x5576a7e8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x5576a7e903f0 con 0x5576a7e8faa0 2024-05-14T11:26:17.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8884c00640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5576a7e8faa0 0x5576a7e8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8883800640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406dde0 0x7f88640702f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8870001220 tx=0x7f8870006000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8884c00640 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5576a7e8faa0 0x5576a7e8feb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8884c00640 1 -- 172.21.15.116:0/1616817378 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f887c00c700 con 0x5576a7e8faa0 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5576a7e8faa0 msgr2=0x5576a7e8feb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5576a7e8faa0 0x5576a7e8feb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406dde0 msgr2=0x7f88640702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f886406dde0 0x7f88640702f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8870001220 tx=0x7f8870006000 comp rx=0 tx=0).stop 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 msgr2=0x5576a7f31ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 --2- 172.21.15.116:0/1616817378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576a7e62c60 0x5576a7f31ae0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f887401c250 tx=0x7f887401a940 comp rx=0 tx=0).stop 2024-05-14T11:26:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f8884c00640 1 -- 172.21.15.116:0/1616817378 reap_dead start 2024-05-14T11:26:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 shutdown_connections 2024-05-14T11:26:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 >> 172.21.15.116:0/1616817378 conn(0x5576a7e4ce10 msgr2=0x5576a7f375d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 shutdown_connections 2024-05-14T11:26:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.422+0000 7f888b00f080 1 -- 172.21.15.116:0/1616817378 wait complete. 2024-05-14T11:26:17.426 INFO:teuthology.orchestra.run.smithi116.stdout:mytag123 2024-05-14T11:26:17.427 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-14T11:26:17.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 Processor -- start 2024-05-14T11:26:17.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 -- start start 2024-05-14T11:26:17.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e1607bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e1608130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d6e15962d0 con 0x55d6e157fa20 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d6e14f5aa0 con 0x55d6e1626630 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cc1120080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d6e1596570 con 0x55d6e15971f0 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52118/0 (socket says 172.21.15.116:52118) 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 -- 172.21.15.116:0/4240814114 learned_addr learned my addr 172.21.15.116:0/4240814114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cb9800640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e1607bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cbac00640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e1608130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 -- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 msgr2=0x55d6e1607bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e1607bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 -- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 msgr2=0x55d6e1608130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e1608130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 -- 172.21.15.116:0/4240814114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e1624b70 con 0x55d6e157fa20 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cb9800640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e1607bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cbac00640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e1608130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.486+0000 7f0cba200640 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0cac01b890 tx=0x7f0cac003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706fea93456504b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cb3e00640 1 -- 172.21.15.116:0/4240814114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b4070 con 0x55d6e157fa20 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cb3e00640 1 -- 172.21.15.116:0/4240814114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac0be070 con 0x55d6e157fa20 2024-05-14T11:26:17.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cb3e00640 1 -- 172.21.15.116:0/4240814114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b9880 con 0x55d6e157fa20 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- 172.21.15.116:0/4240814114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 msgr2=0x55d6e16260f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0cac01b890 tx=0x7f0cac003030 comp rx=0 tx=0).stop 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- 172.21.15.116:0/4240814114 shutdown_connections 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e1608130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e1607bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- 172.21.15.116:0/4240814114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e16260f0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- 172.21.15.116:0/4240814114 >> 172.21.15.116:0/4240814114 conn(0x55d6e153ae10 msgr2=0x55d6e1620780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- 172.21.15.116:0/4240814114 shutdown_connections 2024-05-14T11:26:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- 172.21.15.116:0/4240814114 wait complete. 2024-05-14T11:26:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 Processor -- start 2024-05-14T11:26:17.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- start start 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e157d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e161fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e157e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d6e16250b0 con 0x55d6e157fa20 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d6e1624e10 con 0x55d6e1626630 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cc1120080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55d6e1625350 con 0x55d6e15971f0 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cba200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e157d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cba200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e157d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52130/0 (socket says 172.21.15.116:52130) 2024-05-14T11:26:17.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.490+0000 7f0cba200640 1 -- 172.21.15.116:0/72749135 learned_addr learned my addr 172.21.15.116:0/72749135 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb9800640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e161fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cbac00640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e157e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 -- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 msgr2=0x55d6e161fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55d6e15971f0 0x55d6e161fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 -- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 msgr2=0x55d6e157e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6e1626630 0x55d6e157e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 -- 172.21.15.116:0/72749135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e1620050 con 0x55d6e157fa20 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cba200640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e157d6b0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00a5c0 tx=0x7f0cac01aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb2a00640 1 -- 172.21.15.116:0/72749135 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b4050 con 0x55d6e157fa20 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb2a00640 1 -- 172.21.15.116:0/72749135 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac0b5040 con 0x55d6e157fa20 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d6e157dd70 con 0x55d6e157fa20 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb2a00640 1 -- 172.21.15.116:0/72749135 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b9850 con 0x55d6e157fa20 2024-05-14T11:26:17.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e1593790 con 0x55d6e157fa20 2024-05-14T11:26:17.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb2a00640 1 -- 172.21.15.116:0/72749135 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0cac0cc020 con 0x55d6e157fa20 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.494+0000 7f0cb2a00640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c9c06dde0 0x7f0c9c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cb2a00640 1 -- 172.21.15.116:0/72749135 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f0cac12d6d0 con 0x55d6e157fa20 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cb9800640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c9c06dde0 0x7f0c9c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55d6e1593ed0 0x55d6e15942e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55d6e1594820 con 0x55d6e1593ed0 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cb9800640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c9c06dde0 0x7f0c9c0702f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8000f70 tx=0x7f0ca8001610 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cbac00640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55d6e1593ed0 0x55d6e15942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cbac00640 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55d6e1593ed0 0x55d6e15942e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cbac00640 1 -- 172.21.15.116:0/72749135 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f0cb4011040 con 0x55d6e1593ed0 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55d6e1593ed0 msgr2=0x55d6e15942e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55d6e1593ed0 0x55d6e15942e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c9c06dde0 msgr2=0x7f0c9c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0c9c06dde0 0x7f0c9c0702f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8000f70 tx=0x7f0ca8001610 comp rx=0 tx=0).stop 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 msgr2=0x55d6e157d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 --2- 172.21.15.116:0/72749135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6e157fa20 0x55d6e157d6b0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00a5c0 tx=0x7f0cac01aa00 comp rx=0 tx=0).stop 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cbac00640 1 -- 172.21.15.116:0/72749135 reap_dead start 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 shutdown_connections 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 >> 172.21.15.116:0/72749135 conn(0x55d6e153ae10 msgr2=0x55d6e1624330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 shutdown_connections 2024-05-14T11:26:17.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.498+0000 7f0cc1120080 1 -- 172.21.15.116:0/72749135 wait complete. 2024-05-14T11:26:17.501 INFO:teuthology.orchestra.run.smithi116.stdout:mytag123 2024-05-14T11:26:17.502 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-14T11:26:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 Processor -- start 2024-05-14T11:26:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 -- start start 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x556082047ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820466b0 0x55608203f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55608202cd00 con 0x5560820d1980 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556081fa4aa0 con 0x5560820466b0 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5560820451d0 con 0x556082045420 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56148/0 (socket says 172.21.15.116:56148) 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 -- 172.21.15.116:0/167916654 learned_addr learned my addr 172.21.15.116:0/167916654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5dec00640 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x556082047ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5de200640 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820466b0 0x55608203f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 -- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 msgr2=0x556082047ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x556082047ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 -- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820466b0 msgr2=0x55608203f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820466b0 0x55608203f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 -- 172.21.15.116:0/167916654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608202ded0 con 0x5560820d1980 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5df600640 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb5d801b5e0 tx=0x7fb5d80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d24d4bb0582f7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5dce00640 1 -- 172.21.15.116:0/167916654 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80a8cd0 con 0x5560820d1980 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5dce00640 1 -- 172.21.15.116:0/167916654 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d80b0440 con 0x5560820d1980 2024-05-14T11:26:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.562+0000 7fb5dce00640 1 -- 172.21.15.116:0/167916654 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b07f0 con 0x5560820d1980 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- 172.21.15.116:0/167916654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 msgr2=0x55608203dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb5d801b5e0 tx=0x7fb5d80033b0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- 172.21.15.116:0/167916654 shutdown_connections 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820466b0 0x55608203f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820d1980 0x55608203dc30 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/167916654 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x556082047ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- 172.21.15.116:0/167916654 >> 172.21.15.116:0/167916654 conn(0x556081fe9e10 msgr2=0x55608202e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- 172.21.15.116:0/167916654 shutdown_connections 2024-05-14T11:26:17.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- 172.21.15.116:0/167916654 wait complete. 2024-05-14T11:26:17.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 Processor -- start 2024-05-14T11:26:17.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- start start 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x5560820b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 0x5560820ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820d1980 0x5560820b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556082000250 con 0x5560820466b0 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556082000030 con 0x5560820d1980 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5e5a11080 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5560820004f0 con 0x556082045420 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 0x5560820ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 0x5560820ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56154/0 (socket says 172.21.15.116:56154) 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 -- 172.21.15.116:0/717920624 learned_addr learned my addr 172.21.15.116:0/717920624 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5dec00640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x5560820b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5df600640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820d1980 0x5560820b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 -- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 msgr2=0x5560820b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556082045420 0x5560820b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 -- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820d1980 msgr2=0x5560820b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820d1980 0x5560820b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 -- 172.21.15.116:0/717920624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560820cf2f0 con 0x5560820466b0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5df600640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560820d1980 0x5560820b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.566+0000 7fb5de200640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 0x5560820ce9c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00dfb0 tx=0x7fb5cc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 -- 172.21.15.116:0/717920624 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc014020 con 0x5560820466b0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 -- 172.21.15.116:0/717920624 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5cc010850 con 0x5560820466b0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560820cf590 con 0x5560820466b0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 -- 172.21.15.116:0/717920624 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc010c20 con 0x5560820466b0 2024-05-14T11:26:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560820cfad0 con 0x5560820466b0 2024-05-14T11:26:17.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 -- 172.21.15.116:0/717920624 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5cc017020 con 0x5560820466b0 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5c006dde0 0x7fb5c00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5d7400640 1 -- 172.21.15.116:0/717920624 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fb5cc08e4e0 con 0x5560820466b0 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5dec00640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5c006dde0 0x7fb5c00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5560820b65e0 0x5560820b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x5560820b6f30 con 0x5560820b65e0 2024-05-14T11:26:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.570+0000 7fb5df600640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5560820b65e0 0x5560820b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5dec00640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5c006dde0 0x7fb5c00702f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009730 tx=0x7fb5d0005aa0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5df600640 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5560820b65e0 0x5560820b69f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5df600640 1 -- 172.21.15.116:0/717920624 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb5d80a8d50 con 0x5560820b65e0 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5560820b65e0 msgr2=0x5560820b69f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5560820b65e0 0x5560820b69f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5c006dde0 msgr2=0x7fb5c00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb5c006dde0 0x7fb5c00702f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009730 tx=0x7fb5d0005aa0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 msgr2=0x5560820ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 --2- 172.21.15.116:0/717920624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560820466b0 0x5560820ce9c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00dfb0 tx=0x7fb5cc00b6d0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5df600640 1 -- 172.21.15.116:0/717920624 reap_dead start 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 shutdown_connections 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 >> 172.21.15.116:0/717920624 conn(0x556081fe9e10 msgr2=0x55608203dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 shutdown_connections 2024-05-14T11:26:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.574+0000 7fb5e5a11080 1 -- 172.21.15.116:0/717920624 wait complete. 2024-05-14T11:26:17.577 INFO:teuthology.orchestra.run.smithi116.stdout:mytag123 2024-05-14T11:26:17.577 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-14T11:26:17.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.726+0000 7f234ee00640 1 Processor -- start 2024-05-14T11:26:17.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.726+0000 7f234ee00640 1 -- start start 2024-05-14T11:26:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ee00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2348166750 0x7f2348168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ee00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234806ff20 con 0x7f23480d1670 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ee00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2348070470 con 0x7f234815db30 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ee00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f234806d710 con 0x7f2348166750 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ce00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2348166750 0x7f2348168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f2347400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f23480d1670 0x7f23480d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48302/0 (socket says 172.21.15.116:48302) 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f2347400640 1 -- 172.21.15.116:0/1554750319 learned_addr learned my addr 172.21.15.116:0/1554750319 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ce00640 1 -- 172.21.15.116:0/1554750319 >> v1:172.21.15.149:6789/0 conn(0x7f234815db30 legacy=0x7f23480aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ce00640 1 -- 172.21.15.116:0/1554750319 >> v1:172.21.15.116:6789/0 conn(0x7f23480d1670 legacy=0x7f23480d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.730+0000 7f234ce00640 1 -- 172.21.15.116:0/1554750319 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348070750 con 0x7f2348166750 2024-05-14T11:26:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ce00640 1 --2- 172.21.15.116:0/1554750319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348166750 0x7f2348168b70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2733d8d0dcf00947 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f2346a00640 1 -- 172.21.15.116:0/1554750319 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23400b2070 con 0x7f2348166750 2024-05-14T11:26:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f2346a00640 1 -- 172.21.15.116:0/1554750319 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23400a5710 con 0x7f2348166750 2024-05-14T11:26:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f2346a00640 1 -- 172.21.15.116:0/1554750319 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23400a5ae0 con 0x7f2348166750 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 -- 172.21.15.116:0/1554750319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348166750 msgr2=0x7f2348168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 --2- 172.21.15.116:0/1554750319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348166750 0x7f2348168b70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234001c850 comp rx=0 tx=0).stop 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 -- 172.21.15.116:0/1554750319 shutdown_connections 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 --2- 172.21.15.116:0/1554750319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348166750 0x7f2348168b70 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 -- 172.21.15.116:0/1554750319 >> 172.21.15.116:0/1554750319 conn(0x7f23480b4620 msgr2=0x7f23480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 -- 172.21.15.116:0/1554750319 shutdown_connections 2024-05-14T11:26:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 -- 172.21.15.116:0/1554750319 wait complete. 2024-05-14T11:26:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.734+0000 7f234ee00640 1 Processor -- start 2024-05-14T11:26:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- start start 2024-05-14T11:26:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480d1670 0x7f23480cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2348166750 0x7f2348150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f234816bae0 con 0x7f234815db30 2024-05-14T11:26:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23480ae870 con 0x7f2348166750 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f234816bc60 con 0x7f23480d1670 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56178/0 (socket says 172.21.15.116:56178) 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 -- 172.21.15.116:0/2453180487 learned_addr learned my addr 172.21.15.116:0/2453180487 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ce00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2348166750 0x7f2348150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347e00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480d1670 0x7f23480cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 -- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480d1670 msgr2=0x7f23480cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480d1670 0x7f23480cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 -- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2348166750 msgr2=0x7f2348150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2348166750 0x7f2348150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 -- 172.21.15.116:0/2453180487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348160510 con 0x7f234815db30 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f2347400640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f233400c500 tx=0x7f233400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f233fe00640 1 -- 172.21.15.116:0/2453180487 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334017070 con 0x7f234815db30 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f233fe00640 1 -- 172.21.15.116:0/2453180487 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2334005cf0 con 0x7f234815db30 2024-05-14T11:26:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f233fe00640 1 -- 172.21.15.116:0/2453180487 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23340072d0 con 0x7f234815db30 2024-05-14T11:26:17.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481607f0 con 0x7f234815db30 2024-05-14T11:26:17.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.738+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348160db0 con 0x7f234815db30 2024-05-14T11:26:17.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f233fe00640 1 -- 172.21.15.116:0/2453180487 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f233401a070 con 0x7f234815db30 2024-05-14T11:26:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f23564e4000 1 Processor -- start 2024-05-14T11:26:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f233fe00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 0x7f2328071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f23564e4000 1 -- start start 2024-05-14T11:26:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f233fe00640 1 -- 172.21.15.116:0/2453180487 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f233408e6a0 con 0x7f234815db30 2024-05-14T11:26:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.742+0000 7f234ce00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 0x7f2328071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f234ce00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 0x7f2328071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f23400af000 tx=0x7f23400a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7b6f00 0x55747d930890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d930dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55747d735100 con 0x55747d7c9120 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55747d6edca0 con 0x55747d7b6f00 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55747d7310c0 con 0x55747d3b63e0 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56184/0 (socket says 172.21.15.116:56184) 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 -- 172.21.15.116:0/438007720 learned_addr learned my addr 172.21.15.116:0/438007720 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347400640 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7b6f00 0x55747d930890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f234ce00640 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d930dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 -- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 msgr2=0x55747d930dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d930dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 -- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7b6f00 msgr2=0x55747d930890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7b6f00 0x55747d930890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 -- 172.21.15.116:0/438007720 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747d6f2800 con 0x55747d7c9120 2024-05-14T11:26:17.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f2347e00640 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2338001810 tx=0x7f2338007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ba4feac0c0afcb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f233cc00640 1 -- 172.21.15.116:0/438007720 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380052e0 con 0x55747d7c9120 2024-05-14T11:26:17.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f233cc00640 1 -- 172.21.15.116:0/438007720 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338018070 con 0x55747d7c9120 2024-05-14T11:26:17.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f233cc00640 1 -- 172.21.15.116:0/438007720 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233800f9b0 con 0x55747d7c9120 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- 172.21.15.116:0/438007720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 msgr2=0x55747d930350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2338001810 tx=0x7f2338007e40 comp rx=0 tx=0).stop 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- 172.21.15.116:0/438007720 shutdown_connections 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7b6f00 0x55747d930890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d930dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 --2- 172.21.15.116:0/438007720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7c9120 0x55747d930350 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- 172.21.15.116:0/438007720 >> 172.21.15.116:0/438007720 conn(0x55747d3f6400 msgr2=0x55747d913310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- 172.21.15.116:0/438007720 shutdown_connections 2024-05-14T11:26:17.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.746+0000 7f23564e4000 1 -- 172.21.15.116:0/438007720 wait complete. 2024-05-14T11:26:17.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- start start 2024-05-14T11:26:17.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 Processor -- start 2024-05-14T11:26:17.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d936dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d8e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55747d8002a0 con 0x55747d7b6f00 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55747d7f0f40 con 0x55747d7c9120 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55747d6f1080 con 0x55747d3b63e0 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 10 client.0 _ll_register_callbacks cb 0x55747d8d1c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56188/0 (socket says 172.21.15.116:56188) 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d936dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 -- 172.21.15.116:0/177926231 learned_addr learned my addr 172.21.15.116:0/177926231 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f234ce00640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d8e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 -- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 msgr2=0x55747d936dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d936dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 -- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 msgr2=0x55747d8e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d8e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 -- 172.21.15.116:0/177926231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747d6df860 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2347400640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f23340573f0 tx=0x7f233405b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f171da8f0dcc174 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334072070 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f2334075d40 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2338018070 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f233405f100 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 10 client.8565 fetch_fsmap learned FSMap version 106 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55747d6edef0 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338018070 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.750+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f2334061310 con 0x55747d7b6f00 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 10 client.8565 fetch_fsmap finished waiting for FSMap version 106 2024-05-14T11:26:17.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 10 client.8565 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-14T11:26:17.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 20 client.8565 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:17.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3da930 0x55747d8eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 4 client.8565 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:26:17.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 --> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55747d6cddf0 con 0x55747d3da930 2024-05-14T11:26:17.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2347e00640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3da930 0x55747d8eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2347e00640 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3da930 0x55747d8eacd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3bbd32b9174270 server_cookie=c447d90980d8b81 in_seq=0 out_seq=0 2024-05-14T11:26:17.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2331600640 10 client.8565 ms_handle_connect on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:17.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2331600640 1 -- 172.21.15.116:0/177926231 <== mds.0 v2:172.21.15.116:6836/3756089525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55747d6cddf0 con 0x55747d3da930 2024-05-14T11:26:17.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2331600640 10 client.8565 handle_command_reply: tid=0 2024-05-14T11:26:17.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 client.8565 shutdown 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 msgr2=0x55747d937310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f23340573f0 tx=0x7f233405b3a0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 shutdown_connections 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3da930 0x55747d8eacd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d8e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d937310 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f2331600640 0 client.8565 ms_handle_reset on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 --2- 172.21.15.116:0/177926231 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d936dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 >> 172.21.15.116:0/177926231 conn(0x55747d3f6400 msgr2=0x55747d913200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 shutdown_connections 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 1 -- 172.21.15.116:0/177926231 wait complete. 2024-05-14T11:26:17.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.754+0000 7f23564e4000 20 client.8565 trim_cache size 0 max 16384 2024-05-14T11:26:17.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 Processor -- start 2024-05-14T11:26:17.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 -- start start 2024-05-14T11:26:17.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d99e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d99f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23482052a0 con 0x55747d7b6f00 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f234815f460 con 0x55747d7c9120 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2348204c50 con 0x55747d3b63e0 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56190/0 (socket says 172.21.15.116:56190) 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 -- 172.21.15.116:0/1219923138 learned_addr learned my addr 172.21.15.116:0/1219923138 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f234ce00640 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d99f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347e00640 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d99e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 -- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 msgr2=0x55747d99e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d99e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 -- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 msgr2=0x55747d99f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d99f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 -- 172.21.15.116:0/1219923138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747d8e8070 con 0x55747d7b6f00 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2347400640 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f23340573a0 tx=0x7f233401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9d84db0a86d51c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2333e00640 1 -- 172.21.15.116:0/1219923138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334072070 con 0x55747d7b6f00 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2333e00640 1 -- 172.21.15.116:0/1219923138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2334063b20 con 0x55747d7b6f00 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.770+0000 7f2333e00640 1 -- 172.21.15.116:0/1219923138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334057450 con 0x55747d7b6f00 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- 172.21.15.116:0/1219923138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 msgr2=0x55747d99ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f23340573a0 tx=0x7f233401d000 comp rx=0 tx=0).stop 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- 172.21.15.116:0/1219923138 shutdown_connections 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d7c9120 0x55747d99f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d99ec40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- 172.21.15.116:0/1219923138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d3b63e0 0x55747d99e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- 172.21.15.116:0/1219923138 >> 172.21.15.116:0/1219923138 conn(0x55747d3f6400 msgr2=0x55747d9123a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- 172.21.15.116:0/1219923138 shutdown_connections 2024-05-14T11:26:17.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- 172.21.15.116:0/1219923138 wait complete. 2024-05-14T11:26:17.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- start start 2024-05-14T11:26:17.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 Processor -- start 2024-05-14T11:26:17.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d3b63e0 0x55747d9a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 0x55747d93abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55747d739730 con 0x55747d7b6f00 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55747d734410 con 0x55747d3b63e0 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55747d742500 con 0x55747d7c9120 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f23564e4000 10 client.0 _ll_register_callbacks cb 0x7f2338007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56192/0 (socket says 172.21.15.116:56192) 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 -- 172.21.15.116:0/612562412 learned_addr learned my addr 172.21.15.116:0/612562412 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f234ce00640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 0x55747d93abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 -- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 msgr2=0x55747d93abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 0x55747d93abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 -- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d3b63e0 msgr2=0x55747d9a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d3b63e0 0x55747d9a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f2347400640 1 -- 172.21.15.116:0/612562412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747d7b6340 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.774+0000 7f234ce00640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 0x55747d93abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2347400640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f233408cad0 tx=0x7f233405e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586e51520381d902 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334072070 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f2334056770 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2334063b20 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f2334057450 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 10 client.8571 fetch_fsmap learned FSMap version 106 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55747d6d18e0 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2334063b20 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f2334059ad0 con 0x55747d7b6f00 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 10 client.8571 fetch_fsmap finished waiting for FSMap version 106 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 10 client.8571 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 20 client.8571 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:17.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3c8990 0x55747d93b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 4 client.8571 mds_command: new command op to 8427 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T11:26:17.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 --> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55747d742d10 con 0x55747d3c8990 2024-05-14T11:26:17.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2347e00640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3c8990 0x55747d93b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2347e00640 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3c8990 0x55747d93b580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d8c43ecb185ded server_cookie=9657d48d672a3f29 in_seq=0 out_seq=0 2024-05-14T11:26:17.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.778+0000 7f2331600640 10 client.8571 ms_handle_connect on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:17.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f2331600640 1 -- 172.21.15.116:0/612562412 <== mds.0 v2:172.21.15.116:6836/3756089525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55747d742d10 con 0x55747d3c8990 2024-05-14T11:26:17.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f2331600640 10 client.8571 handle_command_reply: tid=0 2024-05-14T11:26:17.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 client.8571 shutdown 2024-05-14T11:26:17.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 msgr2=0x55747d9a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f233408cad0 tx=0x7f233405e000 comp rx=0 tx=0).stop 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 shutdown_connections 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:6836/3756089525,v1:172.21.15.116:6837/3756089525] conn(0x55747d3c8990 0x55747d93b580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55747d7c9120 0x55747d93abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55747d7b6f00 0x55747d9a3070 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f2331600640 0 client.8571 ms_handle_reset on v2:172.21.15.116:6836/3756089525 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 --2- 172.21.15.116:0/612562412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55747d3b63e0 0x55747d9a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 >> 172.21.15.116:0/612562412 conn(0x55747d3f6400 msgr2=0x55747d912100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 shutdown_connections 2024-05-14T11:26:17.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 1 -- 172.21.15.116:0/612562412 wait complete. 2024-05-14T11:26:17.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f23564e4000 20 client.8571 trim_cache size 0 max 16384 2024-05-14T11:26:17.791 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:17.792 INFO:teuthology.orchestra.run.smithi116.stdout: "message": "", 2024-05-14T11:26:17.792 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0 2024-05-14T11:26:17.792 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:17.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 msgr2=0x7f2328071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 0x7f2328071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f23400af000 tx=0x7f23400a7070 comp rx=0 tx=0).stop 2024-05-14T11:26:17.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 msgr2=0x7f23481501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f233400c500 tx=0x7f233400c9c0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 shutdown_connections 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f232806f430 0x7f2328071940 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2348166750 0x7f2348150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f234815db30 0x7f23481501f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 --2- 172.21.15.116:0/2453180487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f23480d1670 0x7f23480cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 >> 172.21.15.116:0/2453180487 conn(0x7f23480b4620 msgr2=0x7f234815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 shutdown_connections 2024-05-14T11:26:17.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.790+0000 7f234ee00640 1 -- 172.21.15.116:0/2453180487 wait complete. 2024-05-14T11:26:17.808 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:26:17.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.958+0000 7fe5bfc00640 1 Processor -- start 2024-05-14T11:26:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bfc00640 1 -- start start 2024-05-14T11:26:17.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bfc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe5b8168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bfc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b806ff20 con 0x7fe5b80d1670 2024-05-14T11:26:17.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bfc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b8070470 con 0x7fe5b815db30 2024-05-14T11:26:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bfc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b806d710 con 0x7fe5b8166750 2024-05-14T11:26:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe5b80d1670 0x7fe5b80d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48314/0 (socket says 172.21.15.116:48314) 2024-05-14T11:26:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 -- 172.21.15.116:0/485868823 learned_addr learned my addr 172.21.15.116:0/485868823 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bdc00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe5b8168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731961865 0 0) 0x7fe5b806ff20 con 0x7fe5b80d1670 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe59c003530 con 0x7fe5b80d1670 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437511920 0 0) 0x7fe5b8070470 con 0x7fe5b815db30 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5b806ff20 con 0x7fe5b815db30 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2231610231 0 0) 0x7fe59c003530 con 0x7fe5b80d1670 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 msgr2=0x7fe5b8168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe5b8168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 >> v1:172.21.15.149:6789/0 conn(0x7fe5b815db30 legacy=0x7fe5b80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b8070750 con 0x7fe5b80d1670 2024-05-14T11:26:17.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fe5a0001700 con 0x7fe5b80d1670 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 >> v1:172.21.15.116:6789/0 conn(0x7fe5b80d1670 legacy=0x7fe5b80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe59c003c20 0x7fe59c004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c004810 0x7fe59c004ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe59c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c0055e0 con 0x7fe59c004810 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c00bf10 con 0x7fe59c003c20 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe59c00c090 con 0x7fe5b8166750 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c004810 0x7fe59c004ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bd200640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe59c003c20 0x7fe59c004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5bdc00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe59c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 -- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 msgr2=0x7fe59c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe5b8166750 0x7fe59c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 -- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe59c003c20 msgr2=0x7fe59c004070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe59c003c20 0x7fe59c004070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.962+0000 7fe5afe00640 1 -- 172.21.15.116:0/485868823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c003530 con 0x7fe59c004810 2024-05-14T11:26:17.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5afe00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c004810 0x7fe59c004ca0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000c1c0 tx=0x7fe5a001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca82b9576ebd16a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0007c90 con 0x7fe59c004810 2024-05-14T11:26:17.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a00a9020 con 0x7fe59c004810 2024-05-14T11:26:17.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5b7e00640 1 -- 172.21.15.116:0/485868823 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0005ae0 con 0x7fe59c004810 2024-05-14T11:26:17.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bfc00640 1 -- 172.21.15.116:0/485868823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c004810 msgr2=0x7fe59c004ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/485868823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c004810 0x7fe59c004ca0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000c1c0 tx=0x7fe5a001cb40 comp rx=0 tx=0).stop 2024-05-14T11:26:17.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bdc00640 1 -- 172.21.15.116:0/485868823 reap_dead start 2024-05-14T11:26:17.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bfc00640 1 -- 172.21.15.116:0/485868823 shutdown_connections 2024-05-14T11:26:17.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bfc00640 1 -- 172.21.15.116:0/485868823 >> 172.21.15.116:0/485868823 conn(0x7fe5b80b4620 msgr2=0x7fe5b80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:17.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.966+0000 7fe5bfc00640 1 -- 172.21.15.116:0/485868823 shutdown_connections 2024-05-14T11:26:17.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 -- 172.21.15.116:0/485868823 wait complete. 2024-05-14T11:26:17.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 Processor -- start 2024-05-14T11:26:17.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 -- start start 2024-05-14T11:26:17.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe59c004810 0x7fe5b81567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 0x7fe5b81fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b8169590 con 0x7fe5b81fd170 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b80ae870 con 0x7fe5b81fc4b0 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5bfc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b8169710 con 0x7fe59c004810 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.970+0000 7fe5afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56208/0 (socket says 172.21.15.116:56208) 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 -- 172.21.15.116:0/554478821 learned_addr learned my addr 172.21.15.116:0/554478821 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bdc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 0x7fe5b81fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bd200640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe59c004810 0x7fe5b81567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 -- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe59c004810 msgr2=0x7fe5b81567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe59c004810 0x7fe5b81567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 -- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 msgr2=0x7fe5b81fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 0x7fe5b81fc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 -- 172.21.15.116:0/554478821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b81fce00 con 0x7fe5b81fd170 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bdc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 0x7fe5b81fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5afe00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000c160 tx=0x7fe59c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00bc070 con 0x7fe5b81fd170 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a00049f0 con 0x7fe5b81fd170 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b80cd8f0 con 0x7fe5b81fd170 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00010e0 con 0x7fe5b81fd170 2024-05-14T11:26:17.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b80cdeb0 con 0x7fe5b81fd170 2024-05-14T11:26:17.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b806a9c0 con 0x7fe5b81fd170 2024-05-14T11:26:17.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.974+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe5a00bf070 con 0x7fe5b81fd170 2024-05-14T11:26:17.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.978+0000 7fe5b5600640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 0x7fe594071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:17.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.978+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7fe5a01285b0 con 0x7fe5b81fd170 2024-05-14T11:26:17.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.978+0000 7fe5bd200640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 0x7fe594071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:17.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.978+0000 7fe5bd200640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 0x7fe594071940 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8007fd0 tx=0x7fe5a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:17.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:17.982+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe5b81fce00 con 0x7fe5b81fd170 2024-05-14T11:26:18.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.170+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5b80af460 con 0x7fe5b81fd170 2024-05-14T11:26:18.170 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:26:18.170+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:26:18.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.174+0000 7fe5b5600640 1 -- 172.21.15.116:0/554478821 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) ==== 161+0+0 (secure 0 0 0) 0x7fe5a00b4070 con 0x7fe5b81fd170 2024-05-14T11:26:18.172 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 msgr2=0x7fe594071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 0x7fe594071940 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8007fd0 tx=0x7fe5a8009040 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 msgr2=0x7fe5b8152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000c160 tx=0x7fe59c005840 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 shutdown_connections 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe59406f430 0x7fe594071940 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5b81fc4b0 0x7fe5b81fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5b81fd170 0x7fe5b8152060 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 --2- 172.21.15.116:0/554478821 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe59c004810 0x7fe5b81567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 >> 172.21.15.116:0/554478821 conn(0x7fe5b80b4620 msgr2=0x7fe5b8073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 shutdown_connections 2024-05-14T11:26:18.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.178+0000 7fe5bfc00640 1 -- 172.21.15.116:0/554478821 wait complete. 2024-05-14T11:26:18.183 INFO:tasks.ceph.mds.c.smithi116.stderr: -2213> 2024-05-14T11:26:07.614+0000 7f7748e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:26:18.196 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-14T11:26:18.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- start start 2024-05-14T11:26:18.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 Processor -- start 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c3b00 0x555cf86ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4400 0x555cf86ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f4480 con 0x555cf79c3680 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f4000 con 0x555cf79c4400 2024-05-14T11:26:18.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555cf86f4600 con 0x555cf79c3b00 2024-05-14T11:26:18.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56226/0 (socket says 172.21.15.116:56226) 2024-05-14T11:26:18.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 -- 172.21.15.116:0/405343107 learned_addr learned my addr 172.21.15.116:0/405343107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c80000640 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4400 0x555cf86ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c3b00 0x555cf86ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 -- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c3b00 msgr2=0x555cf86ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c3b00 0x555cf86ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 -- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4400 msgr2=0x555cf86ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4400 0x555cf86ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 -- 172.21.15.116:0/405343107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf86c8b40 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7f600640 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x555cf876d890 tx=0x555cf7a06480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad3591347eb9a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7c400640 1 -- 172.21.15.116:0/405343107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf79224e0 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7c400640 1 -- 172.21.15.116:0/405343107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555cf79361c0 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7c400640 1 -- 172.21.15.116:0/405343107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf7922b60 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf86c8d20 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555cf86c8f00 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 4 waiting for MDS map... 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7c400640 1 -- 172.21.15.116:0/405343107 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x555cf8818600 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c7c400640 1 -- 172.21.15.116:0/405343107 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x555cf7943180 con 0x555cf79c3680 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 4 Got MDS map 107 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 10 main: JournalTool::main 2024-05-14T11:26:18.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.262+0000 7f4c8224a400 4 main: JournalTool: connecting to RADOS... 2024-05-14T11:26:18.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 Processor -- start 2024-05-14T11:26:18.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 -- start start 2024-05-14T11:26:18.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cf794c180 con 0x555cf79c2900 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cf794c480 con 0x555cf79c2480 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555cf794c900 con 0x555cf79c2d80 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56238/0 (socket says 172.21.15.116:56238) 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.266+0000 7f4c7f600640 1 -- 172.21.15.116:0/3024124284 learned_addr learned my addr 172.21.15.116:0/3024124284 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c80000640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 -- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 msgr2=0x555cf86cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 -- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 msgr2=0x555cf86cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 -- 172.21.15.116:0/3024124284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf86c8d20 con 0x555cf79c2900 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:18.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c7f600640 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x555cf8826c60 tx=0x555cf7a06960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781313af789842a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c79c00640 1 -- 172.21.15.116:0/3024124284 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf7922340 con 0x555cf79c2900 2024-05-14T11:26:18.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c79c00640 1 -- 172.21.15.116:0/3024124284 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555cf7936e00 con 0x555cf79c2900 2024-05-14T11:26:18.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c79c00640 1 -- 172.21.15.116:0/3024124284 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf79224e0 con 0x555cf79c2900 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- 172.21.15.116:0/3024124284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 msgr2=0x555cf86cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x555cf8826c60 tx=0x555cf7a06960 comp rx=0 tx=0).stop 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- 172.21.15.116:0/3024124284 shutdown_connections 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 --2- 172.21.15.116:0/3024124284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- 172.21.15.116:0/3024124284 >> 172.21.15.116:0/3024124284 conn(0x555cf79c3200 msgr2=0x555cf7c18000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- 172.21.15.116:0/3024124284 shutdown_connections 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- 172.21.15.116:0/3024124284 wait complete. 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 Processor -- start 2024-05-14T11:26:18.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.270+0000 7f4c8224a400 1 -- start start 2024-05-14T11:26:18.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f5b00 con 0x555cf79c2480 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f5200 con 0x555cf79c2900 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555cf86f5500 con 0x555cf79c2d80 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56250/0 (socket says 172.21.15.116:56250) 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 -- 172.21.15.116:0/1509312069 learned_addr learned my addr 172.21.15.116:0/1509312069 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 -- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 msgr2=0x555cf86cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 -- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 msgr2=0x555cf86cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 -- 172.21.15.116:0/1509312069 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf86c9c20 con 0x555cf79c2480 2024-05-14T11:26:18.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c80000640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x555cf8827980 tx=0x555cf8830e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c77e00640 1 -- 172.21.15.116:0/1509312069 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf7922340 con 0x555cf79c2480 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c77e00640 1 -- 172.21.15.116:0/1509312069 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555cf7936540 con 0x555cf79c2480 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c77e00640 1 -- 172.21.15.116:0/1509312069 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf79224e0 con 0x555cf79c2480 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cf86c9e00 con 0x555cf79c2480 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf86c9e00 con 0x555cf79c2480 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c8224a400 4 main: JournalTool: resolving pool 17 2024-05-14T11:26:18.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.274+0000 7f4c77e00640 1 -- 172.21.15.116:0/1509312069 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x555cf7c2ec00 con 0x555cf79c2480 2024-05-14T11:26:18.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c77e00640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 0x555cf86cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c77e00640 1 -- 172.21.15.116:0/1509312069 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x555cf7c34a00 con 0x555cf79c2480 2024-05-14T11:26:18.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 4 main: JournalTool: creating IoCtx.. 2024-05-14T11:26:18.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 4 main: Executing for rank 0 2024-05-14T11:26:18.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 0x555cf86cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 0x555cf86cc100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x555cf8826c60 tx=0x555cf8830c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 -- start start 2024-05-14T11:26:18.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 Processor -- start 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 0x555cf86cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 0x555cf86ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c5600 0x555cf86cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f5080 con 0x555cf79c4d00 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cf86f5680 con 0x555cf79c4880 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c8224a400 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x555cf794c180 con 0x555cf79c5600 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 0x555cf86ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7f600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 0x555cf86ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56262/0 (socket says 172.21.15.116:56262) 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c80000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 0x555cf86cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7f600640 1 -- 172.21.15.116:0/1708696102 learned_addr learned my addr 172.21.15.116:0/1708696102 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.278+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c5600 0x555cf86cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c5600 msgr2=0x555cf86cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c5600 0x555cf86cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 msgr2=0x555cf86ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 0x555cf86ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7d74960 con 0x555cf79c4880 2024-05-14T11:26:18.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c4d00 0x555cf86ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:18.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c80000640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 0x555cf86cc680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x555cf7c5f8c0 tx=0x555cf88317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e5a83041742e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c74200640 1 -- 172.21.15.116:0/1708696102 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf7cbd1e0 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c74200640 1 -- 172.21.15.116:0/1708696102 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555cf7936e00 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c74200640 1 -- 172.21.15.116:0/1708696102 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf7cbd380 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf7d74b40 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555cf7d74d20 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 4 waiting for MDS map... 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c74200640 1 -- 172.21.15.116:0/1708696102 <== mon.1 v2:172.21.15.149:3300/0 4 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x555cf7d86200 con 0x555cf79c4880 2024-05-14T11:26:18.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c74200640 1 -- 172.21.15.116:0/1708696102 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x555cf7943500 con 0x555cf79c4880 2024-05-14T11:26:18.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 4 Got MDS map 107 2024-05-14T11:26:18.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x555cf79c5a80 0x555cf86cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7d89800 con 0x555cf79c5a80 2024-05-14T11:26:18.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x555cf79c5a80 0x555cf86cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.282+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x555cf79c5a80 0x555cf86cd700 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=3d22c75df8137d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.286+0000 7f4c7f600640 1 -- 172.21.15.116:0/1708696102 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v115'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec000 con 0x555cf79c5a80 2024-05-14T11:26:18.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.286+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x555cf7da2000 0x555cf7daa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.286+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7d89c00 con 0x555cf7da2000 2024-05-14T11:26:18.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.286+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x555cf7da2000 0x555cf7daa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.286+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x555cf7da2000 0x555cf7daa000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=73a15874fce9716 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7ec00640 1 -- 172.21.15.116:0/1708696102 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v115'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec500 con 0x555cf7da2000 2024-05-14T11:26:18.291 INFO:teuthology.orchestra.run.smithi116.stdout:writing EResetJournal entry 2024-05-14T11:26:18.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7dae000 con 0x555cf7da2000 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 0x555cf7daa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7dae400 con 0x555cf7da2480 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x555cf7da2900 0x555cf7daab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7dae800 con 0x555cf7da2900 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 0x555cf7dab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7d89c00 con 0x555cf7da2d80 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf38bdc00 con 0x555cf7da2900 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf38bd400 con 0x555cf79c5a80 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x555cf7da2900 0x555cf7daab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c80000640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 0x555cf7daa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 0x555cf7dab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7f600640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x555cf7da2900 0x555cf7daab00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=4389a353745e3c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7ec00640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 0x555cf7dab080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=e005b8c72f38f673 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c80000640 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 0x555cf7daa580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=17cea69f210022ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7ec00640 1 -- 172.21.15.116:0/1708696102 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(3 200.00000001 [delete] v115'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec500 con 0x555cf7da2000 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7f600640 1 -- 172.21.15.116:0/1708696102 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(8 200.00000006 [delete] v115'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec000 con 0x555cf79c5a80 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7f600640 1 -- 172.21.15.116:0/1708696102 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(7 200.00000005 [delete] v115'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec000 con 0x555cf7da2900 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.294+0000 7f4c7f600640 1 -- 172.21.15.116:0/1708696102 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(5 200.00000003 [delete] v115'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec000 con 0x555cf7da2900 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c80000640 1 -- 172.21.15.116:0/1708696102 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(4 200.00000002 [delete] v115'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec780 con 0x555cf7da2480 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c7ec00640 1 -- 172.21.15.116:0/1708696102 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(6 200.00000004 [delete] v115'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec500 con 0x555cf7da2d80 2024-05-14T11:26:18.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c72400640 1 -- 172.21.15.116:0/1708696102 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x555cf7c37400 con 0x555cf7da2000 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c7ec00640 1 -- 172.21.15.116:0/1708696102 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v115'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x555cf79ec500 con 0x555cf7da2000 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x555cf7da2900 msgr2=0x555cf7daab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x555cf7da2900 0x555cf7daab00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x555cf79c5a80 msgr2=0x555cf86cd700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x555cf79c5a80 0x555cf86cd700 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x555cf7da2000 msgr2=0x555cf7daa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x555cf7da2000 0x555cf7daa000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 msgr2=0x555cf7daa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 0x555cf7daa580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c7ec00640 1 -- 172.21.15.116:0/1708696102 reap_dead start 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 msgr2=0x555cf7dab080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 0x555cf7dab080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 msgr2=0x555cf86cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 0x555cf86cc680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x555cf7c5f8c0 tx=0x555cf88317a0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 shutdown_connections 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x555cf7da2480 0x555cf7daa580 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x555cf7da2d80 0x555cf7dab080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1708696102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4880 0x555cf86cc680 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 >> 172.21.15.116:0/1708696102 conn(0x555cf79c2000 msgr2=0x555cf7c19000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 shutdown_connections 2024-05-14T11:26:18.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1708696102 wait complete. 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 msgr2=0x555cf86cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 0x555cf86cc100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x555cf8826c60 tx=0x555cf8830c00 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 msgr2=0x555cf86cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x555cf8827980 tx=0x555cf8830e40 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 shutdown_connections 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x555cf79c5180 0x555cf86cc100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c2900 0x555cf86cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c2d80 0x555cf86cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 --2- 172.21.15.116:0/1509312069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c2480 0x555cf86cbb80 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 >> 172.21.15.116:0/1509312069 conn(0x555cf79c3200 msgr2=0x555cf7c18000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 shutdown_connections 2024-05-14T11:26:18.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.298+0000 7f4c8224a400 1 -- 172.21.15.116:0/1509312069 wait complete. 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 msgr2=0x555cf86cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x555cf876d890 tx=0x555cf7a06480 comp rx=0 tx=0).stop 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 shutdown_connections 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cf79c4400 0x555cf86ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x555cf79c3b00 0x555cf86ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 --2- 172.21.15.116:0/405343107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cf79c3680 0x555cf86cab00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 >> 172.21.15.116:0/405343107 conn(0x555cf79c3f80 msgr2=0x555cf38eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 shutdown_connections 2024-05-14T11:26:18.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.302+0000 7f4c8224a400 1 -- 172.21.15.116:0/405343107 wait complete. 2024-05-14T11:26:18.308 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.112122, result: writing EResetJournal entry 2024-05-14T11:26:18.309 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-14T11:26:18.321 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:26:18.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.366+0000 7f65803208c0 1 -- start start 2024-05-14T11:26:18.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.366+0000 7f65803208c0 1 Processor -- start 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29d680 0x56557f006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29db00 0x56557f006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56557f00e180 con 0x56557e29df80 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56557f00e300 con 0x56557e29d680 2024-05-14T11:26:18.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56557f00e480 con 0x56557e29db00 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56298/0 (socket says 172.21.15.116:56298) 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 -- 172.21.15.116:0/4144960172 learned_addr learned my addr 172.21.15.116:0/4144960172 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657da00640 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29db00 0x56557f006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657e400640 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29d680 0x56557f006580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 -- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29db00 msgr2=0x56557f006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29db00 0x56557f006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 -- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29d680 msgr2=0x56557f006580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29d680 0x56557f006580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 -- 172.21.15.116:0/4144960172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56557a2201e0 con 0x56557e29df80 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657ee00640 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x56557f097350 tx=0x56557e345ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf79ea655c3d177 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657b200640 1 -- 172.21.15.116:0/4144960172 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e2609c0 con 0x56557e29df80 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657b200640 1 -- 172.21.15.116:0/4144960172 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56557e274e00 con 0x56557e29df80 2024-05-14T11:26:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657b200640 1 -- 172.21.15.116:0/4144960172 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e2604e0 con 0x56557e29df80 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56557a2214a0 con 0x56557e29df80 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56557a2203c0 con 0x56557e29df80 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 4 waiting for MDS map... 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657b200640 1 -- 172.21.15.116:0/4144960172 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x56557f156200 con 0x56557e29df80 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f657b200640 1 -- 172.21.15.116:0/4144960172 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x56557e281180 con 0x56557e29df80 2024-05-14T11:26:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 4 Got MDS map 107 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 4 datascan.main: Using metadata pool output 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.370+0000 7f65803208c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 Processor -- start 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- start start 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56557f00e600 con 0x56557e29cd80 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56557f00e780 con 0x56557e29e880 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56557f00e900 con 0x56557e29c900 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56312/0 (socket says 172.21.15.116:56312) 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 -- 172.21.15.116:0/4209933575 learned_addr learned my addr 172.21.15.116:0/4209933575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657e400640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 -- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 msgr2=0x56557f007600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 -- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 msgr2=0x56557f007b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 -- 172.21.15.116:0/4209933575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56557a2201e0 con 0x56557e29cd80 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657e400640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f657da00640 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x56557f1631a0 tx=0x56557f166c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e6f6d0c711b776 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f6578a00640 1 -- 172.21.15.116:0/4209933575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e2609c0 con 0x56557e29cd80 2024-05-14T11:26:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f6578a00640 1 -- 172.21.15.116:0/4209933575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56557e274540 con 0x56557e29cd80 2024-05-14T11:26:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f6578a00640 1 -- 172.21.15.116:0/4209933575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e2604e0 con 0x56557e29cd80 2024-05-14T11:26:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- 172.21.15.116:0/4209933575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 msgr2=0x56557f007080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x56557f1631a0 tx=0x56557f166c60 comp rx=0 tx=0).stop 2024-05-14T11:26:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- 172.21.15.116:0/4209933575 shutdown_connections 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 --2- 172.21.15.116:0/4209933575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- 172.21.15.116:0/4209933575 >> 172.21.15.116:0/4209933575 conn(0x56557e29d200 msgr2=0x56557e546000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- 172.21.15.116:0/4209933575 shutdown_connections 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.374+0000 7f65803208c0 1 -- 172.21.15.116:0/4209933575 wait complete. 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 Processor -- start 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- start start 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56557f00fc80 con 0x56557e29c900 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56557f00fb00 con 0x56557e29e880 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x56557f00f980 con 0x56557e29cd80 2024-05-14T11:26:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56326/0 (socket says 172.21.15.116:56326) 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 learned_addr learned my addr 172.21.15.116:0/194877954 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29cd80 msgr2=0x56557f007080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29cd80 0x56557f007080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 msgr2=0x56557f007b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29e880 0x56557f007b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56557a2212c0 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x56557f163e60 tx=0x56557f166c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f6576c00640 1 -- 172.21.15.116:0/194877954 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e260000 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f6576c00640 1 -- 172.21.15.116:0/194877954 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56557e274a80 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56557a221a40 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f6576c00640 1 -- 172.21.15.116:0/194877954 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56557e2609c0 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56557a221c20 con 0x56557e29c900 2024-05-14T11:26:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.378+0000 7f65803208c0 4 datascan.init: resolving metadata pool 17 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f6576c00640 1 -- 172.21.15.116:0/194877954 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x56557e570c00 con 0x56557e29c900 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f6576c00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 0x56557f008100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f6576c00640 1 -- 172.21.15.116:0/194877954 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x56557e566a00 con 0x56557e29c900 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f65803208c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f65803208c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f65803208c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 0x56557f008100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x56557e29ed00 0x56557f008680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e000 con 0x56557e29ed00 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x56557e29ed00 0x56557f008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 0x56557f008100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x56557e347080 tx=0x56557e670000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x56557e29ed00 0x56557f008680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(1 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f6578000640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x56557e682000 0x56557e686000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f400 con 0x56557e682000 2024-05-14T11:26:18.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x56557e682000 0x56557e686000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.382+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x56557e682000 0x56557e686000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.386+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(2 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.386+0000 7f6578a00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 0x56557e686580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.386+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f800 con 0x56557e682480 2024-05-14T11:26:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.386+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 0x56557e686580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.386+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 0x56557e686580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(3 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e682480 2024-05-14T11:26:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66fc00 con 0x56557e29ed00 2024-05-14T11:26:18.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(4 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e690000 con 0x56557e29ed00 2024-05-14T11:26:18.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(5 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e690800 con 0x56557e682000 2024-05-14T11:26:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(6 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578a00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56557e682900 0x56557e686b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e690400 con 0x56557e682900 2024-05-14T11:26:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56557e682900 0x56557e686b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56557e682900 0x56557e686b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(7 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578000640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x56557e683200 0x56557e687600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e690c00 con 0x56557e683200 2024-05-14T11:26:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x56557e683200 0x56557e687600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x56557e683200 0x56557e687600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(8 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310c00 con 0x56557e29ed00 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(9 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.390+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e691000 con 0x56557e29ed00 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(10 [pgnls start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310800 con 0x56557e682480 2024-05-14T11:26:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(11 [pgnls start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e682480 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e690800 con 0x56557e29ed00 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(12 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310400 con 0x56557e682000 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(13 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578000640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x56557e683680 0x56557e687b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f400 con 0x56557e683680 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x56557e683680 0x56557e687b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657e400640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x56557e683680 0x56557e687b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(14 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f000 con 0x56557e682900 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(15 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578000640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 0x56557e688100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e000 con 0x56557e683b00 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 0x56557e688100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.394+0000 7f657da00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 0x56557e688100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(16 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f800 con 0x56557e683b00 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(17 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66fc00 con 0x56557e683200 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(18 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f400 con 0x56557e683680 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(19 [pgnls start_epoch 115 out=81b] v114'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e400 con 0x56557e683680 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f65803208c0 10 datascan.operator(): handling object 1099511627778.0 2024-05-14T11:26:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f000 con 0x56557e683680 2024-05-14T11:26:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(21 10000000002.00000000 [call] v115'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.398+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e000 con 0x56557e683200 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(22 [pgnls start_epoch 115 out=49b] v113'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310800 con 0x56557e683200 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(23 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578000640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x56557e683f80 0x56557e688680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2000 con 0x56557e683f80 2024-05-14T11:26:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x56557e683f80 0x56557e688680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x56557e683f80 0x56557e688680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(24 [pgnls start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3400 con 0x56557e683200 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 5 ==== osd_op_reply(25 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2400 con 0x56557e29ed00 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(26 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3800 con 0x56557e683b00 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(27 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2800 con 0x56557e683680 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 5 ==== osd_op_reply(28 [pgnls start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3c00 con 0x56557e683200 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 6 ==== osd_op_reply(29 [pgnls start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2c00 con 0x56557e682900 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(30 [pgnls start_epoch 115 out=81b] v113'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.402+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56557e310c00 con 0x56557e682900 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f65803208c0 10 datascan.operator(): handling object 1099511628279.0 2024-05-14T11:26:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56557e310400 con 0x56557e682900 2024-05-14T11:26:18.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v115'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310800 con 0x56557e29ed00 2024-05-14T11:26:18.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(33 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3000 con 0x56557e683200 2024-05-14T11:26:18.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 7 ==== osd_op_reply(34 [pgnls start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2400 con 0x56557e682000 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(35 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2000 con 0x56557e683200 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 8 ==== osd_op_reply(36 [pgnls start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3800 con 0x56557e683200 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 9 ==== osd_op_reply(37 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.406+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3400 con 0x56557e682000 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(38 [pgnls start_epoch 115 out=49b] v113'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2c00 con 0x56557e682900 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(39 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3c00 con 0x56557e683f80 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(40 [pgnls start_epoch 115 out=81b] v113'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56557e66fc00 con 0x56557e683f80 2024-05-14T11:26:18.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f65803208c0 10 datascan.operator(): handling object 1099511627780.1 2024-05-14T11:26:18.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e000 con 0x56557e682900 2024-05-14T11:26:18.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(42 10000000004.00000000 [call] v115'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0000 con 0x56557e683200 2024-05-14T11:26:18.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 10 ==== osd_op_reply(43 [pgnls start_epoch 115 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66fc00 con 0x56557e683b00 2024-05-14T11:26:18.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(44 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.410+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310c00 con 0x56557e683f80 2024-05-14T11:26:18.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(45 [pgnls start_epoch 115 out=81b] v114'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0400 con 0x56557e683f80 2024-05-14T11:26:18.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 10 datascan.operator(): handling object 1099511628280.0 2024-05-14T11:26:18.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0800 con 0x56557e683f80 2024-05-14T11:26:18.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v115'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0c00 con 0x56557e29ed00 2024-05-14T11:26:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 9 ==== osd_op_reply(48 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310400 con 0x56557e682900 2024-05-14T11:26:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 8 ==== osd_op_reply(49 [pgnls start_epoch 115 out=81b] v115'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1000 con 0x56557e682900 2024-05-14T11:26:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 10 datascan.operator(): handling object 1099511627780.0 2024-05-14T11:26:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.414+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0c00 con 0x56557e682900 2024-05-14T11:26:18.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 10 ==== osd_op_reply(51 10000000004.00000000 [call] v115'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310800 con 0x56557e682480 2024-05-14T11:26:18.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(52 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e682480 2024-05-14T11:26:18.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f800 con 0x56557e682000 2024-05-14T11:26:18.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(53 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3000 con 0x56557e683f80 2024-05-14T11:26:18.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 7 ==== osd_op_reply(54 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1000 con 0x56557e683f80 2024-05-14T11:26:18.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 8 ==== osd_op_reply(55 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2400 con 0x56557e682480 2024-05-14T11:26:18.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(56 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e682480 2024-05-14T11:26:18.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0000 con 0x56557e29ed00 2024-05-14T11:26:18.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.0 v2:172.21.15.116:6808/772722056 10 ==== osd_op_reply(57 [pgnls start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e29ed00 2024-05-14T11:26:18.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2000 con 0x56557e683200 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.3 v2:172.21.15.116:6816/1815220698 11 ==== osd_op_reply(58 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683200 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.418+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e310800 con 0x56557e682900 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 11 ==== osd_op_reply(59 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3800 con 0x56557e683680 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 6 ==== osd_op_reply(60 [pgnls start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a2800 con 0x56557e683b00 2024-05-14T11:26:18.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(61 [pgnls start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3400 con 0x56557e682900 2024-05-14T11:26:18.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 12 ==== osd_op_reply(62 [pgnls start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f400 con 0x56557e682000 2024-05-14T11:26:18.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(63 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6a3c00 con 0x56557e683b00 2024-05-14T11:26:18.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(64 [pgnls start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66e400 con 0x56557e682900 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 13 ==== osd_op_reply(65 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1400 con 0x56557e683f80 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 9 ==== osd_op_reply(66 [pgnls start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e66f000 con 0x56557e683680 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 <== osd.2 v2:172.21.15.116:6817/2784504541 7 ==== osd_op_reply(67 [pgnls start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2780 con 0x56557e683680 2024-05-14T11:26:18.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1800 con 0x56557e682000 2024-05-14T11:26:18.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(68 [pgnls start_epoch 115 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.422+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0400 con 0x56557e682000 2024-05-14T11:26:18.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 9 ==== osd_op_reply(69 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1c00 con 0x56557e683b00 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(70 [pgnls start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0800 con 0x56557e682000 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.5 v2:172.21.15.149:6824/4173680411 10 ==== osd_op_reply(71 [pgnls start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e682000 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0c00 con 0x56557e682900 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.1 v2:172.21.15.116:6800/4228261718 14 ==== osd_op_reply(72 [pgnls start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e682900 2024-05-14T11:26:18.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f6578a00640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b1000 con 0x56557e683b00 2024-05-14T11:26:18.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657da00640 1 -- 172.21.15.116:0/194877954 <== osd.7 v2:172.21.15.149:6808/2875938804 8 ==== osd_op_reply(73 [pgnls start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2000 con 0x56557e683b00 2024-05-14T11:26:18.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f6578000640 1 -- 172.21.15.116:0/194877954 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x56557e6b0000 con 0x56557e683f80 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657ee00640 1 -- 172.21.15.116:0/194877954 <== osd.4 v2:172.21.15.149:6809/3313182122 10 ==== osd_op_reply(74 [pgnls start_epoch 115 out=49b] v113'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56557e3a2500 con 0x56557e683f80 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x56557e29ed00 msgr2=0x56557f008680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x56557e29ed00 0x56557f008680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56557e682900 msgr2=0x56557e686b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x56557e682900 0x56557e686b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x56557e683680 msgr2=0x56557e687b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x56557e683680 0x56557e687b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x56557e683200 msgr2=0x56557e687600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x56557e683200 0x56557e687600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 reap_dead start 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x56557e683f80 msgr2=0x56557e688680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x56557e683f80 0x56557e688680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x56557e682000 msgr2=0x56557e686000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x56557e682000 0x56557e686000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f657e400640 1 -- 172.21.15.116:0/194877954 reap_dead start 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 msgr2=0x56557e686580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 0x56557e686580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 msgr2=0x56557e688100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 0x56557e688100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 msgr2=0x56557f008100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 0x56557f008100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x56557e347080 tx=0x56557e670000 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 msgr2=0x56557f007600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x56557f163e60 tx=0x56557f166c00 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 shutdown_connections 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x56557e682480 0x56557e686580 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x56557e683b00 0x56557e688100 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x56557e29fa80 0x56557f008100 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/194877954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29c900 0x56557f007600 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 >> 172.21.15.116:0/194877954 conn(0x56557e29d200 msgr2=0x56557e546000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 shutdown_connections 2024-05-14T11:26:18.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/194877954 wait complete. 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 msgr2=0x56557f006b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x56557f097350 tx=0x56557e345ec0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 shutdown_connections 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56557e29df80 0x56557f006b00 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x56557e29db00 0x56557f006000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 --2- 172.21.15.116:0/4144960172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56557e29d680 0x56557f006580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 >> 172.21.15.116:0/4144960172 conn(0x56557e29e400 msgr2=0x56557a229800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 shutdown_connections 2024-05-14T11:26:18.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.426+0000 7f65803208c0 1 -- 172.21.15.116:0/4144960172 wait complete. 2024-05-14T11:26:18.432 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.123510, result: 2024-05-14T11:26:18.432 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-14T11:26:18.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.530+0000 7fd7534498c0 1 -- start start 2024-05-14T11:26:18.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.530+0000 7fd7534498c0 1 Processor -- start 2024-05-14T11:26:18.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 0x5641d3bb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4db00 0x5641d3bb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 0x5641d3bb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5641d3bda180 con 0x5641d2e4df80 2024-05-14T11:26:18.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d2e3a900 con 0x5641d2e4db00 2024-05-14T11:26:18.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5641d2e3a480 con 0x5641d2e4d680 2024-05-14T11:26:18.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 0x5641d3bb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd750a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 0x5641d3bb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd750000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4db00 0x5641d3bb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd750a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 0x5641d3bb6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:40686/0 (socket says 172.21.15.116:40686) 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd750a00640 1 -- 172.21.15.116:0/2304786854 learned_addr learned my addr 172.21.15.116:0/2304786854 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 -- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 msgr2=0x5641d3bb6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 0x5641d3bb6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 -- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4db00 msgr2=0x5641d3bb6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4db00 0x5641d3bb6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 -- 172.21.15.116:0/2304786854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641cedd01e0 con 0x5641d2e4df80 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd751400640 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 0x5641d3bb6000 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x5641d3c47b60 tx=0x5641d2ef4480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebf1e446fbc90f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd74d800640 1 -- 172.21.15.116:0/2304786854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e104e0 con 0x5641d2e4df80 2024-05-14T11:26:18.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd74d800640 1 -- 172.21.15.116:0/2304786854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5641d2e241c0 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd74d800640 1 -- 172.21.15.116:0/2304786854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e10b60 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641cedd12c0 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5641cedd1a40 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 4 waiting for MDS map... 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd74d800640 1 -- 172.21.15.116:0/2304786854 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x5641d3d08600 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd74d800640 1 -- 172.21.15.116:0/2304786854 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x5641d2e31180 con 0x5641d2e4df80 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 4 Got MDS map 107 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 4 datascan.main: Using metadata pool output 2024-05-14T11:26:18.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.534+0000 7fd7534498c0 1 Processor -- start 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- start start 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5641d2e3a180 con 0x5641d2e4cd80 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d3bdad80 con 0x5641d2e4e880 2024-05-14T11:26:18.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5641d3bdaf00 con 0x5641d2e4c900 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56350/0 (socket says 172.21.15.116:56350) 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750000640 1 -- 172.21.15.116:0/3654570989 learned_addr learned my addr 172.21.15.116:0/3654570989 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750000640 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 -- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4e880 msgr2=0x5641d3bb7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 -- 172.21.15.116:0/3654570989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 msgr2=0x5641d3bb7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 -- 172.21.15.116:0/3654570989 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641cedd1c20 con 0x5641d2e4c900 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd750a00640 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 0x5641d3bb7600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1d0e0 tx=0x5641d2ef4840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a292ac9a0cc18e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd74b000640 1 -- 172.21.15.116:0/3654570989 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e109c0 con 0x5641d2e4c900 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd74b000640 1 -- 172.21.15.116:0/3654570989 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5641d2e24e00 con 0x5641d2e4c900 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd74b000640 1 -- 172.21.15.116:0/3654570989 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e104e0 con 0x5641d2e4c900 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 msgr2=0x5641d3bb7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 0x5641d3bb7600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1d0e0 tx=0x5641d2ef4840 comp rx=0 tx=0).stop 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- 172.21.15.116:0/3654570989 shutdown_connections 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 --2- 172.21.15.116:0/3654570989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- 172.21.15.116:0/3654570989 >> 172.21.15.116:0/3654570989 conn(0x5641d2e4d200 msgr2=0x5641d3108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- 172.21.15.116:0/3654570989 shutdown_connections 2024-05-14T11:26:18.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.538+0000 7fd7534498c0 1 -- 172.21.15.116:0/3654570989 wait complete. 2024-05-14T11:26:18.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 Processor -- start 2024-05-14T11:26:18.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- start start 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5641d3bdb200 con 0x5641d2e4e880 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d3bdb080 con 0x5641d2e4c900 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x5641d3bdb380 con 0x5641d2e4cd80 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:34388/0 (socket says 172.21.15.116:34388) 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 learned_addr learned my addr 172.21.15.116:0/2778599894 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4cd80 msgr2=0x5641d3bb7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4cd80 0x5641d3bb7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4e880 msgr2=0x5641d3bb7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4e880 0x5641d3bb7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641cedd14a0 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1dec0 tx=0x5641d2ef4960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd749200640 1 -- 172.21.15.116:0/2778599894 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e109c0 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd749200640 1 -- 172.21.15.116:0/2778599894 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5641d2e24540 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd749200640 1 -- 172.21.15.116:0/2778599894 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641d2e104e0 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5641cedd03c0 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641cedd1c20 con 0x5641d2e4c900 2024-05-14T11:26:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.542+0000 7fd7534498c0 4 datascan.init: resolving metadata pool 17 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd749200640 1 -- 172.21.15.116:0/2778599894 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x5641d3bc0c00 con 0x5641d2e4c900 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd749200640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 0x5641d3bb8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd749200640 1 -- 172.21.15.116:0/2778599894 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x5641d3130600 con 0x5641d2e4c900 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5641d2e4ed00 0x5641d3bb8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d2e4ed00 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 0x5641d3bb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5641d2e4ed00 0x5641d3bb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 0x5641d3bb8100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1d0e0 tx=0x5641d2ef4b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5641d2e4ed00 0x5641d3bb8680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5641d2e4f180 0x5641d3bb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d2e4f180 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.546+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5641d2e4f180 0x5641d3bb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5641d2e4f180 0x5641d3bb8c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5641d2e4f600 0x5641d3bb9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d2e4f600 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5641d2e4f600 0x5641d3bb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5641d2e4f600 0x5641d3bb9180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd74a600640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5641d3234000 0x5641d323c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133c00 con 0x5641d3234000 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5641d3234000 0x5641d323c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5641d3234000 0x5641d323c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3246000 con 0x5641d2e4f180 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.550+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3246400 con 0x5641d2e4f600 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3246c00 con 0x5641d2e4f600 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3246800 con 0x5641d3234000 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5641d3234480 0x5641d323c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3247000 con 0x5641d3234480 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5641d3234480 0x5641d323c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5641d3234480 0x5641d323c580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d2e4ed00 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d2e4f600 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05400 con 0x5641d2e4f600 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133c00 con 0x5641d2e4f180 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05000 con 0x5641d2e4f600 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.554+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d3234000 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74a600640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5641d3234900 0x5641d323cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d3234900 2024-05-14T11:26:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5641d3234900 0x5641d323cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750a00640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5641d3234900 0x5641d323cb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e000 con 0x5641d3234480 2024-05-14T11:26:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74a600640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5641d3234d80 0x5641d323d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d3234d80 2024-05-14T11:26:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5641d3234d80 0x5641d323d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5641d3234d80 0x5641d323d080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e400 con 0x5641d3234d80 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641cedabc00 con 0x5641d2e4ed00 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.558+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 115 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e800 con 0x5641d3234900 2024-05-14T11:26:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 115 out=81b] v115'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133400 con 0x5641d2e4f600 2024-05-14T11:26:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324ec00 con 0x5641d3234000 2024-05-14T11:26:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e400 con 0x5641d2e4f180 2024-05-14T11:26:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f000 con 0x5641d2e4f600 2024-05-14T11:26:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324ec00 con 0x5641d2e4f600 2024-05-14T11:26:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f400 con 0x5641d3234000 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641cedabc00 con 0x5641d3234480 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f800 con 0x5641d2e4ed00 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d2e4f600 2024-05-14T11:26:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.562+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324fc00 con 0x5641d2e4f600 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d2e4f180 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e800 con 0x5641d2e4f600 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256000 con 0x5641d3234000 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f000 con 0x5641d3234900 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256400 con 0x5641d3234480 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d3234d80 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256800 con 0x5641d3234d80 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133400 con 0x5641d2e4ed00 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256c00 con 0x5641d3234900 2024-05-14T11:26:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 115 out=49b] v115'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.566+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05400 con 0x5641d2e4ed00 2024-05-14T11:26:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 115 out=49b] v113'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257000 con 0x5641d2e4ed00 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74b000640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5641d3235200 0x5641d323d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05000 con 0x5641d3235200 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5641d3235200 0x5641d323d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd751400640 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5641d3235200 0x5641d323d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257400 con 0x5641d2e4ed00 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d2e4f600 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d3234d80 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133400 con 0x5641d3234900 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d2e4ed00 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641cedabc00 con 0x5641d3234480 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.570+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 115 out=81b] v115'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 10 datascan.operator(): handling object 100000001f7.0 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256c00 con 0x5641d3234480 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256000 con 0x5641d2e4f600 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256800 con 0x5641d2e4ed00 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv16 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.get_frag_of: frag is * 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d2e4f600 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256400 con 0x5641d2e4f600 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.read_dentry: bravo_head not found in result 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-14T11:26:18.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257800 con 0x5641d2e4f600 2024-05-14T11:26:18.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v115'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-14T11:26:18.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257c00 con 0x5641d2e4f600 2024-05-14T11:26:18.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.574+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f400 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133c00 con 0x5641d3234000 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f800 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324ec00 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e000 con 0x5641d3234000 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 115 out=49b] v113'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324fc00 con 0x5641d3234480 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e400 con 0x5641d3235200 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 115 out=49b] v113'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e800 con 0x5641d2e4ed00 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 115 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f400 con 0x5641d3234d80 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f000 con 0x5641d3235200 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 115 out=49b] v115'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.578+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257000 con 0x5641d2e4f600 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324fc00 con 0x5641d3234480 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 115 out=49b] v115'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257400 con 0x5641d2e4f180 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e400 con 0x5641d3234000 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256800 con 0x5641d3235200 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f000 con 0x5641d3235200 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132c00 con 0x5641d2e4f180 2024-05-14T11:26:18.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.6 v2:172.21.15.149:6800/3938301688 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d2e4f180 2024-05-14T11:26:18.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133800 con 0x5641d2e4f600 2024-05-14T11:26:18.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.0 v2:172.21.15.116:6808/772722056 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d2e4f600 2024-05-14T11:26:18.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d2e4ed00 2024-05-14T11:26:18.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.3 v2:172.21.15.116:6816/1815220698 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d2e4ed00 2024-05-14T11:26:18.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256c00 con 0x5641d3234480 2024-05-14T11:26:18.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05400 con 0x5641d3234900 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.582+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 115 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641cedabc00 con 0x5641d3234d80 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 115 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256000 con 0x5641d3234480 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 115 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3d05000 con 0x5641d3234000 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3256400 con 0x5641d3234d80 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133c00 con 0x5641d3234480 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3257800 con 0x5641d3235200 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 115 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133000 con 0x5641d3234900 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 <== osd.2 v2:172.21.15.116:6817/2784504541 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 115 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52780 con 0x5641d3234900 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324f800 con 0x5641d3234000 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 115 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133400 con 0x5641d3234000 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324ec00 con 0x5641d3234d80 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 115 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.586+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3132800 con 0x5641d3234000 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.5 v2:172.21.15.149:6824/4173680411 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 115 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234000 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e000 con 0x5641d3234480 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.1 v2:172.21.15.116:6800/4228261718 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 115 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3234480 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd74a600640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d3133800 con 0x5641d3234d80 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd750000640 1 -- 172.21.15.116:0/2778599894 <== osd.7 v2:172.21.15.149:6808/2875938804 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 115 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52000 con 0x5641d3234d80 2024-05-14T11:26:18.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd74b000640 1 -- 172.21.15.116:0/2778599894 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 115 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x5641d324e800 con 0x5641d3235200 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd751400640 1 -- 172.21.15.116:0/2778599894 <== osd.4 v2:172.21.15.149:6809/3313182122 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 115 out=49b] v113'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5641d2f52500 con 0x5641d3235200 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5641d2e4f600 msgr2=0x5641d3bb9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x5641d2e4f600 0x5641d3bb9180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5641d3234480 msgr2=0x5641d323c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x5641d3234480 0x5641d323c580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5641d3234900 msgr2=0x5641d323cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x5641d3234900 0x5641d323cb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5641d2e4ed00 msgr2=0x5641d3bb8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x5641d2e4ed00 0x5641d3bb8680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5641d3235200 msgr2=0x5641d323d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x5641d3235200 0x5641d323d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5641d3234000 msgr2=0x5641d323c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x5641d3234000 0x5641d323c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5641d2e4f180 msgr2=0x5641d3bb8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x5641d2e4f180 0x5641d3bb8c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5641d3234d80 msgr2=0x5641d323d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x5641d3234d80 0x5641d323d080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 reap_dead start 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd750a00640 1 -- 172.21.15.116:0/2778599894 reap_dead start 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 msgr2=0x5641d3bb8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 0x5641d3bb8100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1d0e0 tx=0x5641d2ef4b40 comp rx=0 tx=0).stop 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 msgr2=0x5641d3bb7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5641d3d1dec0 tx=0x5641d2ef4960 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 shutdown_connections 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x5641d2e4fa80 0x5641d3bb8100 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2778599894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4c900 0x5641d3bb7600 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 >> 172.21.15.116:0/2778599894 conn(0x5641d2e4d200 msgr2=0x5641d3108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 shutdown_connections 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2778599894 wait complete. 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 msgr2=0x5641d3bb6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 0x5641d3bb6000 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x5641d3c47b60 tx=0x5641d2ef4480 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 shutdown_connections 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d2e4db00 0x5641d3bb6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5641d2e4df80 0x5641d3bb6000 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 --2- 172.21.15.116:0/2304786854 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x5641d2e4d680 0x5641d3bb6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 >> 172.21.15.116:0/2304786854 conn(0x5641d2e4e400 msgr2=0x5641cedd9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 shutdown_connections 2024-05-14T11:26:18.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.590+0000 7fd7534498c0 1 -- 172.21.15.116:0/2304786854 wait complete. 2024-05-14T11:26:18.595 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.162403, result: 2024-05-14T11:26:18.595 DEBUG:teuthology.orchestra.run.smithi116:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-14T11:26:18.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- start start 2024-05-14T11:26:18.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 Processor -- start 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 0x5572202f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 0x5572202f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55721f5fe180 con 0x55721f58f680 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55721f5fe300 con 0x55721f58ff80 2024-05-14T11:26:18.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55721f5fe480 con 0x55721f58fb00 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56380/0 (socket says 172.21.15.116:56380) 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 -- 172.21.15.116:0/1877113115 learned_addr learned my addr 172.21.15.116:0/1877113115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3400640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 0x5572202f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a4800640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 0x5572202f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 -- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 msgr2=0x5572202f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 0x5572202f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 -- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 msgr2=0x5572202f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 0x5572202f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 -- 172.21.15.116:0/1877113115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721b5121e0 con 0x55721f58f680 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3400640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 0x5572202f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a4800640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 0x5572202f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x557220389380 tx=0x55721f637ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11653709d2a7973 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a0c00640 1 -- 172.21.15.116:0/1877113115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f5529c0 con 0x55721f58f680 2024-05-14T11:26:18.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a0c00640 1 -- 172.21.15.116:0/1877113115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55721f566e00 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a0c00640 1 -- 172.21.15.116:0/1877113115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f5524e0 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721b5134a0 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55721b5123c0 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 4 waiting for MDS map... 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a0c00640 1 -- 172.21.15.116:0/1877113115 <== mon.0 v2:172.21.15.116:3300/0 4 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x557220446200 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a0c00640 1 -- 172.21.15.116:0/1877113115 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x55721f573180 con 0x55721f58f680 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 4 Got MDS map 107 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 4 datascan.main: Using metadata pool output 2024-05-14T11:26:18.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.654+0000 7fb4a5c3d8c0 4 datascan.main: connecting to RADOS... 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 Processor -- start 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 -- start start 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55721f5fe600 con 0x55721f58e900 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55721f5fe780 con 0x55721f590880 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55721f5fe900 con 0x55721f58ed80 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56390/0 (socket says 172.21.15.116:56390) 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 -- 172.21.15.116:0/3317619172 learned_addr learned my addr 172.21.15.116:0/3317619172 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3400640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a4800640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 -- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 msgr2=0x5572202f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 -- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 msgr2=0x5572202f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 -- 172.21.15.116:0/3317619172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721b5121e0 con 0x55721f58e900 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a4800640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.658+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x557220302930 tx=0x55721f637f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff4a0c54afeef64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb49e400640 1 -- 172.21.15.116:0/3317619172 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f552b60 con 0x55721f58e900 2024-05-14T11:26:18.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb49e400640 1 -- 172.21.15.116:0/3317619172 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55721f566c40 con 0x55721f58e900 2024-05-14T11:26:18.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb49e400640 1 -- 172.21.15.116:0/3317619172 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f552340 con 0x55721f58e900 2024-05-14T11:26:18.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/3317619172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 msgr2=0x5572202f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x557220302930 tx=0x55721f637f20 comp rx=0 tx=0).stop 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/3317619172 shutdown_connections 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/3317619172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/3317619172 >> 172.21.15.116:0/3317619172 conn(0x55721f58f200 msgr2=0x55721f830000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/3317619172 shutdown_connections 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/3317619172 wait complete. 2024-05-14T11:26:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.662+0000 7fb4a5c3d8c0 1 Processor -- start 2024-05-14T11:26:18.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- start start 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55721f5ff500 con 0x55721f58ed80 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55721f5fea80 con 0x55721f590880 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55721f5ff680 con 0x55721f58e900 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56398/0 (socket says 172.21.15.116:56398) 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 learned_addr learned my addr 172.21.15.116:0/4012076522 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58e900 msgr2=0x5572202f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58e900 0x5572202f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 msgr2=0x5572202f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f590880 0x5572202f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721b5132c0 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x5572203031a0 tx=0x55721f637e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb49c600640 1 -- 172.21.15.116:0/4012076522 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f552b60 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb49c600640 1 -- 172.21.15.116:0/4012076522 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55721f566a80 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb49c600640 1 -- 172.21.15.116:0/4012076522 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55721f552340 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55721b513a40 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721b513c20 con 0x55721f58ed80 2024-05-14T11:26:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.666+0000 7fb4a5c3d8c0 4 datascan.init: resolving metadata pool 17 2024-05-14T11:26:18.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb49c600640 1 -- 172.21.15.116:0/4012076522 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55721f842c00 con 0x55721f58ed80 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb49c600640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 0x5572202fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb49c600640 1 -- 172.21.15.116:0/4012076522 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x55721f83ac00 con 0x55721f58ed80 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a5c3d8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a5c3d8c0 4 datascan.main: resolving metadata pool 17 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 0x5572202fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55721f590d00 0x5572202fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55721f590d00 0x5572202fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 0x5572202fa100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55721f639080 tx=0x557220319f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55721f590d00 0x5572202fa680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== osd_op_reply(1 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55721f964000 0x55721f968000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55721f964000 0x55721f968000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.670+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55721f964000 0x55721f968000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== osd_op_reply(2 [pgnls start_epoch 115 out=177b] v114'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== osd_op_reply(3 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55721f964480 0x55721f968580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55721f964480 0x55721f968580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55721f964480 0x55721f968580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== osd_op_reply(4 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== osd_op_reply(5 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== osd_op_reply(6 [pgnls start_epoch 115 out=149b] v115'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 3 ==== osd_op_reply(7 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55721f964900 0x55721f968b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964900 2024-05-14T11:26:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.674+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55721f964900 0x55721f968b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55721f964900 0x55721f968b00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== osd_op_reply(8 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55721f964d80 0x55721f969080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964d80 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55721f964d80 0x55721f969080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55721f964d80 0x55721f969080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== osd_op_reply(9 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 3 ==== osd_op_reply(10 [pgnls start_epoch 115 out=81b] v114'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 4 ==== osd_op_reply(11 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 3 ==== osd_op_reply(12 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55721f965b00 0x55721f96a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965b00 2024-05-14T11:26:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55721f965b00 0x55721f96a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.678+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55721f965b00 0x55721f96a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== osd_op_reply(13 [pgnls start_epoch 115 out=77b] v114'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3e00640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55721f965f80 0x55721f96a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965f80 2024-05-14T11:26:18.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55721f965f80 0x55721f96a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55721f965f80 0x55721f96a680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== osd_op_reply(14 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55721f965200 0x55721f969600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:18.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965200 2024-05-14T11:26:18.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55721f965200 0x55721f969600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:18.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55721f965200 0x55721f969600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:18.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== osd_op_reply(15 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694500 con 0x55721f965200 2024-05-14T11:26:18.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964900 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== osd_op_reply(16 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 4 ==== osd_op_reply(17 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 4 ==== osd_op_reply(18 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.682+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 5 ==== osd_op_reply(19 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965f80 2024-05-14T11:26:18.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== osd_op_reply(20 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965b00 2024-05-14T11:26:18.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== osd_op_reply(21 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965f80 2024-05-14T11:26:18.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 3 ==== osd_op_reply(22 [pgnls start_epoch 115 out=97b] v114'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 5 ==== osd_op_reply(23 [pgnls start_epoch 115 out=74b] v112'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f590d00 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 6 ==== osd_op_reply(24 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964d80 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== osd_op_reply(25 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964000 2024-05-14T11:26:18.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 6 ==== osd_op_reply(26 [pgnls start_epoch 115 out=98b] v114'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 5 ==== osd_op_reply(27 [pgnls start_epoch 115 out=76b] v115'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964d80 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 3 ==== osd_op_reply(28 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964480 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 6 ==== osd_op_reply(29 [pgnls start_epoch 115 out=105b] v115'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.686+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965b00 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 3 ==== osd_op_reply(30 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f964900 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 3 ==== osd_op_reply(31 [pgnls start_epoch 115 out=73b] v115'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x557220443c00 con 0x55721f965b00 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 4 ==== osd_op_reply(32 [pgnls start_epoch 115 out=101b] v115'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95e800 con 0x55721f964000 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-14T11:26:18.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95ec00 con 0x55721f964000 2024-05-14T11:26:18.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-14T11:26:18.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f000 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv16 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f400 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f800 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-14T11:26:18.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95fc00 con 0x55721f964000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95e800 con 0x55721f964900 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95ec00 con 0x55721f964000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f000 con 0x55721f965200 2024-05-14T11:26:18.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f965200 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.690+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f400 con 0x55721f965f80 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990000 con 0x55721f590d00 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-14T11:26:18.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990400 con 0x55721f964000 2024-05-14T11:26:18.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-14T11:26:18.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-14T11:26:18.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-14T11:26:18.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990800 con 0x55721f964480 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990c00 con 0x55721f964480 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 20 datascan.scan_links: injected ino detected 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-14T11:26:18.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f991000 con 0x55721f965b00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 8 ==== osd_op_reply(48 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.694+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 15 ==== osd_op_reply(49 [pgnls start_epoch 115 out=177b] v114'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 9 ==== osd_op_reply(50 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 9 ==== osd_op_reply(51 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 10 ==== osd_op_reply(52 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 16 ==== osd_op_reply(53 [pgnls start_epoch 115 out=149b] v115'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 10 ==== osd_op_reply(54 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964900 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 5 ==== osd_op_reply(55 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964d80 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 4 ==== osd_op_reply(56 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 17 ==== osd_op_reply(57 [pgnls start_epoch 115 out=81b] v114'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 11 ==== osd_op_reply(58 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 11 ==== osd_op_reply(59 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.698+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965b00 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 6 ==== osd_op_reply(60 [pgnls start_epoch 115 out=77b] v114'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965f80 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 5 ==== osd_op_reply(61 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965200 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.2 v2:172.21.15.116:6817/2784504541 3 ==== osd_op_reply(62 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694500 con 0x55721f965200 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964900 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 6 ==== osd_op_reply(63 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 12 ==== osd_op_reply(64 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 18 ==== osd_op_reply(65 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 19 ==== osd_op_reply(66 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965f80 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 6 ==== osd_op_reply(67 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965b00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 7 ==== osd_op_reply(68 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965f80 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 7 ==== osd_op_reply(69 [pgnls start_epoch 115 out=97b] v114'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 12 ==== osd_op_reply(70 [pgnls start_epoch 115 out=74b] v112'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.702+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f590d00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 13 ==== osd_op_reply(71 [pgnls start_epoch 115 out=73b] v114'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964d80 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 5 ==== osd_op_reply(72 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 20 ==== osd_op_reply(73 [pgnls start_epoch 115 out=98b] v114'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 13 ==== osd_op_reply(74 [pgnls start_epoch 115 out=76b] v115'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964d80 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.4 v2:172.21.15.149:6809/3313182122 6 ==== osd_op_reply(75 [pgnls start_epoch 115 out=49b] v115'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694500 con 0x55721f964d80 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964480 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 14 ==== osd_op_reply(76 [pgnls start_epoch 115 out=105b] v115'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965b00 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 8 ==== osd_op_reply(77 [pgnls start_epoch 115 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964900 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 7 ==== osd_op_reply(78 [pgnls start_epoch 115 out=73b] v115'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 115 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f965b00 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 9 ==== osd_op_reply(79 [pgnls start_epoch 115 out=101b] v115'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-14T11:26:18.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990400 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.706+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f800 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95fc00 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv16 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-14T11:26:18.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95ec00 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f000 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95e800 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-14T11:26:18.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f400 con 0x55721f964900 2024-05-14T11:26:18.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.1 v2:172.21.15.116:6800/4228261718 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f964900 2024-05-14T11:26:18.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-14T11:26:18.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990400 con 0x55721f964000 2024-05-14T11:26:18.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990000 con 0x55721f965200 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.2 v2:172.21.15.116:6817/2784504541 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694500 con 0x55721f965200 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f991800 con 0x55721f965f80 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.5 v2:172.21.15.149:6824/4173680411 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f965f80 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f800 con 0x55721f590d00 2024-05-14T11:26:18.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.710+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95fc00 con 0x55721f964000 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95ec00 con 0x55721f964480 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95f000 con 0x55721f964480 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.scan_links: injected ino detected 2024-05-14T11:26:18.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f95e800 con 0x55721f965b00 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.6 v2:172.21.15.149:6800/3938301688 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55721f694780 con 0x55721f965b00 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f990800 con 0x55721f964480 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-14T11:26:18.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.scan_links: first is now 2 2024-05-14T11:26:18.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55721f990c00 con 0x55721f964480 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a3e00640 1 -- 172.21.15.116:0/4012076522 <== osd.0 v2:172.21.15.116:6808/772722056 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v115'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55721f694780 con 0x55721f964480 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 10 datascan.scan_links: updating inotable 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.714+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f991000 con 0x55721f964000 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55721f991400 con 0x55721f964000 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a4800640 1 -- 172.21.15.116:0/4012076522 <== osd.3 v2:172.21.15.116:6816/1815220698 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v115'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55721f694500 con 0x55721f964000 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 10 datascan.scan_links: updating snaptable 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55721f991c00 con 0x55721f590d00 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 <== osd.7 v2:172.21.15.149:6808/2875938804 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55721f694000 con 0x55721f590d00 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55721f964480 msgr2=0x55721f968580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x55721f964480 0x55721f968580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55721f964900 msgr2=0x55721f968b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x55721f964900 0x55721f968b00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55721f965200 msgr2=0x55721f969600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x55721f965200 0x55721f969600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55721f964000 msgr2=0x55721f968000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x55721f964000 0x55721f968000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55721f964d80 msgr2=0x55721f969080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x55721f964d80 0x55721f969080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55721f965f80 msgr2=0x55721f96a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x55721f965f80 0x55721f96a680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55721f965b00 msgr2=0x55721f96a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x55721f965b00 0x55721f96a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55721f590d00 msgr2=0x5572202fa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x55721f590d00 0x5572202fa680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 reap_dead start 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 reap_dead start 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 reap_dead start 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a3400640 1 -- 172.21.15.116:0/4012076522 reap_dead start 2024-05-14T11:26:18.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 msgr2=0x5572202fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 0x5572202fa100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55721f639080 tx=0x557220319f80 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 msgr2=0x5572202f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x5572203031a0 tx=0x55721f637e60 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 shutdown_connections 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x55721f591a80 0x5572202fa100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/4012076522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58ed80 0x5572202f9080 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 >> 172.21.15.116:0/4012076522 conn(0x55721f58f200 msgr2=0x55721f830000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 shutdown_connections 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/4012076522 wait complete. 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 msgr2=0x5572202f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x557220389380 tx=0x55721f637ec0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 shutdown_connections 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55721f58ff80 0x5572202f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55721f58fb00 0x5572202f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 --2- 172.21.15.116:0/1877113115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55721f58f680 0x5572202f8580 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:18.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 >> 172.21.15.116:0/1877113115 conn(0x55721f590400 msgr2=0x55721b51b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:18.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 shutdown_connections 2024-05-14T11:26:18.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:18.718+0000 7fb4a5c3d8c0 1 -- 172.21.15.116:0/1877113115 wait complete. 2024-05-14T11:26:18.723 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.128345, result: 2024-05-14T11:26:18.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:18.727 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:18.766 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-05-14T11:26:18.766 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:18.767 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-05-14T11:26:18.769 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-05-14T11:26:18.770 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-05-14T11:26:18.771 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.772 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.773 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-05-14T11:26:18.774 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:26:18.775 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = False 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.776 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-14T11:26:18.777 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:18.778 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:18.779 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-14T11:26:18.780 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-14T11:26:18.781 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 20 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 20 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 10 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.782 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.783 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.784 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: mon op complaint time = 120 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: client mount timeout = 600 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-14T11:26:18.785 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout:[mds] 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds = 20 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds balancer = 20 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug frag = True 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug scatterstat = False 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: mds op complaint time = 180 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: mds verify scatter = True 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:18.786 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.787 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.789 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:18.832 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-14T11:26:18.871 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-05-14T11:26:18.871 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:18.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:18.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.874 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-05-14T11:26:18.876 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-05-14T11:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.878 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.879 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:18.880 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-05-14T11:26:18.881 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-05-14T11:26:18.882 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.883 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-14T11:26:18.884 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-14T11:26:18.885 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.116,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 8ea35704-38e5-41a3-b9f5-7d0f3a8c4277 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-14T11:26:18.886 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.887 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 20 2024-05-14T11:26:18.888 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 20 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 10 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-14T11:26:18.889 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-14T11:26:18.890 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-14T11:26:18.891 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: mon op complaint time = 120 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: client mount timeout = 600 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: debug client = 20 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.892 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout:[mds] 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds = 20 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: debug mds balancer = 20 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug frag = True 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: mds debug scatterstat = True 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: mds op complaint time = 180 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: mds verify scatter = True 2024-05-14T11:26:18.893 INFO:teuthology.orchestra.run.smithi149.stdout: osd op complaint time = 180 2024-05-14T11:26:18.894 INFO:teuthology.orchestra.run.smithi149.stdout: rados mon op timeout = 900 2024-05-14T11:26:18.894 INFO:teuthology.orchestra.run.smithi149.stdout: rados osd op timeout = 900 2024-05-14T11:26:18.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-14T11:26:19.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.086+0000 7f8d21200640 1 Processor -- start 2024-05-14T11:26:19.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d21200640 1 -- start start 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d21200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d21200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1c06f0a0 con 0x7f8d1c06c700 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d21200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1c0cbb60 con 0x7f8d1c0d1730 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d21200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8d1c06f420 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:19.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56406/0 (socket says 172.21.15.116:56406) 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d1a800640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8d1c06c700 0x7f8d1c06cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48328/0 (socket says 172.21.15.116:48328) 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 -- 172.21.15.116:0/1178363633 learned_addr learned my addr 172.21.15.116:0/1178363633 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 -- 172.21.15.116:0/1178363633 >> v1:172.21.15.149:6789/0 conn(0x7f8d1c0d1730 legacy=0x7f8d1c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 -- 172.21.15.116:0/1178363633 >> v1:172.21.15.116:6789/0 conn(0x7f8d1c06c700 legacy=0x7f8d1c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19e00640 1 -- 172.21.15.116:0/1178363633 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c06f700 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.090+0000 7f8d19400640 1 -- 172.21.15.116:0/1178363633 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087360172 0 0) 0x7f8d1c06f0a0 con 0x7f8d1c06c700 2024-05-14T11:26:19.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d19e00640 1 --2- 172.21.15.116:0/1178363633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009870 tx=0x7f8d0401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f769835002b5b50b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:19.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d19400640 1 -- 172.21.15.116:0/1178363633 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040a9620 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d19400640 1 -- 172.21.15.116:0/1178363633 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d040a97c0 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d19400640 1 -- 172.21.15.116:0/1178363633 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040a9b90 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 -- 172.21.15.116:0/1178363633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c0cd1c0 msgr2=0x7f8d1c0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 --2- 172.21.15.116:0/1178363633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009870 tx=0x7f8d0401c560 comp rx=0 tx=0).stop 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 -- 172.21.15.116:0/1178363633 shutdown_connections 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 --2- 172.21.15.116:0/1178363633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0cd5d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 -- 172.21.15.116:0/1178363633 >> 172.21.15.116:0/1178363633 conn(0x7f8d1c0b4720 msgr2=0x7f8d1c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 -- 172.21.15.116:0/1178363633 shutdown_connections 2024-05-14T11:26:19.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.094+0000 7f8d21200640 1 -- 172.21.15.116:0/1178363633 wait complete. 2024-05-14T11:26:19.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 Processor -- start 2024-05-14T11:26:19.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- start start 2024-05-14T11:26:19.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:19.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:19.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 0x7f8d1c072d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:19.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c16a9c0 con 0x7f8d1c06c700 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c15cf10 con 0x7f8d1c0d1730 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1c16ab40 con 0x7f8d1c0cd1c0 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56420/0 (socket says 172.21.15.116:56420) 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 -- 172.21.15.116:0/138865411 learned_addr learned my addr 172.21.15.116:0/138865411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d11400640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d1a800640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 0x7f8d1c072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 -- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8d1c0cd1c0 msgr2=0x7f8d1c0b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 -- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 msgr2=0x7f8d1c072d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 0x7f8d1c072d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 -- 172.21.15.116:0/138865411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c0caef0 con 0x7f8d1c06c700 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d1a800640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 0x7f8d1c072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:19.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d19e00640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401a190 tx=0x7f8d04019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:19.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004020 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d040041c0 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04003620 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c0ca370 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.098+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c0ca930 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c06a9c0 con 0x7f8d1c06c700 2024-05-14T11:26:19.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d040037c0 con 0x7f8d1c06c700 2024-05-14T11:26:19.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d12a00640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 0x7f8cf8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:19.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f8d04129550 con 0x7f8d1c06c700 2024-05-14T11:26:19.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d11400640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 0x7f8cf8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:19.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.102+0000 7f8d11400640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 0x7f8cf8071940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c1601c0 tx=0x7f8d0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:19.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.106+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8d040b7070 con 0x7f8d1c06c700 2024-05-14T11:26:19.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:19.310+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8d1c069de0 con 0x7f8d1c06c700 2024-05-14T11:26:20.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.190+0000 7f8d12a00640 1 -- 172.21.15.116:0/138865411 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v109) ==== 162+0+0 (secure 0 0 0) 0x7f8d040ad020 con 0x7f8d1c06c700 2024-05-14T11:26:20.190 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 msgr2=0x7f8cf8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 0x7f8cf8071940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c1601c0 tx=0x7f8d0c009040 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 msgr2=0x7f8d1c0b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8d0401a190 tx=0x7f8d04019300 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 shutdown_connections 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f8cf806f430 0x7f8cf8071940 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d1c0d1730 0x7f8d1c072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8d1c0cd1c0 0x7f8d1c0b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 --2- 172.21.15.116:0/138865411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c06c700 0x7f8d1c0b2610 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 >> 172.21.15.116:0/138865411 conn(0x7f8d1c0b4720 msgr2=0x7f8d1c15b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:20.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 shutdown_connections 2024-05-14T11:26:20.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.194+0000 7f8d21200640 1 -- 172.21.15.116:0/138865411 wait complete. 2024-05-14T11:26:20.221 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 Processor -- start 2024-05-14T11:26:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 -- start start 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f16d40d1670 0x7f16d40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16d40af380 0x7f16d40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d40d1fc0 con 0x7f16d4156fa0 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f16d40706f0 con 0x7f16d40af380 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16dbe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f16d41573b0 con 0x7f16d40d1670 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16d40af380 0x7f16d40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f16d4156fa0 0x7f16d406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48342/0 (socket says 172.21.15.116:48342) 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16d40af380 0x7f16d40af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56434/0 (socket says 172.21.15.116:56434) 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9400640 1 -- 172.21.15.116:0/1471342912 learned_addr learned my addr 172.21.15.116:0/1471342912 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:20.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d8a00640 1 --2- 172.21.15.116:0/1471342912 >> v2:172.21.15.149:3300/0 conn(0x7f16d40d1670 0x7f16d40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 -- 172.21.15.116:0/1471342912 >> v2:172.21.15.149:3300/0 conn(0x7f16d40d1670 msgr2=0x7f16d40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:20.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 --2- 172.21.15.116:0/1471342912 >> v2:172.21.15.149:3300/0 conn(0x7f16d40d1670 0x7f16d40d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 -- 172.21.15.116:0/1471342912 >> v1:172.21.15.116:6789/0 conn(0x7f16d4156fa0 legacy=0x7f16d406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d9e00640 1 -- 172.21.15.116:0/1471342912 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d4160510 con 0x7f16d40af380 2024-05-14T11:26:20.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.374+0000 7f16d3e00640 1 -- 172.21.15.116:0/1471342912 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2828631419 0 0) 0x7f16c8001120 con 0x7f16d4156fa0 2024-05-14T11:26:20.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16d9e00640 1 --2- 172.21.15.116:0/1471342912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d40af790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f16d4070c70 tx=0x7f16cc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b406377c38338557 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:20.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16d3e00640 1 -- 172.21.15.116:0/1471342912 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0b2070 con 0x7f16d40af380 2024-05-14T11:26:20.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16d3e00640 1 -- 172.21.15.116:0/1471342912 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc0a58e0 con 0x7f16d40af380 2024-05-14T11:26:20.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16d3e00640 1 -- 172.21.15.116:0/1471342912 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0a5cb0 con 0x7f16d40af380 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 -- 172.21.15.116:0/1471342912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 msgr2=0x7f16d40af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1471342912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d40af790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f16d4070c70 tx=0x7f16cc01c930 comp rx=0 tx=0).stop 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 -- 172.21.15.116:0/1471342912 shutdown_connections 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1471342912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d40af790 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1471342912 >> v2:172.21.15.149:3300/0 conn(0x7f16d40d1670 0x7f16d40d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 -- 172.21.15.116:0/1471342912 >> 172.21.15.116:0/1471342912 conn(0x7f16d40b4620 msgr2=0x7f16d40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:20.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 -- 172.21.15.116:0/1471342912 shutdown_connections 2024-05-14T11:26:20.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.378+0000 7f16dbe00640 1 -- 172.21.15.116:0/1471342912 wait complete. 2024-05-14T11:26:20.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 Processor -- start 2024-05-14T11:26:20.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- start start 2024-05-14T11:26:20.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 0x7f16d4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d4156fa0 0x7f16d41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f16d40b0660 con 0x7f16d40af380 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16d40b04e0 con 0x7f16d4156fa0 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f16d4169810 con 0x7f16d40d1670 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56448/0 (socket says 172.21.15.116:56448) 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 -- 172.21.15.116:0/1223910585 learned_addr learned my addr 172.21.15.116:0/1223910585 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9e00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d4156fa0 0x7f16d41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d8a00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 0x7f16d4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 -- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 msgr2=0x7f16d4155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 0x7f16d4155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 -- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d4156fa0 msgr2=0x7f16d41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d4156fa0 0x7f16d41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 -- 172.21.15.116:0/1223910585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d414c100 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d8a00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 0x7f16d4155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d9400640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c430 tx=0x7f16c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c80074e0 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c80047f0 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d4072ad0 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8007680 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.382+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d4073090 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.386+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d406a9c0 con 0x7f16d40af380 2024-05-14T11:26:20.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.386+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16c8011070 con 0x7f16d40af380 2024-05-14T11:26:20.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.386+0000 7f16d1600640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 0x7f16b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:20.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.386+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f16c808f0a0 con 0x7f16d40af380 2024-05-14T11:26:20.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.386+0000 7f16d8a00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 0x7f16b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:20.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.390+0000 7f16d8a00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 0x7f16b4071940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f16d40b1b90 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:20.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.390+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f16d40d1fc0 con 0x7f16d40af380 2024-05-14T11:26:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d4075dc0 con 0x7f16d40af380 2024-05-14T11:26:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16d1600640 1 -- 172.21.15.116:0/1223910585 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5193 (secure 0 0 0) 0x7f16c805bcb0 con 0x7f16d40af380 2024-05-14T11:26:20.584 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:20.584 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:20.197240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-14T11:26:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 110 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 msgr2=0x7f16b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 0x7f16b4071940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f16d40b1b90 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 msgr2=0x7f16d414b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f16c800c430 tx=0x7f16c800c8f0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 shutdown_connections 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f16b406f430 0x7f16b4071940 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d4156fa0 0x7f16d41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16d40d1670 0x7f16d4155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 --2- 172.21.15.116:0/1223910585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16d40af380 0x7f16d414b7a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 >> 172.21.15.116:0/1223910585 conn(0x7f16d40b4620 msgr2=0x7f16d406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 shutdown_connections 2024-05-14T11:26:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:20.586+0000 7f16dbe00640 1 -- 172.21.15.116:0/1223910585 wait complete. 2024-05-14T11:26:21.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:20.197240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8598': {'gid': 8598, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.116:6837/2487260442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 2487260442}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 2487260442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:26:21.008 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8598:up:replay 2024-05-14T11:26:22.008 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.166+0000 7ff14fe00640 1 Processor -- start 2024-05-14T11:26:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 -- start start 2024-05-14T11:26:22.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff148156fa0 0x7ff14806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 0x7ff1480d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1480d1fc0 con 0x7ff14806e9e0 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1481573b0 con 0x7ff148156fa0 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14fe00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7ff148161120 con 0x7ff1480d1670 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff148156fa0 0x7ff14806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff148156fa0 0x7ff14806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56458/0 (socket says 172.21.15.116:56458) 2024-05-14T11:26:22.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 -- 172.21.15.116:0/695475640 learned_addr learned my addr 172.21.15.116:0/695475640 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14ca00640 1 --2- 172.21.15.116:0/695475640 >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 0x7ff1480d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 -- 172.21.15.116:0/695475640 >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 msgr2=0x7ff1480d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 --2- 172.21.15.116:0/695475640 >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 0x7ff1480d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 -- 172.21.15.116:0/695475640 >> v1:172.21.15.149:6789/0 conn(0x7ff14806e9e0 legacy=0x7ff14806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14d400640 1 -- 172.21.15.116:0/695475640 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff148160510 con 0x7ff148156fa0 2024-05-14T11:26:22.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.170+0000 7ff14ca00640 1 --2- 172.21.15.116:0/695475640 >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 0x7ff1480d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:22.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14d400640 1 --2- 172.21.15.116:0/695475640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff14806ffe0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff148070c70 tx=0x7ff13c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d3bd269f001a1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:22.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff147e00640 1 -- 172.21.15.116:0/695475640 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0a5660 con 0x7ff148156fa0 2024-05-14T11:26:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff147e00640 1 -- 172.21.15.116:0/695475640 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13c0a5800 con 0x7ff148156fa0 2024-05-14T11:26:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff147e00640 1 -- 172.21.15.116:0/695475640 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0a5bd0 con 0x7ff148156fa0 2024-05-14T11:26:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 -- 172.21.15.116:0/695475640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 msgr2=0x7ff14806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 --2- 172.21.15.116:0/695475640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff14806ffe0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff148070c70 tx=0x7ff13c01c7e0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 -- 172.21.15.116:0/695475640 shutdown_connections 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 --2- 172.21.15.116:0/695475640 >> v2:172.21.15.149:3300/0 conn(0x7ff1480d1670 0x7ff1480d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 --2- 172.21.15.116:0/695475640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff14806ffe0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 -- 172.21.15.116:0/695475640 >> 172.21.15.116:0/695475640 conn(0x7ff1480b4620 msgr2=0x7ff1480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 -- 172.21.15.116:0/695475640 shutdown_connections 2024-05-14T11:26:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.174+0000 7ff14fe00640 1 -- 172.21.15.116:0/695475640 wait complete. 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 Processor -- start 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 -- start start 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff14806e9e0 0x7ff1480ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1480d1670 0x7ff1481510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff148070520 con 0x7ff148156fa0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1480b07a0 con 0x7ff1480d1670 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14fe00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1480706a0 con 0x7ff14806e9e0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56466/0 (socket says 172.21.15.116:56466) 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 -- 172.21.15.116:0/3468948869 learned_addr learned my addr 172.21.15.116:0/3468948869 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14ca00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1480d1670 0x7ff1481510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14d400640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff14806e9e0 0x7ff1480ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 -- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff14806e9e0 msgr2=0x7ff1480ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff14806e9e0 0x7ff1480ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 -- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1480d1670 msgr2=0x7ff1481510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1480d1670 0x7ff1481510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 -- 172.21.15.116:0/3468948869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1480cd6f0 con 0x7ff148156fa0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff14de00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b490 tx=0x7ff14000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.178+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140015020 con 0x7ff148156fa0 2024-05-14T11:26:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140011830 con 0x7ff148156fa0 2024-05-14T11:26:22.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1481f9da0 con 0x7ff148156fa0 2024-05-14T11:26:22.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140011c60 con 0x7ff148156fa0 2024-05-14T11:26:22.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1480cd960 con 0x7ff148156fa0 2024-05-14T11:26:22.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1480ce600 con 0x7ff148156fa0 2024-05-14T11:26:22.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff140018020 con 0x7ff148156fa0 2024-05-14T11:26:22.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff145600640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 0x7ff128071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:22.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7ff14008f110 con 0x7ff148156fa0 2024-05-14T11:26:22.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.182+0000 7ff14d400640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 0x7ff128071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:22.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.186+0000 7ff14d400640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 0x7ff128071940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0ae000 tx=0x7ff13c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:22.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.190+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff14001f030 con 0x7ff148156fa0 2024-05-14T11:26:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff14806a9c0 con 0x7ff148156fa0 2024-05-14T11:26:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff145600640 1 -- 172.21.15.116:0/3468948869 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5193 (secure 0 0 0) 0x7ff14005bd20 con 0x7ff148156fa0 2024-05-14T11:26:22.380 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:22.380 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:21.210210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-14T11:26:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 112 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 msgr2=0x7ff128071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 0x7ff128071940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0ae000 tx=0x7ff13c0a7070 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 msgr2=0x7ff1481fa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b490 tx=0x7ff14000b950 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 shutdown_connections 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff12806f430 0x7ff128071940 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1480d1670 0x7ff1481510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148156fa0 0x7ff1481fa530 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 --2- 172.21.15.116:0/3468948869 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff14806e9e0 0x7ff1480ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 >> 172.21.15.116:0/3468948869 conn(0x7ff1480b4620 msgr2=0x7ff14815b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:22.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 shutdown_connections 2024-05-14T11:26:22.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:22.382+0000 7ff14fe00640 1 -- 172.21.15.116:0/3468948869 wait complete. 2024-05-14T11:26:22.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:21.210210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8598': {'gid': 8598, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.116:6837/2487260442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 2487260442}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 2487260442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-14T11:26:22.804 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8598:up:rejoin 2024-05-14T11:26:23.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.965+0000 7f100b000640 1 Processor -- start 2024-05-14T11:26:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.965+0000 7f100b000640 1 -- start start 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f100b000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f10040d1670 0x7f10040d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f100b000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10040af380 0x7f10040af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f100b000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10040d1fc0 con 0x7f1004156fa0 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f100b000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f10040706f0 con 0x7f10040af380 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f100b000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f10041573b0 con 0x7f10040d1670 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1009000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10040af380 0x7f10040af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1003e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1004156fa0 0x7f100406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:48356/0 (socket says 172.21.15.116:48356) 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1003e00640 1 -- 172.21.15.116:0/296422491 learned_addr learned my addr 172.21.15.116:0/296422491 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1003400640 1 --2- 172.21.15.116:0/296422491 >> v2:172.21.15.149:3300/0 conn(0x7f10040d1670 0x7f10040d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1009000640 1 -- 172.21.15.116:0/296422491 >> v2:172.21.15.149:3300/0 conn(0x7f10040d1670 msgr2=0x7f10040d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1009000640 1 --2- 172.21.15.116:0/296422491 >> v2:172.21.15.149:3300/0 conn(0x7f10040d1670 0x7f10040d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1009000640 1 -- 172.21.15.116:0/296422491 >> v1:172.21.15.116:6789/0 conn(0x7f1004156fa0 legacy=0x7f100406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1009000640 1 -- 172.21.15.116:0/296422491 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1004160510 con 0x7f10040af380 2024-05-14T11:26:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.969+0000 7f1002a00640 1 -- 172.21.15.116:0/296422491 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280388432 0 0) 0x7f0ff40014a0 con 0x7f1004156fa0 2024-05-14T11:26:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f1009000640 1 --2- 172.21.15.116:0/296422491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f10040af790 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f1004070c70 tx=0x7f0ffc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeba05b632e91b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f1002a00640 1 -- 172.21.15.116:0/296422491 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0b0070 con 0x7f10040af380 2024-05-14T11:26:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f1002a00640 1 -- 172.21.15.116:0/296422491 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc0ab600 con 0x7f10040af380 2024-05-14T11:26:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f1002a00640 1 -- 172.21.15.116:0/296422491 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0ab9d0 con 0x7f10040af380 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- 172.21.15.116:0/296422491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 msgr2=0x7f10040af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 --2- 172.21.15.116:0/296422491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f10040af790 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f1004070c70 tx=0x7f0ffc01c480 comp rx=0 tx=0).stop 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- 172.21.15.116:0/296422491 shutdown_connections 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 --2- 172.21.15.116:0/296422491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f10040af790 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 --2- 172.21.15.116:0/296422491 >> v2:172.21.15.149:3300/0 conn(0x7f10040d1670 0x7f10040d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- 172.21.15.116:0/296422491 >> 172.21.15.116:0/296422491 conn(0x7f10040b4620 msgr2=0x7f10040b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- 172.21.15.116:0/296422491 shutdown_connections 2024-05-14T11:26:23.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- 172.21.15.116:0/296422491 wait complete. 2024-05-14T11:26:23.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 Processor -- start 2024-05-14T11:26:23.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.973+0000 7f100b000640 1 -- start start 2024-05-14T11:26:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10040d1670 0x7f1004156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1004156fa0 0x7f100414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f10040b0660 con 0x7f10040af380 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f10040b04e0 con 0x7f10040d1670 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1004169810 con 0x7f1004156fa0 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56492/0 (socket says 172.21.15.116:56492) 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 -- 172.21.15.116:0/2281280511 learned_addr learned my addr 172.21.15.116:0/2281280511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1009000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1004156fa0 0x7f100414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003400640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10040d1670 0x7f1004156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 -- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1004156fa0 msgr2=0x7f100414f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1004156fa0 0x7f100414f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 -- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10040d1670 msgr2=0x7f1004156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10040d1670 0x7f1004156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 -- 172.21.15.116:0/2281280511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1004150e70 con 0x7f10040af380 2024-05-14T11:26:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f1003e00640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400c770 tx=0x7f0ff400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4018070 con 0x7f10040af380 2024-05-14T11:26:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff400a640 con 0x7f10040af380 2024-05-14T11:26:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100414eac0 con 0x7f10040af380 2024-05-14T11:26:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400a9f0 con 0x7f10040af380 2024-05-14T11:26:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100414f080 con 0x7f10040af380 2024-05-14T11:26:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.977+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100406a9c0 con 0x7f10040af380 2024-05-14T11:26:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.981+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ff401b070 con 0x7f10040af380 2024-05-14T11:26:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.981+0000 7f0ffbe00640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 0x7f0fe4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.981+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f0ff408eb90 con 0x7f10040af380 2024-05-14T11:26:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.981+0000 7f1003400640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 0x7f0fe4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.981+0000 7f1003400640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 0x7f0fe4071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f100415a070 tx=0x7f0ff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:23.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:23.985+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f10040d1fc0 con 0x7f10040af380 2024-05-14T11:26:24.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10040b2830 con 0x7f10040af380 2024-05-14T11:26:24.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f0ffbe00640 1 -- 172.21.15.116:0/2281280511 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5200 (secure 0 0 0) 0x7f0ff405b8d0 con 0x7f10040af380 2024-05-14T11:26:24.176 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:24.176 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:23.215968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8598,"qdb_cluster":[8598]},"id":8}]} 2024-05-14T11:26:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 113 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 msgr2=0x7f0fe4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 0x7f0fe4071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f100415a070 tx=0x7f0ff0008040 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 msgr2=0x7f100415c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400c770 tx=0x7f0ff400cc30 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 shutdown_connections 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f0fe406f400 0x7f0fe4071910 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10040d1670 0x7f1004156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1004156fa0 0x7f100414f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 --2- 172.21.15.116:0/2281280511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10040af380 0x7f100415c5d0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 >> 172.21.15.116:0/2281280511 conn(0x7f10040b4620 msgr2=0x7f10040b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 shutdown_connections 2024-05-14T11:26:24.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.177+0000 7f100b000640 1 -- 172.21.15.116:0/2281280511 wait complete. 2024-05-14T11:26:24.187 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-05-14T11:26:24.189+0000 7fa9fc000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:26:24.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T11:25:41.831155+0000', 'modified': '2024-05-14T11:26:23.215968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8598}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8598': {'gid': 8598, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6837/2487260442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 2487260442}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 2487260442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8598, 'qdb_cluster': [8598]} 2024-05-14T11:26:24.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T11:26:24.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-14T11:26:24.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 Processor -- start 2024-05-14T11:26:24.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 -- start start 2024-05-14T11:26:24.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 0x7f43a40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f43a40d1670 0x7f43a40d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a406ff20 con 0x7f43a4161600 2024-05-14T11:26:24.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f43a406d710 con 0x7f43a40d1670 2024-05-14T11:26:24.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43ab200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f43a40ae100 con 0x7f43a415db30 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43a3400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f43a40d1670 0x7f43a40d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 0x7f43a40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43a9200640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f43a4161600 0x7f43a4161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:35992/0 (socket says 172.21.15.116:35992) 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.749+0000 7f43a9200640 1 -- 172.21.15.116:0/1718536876 learned_addr learned my addr 172.21.15.116:0/1718536876 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a2a00640 1 -- 172.21.15.116:0/1718536876 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1693068966 0 0) 0x7f43a406ff20 con 0x7f43a4161600 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a2a00640 1 -- 172.21.15.116:0/1718536876 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4388003530 con 0x7f43a4161600 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3400640 1 -- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 msgr2=0x7f43a40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3400640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 0x7f43a40aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3400640 1 -- 172.21.15.116:0/1718536876 >> v1:172.21.15.149:6789/0 conn(0x7f43a4161600 legacy=0x7f43a4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3400640 1 -- 172.21.15.116:0/1718536876 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a40d1fc0 con 0x7f43a40d1670 2024-05-14T11:26:24.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3e00640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 0x7f43a40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:24.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a3400640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a40d1670 0x7f43a40d1a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43900097c0 tx=0x7f439001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294d7db197b8d779 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a2a00640 1 -- 172.21.15.116:0/1718536876 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900b0070 con 0x7f43a40d1670 2024-05-14T11:26:24.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a2a00640 1 -- 172.21.15.116:0/1718536876 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43900ab600 con 0x7f43a40d1670 2024-05-14T11:26:24.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.753+0000 7f43a2a00640 1 -- 172.21.15.116:0/1718536876 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900ab9d0 con 0x7f43a40d1670 2024-05-14T11:26:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a40d1670 msgr2=0x7f43a40d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a40d1670 0x7f43a40d1a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43900097c0 tx=0x7f439001c410 comp rx=0 tx=0).stop 2024-05-14T11:26:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- 172.21.15.116:0/1718536876 shutdown_connections 2024-05-14T11:26:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a40d1670 0x7f43a40d1a80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 --2- 172.21.15.116:0/1718536876 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a415db30 0x7f43a40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- 172.21.15.116:0/1718536876 >> 172.21.15.116:0/1718536876 conn(0x7f43a40b4620 msgr2=0x7f43a40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- 172.21.15.116:0/1718536876 shutdown_connections 2024-05-14T11:26:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- 172.21.15.116:0/1718536876 wait complete. 2024-05-14T11:26:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 Processor -- start 2024-05-14T11:26:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.757+0000 7f43ab200640 1 -- start start 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 0x7f43a414c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 0x7f43a415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43a415df40 con 0x7f43a415db30 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40aefe0 con 0x7f43a4161600 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f43a416af80 con 0x7f43a40d1670 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56504/0 (socket says 172.21.15.116:56504) 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 -- 172.21.15.116:0/677641379 learned_addr learned my addr 172.21.15.116:0/677641379 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3e00640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 0x7f43a414c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a9200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 0x7f43a415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 -- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 msgr2=0x7f43a414c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 0x7f43a414c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 -- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 msgr2=0x7f43a415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 0x7f43a415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 -- 172.21.15.116:0/677641379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a414fc30 con 0x7f43a415db30 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3e00640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 0x7f43a414c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a9200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 0x7f43a415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43a3400640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f439001a040 tx=0x7f43900191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f439be00640 1 -- 172.21.15.116:0/677641379 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900b8020 con 0x7f43a415db30 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f439be00640 1 -- 172.21.15.116:0/677641379 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43900b4600 con 0x7f43a415db30 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f439be00640 1 -- 172.21.15.116:0/677641379 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900b0040 con 0x7f43a415db30 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a414bdc0 con 0x7f43a415db30 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.761+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a414cde0 con 0x7f43a415db30 2024-05-14T11:26:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f439be00640 1 -- 172.21.15.116:0/677641379 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43900bb020 con 0x7f43a415db30 2024-05-14T11:26:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f439be00640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 0x7f4384071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f439be00640 1 -- 172.21.15.116:0/677641379 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f4390128c10 con 0x7f43a415db30 2024-05-14T11:26:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43a3e00640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 0x7f4384071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 Processor -- start 2024-05-14T11:26:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 -- start start 2024-05-14T11:26:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43a3e00640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 0x7f4384071910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4394009b10 tx=0x7f4394008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe94e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fe9384620 con 0x556fe8fad7c0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556fe929f740 con 0x556fe937f6b0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.765+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556fe92ed090 con 0x556fe9395f80 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56520/0 (socket says 172.21.15.116:56520) 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 -- 172.21.15.116:0/639704803 learned_addr learned my addr 172.21.15.116:0/639704803 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a9200640 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3400640 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe94e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 -- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 msgr2=0x556fe94e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe94e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 -- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 msgr2=0x556fe94e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 -- 172.21.15.116:0/639704803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe94810a0 con 0x556fe8fad7c0 2024-05-14T11:26:24.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43a3e00640 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4394011de0 tx=0x7f4394014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc6ab34fb0d0f45 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f4398c00640 1 -- 172.21.15.116:0/639704803 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394011e10 con 0x556fe8fad7c0 2024-05-14T11:26:24.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f4398c00640 1 -- 172.21.15.116:0/639704803 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394026070 con 0x556fe8fad7c0 2024-05-14T11:26:24.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f4398c00640 1 -- 172.21.15.116:0/639704803 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394021640 con 0x556fe8fad7c0 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- 172.21.15.116:0/639704803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 msgr2=0x556fe94e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4394011de0 tx=0x7f4394014000 comp rx=0 tx=0).stop 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- 172.21.15.116:0/639704803 shutdown_connections 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe94e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 --2- 172.21.15.116:0/639704803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94e7320 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- 172.21.15.116:0/639704803 >> 172.21.15.116:0/639704803 conn(0x556fe946c340 msgr2=0x556fe949bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- 172.21.15.116:0/639704803 shutdown_connections 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- 172.21.15.116:0/639704803 wait complete. 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 -- start start 2024-05-14T11:26:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.769+0000 7f43b296e000 1 Processor -- start 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe94ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe94edec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fe92e9050 con 0x556fe9395f80 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556fe92a5ec0 con 0x556fe8fad7c0 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556fe928cc90 con 0x556fe937f6b0 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 10 client.0 _ll_register_callbacks cb 0x556fe9488e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56534/0 (socket says 172.21.15.116:56534) 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 -- 172.21.15.116:0/2581384325 learned_addr learned my addr 172.21.15.116:0/2581384325 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a3e00640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe94ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a3400640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe94edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 -- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 msgr2=0x556fe94edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe94edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 -- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 msgr2=0x556fe94ed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe94ed980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 -- 172.21.15.116:0/2581384325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe93976f0 con 0x556fe9395f80 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a9200640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f439c007580 tx=0x7f439c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74be2dd0906cafc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c018020 con 0x556fe9395f80 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f439c010c90 con 0x556fe9395f80 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4394026070 con 0x556fe9395f80 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f439c022050 con 0x556fe9395f80 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 10 client.8676 fetch_fsmap learned FSMap version 113 2024-05-14T11:26:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x556fe93936a0 con 0x556fe9395f80 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394026070 con 0x556fe9395f80 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f439c025020 con 0x556fe9395f80 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 10 client.8676 fetch_fsmap finished waiting for FSMap version 113 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 10 client.8676 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 20 client.8676 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f927d0 0x556fe94cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 4 client.8676 mds_command: new command op to 8598 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T11:26:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 --> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe9285be0 con 0x556fe8f927d0 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a3e00640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f927d0 0x556fe94cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f438d600640 10 client.8676 ms_handle_connect on v2:172.21.15.116:6836/2487260442 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.773+0000 7f43a3e00640 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f927d0 0x556fe94cc460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8502b4f955660ef4 server_cookie=2e5476cb9d352c25 in_seq=0 out_seq=0 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f438d600640 1 -- 172.21.15.116:0/2581384325 <== mds.0 v2:172.21.15.116:6836/2487260442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556fe9285be0 con 0x556fe8f927d0 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f438d600640 10 client.8676 handle_command_reply: tid=0 2024-05-14T11:26:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 client.8676 shutdown 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 msgr2=0x556fe94cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f439c007580 tx=0x7f439c007a40 comp rx=0 tx=0).stop 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 shutdown_connections 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f927d0 0x556fe94cc460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe94cb720 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe94edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 --2- 172.21.15.116:0/2581384325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe94ed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f438d600640 0 client.8676 ms_handle_reset on v2:172.21.15.116:6836/2487260442 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 >> 172.21.15.116:0/2581384325 conn(0x556fe946c340 msgr2=0x556fe949c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 shutdown_connections 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 1 -- 172.21.15.116:0/2581384325 wait complete. 2024-05-14T11:26:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.777+0000 7f43b296e000 20 client.8676 trim_cache size 0 max 16384 2024-05-14T11:26:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.793+0000 7f43b296e000 1 Processor -- start 2024-05-14T11:26:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.793+0000 7f43b296e000 1 -- start start 2024-05-14T11:26:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe9554e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe95553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43a42057e0 con 0x556fe9395f80 2024-05-14T11:26:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43a414ebc0 con 0x556fe8fad7c0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f43a414f210 con 0x556fe937f6b0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56548/0 (socket says 172.21.15.116:56548) 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 -- 172.21.15.116:0/2339946985 learned_addr learned my addr 172.21.15.116:0/2339946985 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a3e00640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe9554e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a3400640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe95553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 -- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 msgr2=0x556fe95553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe95553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 -- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 msgr2=0x556fe9554e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe9554e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 -- 172.21.15.116:0/2339946985 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe92c6af0 con 0x556fe9395f80 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a3400640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe95553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43a9200640 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f439c001770 tx=0x7f439c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281b555194900a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f438fe00640 1 -- 172.21.15.116:0/2339946985 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c02b070 con 0x556fe9395f80 2024-05-14T11:26:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f438fe00640 1 -- 172.21.15.116:0/2339946985 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c010c90 con 0x556fe9395f80 2024-05-14T11:26:24.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f438fe00640 1 -- 172.21.15.116:0/2339946985 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c018030 con 0x556fe9395f80 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- 172.21.15.116:0/2339946985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 msgr2=0x556fe9555910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f439c001770 tx=0x7f439c00c000 comp rx=0 tx=0).stop 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- 172.21.15.116:0/2339946985 shutdown_connections 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe9395f80 0x556fe9555910 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe937f6b0 0x556fe95553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 --2- 172.21.15.116:0/2339946985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe8fad7c0 0x556fe9554e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- 172.21.15.116:0/2339946985 >> 172.21.15.116:0/2339946985 conn(0x556fe946c340 msgr2=0x556fe94f37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- 172.21.15.116:0/2339946985 shutdown_connections 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- 172.21.15.116:0/2339946985 wait complete. 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 -- start start 2024-05-14T11:26:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.797+0000 7f43b296e000 1 Processor -- start 2024-05-14T11:26:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe9516480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fe929f910 con 0x556fe8fad7c0 2024-05-14T11:26:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556fe92827d0 con 0x556fe937f6b0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x556fe92c33a0 con 0x556fe9395f80 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 10 client.0 _ll_register_callbacks cb 0x7f439c001b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56560/0 (socket says 172.21.15.116:56560) 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 -- 172.21.15.116:0/3353751409 learned_addr learned my addr 172.21.15.116:0/3353751409 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a9200640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe9516480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3400640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 -- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 msgr2=0x556fe9516480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe9516480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 -- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 msgr2=0x556fe94ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 -- 172.21.15.116:0/3353751409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe92c5e40 con 0x556fe8fad7c0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a9200640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe9516480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3e00640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4394023040 tx=0x7f4394004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69763031f6d538e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439403e070 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f4394037420 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439c010c90 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f4394049050 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 10 client.8682 fetch_fsmap learned FSMap version 113 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x556fe936d4e0 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c010c90 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f439404c020 con 0x556fe8fad7c0 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 10 client.8682 fetch_fsmap finished waiting for FSMap version 113 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 10 client.8682 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 20 client.8682 populate_metadata read hostname 'smithi116' 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f80830 0x556fe95169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 4 client.8682 mds_command: new command op to 8598 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 --> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] -- command(tid 0: {"prefix": "status"}) -- 0x556fe92faca0 con 0x556fe8f80830 2024-05-14T11:26:24.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.801+0000 7f43a3400640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f80830 0x556fe95169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43a3400640 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f80830 0x556fe95169c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ce5e2769d88495 server_cookie=849b6dbfde190b6b in_seq=0 out_seq=0 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f438d600640 10 client.8682 ms_handle_connect on v2:172.21.15.116:6836/2487260442 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f438d600640 1 -- 172.21.15.116:0/3353751409 <== mds.0 v2:172.21.15.116:6836/2487260442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556fe92faca0 con 0x556fe8f80830 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f438d600640 10 client.8682 handle_command_reply: tid=0 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 client.8682 shutdown 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 msgr2=0x556fe94eb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4394023040 tx=0x7f4394004740 comp rx=0 tx=0).stop 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 shutdown_connections 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:6836/2487260442,v1:172.21.15.116:6837/2487260442] conn(0x556fe8f80830 0x556fe95169c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556fe937f6b0 0x556fe94ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x556fe9395f80 0x556fe9516480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f438d600640 0 client.8682 ms_handle_reset on v2:172.21.15.116:6836/2487260442 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 --2- 172.21.15.116:0/3353751409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fe8fad7c0 0x556fe94eb910 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 >> 172.21.15.116:0/3353751409 conn(0x556fe946c340 msgr2=0x556fe94ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 shutdown_connections 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 1 -- 172.21.15.116:0/3353751409 wait complete. 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43b296e000 20 client.8682 trim_cache size 0 max 16384 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "8ea35704-38e5-41a3-b9f5-7d0f3a8c4277", 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-05-14T11:26:24.807 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8598, 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 4.6078951549999996, 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 113, 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 115, 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 115, 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 6.478597583 2024-05-14T11:26:24.808 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 msgr2=0x7f4384071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 0x7f4384071910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4394009b10 tx=0x7f4394008040 comp rx=0 tx=0).stop 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 msgr2=0x7f43a4149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f439001a040 tx=0x7f43900191b0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 shutdown_connections 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f438406f400 0x7f4384071910 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a4161600 0x7f43a415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a415db30 0x7f43a4149cd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 --2- 172.21.15.116:0/677641379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f43a40d1670 0x7f43a414c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 >> 172.21.15.116:0/677641379 conn(0x7f43a40b4620 msgr2=0x7f43a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 shutdown_connections 2024-05-14T11:26:24.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.805+0000 7f43ab200640 1 -- 172.21.15.116:0/677641379 wait complete. 2024-05-14T11:26:24.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-14T11:26:24.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.969+0000 7f250fc00640 1 Processor -- start 2024-05-14T11:26:24.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 -- start start 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f250815db30 0x7f25080aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25080d1670 0x7f25080d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f2508161600 0x7f2508161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f25080ae100 con 0x7f25080d1670 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f25080ae870 con 0x7f2508161600 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f250806ff20 con 0x7f250815db30 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25080d1670 0x7f25080d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25080d1670 0x7f25080d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56570/0 (socket says 172.21.15.116:56570) 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 -- 172.21.15.116:0/1564550811 learned_addr learned my addr 172.21.15.116:0/1564550811 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250dc00640 1 --2- 172.21.15.116:0/1564550811 >> v2:172.21.15.149:3300/0 conn(0x7f2508161600 0x7f2508161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f250d200640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f250815db30 0x7f25080aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 -- 172.21.15.116:0/1564550811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f250815db30 msgr2=0x7f25080aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f250815db30 0x7f25080aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 -- 172.21.15.116:0/1564550811 >> v2:172.21.15.149:3300/0 conn(0x7f2508161600 msgr2=0x7f2508161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:24.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 --2- 172.21.15.116:0/1564550811 >> v2:172.21.15.149:3300/0 conn(0x7f2508161600 0x7f2508161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.973+0000 7f24ffe00640 1 -- 172.21.15.116:0/1564550811 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25080d1fc0 con 0x7f25080d1670 2024-05-14T11:26:24.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f24ffe00640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f25080d1a80 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f25080b2db0 tx=0x7f24f001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a36a5856abd428 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f2507e00640 1 -- 172.21.15.116:0/1564550811 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f00b0070 con 0x7f25080d1670 2024-05-14T11:26:24.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f2507e00640 1 -- 172.21.15.116:0/1564550811 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f00ab600 con 0x7f25080d1670 2024-05-14T11:26:24.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f2507e00640 1 -- 172.21.15.116:0/1564550811 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f00ab9d0 con 0x7f25080d1670 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 -- 172.21.15.116:0/1564550811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 msgr2=0x7f25080d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f25080d1a80 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f25080b2db0 tx=0x7f24f001c490 comp rx=0 tx=0).stop 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 -- 172.21.15.116:0/1564550811 shutdown_connections 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f25080d1a80 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 --2- 172.21.15.116:0/1564550811 >> v2:172.21.15.149:3300/0 conn(0x7f2508161600 0x7f2508161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 --2- 172.21.15.116:0/1564550811 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f250815db30 0x7f25080aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 -- 172.21.15.116:0/1564550811 >> 172.21.15.116:0/1564550811 conn(0x7f25080b4620 msgr2=0x7f25080b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 -- 172.21.15.116:0/1564550811 shutdown_connections 2024-05-14T11:26:24.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.977+0000 7f250fc00640 1 -- 172.21.15.116:0/1564550811 wait complete. 2024-05-14T11:26:24.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 Processor -- start 2024-05-14T11:26:24.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- start start 2024-05-14T11:26:24.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 0x7f25080b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 0x7f25080b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2508162360 con 0x7f25080d1670 2024-05-14T11:26:24.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25080aefe0 con 0x7f250815db30 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f250815df40 con 0x7f2508161600 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56572/0 (socket says 172.21.15.116:56572) 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 -- 172.21.15.116:0/648334807 learned_addr learned my addr 172.21.15.116:0/648334807 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 -- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 msgr2=0x7f25080b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f24ffe00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 0x7f25080b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250dc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 0x7f25080b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 0x7f25080b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 -- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 msgr2=0x7f25080b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 0x7f25080b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 -- 172.21.15.116:0/648334807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25080b2f90 con 0x7f25080d1670 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f24ffe00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 0x7f25080b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250dc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 0x7f25080b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250d200640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e870 tx=0x7f24f800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8017070 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f800c940 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800ccf0 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250814bce0 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.981+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25080b3550 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2508156ef0 con 0x7f25080d1670 2024-05-14T11:26:24.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24f801a070 con 0x7f25080d1670 2024-05-14T11:26:24.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f2505600640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 0x7f24e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:24.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f24f808f400 con 0x7f25080d1670 2024-05-14T11:26:24.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f24ffe00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 0x7f24e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:24.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.985+0000 7f24ffe00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 0x7f24e4071910 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f25080d2280 tx=0x7f24f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:24.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:24.989+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f24f805c0b0 con 0x7f25080d1670 2024-05-14T11:26:25.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2508158420 con 0x7f25080d1670 2024-05-14T11:26:25.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f2505600640 1 -- 172.21.15.116:0/648334807 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) ==== 53+0+73 (secure 0 0 0) 0x7f24f805b9b0 con 0x7f25080d1670 2024-05-14T11:26:25.184 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T11:26:25.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 msgr2=0x7f24e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:25.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 0x7f24e4071910 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f25080d2280 tx=0x7f24f0008040 comp rx=0 tx=0).stop 2024-05-14T11:26:25.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 msgr2=0x7f250814b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e870 tx=0x7f24f800ec70 comp rx=0 tx=0).stop 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 shutdown_connections 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f24e406f400 0x7f24e4071910 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f250815db30 0x7f25080b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2508161600 0x7f25080b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 --2- 172.21.15.116:0/648334807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25080d1670 0x7f250814b1a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 >> 172.21.15.116:0/648334807 conn(0x7f25080b4620 msgr2=0x7f2508158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 shutdown_connections 2024-05-14T11:26:25.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25.185+0000 7f250fc00640 1 -- 172.21.15.116:0/648334807 wait complete. 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T11:26:25.205 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:25.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.206 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-05-14T11:26:25.214 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T11:26:25.214 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T11:26:25.214 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T11:26:25.214 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 41817sec preferred_lft 41817sec 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.215 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:ab:9b:c3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:25.216 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-14T11:26:25.217 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-05-14T11:26:25.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.217 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T11:26:25.277 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:26:25.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.279 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-05-14T11:26:25.334 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-05-14T11:26:25.334 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T11:26:25.334 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:25.335 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:25.336 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-14T11:26:25.336 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T11:26:25.336 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T11:26:25.337 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-05-14T11:26:25.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T11:26:25.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T11:26:25.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T11:26:25.337 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-05-14T11:26:25.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T11:26:25.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:25.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.473 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:26:25.526 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:26:25.526 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:26:25.527 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:25.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.528 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-05-14T11:26:25.590 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-14T11:26:25.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:26:25.590 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:25.646 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T11:26:25.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T11:26:25.647 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:25.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:26:25.763 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:26:25.816 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:26:25.816 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:26:25.816 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:26:25.817 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:26:25.817 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:26:25.817 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:26:25.817 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:26:25.817 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:26:25.818 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:26:25.818 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:26:25.818 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:26:25.818 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:26:25.819 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:26:25.820 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:25.821 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:25.823 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:26:25.878 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T11:26:25.878 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-14T11:26:25.925 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-05-14T11:26:25.947 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T11:26:26.000 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-05-14T11:26:26.001 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-05-14T11:26:26.002 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-05-14T11:26:26.002 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-05-14T11:26:26.002 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-05-14T11:26:26.002 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-05-14T11:26:26.002 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T11:26:26.003 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-05-14T11:26:26.003 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-05-14T11:26:26.003 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-05-14T11:26:26.003 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-05-14T11:26:26.003 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-05-14T11:26:26.004 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-05-14T11:26:26.004 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-05-14T11:26:26.004 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-05-14T11:26:26.004 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-05-14T11:26:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-05-14T11:26:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-05-14T11:26:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:26.006 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:26.008 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-05-14T11:26:26.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:2024-05-14T11:26:26.033+0000 7fa61edab580 -1 init, newargv = 0x7fa604005830 newargc=15 2024-05-14T11:26:26.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[50389]: starting ceph client 2024-05-14T11:26:26.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[50389]: starting fuse 2024-05-14T11:26:26.043 INFO:teuthology.orchestra.run.smithi116.stdout:44 2024-05-14T11:26:26.044 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-05-14T11:26:26.044 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-05-14T11:26:26.044 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-05-14T11:26:26.044 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-14T11:26:26.045 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-05-14T11:26:26.046 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-05-14T11:26:26.047 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.047 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-05-14T11:26:26.047 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T11:26:26.047 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T11:26:26.047 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.0")) 2024-05-14T11:26:26.048 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:26.170 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.0.50389.asok 2024-05-14T11:26:26.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:26.182 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50389.asok 2024-05-14T11:26:26.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:26.182 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50389.asok status 2024-05-14T11:26:26.357 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "0", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "50389", 2024-05-14T11:26:26.358 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 8694, 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-05-14T11:26:26.359 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 8694 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:26:26.360 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1837909052 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-05-14T11:26:26.361 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1837909052 2024-05-14T11:26:26.362 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-05-14T11:26:26.362 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.8694 192.168.0.1:0/1837909052", 2024-05-14T11:26:26.362 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/1837909052", 2024-05-14T11:26:26.362 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-05-14T11:26:26.363 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 113, 2024-05-14T11:26:26.363 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 115, 2024-05-14T11:26:26.363 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-05-14T11:26:26.363 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-05-14T11:26:26.364 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-05-14T11:26:26.364 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:26.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:26.374 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-05-14T11:26:26.374 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:26.383 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-05-14T11:26:26.383 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:26.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:26.384 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:26.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:26.445 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./ 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/charlie 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/alpha 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/flushed/bravo 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/unflushed 2024-05-14T11:26:26.508 INFO:teuthology.orchestra.run.smithi116.stdout:./parent/unflushed/jfile 2024-05-14T11:26:26.509 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:26.509 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.509 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:26.510 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.510 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-14T11:26:26.510 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:26.615 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-05-14T11:26:26.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:26.624 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:26.625 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.625 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:26.625 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.625 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-14T11:26:26.625 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:26.733 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627776 2024-05-14T11:26:26.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-14T11:26:26.742 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:26.850 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627777 2024-05-14T11:26:26.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-14T11:26:26.859 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:26.966 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628280 2024-05-14T11:26:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-14T11:26:26.975 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:27.082 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627778 2024-05-14T11:26:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-14T11:26:27.091 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:27.198 INFO:teuthology.orchestra.run.smithi116.stdout:1099511628279 2024-05-14T11:26:27.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-14T11:26:27.207 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:27.313 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627779 2024-05-14T11:26:27.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:27.322 DEBUG:teuthology.orchestra.run.smithi116:> stdin-killer -- python3 -c ' 2024-05-14T11:26:27.322 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-05-14T11:26:27.322 DEBUG:teuthology.orchestra.run.smithi116:> import stat 2024-05-14T11:26:27.323 DEBUG:teuthology.orchestra.run.smithi116:> 2024-05-14T11:26:27.323 DEBUG:teuthology.orchestra.run.smithi116:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-14T11:26:27.323 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-05-14T11:26:27.430 INFO:teuthology.orchestra.run.smithi116.stdout:1099511627780 2024-05-14T11:26:27.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T11:26:27.439 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-14T11:26:27.439 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:26:27.553 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-14T11:26:27.655 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-14T11:26:27.655 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:26:27.655 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:27.655 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:27.664 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:27.664 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:27.714 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-14T11:26:27.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 Processor -- start 2024-05-14T11:26:27.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 -- start start 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff9500d24d0 0x7ff9500b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d09c0 0x7ff95015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9500d28e0 con 0x7ff9500cdcf0 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff9500d2040 con 0x7ff9500d24d0 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff955c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff9500d1430 con 0x7ff9500d09c0 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff9500d24d0 0x7ff9500b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff9500d24d0 0x7ff9500b9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40646/0 (socket says 172.21.15.116:40646) 2024-05-14T11:26:27.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 -- 172.21.15.116:0/4178902446 learned_addr learned my addr 172.21.15.116:0/4178902446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:27.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 -- 172.21.15.116:0/4178902446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d09c0 msgr2=0x7ff95015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:27.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 --2- 172.21.15.116:0/4178902446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d09c0 0x7ff95015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:27.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 -- 172.21.15.116:0/4178902446 >> v1:172.21.15.149:6789/0 conn(0x7ff9500cdcf0 legacy=0x7ff9500d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:27.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.901+0000 7ff94fe00640 1 -- 172.21.15.116:0/4178902446 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500d1710 con 0x7ff9500d24d0 2024-05-14T11:26:27.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff94fe00640 1 --2- 172.21.15.116:0/4178902446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9500d24d0 0x7ff9500b9020 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff940009e50 tx=0x7ff94001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33a21ffce373d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:27.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff94ea00640 1 -- 172.21.15.116:0/4178902446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400a5580 con 0x7ff9500d24d0 2024-05-14T11:26:27.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff94ea00640 1 -- 172.21.15.116:0/4178902446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9400a5720 con 0x7ff9500d24d0 2024-05-14T11:26:27.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff94ea00640 1 -- 172.21.15.116:0/4178902446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400a5af0 con 0x7ff9500d24d0 2024-05-14T11:26:27.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 -- 172.21.15.116:0/4178902446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9500d24d0 msgr2=0x7ff9500b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:27.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 --2- 172.21.15.116:0/4178902446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9500d24d0 0x7ff9500b9020 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff940009e50 tx=0x7ff94001c850 comp rx=0 tx=0).stop 2024-05-14T11:26:27.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 -- 172.21.15.116:0/4178902446 shutdown_connections 2024-05-14T11:26:27.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 --2- 172.21.15.116:0/4178902446 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d09c0 0x7ff95015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:27.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 --2- 172.21.15.116:0/4178902446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9500d24d0 0x7ff9500b9020 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:27.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.905+0000 7ff955c00640 1 -- 172.21.15.116:0/4178902446 >> 172.21.15.116:0/4178902446 conn(0x7ff9500b51f0 msgr2=0x7ff9500b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:27.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- 172.21.15.116:0/4178902446 shutdown_connections 2024-05-14T11:26:27.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- 172.21.15.116:0/4178902446 wait complete. 2024-05-14T11:26:27.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 Processor -- start 2024-05-14T11:26:27.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- start start 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9500cdcf0 0x7ff95014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d24d0 0x7ff95006e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff95015d260 con 0x7ff95014fe90 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff95015d0e0 con 0x7ff9500cdcf0 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.909+0000 7ff955c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff95015d3e0 con 0x7ff9500d24d0 2024-05-14T11:26:27.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40650/0 (socket says 172.21.15.116:40650) 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 -- 172.21.15.116:0/4004254880 learned_addr learned my addr 172.21.15.116:0/4004254880 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff94fe00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9500cdcf0 0x7ff95014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff94f400640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d24d0 0x7ff95006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 -- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d24d0 msgr2=0x7ff95006e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d24d0 0x7ff95006e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 -- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9500cdcf0 msgr2=0x7ff95014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9500cdcf0 0x7ff95014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 -- 172.21.15.116:0/4004254880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95006f680 con 0x7ff95014fe90 2024-05-14T11:26:27.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955200640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b410 tx=0x7ff94800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948015020 con 0x7ff95014fe90 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff948011830 con 0x7ff95014fe90 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95006f960 con 0x7ff95014fe90 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948011c20 con 0x7ff95014fe90 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95006ff20 con 0x7ff95014fe90 2024-05-14T11:26:27.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950050880 con 0x7ff95014fe90 2024-05-14T11:26:27.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.913+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff948018020 con 0x7ff95014fe90 2024-05-14T11:26:27.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.917+0000 7ff947e00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 0x7ff930071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:27.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.917+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7ff94808f2a0 con 0x7ff95014fe90 2024-05-14T11:26:27.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.917+0000 7ff94fe00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 0x7ff930071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:27.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.917+0000 7ff94fe00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 0x7ff930071910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ae000 tx=0x7ff9400a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:27.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:27.921+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff94801f020 con 0x7ff95014fe90 2024-05-14T11:26:28.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff95006a3d0 con 0x7ff95014fe90 2024-05-14T11:26:28.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff947e00640 1 -- 172.21.15.116:0/4004254880 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v115) ==== 80+0+248 (secure 0 0 0) 0x7ff94805b980 con 0x7ff95014fe90 2024-05-14T11:26:28.088 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6837/3756089525 2024-05-15T11:26:18.174966+0000 2024-05-14T11:26:28.089 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/3756089525 2024-05-15T11:26:18.174966+0000 2024-05-14T11:26:28.089 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6837/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:28.089 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6836/1742968683 2024-05-15T11:26:01.594455+0000 2024-05-14T11:26:28.089 INFO:teuthology.orchestra.run.smithi116.stderr:listed 4 entries 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 msgr2=0x7ff930071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 0x7ff930071910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ae000 tx=0x7ff9400a7070 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 msgr2=0x7ff95006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b410 tx=0x7ff94800b8d0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 shutdown_connections 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7ff93006f400 0x7ff930071910 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff95014fe90 0x7ff95006ee40 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff9500d24d0 0x7ff95006e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 --2- 172.21.15.116:0/4004254880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9500cdcf0 0x7ff95014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 >> 172.21.15.116:0/4004254880 conn(0x7ff9500b51f0 msgr2=0x7ff950161660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 shutdown_connections 2024-05-14T11:26:28.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.089+0000 7ff955c00640 1 -- 172.21.15.116:0/4004254880 wait complete. 2024-05-14T11:26:28.110 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-05-14T11:26:28.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:28.110 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-14T11:26:28.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi116.stderr:ceph-fuse[50389]: fuse finished with error 0 and tester_r 0 2024-05-14T11:26:28.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:26:28.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:28.130 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T11:26:28.204 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:26:28.205 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:26:28.205 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:28.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:26:28.214 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:26:28.214 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:28.214 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:28.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:26:28.254 INFO:tasks.cephfs.mount:Terminating background process 2024-05-14T11:26:28.255 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:26:28.255 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:28.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:26:28.264 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:28.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.449+0000 7f6211400640 1 Processor -- start 2024-05-14T11:26:28.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.449+0000 7f6211400640 1 -- start start 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f620c15db30 0x7f620c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f620c0d1670 0x7f620c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c161600 0x7f620c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f620c0ae100 con 0x7f620c0d1670 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f620c0ae870 con 0x7f620c15db30 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6211400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f620c06ff20 con 0x7f620c161600 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f620c0d1670 0x7f620c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f620c0d1670 0x7f620c0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40654/0 (socket says 172.21.15.116:40654) 2024-05-14T11:26:28.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 -- 172.21.15.116:0/2833938011 learned_addr learned my addr 172.21.15.116:0/2833938011 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f620b200640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c161600 0x7f620c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f620a800640 1 --2- 172.21.15.116:0/2833938011 >> v2:172.21.15.149:3300/0 conn(0x7f620c15db30 0x7f620c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 -- 172.21.15.116:0/2833938011 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c161600 msgr2=0x7f620c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c161600 0x7f620c161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 -- 172.21.15.116:0/2833938011 >> v2:172.21.15.149:3300/0 conn(0x7f620c15db30 msgr2=0x7f620c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 --2- 172.21.15.116:0/2833938011 >> v2:172.21.15.149:3300/0 conn(0x7f620c15db30 0x7f620c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.453+0000 7f6201e00640 1 -- 172.21.15.116:0/2833938011 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c0d1fc0 con 0x7f620c0d1670 2024-05-14T11:26:28.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6201e00640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c0d1670 0x7f620c0d1a80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f620c071f20 tx=0x7f61fc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dc380322536a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6209e00640 1 -- 172.21.15.116:0/2833938011 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0b0070 con 0x7f620c0d1670 2024-05-14T11:26:28.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6209e00640 1 -- 172.21.15.116:0/2833938011 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc0ab600 con 0x7f620c0d1670 2024-05-14T11:26:28.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6209e00640 1 -- 172.21.15.116:0/2833938011 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0ab9d0 con 0x7f620c0d1670 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- 172.21.15.116:0/2833938011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c0d1670 msgr2=0x7f620c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c0d1670 0x7f620c0d1a80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f620c071f20 tx=0x7f61fc01c490 comp rx=0 tx=0).stop 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- 172.21.15.116:0/2833938011 shutdown_connections 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c0d1670 0x7f620c0d1a80 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 --2- 172.21.15.116:0/2833938011 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c161600 0x7f620c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 --2- 172.21.15.116:0/2833938011 >> v2:172.21.15.149:3300/0 conn(0x7f620c15db30 0x7f620c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- 172.21.15.116:0/2833938011 >> 172.21.15.116:0/2833938011 conn(0x7f620c0b4620 msgr2=0x7f620c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- 172.21.15.116:0/2833938011 shutdown_connections 2024-05-14T11:26:28.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- 172.21.15.116:0/2833938011 wait complete. 2024-05-14T11:26:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 Processor -- start 2024-05-14T11:26:28.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.457+0000 7f6211400640 1 -- start start 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f620c0d1670 0x7f620c09a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c15db30 0x7f620c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f620c162360 con 0x7f620c161600 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0aefe0 con 0x7f620c0d1670 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f620c15df40 con 0x7f620c15db30 2024-05-14T11:26:28.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40668/0 (socket says 172.21.15.116:40668) 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 -- 172.21.15.116:0/1321970867 learned_addr learned my addr 172.21.15.116:0/1321970867 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6201e00640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c15db30 0x7f620c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620a800640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f620c0d1670 0x7f620c09a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 -- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c15db30 msgr2=0x7f620c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c15db30 0x7f620c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 -- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f620c0d1670 msgr2=0x7f620c09a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f620c0d1670 0x7f620c09a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 -- 172.21.15.116:0/1321970867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c09ae60 con 0x7f620c161600 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f620b200640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6204007d70 tx=0x7f620400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204014420 con 0x7f620c161600 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6204019070 con 0x7f620c161600 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204014820 con 0x7f620c161600 2024-05-14T11:26:28.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c073530 con 0x7f620c161600 2024-05-14T11:26:28.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c073af0 con 0x7f620c161600 2024-05-14T11:26:28.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.461+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c0732a0 con 0x7f620c161600 2024-05-14T11:26:28.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.465+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f62040149c0 con 0x7f620c161600 2024-05-14T11:26:28.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.465+0000 7f6203400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 0x7f61e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.465+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f620408fc00 con 0x7f620c161600 2024-05-14T11:26:28.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.465+0000 7f620a800640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 0x7f61e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.465+0000 7f620a800640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 0x7f61e8071910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f61f40079c0 tx=0x7f61f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.469+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f620405c1c0 con 0x7f620c161600 2024-05-14T11:26:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620c0b4db0 con 0x7f620c161600 2024-05-14T11:26:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6203400640 1 -- 172.21.15.116:0/1321970867 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5200 (secure 0 0 0) 0x7f62040604c0 con 0x7f620c161600 2024-05-14T11:26:28.660 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:28.660 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:23.215968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8598,"qdb_cluster":[8598]},"id":8}]} 2024-05-14T11:26:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 113 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 msgr2=0x7f61e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 0x7f61e8071910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f61f40079c0 tx=0x7f61f4008040 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 msgr2=0x7f620c15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6204007d70 tx=0x7f620400e4b0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 shutdown_connections 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f61e806f400 0x7f61e8071910 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f620c161600 0x7f620c15f230 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f620c15db30 0x7f620c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 --2- 172.21.15.116:0/1321970867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f620c0d1670 0x7f620c09a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 >> 172.21.15.116:0/1321970867 conn(0x7f620c0b4620 msgr2=0x7f620c157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 shutdown_connections 2024-05-14T11:26:28.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.661+0000 7f6211400640 1 -- 172.21.15.116:0/1321970867 wait complete. 2024-05-14T11:26:28.681 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 Processor -- start 2024-05-14T11:26:28.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 -- start start 2024-05-14T11:26:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7f415db30 0x7fd7f40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 0x7fd7f406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f406ff20 con 0x7fd7f40d1670 2024-05-14T11:26:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40ae100 con 0x7fd7f415db30 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f9400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f40ae870 con 0x7fd7f406c700 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7f415db30 0x7fd7f40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7f415db30 0x7fd7f40aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40672/0 (socket says 172.21.15.116:40672) 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 -- 172.21.15.116:0/3505812209 learned_addr learned my addr 172.21.15.116:0/3505812209 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f3200640 1 --2- 172.21.15.116:0/3505812209 >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 0x7fd7f406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 -- 172.21.15.116:0/3505812209 >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 msgr2=0x7fd7f406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 --2- 172.21.15.116:0/3505812209 >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 0x7fd7f406cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 -- 172.21.15.116:0/3505812209 >> v1:172.21.15.149:6789/0 conn(0x7fd7f40d1670 legacy=0x7fd7f40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f2800640 1 -- 172.21.15.116:0/3505812209 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f4070470 con 0x7fd7f415db30 2024-05-14T11:26:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.829+0000 7fd7f3200640 1 --2- 172.21.15.116:0/3505812209 >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 0x7fd7f406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f2800640 1 --2- 172.21.15.116:0/3505812209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f415db30 0x7fd7f40aeaa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd7f41626a0 tx=0x7fd7e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8832356a8fdb4788 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f1400640 1 -- 172.21.15.116:0/3505812209 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b0070 con 0x7fd7f415db30 2024-05-14T11:26:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f1400640 1 -- 172.21.15.116:0/3505812209 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40ab600 con 0x7fd7f415db30 2024-05-14T11:26:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f1400640 1 -- 172.21.15.116:0/3505812209 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40ab9d0 con 0x7fd7f415db30 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 -- 172.21.15.116:0/3505812209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f415db30 msgr2=0x7fd7f40aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3505812209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f415db30 0x7fd7f40aeaa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd7f41626a0 tx=0x7fd7e401c450 comp rx=0 tx=0).stop 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 -- 172.21.15.116:0/3505812209 shutdown_connections 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3505812209 >> v2:172.21.15.149:3300/0 conn(0x7fd7f406c700 0x7fd7f406cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3505812209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f415db30 0x7fd7f40aeaa0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 -- 172.21.15.116:0/3505812209 >> 172.21.15.116:0/3505812209 conn(0x7fd7f40b4620 msgr2=0x7fd7f40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 -- 172.21.15.116:0/3505812209 shutdown_connections 2024-05-14T11:26:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.833+0000 7fd7f9400640 1 -- 172.21.15.116:0/3505812209 wait complete. 2024-05-14T11:26:28.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 Processor -- start 2024-05-14T11:26:28.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 -- start start 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 0x7fd7f414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7f415db30 0x7fd7f4150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f416a770 con 0x7fd7f406c700 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f415df40 con 0x7fd7f415db30 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f9400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f416a8f0 con 0x7fd7f40d1670 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f2800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40686/0 (socket says 172.21.15.116:40686) 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f2800640 1 -- 172.21.15.116:0/3536993650 learned_addr learned my addr 172.21.15.116:0/3536993650 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:28.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f3200640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7f415db30 0x7fd7f4150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.837+0000 7fd7f1e00640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 0x7fd7f414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 -- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 msgr2=0x7fd7f414f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 0x7fd7f414f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 -- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7f415db30 msgr2=0x7fd7f4150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7f415db30 0x7fd7f4150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 -- 172.21.15.116:0/3536993650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f409ae60 con 0x7fd7f406c700 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f1e00640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 0x7fd7f414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:28.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f2800640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ae040 tx=0x7fd7e40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b0040 con 0x7fd7f406c700 2024-05-14T11:26:28.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40b9070 con 0x7fd7f406c700 2024-05-14T11:26:28.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f414fdb0 con 0x7fd7f406c700 2024-05-14T11:26:28.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b48a0 con 0x7fd7f406c700 2024-05-14T11:26:28.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f4155450 con 0x7fd7f406c700 2024-05-14T11:26:28.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f406a9c0 con 0x7fd7f406c700 2024-05-14T11:26:28.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7e40ab4c0 con 0x7fd7f406c700 2024-05-14T11:26:28.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.841+0000 7fd7eaa00640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 0x7fd7d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:28.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.845+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7fd7e4129be0 con 0x7fd7f406c700 2024-05-14T11:26:28.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.845+0000 7fd7f1e00640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 0x7fd7d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:28.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.845+0000 7fd7f1e00640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 0x7fd7d4071940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40b2c50 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:28.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:28.845+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd7e40160d0 con 0x7fd7f406c700 2024-05-14T11:26:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f40cd100 con 0x7fd7f406c700 2024-05-14T11:26:29.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7eaa00640 1 -- 172.21.15.116:0/3536993650 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5200 (secure 0 0 0) 0x7fd7e40f6970 con 0x7fd7f406c700 2024-05-14T11:26:29.036 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:29.036 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:23.215968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8598,"qdb_cluster":[8598]},"id":8}]} 2024-05-14T11:26:29.037 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 113 2024-05-14T11:26:29.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 msgr2=0x7fd7d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 0x7fd7d4071940 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40b2c50 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 msgr2=0x7fd7f409a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ae040 tx=0x7fd7e40191f0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 shutdown_connections 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd7d406f430 0x7fd7d4071940 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7f415db30 0x7fd7f4150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7f40d1670 0x7fd7f414f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 --2- 172.21.15.116:0/3536993650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f406c700 0x7fd7f409a920 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 >> 172.21.15.116:0/3536993650 conn(0x7fd7f40b4620 msgr2=0x7fd7f4151400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 shutdown_connections 2024-05-14T11:26:29.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.037+0000 7fd7f9400640 1 -- 172.21.15.116:0/3536993650 wait complete. 2024-05-14T11:26:29.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:26:29.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 Processor -- start 2024-05-14T11:26:29.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 -- start start 2024-05-14T11:26:29.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f304c0cdf80 0x7f304c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0d1510 0x7f304c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f304c0b40c0 0x7f304c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f304c0d1e60 con 0x7f304c0b40c0 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f304c0b4a10 con 0x7f304c0cdf80 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3054800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f304c0b3060 con 0x7f304c0d1510 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3052800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f304c0b40c0 0x7f304c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3052800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f304c0b40c0 0x7f304c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40700/0 (socket says 172.21.15.116:40700) 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3052800640 1 -- 172.21.15.116:0/520247708 learned_addr learned my addr 172.21.15.116:0/520247708 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3051400640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0d1510 0x7f304c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.205+0000 7f3051e00640 1 --2- 172.21.15.116:0/520247708 >> v2:172.21.15.149:3300/0 conn(0x7f304c0cdf80 0x7f304c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 -- 172.21.15.116:0/520247708 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0d1510 msgr2=0x7f304c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0d1510 0x7f304c0d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 -- 172.21.15.116:0/520247708 >> v2:172.21.15.149:3300/0 conn(0x7f304c0cdf80 msgr2=0x7f304c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 --2- 172.21.15.116:0/520247708 >> v2:172.21.15.149:3300/0 conn(0x7f304c0cdf80 0x7f304c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 -- 172.21.15.116:0/520247708 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c0b3300 con 0x7f304c0b40c0 2024-05-14T11:26:29.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3052800640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c0b44d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f304c074690 tx=0x7f304801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d39183129e2b270 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3050a00640 1 -- 172.21.15.116:0/520247708 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480a5840 con 0x7f304c0b40c0 2024-05-14T11:26:29.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3050a00640 1 -- 172.21.15.116:0/520247708 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30480a59e0 con 0x7f304c0b40c0 2024-05-14T11:26:29.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.209+0000 7f3050a00640 1 -- 172.21.15.116:0/520247708 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480a5db0 con 0x7f304c0b40c0 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- 172.21.15.116:0/520247708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 msgr2=0x7f304c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c0b44d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f304c074690 tx=0x7f304801c9c0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- 172.21.15.116:0/520247708 shutdown_connections 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c0b44d0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 --2- 172.21.15.116:0/520247708 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0d1510 0x7f304c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 --2- 172.21.15.116:0/520247708 >> v2:172.21.15.149:3300/0 conn(0x7f304c0cdf80 0x7f304c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- 172.21.15.116:0/520247708 >> 172.21.15.116:0/520247708 conn(0x7f304c0d0770 msgr2=0x7f304c0d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- 172.21.15.116:0/520247708 shutdown_connections 2024-05-14T11:26:29.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- 172.21.15.116:0/520247708 wait complete. 2024-05-14T11:26:29.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 Processor -- start 2024-05-14T11:26:29.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.213+0000 7f3054800640 1 -- start start 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0cdf80 0x7f304c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304c0d1510 0x7f304c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f304c160320 con 0x7f304c0b40c0 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1601a0 con 0x7f304c0d1510 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f304c06fdc0 con 0x7f304c0cdf80 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40710/0 (socket says 172.21.15.116:40710) 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 -- 172.21.15.116:0/433143975 learned_addr learned my addr 172.21.15.116:0/433143975 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051400640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0cdf80 0x7f304c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3052800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304c0d1510 0x7f304c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 -- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0cdf80 msgr2=0x7f304c154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0cdf80 0x7f304c154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 -- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304c0d1510 msgr2=0x7f304c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304c0d1510 0x7f304c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 -- 172.21.15.116:0/433143975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c15c230 con 0x7f304c0b40c0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3051e00640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f304000d830 tx=0x7f304000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040018030 con 0x7f304c0b40c0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304000be10 con 0x7f304c0b40c0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c1609b0 con 0x7f304c0b40c0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400098b0 con 0x7f304c0b40c0 2024-05-14T11:26:29.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c160f70 con 0x7f304c0b40c0 2024-05-14T11:26:29.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c06a9c0 con 0x7f304c0b40c0 2024-05-14T11:26:29.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.217+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f304001b070 con 0x7f304c0b40c0 2024-05-14T11:26:29.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.221+0000 7f3046000640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 0x7f3030071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.221+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f304008f760 con 0x7f304c0b40c0 2024-05-14T11:26:29.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.221+0000 7f3051400640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 0x7f3030071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.221+0000 7f3051400640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 0x7f3030071940 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f304c15c5c0 tx=0x7f303c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.225+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f304005c400 con 0x7f304c0b40c0 2024-05-14T11:26:29.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.385+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f304c0b9590 con 0x7f304c0b40c0 2024-05-14T11:26:29.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.385+0000 7f3046000640 1 -- 172.21.15.116:0/433143975 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17263 (secure 0 0 0) 0x7f304005bd00 con 0x7f304c0b40c0 2024-05-14T11:26:29.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:29.385 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":115,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:26:18.174998+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:25:39.509510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-14T11:25:40.801825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6837/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6836/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6837/1742968683":"2024-05-15T11:26:01.594455+0000","172.21.15.116:6836/1742968683":"2024-05-15T11:26:01.594455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 msgr2=0x7f3030071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 0x7f3030071940 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f304c15c5c0 tx=0x7f303c008040 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 msgr2=0x7f304c15b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f304000d830 tx=0x7f304000dcf0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 shutdown_connections 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f303006f430 0x7f3030071940 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304c0d1510 0x7f304c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304c0cdf80 0x7f304c154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 --2- 172.21.15.116:0/433143975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c0b40c0 0x7f304c15b930 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 >> 172.21.15.116:0/433143975 conn(0x7f304c0d0770 msgr2=0x7f304c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 shutdown_connections 2024-05-14T11:26:29.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.389+0000 7f3054800640 1 -- 172.21.15.116:0/433143975 wait complete. 2024-05-14T11:26:29.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:26:29.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.557+0000 7f5050e00640 1 Processor -- start 2024-05-14T11:26:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.557+0000 7f5050e00640 1 -- start start 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.557+0000 7f5050e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f504c15db30 0x7f504c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5050e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5050e00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f504c06ff20 con 0x7f504c161600 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5050e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f504c06d710 con 0x7f504c15db30 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5050e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f504c0ae100 con 0x7f504c0d1670 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f504a800640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f504c161600 0x7f504c161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:45770/0 (socket says 172.21.15.116:45770) 2024-05-14T11:26:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f504a800640 1 -- 172.21.15.116:0/3017369127 learned_addr learned my addr 172.21.15.116:0/3017369127 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f504c15db30 0x7f504c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5048a00640 1 -- 172.21.15.116:0/3017369127 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938073038 0 0) 0x7f504c06ff20 con 0x7f504c161600 2024-05-14T11:26:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5048a00640 1 -- 172.21.15.116:0/3017369127 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5030003530 con 0x7f504c161600 2024-05-14T11:26:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049e00640 1 -- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 msgr2=0x7f504c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049e00640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049e00640 1 -- 172.21.15.116:0/3017369127 >> v1:172.21.15.149:6789/0 conn(0x7f504c161600 legacy=0x7f504c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049e00640 1 -- 172.21.15.116:0/3017369127 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c0d1fc0 con 0x7f504c15db30 2024-05-14T11:26:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.561+0000 7f5049400640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5049e00640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c0aeaa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f503c0097c0 tx=0x7f503c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b18d28c48ec88a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5048a00640 1 -- 172.21.15.116:0/3017369127 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0b0070 con 0x7f504c15db30 2024-05-14T11:26:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5048a00640 1 -- 172.21.15.116:0/3017369127 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503c0ab600 con 0x7f504c15db30 2024-05-14T11:26:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5048a00640 1 -- 172.21.15.116:0/3017369127 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0ab9b0 con 0x7f504c15db30 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 msgr2=0x7f504c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c0aeaa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f503c0097c0 tx=0x7f503c01c410 comp rx=0 tx=0).stop 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- 172.21.15.116:0/3017369127 shutdown_connections 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 --2- 172.21.15.116:0/3017369127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c0aeaa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- 172.21.15.116:0/3017369127 >> 172.21.15.116:0/3017369127 conn(0x7f504c0b4620 msgr2=0x7f504c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- 172.21.15.116:0/3017369127 shutdown_connections 2024-05-14T11:26:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- 172.21.15.116:0/3017369127 wait complete. 2024-05-14T11:26:29.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 Processor -- start 2024-05-14T11:26:29.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.565+0000 7f5050e00640 1 -- start start 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f504c15df40 con 0x7f504c161600 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f504c0aefe0 con 0x7f504c15db30 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f504c16af80 con 0x7f504c0d1670 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40726/0 (socket says 172.21.15.116:40726) 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 -- 172.21.15.116:0/3968453923 learned_addr learned my addr 172.21.15.116:0/3968453923 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:29.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5049400640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5049e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 -- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 msgr2=0x7f504c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 -- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 msgr2=0x7f504c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 -- 172.21.15.116:0/3968453923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c160060 con 0x7f504c161600 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f504a800640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f504400caa0 tx=0x7f504400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504400ab40 con 0x7f504c161600 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504400ace0 con 0x7f504c161600 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c160340 con 0x7f504c161600 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044012600 con 0x7f504c161600 2024-05-14T11:26:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c151ec0 con 0x7f504c161600 2024-05-14T11:26:29.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.569+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c06a9c0 con 0x7f504c161600 2024-05-14T11:26:29.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.573+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50440127a0 con 0x7f504c161600 2024-05-14T11:26:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.573+0000 7f5042000640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 0x7f502c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.573+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f504408f920 con 0x7f504c161600 2024-05-14T11:26:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.573+0000 7f5049e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 0x7f502c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.573+0000 7f5049e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 0x7f502c071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f503c0ad000 tx=0x7f503c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.577+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f504405c000 con 0x7f504c161600 2024-05-14T11:26:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.765+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f504c0b0f10 con 0x7f504c161600 2024-05-14T11:26:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.765+0000 7f5042000640 1 -- 172.21.15.116:0/3968453923 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5200 (secure 0 0 0) 0x7f5044060300 con 0x7f504c161600 2024-05-14T11:26:29.766 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:29.766 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:23.215968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8598,"qdb_cluster":[8598]},"id":8}]} 2024-05-14T11:26:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 113 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 msgr2=0x7f502c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 0x7f502c071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f503c0ad000 tx=0x7f503c0a6000 comp rx=0 tx=0).stop 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 msgr2=0x7f504c15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f504400caa0 tx=0x7f504400cf60 comp rx=0 tx=0).stop 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 shutdown_connections 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f502c06f400 0x7f502c071910 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f504c15db30 0x7f504c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f504c161600 0x7f504c15c150 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 --2- 172.21.15.116:0/3968453923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f504c0d1670 0x7f504c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 >> 172.21.15.116:0/3968453923 conn(0x7f504c0b4620 msgr2=0x7f504c0b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 shutdown_connections 2024-05-14T11:26:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.769+0000 7f5050e00640 1 -- 172.21.15.116:0/3968453923 wait complete. 2024-05-14T11:26:29.788 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.933+0000 7f6abfc00640 1 Processor -- start 2024-05-14T11:26:29.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.933+0000 7f6abfc00640 1 -- start start 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abfc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6ab80d1670 0x7f6ab80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abfc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab806ff20 con 0x7f6ab815db30 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abfc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab80d1fc0 con 0x7f6ab806c700 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abfc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab80ae100 con 0x7f6ab80d1670 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abd200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ab815db30 0x7f6ab80aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36846/0 (socket says 172.21.15.116:36846) 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6abd200640 1 -- 172.21.15.116:0/324914947 learned_addr learned my addr 172.21.15.116:0/324914947 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:29.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7e00640 1 --2- 172.21.15.116:0/324914947 >> v2:172.21.15.149:3300/0 conn(0x7f6ab80d1670 0x7f6ab80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717909083 0 0) 0x7f6ab806ff20 con 0x7f6ab815db30 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a9c003530 con 0x7f6ab815db30 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451337566 0 0) 0x7f6ab0001120 con 0x7f6ab806c700 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab806ff20 con 0x7f6ab806c700 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7e00640 1 -- 172.21.15.116:0/324914947 >> v1:172.21.15.149:6789/0 conn(0x7f6ab806c700 legacy=0x7f6ab806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7e00640 1 -- 172.21.15.116:0/324914947 >> v1:172.21.15.116:6789/0 conn(0x7f6ab815db30 legacy=0x7f6ab80aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1728087737 0 0) 0x7f6a9c003530 con 0x7f6ab815db30 2024-05-14T11:26:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.937+0000 7f6ab7e00640 1 -- 172.21.15.116:0/324914947 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab8070470 con 0x7f6ab80d1670 2024-05-14T11:26:29.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6ab7e00640 1 --2- 172.21.15.116:0/324914947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab80d1a80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80ae280 tx=0x7f6aa401c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baa00251d37a9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40b0070 con 0x7f6ab80d1670 2024-05-14T11:26:29.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa40ab400 con 0x7f6ab80d1670 2024-05-14T11:26:29.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6ab7400640 1 -- 172.21.15.116:0/324914947 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8001a80 con 0x7f6ab80d1670 2024-05-14T11:26:29.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 -- 172.21.15.116:0/324914947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 msgr2=0x7f6ab80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 --2- 172.21.15.116:0/324914947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab80d1a80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80ae280 tx=0x7f6aa401c2a0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 -- 172.21.15.116:0/324914947 shutdown_connections 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 --2- 172.21.15.116:0/324914947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab80d1a80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 -- 172.21.15.116:0/324914947 >> 172.21.15.116:0/324914947 conn(0x7f6ab80b4620 msgr2=0x7f6ab80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 -- 172.21.15.116:0/324914947 shutdown_connections 2024-05-14T11:26:29.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.941+0000 7f6abfc00640 1 -- 172.21.15.116:0/324914947 wait complete. 2024-05-14T11:26:29.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 Processor -- start 2024-05-14T11:26:29.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- start start 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab8147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 0x7f6ab81479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab816bf90 con 0x7f6ab806c700 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab815df40 con 0x7f6ab80d1670 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab816c110 con 0x7f6ab815db30 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40732/0 (socket says 172.21.15.116:40732) 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 -- 172.21.15.116:0/3656534376 learned_addr learned my addr 172.21.15.116:0/3656534376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abdc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 0x7f6ab81479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6ab7e00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab8147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 -- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 msgr2=0x7f6ab81479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 0x7f6ab81479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 -- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 msgr2=0x7f6ab8147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab8147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 -- 172.21.15.116:0/3656534376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab8155550 con 0x7f6ab806c700 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abdc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 0x7f6ab81479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abd200640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800cc70 tx=0x7f6aa800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8017070 con 0x7f6ab806c700 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa8012400 con 0x7f6ab806c700 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa80127b0 con 0x7f6ab806c700 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab8073a70 con 0x7f6ab806c700 2024-05-14T11:26:29.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.945+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81557f0 con 0x7f6ab806c700 2024-05-14T11:26:29.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab806a9c0 con 0x7f6ab806c700 2024-05-14T11:26:29.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6aa801a070 con 0x7f6ab806c700 2024-05-14T11:26:29.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6ab4c00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 0x7f6a98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:29.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f6aa808f2d0 con 0x7f6ab806c700 2024-05-14T11:26:29.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6ab7e00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 0x7f6a98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:29.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.949+0000 7f6ab7e00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 0x7f6a98071910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ad000 tx=0x7f6aa4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:29.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:29.953+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6aa805b920 con 0x7f6ab806c700 2024-05-14T11:26:30.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.145+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ab80b10c0 con 0x7f6ab806c700 2024-05-14T11:26:30.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.145+0000 7f6ab4c00640 1 -- 172.21.15.116:0/3656534376 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5200 (secure 0 0 0) 0x7f6aa805fc20 con 0x7f6ab806c700 2024-05-14T11:26:30.145 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:30.145 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T11:25:41.831155+0000","modified":"2024-05-14T11:26:23.215968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8598},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8598":{"gid":8598,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6837/2487260442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":2487260442},{"type":"v1","addr":"172.21.15.116:6837","nonce":2487260442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8598,"qdb_cluster":[8598]},"id":8}]} 2024-05-14T11:26:30.145 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 113 2024-05-14T11:26:30.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 msgr2=0x7f6a98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 0x7f6a98071910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40ad000 tx=0x7f6aa4008040 comp rx=0 tx=0).stop 2024-05-14T11:26:30.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 msgr2=0x7f6ab814b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800cc70 tx=0x7f6aa800a430 comp rx=0 tx=0).stop 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 shutdown_connections 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6a9806f400 0x7f6a98071910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ab80d1670 0x7f6ab8147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6ab815db30 0x7f6ab81479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 --2- 172.21.15.116:0/3656534376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab806c700 0x7f6ab814b9c0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 >> 172.21.15.116:0/3656534376 conn(0x7f6ab80b4620 msgr2=0x7f6ab80b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 shutdown_connections 2024-05-14T11:26:30.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.149+0000 7f6abfc00640 1 -- 172.21.15.116:0/3656534376 wait complete. 2024-05-14T11:26:30.167 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:26:30.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.321+0000 7f1b71000640 1 Processor -- start 2024-05-14T11:26:30.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.321+0000 7f1b71000640 1 -- start start 2024-05-14T11:26:30.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b71000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1b6c0d1670 0x7f1b6c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b71000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f1b6c0af380 0x7f1b6c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b71000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c0d1fc0 con 0x7f1b6c156fa0 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b71000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c0706f0 con 0x7f1b6c0d1670 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b71000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c1573b0 con 0x7f1b6c0af380 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b69e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1b6c156fa0 0x7f1b6c06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36848/0 (socket says 172.21.15.116:36848) 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b69e00640 1 -- 172.21.15.116:0/568743568 learned_addr learned my addr 172.21.15.116:0/568743568 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b61400640 1 --2- 172.21.15.116:0/568743568 >> v2:172.21.15.116:3300/0 conn(0x7f1b6c0d1670 0x7f1b6c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b6a800640 1 --2- 172.21.15.116:0/568743568 >> v2:172.21.15.149:3300/0 conn(0x7f1b6c0af380 0x7f1b6c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b69400640 1 -- 172.21.15.116:0/568743568 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102318972 0 0) 0x7f1b54001120 con 0x7f1b6c156fa0 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b69400640 1 -- 172.21.15.116:0/568743568 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b50003530 con 0x7f1b6c156fa0 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b61400640 1 -- 172.21.15.116:0/568743568 >> v2:172.21.15.149:3300/0 conn(0x7f1b6c0af380 msgr2=0x7f1b6c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b61400640 1 --2- 172.21.15.116:0/568743568 >> v2:172.21.15.149:3300/0 conn(0x7f1b6c0af380 0x7f1b6c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b61400640 1 -- 172.21.15.116:0/568743568 >> v1:172.21.15.116:6789/0 conn(0x7f1b6c156fa0 legacy=0x7f1b6c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.325+0000 7f1b61400640 1 -- 172.21.15.116:0/568743568 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c160510 con 0x7f1b6c0d1670 2024-05-14T11:26:30.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b61400640 1 --2- 172.21.15.116:0/568743568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c0d1a80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c074e70 tx=0x7f1b5c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b9a0e570feab32 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b69400640 1 -- 172.21.15.116:0/568743568 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0b0070 con 0x7f1b6c0d1670 2024-05-14T11:26:30.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b69400640 1 -- 172.21.15.116:0/568743568 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c0ab600 con 0x7f1b6c0d1670 2024-05-14T11:26:30.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b69400640 1 -- 172.21.15.116:0/568743568 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0ab9b0 con 0x7f1b6c0d1670 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 -- 172.21.15.116:0/568743568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 msgr2=0x7f1b6c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 --2- 172.21.15.116:0/568743568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c0d1a80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c074e70 tx=0x7f1b5c01c480 comp rx=0 tx=0).stop 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 -- 172.21.15.116:0/568743568 shutdown_connections 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 --2- 172.21.15.116:0/568743568 >> v2:172.21.15.149:3300/0 conn(0x7f1b6c0af380 0x7f1b6c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 --2- 172.21.15.116:0/568743568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c0d1a80 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 -- 172.21.15.116:0/568743568 >> 172.21.15.116:0/568743568 conn(0x7f1b6c0b4620 msgr2=0x7f1b6c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 -- 172.21.15.116:0/568743568 shutdown_connections 2024-05-14T11:26:30.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.329+0000 7f1b71000640 1 -- 172.21.15.116:0/568743568 wait complete. 2024-05-14T11:26:30.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 Processor -- start 2024-05-14T11:26:30.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- start start 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1b6c0af380 0x7f1b6c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b6c156fa0 0x7f1b6c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0b0660 con 0x7f1b6c0d1670 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0b04e0 con 0x7f1b6c156fa0 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c06c700 con 0x7f1b6c0af380 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40760/0 (socket says 172.21.15.116:40760) 2024-05-14T11:26:30.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 -- 172.21.15.116:0/835241364 learned_addr learned my addr 172.21.15.116:0/835241364 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b69e00640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1b6c0af380 0x7f1b6c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b6a800640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b6c156fa0 0x7f1b6c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 -- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1b6c0af380 msgr2=0x7f1b6c14d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1b6c0af380 0x7f1b6c14d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 -- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b6c156fa0 msgr2=0x7f1b6c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b6c156fa0 0x7f1b6c1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 -- 172.21.15.116:0/835241364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c15c530 con 0x7f1b6c0d1670 2024-05-14T11:26:30.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b61400640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0ae040 tx=0x7f1b5c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0bc070 con 0x7f1b6c0d1670 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c0b1040 con 0x7f1b6c0d1670 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c0737b0 con 0x7f1b6c0d1670 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0b0050 con 0x7f1b6c0d1670 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.333+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c073d70 con 0x7f1b6c0d1670 2024-05-14T11:26:30.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c06a9c0 con 0x7f1b6c0d1670 2024-05-14T11:26:30.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1b5c0bf070 con 0x7f1b6c0d1670 2024-05-14T11:26:30.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b62a00640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 0x7f1b48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f1b5c128f00 con 0x7f1b6c0d1670 2024-05-14T11:26:30.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b69e00640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 0x7f1b48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.337+0000 7f1b69e00640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 0x7f1b48071910 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007fd0 tx=0x7f1b54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.341+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1b5c016070 con 0x7f1b6c0d1670 2024-05-14T11:26:30.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.505+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b6c073540 con 0x7f1b6c0d1670 2024-05-14T11:26:30.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b62a00640 1 -- 172.21.15.116:0/835241364 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17263 (secure 0 0 0) 0x7f1b5c0f5ba0 con 0x7f1b6c0d1670 2024-05-14T11:26:30.507 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:30.507 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":115,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:26:18.174998+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-14T11:25:39.509510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-14T11:25:40.801825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6837/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6836/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6837/1742968683":"2024-05-15T11:26:01.594455+0000","172.21.15.116:6836/1742968683":"2024-05-15T11:26:01.594455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:26:30.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 msgr2=0x7f1b48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 0x7f1b48071910 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007fd0 tx=0x7f1b54009040 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 msgr2=0x7f1b6c14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0ae040 tx=0x7f1b5c01acc0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 shutdown_connections 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f1b4806f400 0x7f1b48071910 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b6c156fa0 0x7f1b6c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b6c0d1670 0x7f1b6c14da10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 --2- 172.21.15.116:0/835241364 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1b6c0af380 0x7f1b6c14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 >> 172.21.15.116:0/835241364 conn(0x7f1b6c0b4620 msgr2=0x7f1b6c0b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 shutdown_connections 2024-05-14T11:26:30.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.509+0000 7f1b71000640 1 -- 172.21.15.116:0/835241364 wait complete. 2024-05-14T11:26:30.530 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-14T11:26:30.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 Processor -- start 2024-05-14T11:26:30.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 -- start start 2024-05-14T11:26:30.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f08640d1670 0x7f08640d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 0x7f08640af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640d1fc0 con 0x7f0864156fa0 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f08640706f0 con 0x7f08640d1670 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086c400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f08641573b0 con 0x7f08640af380 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f0869a00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f0864156fa0 0x7f086406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:45792/0 (socket says 172.21.15.116:45792) 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f0869a00640 1 -- 172.21.15.116:0/1880298671 learned_addr learned my addr 172.21.15.116:0/1880298671 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086a400640 1 --2- 172.21.15.116:0/1880298671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 0x7f08640af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f0869000640 1 --2- 172.21.15.116:0/1880298671 >> v2:172.21.15.149:3300/0 conn(0x7f08640d1670 0x7f08640d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f0863e00640 1 -- 172.21.15.116:0/1880298671 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078923338 0 0) 0x7f0858001120 con 0x7f0864156fa0 2024-05-14T11:26:30.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086a400640 1 -- 172.21.15.116:0/1880298671 >> v2:172.21.15.149:3300/0 conn(0x7f08640d1670 msgr2=0x7f08640d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:30.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086a400640 1 --2- 172.21.15.116:0/1880298671 >> v2:172.21.15.149:3300/0 conn(0x7f08640d1670 0x7f08640d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086a400640 1 -- 172.21.15.116:0/1880298671 >> v1:172.21.15.149:6789/0 conn(0x7f0864156fa0 legacy=0x7f086406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.681+0000 7f086a400640 1 -- 172.21.15.116:0/1880298671 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0864160510 con 0x7f08640af380 2024-05-14T11:26:30.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086a400640 1 --2- 172.21.15.116:0/1880298671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 0x7f08640af790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f085c009d90 tx=0x7f085c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ba6bbc55070338 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f0863e00640 1 -- 172.21.15.116:0/1880298671 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0b2070 con 0x7f08640af380 2024-05-14T11:26:30.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f0863e00640 1 -- 172.21.15.116:0/1880298671 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c0a5920 con 0x7f08640af380 2024-05-14T11:26:30.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f0863e00640 1 -- 172.21.15.116:0/1880298671 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0a5cf0 con 0x7f08640af380 2024-05-14T11:26:30.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 -- 172.21.15.116:0/1880298671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 msgr2=0x7f08640af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 --2- 172.21.15.116:0/1880298671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 0x7f08640af790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f085c009d90 tx=0x7f085c01c970 comp rx=0 tx=0).stop 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 -- 172.21.15.116:0/1880298671 shutdown_connections 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 --2- 172.21.15.116:0/1880298671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640af380 0x7f08640af790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 --2- 172.21.15.116:0/1880298671 >> v2:172.21.15.149:3300/0 conn(0x7f08640d1670 0x7f08640d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 -- 172.21.15.116:0/1880298671 >> 172.21.15.116:0/1880298671 conn(0x7f08640b4620 msgr2=0x7f08640b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 -- 172.21.15.116:0/1880298671 shutdown_connections 2024-05-14T11:26:30.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.685+0000 7f086c400640 1 -- 172.21.15.116:0/1880298671 wait complete. 2024-05-14T11:26:30.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 Processor -- start 2024-05-14T11:26:30.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 -- start start 2024-05-14T11:26:30.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640d1670 0x7f086415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0864156fa0 0x7f086415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f08640b0660 con 0x7f08640af380 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08640b04e0 con 0x7f0864156fa0 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086c400640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0864169810 con 0x7f08640d1670 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f0869a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f0869a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40772/0 (socket says 172.21.15.116:40772) 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f0869a00640 1 -- 172.21.15.116:0/843765206 learned_addr learned my addr 172.21.15.116:0/843765206 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f0869000640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640d1670 0x7f086415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.689+0000 7f086a400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0864156fa0 0x7f086415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 -- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640d1670 msgr2=0x7f086415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640d1670 0x7f086415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 -- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0864156fa0 msgr2=0x7f086415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0864156fa0 0x7f086415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 -- 172.21.15.116:0/843765206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08641501b0 con 0x7f08640af380 2024-05-14T11:26:30.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0869a00640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f085800c3c0 tx=0x7f085800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580074e0 con 0x7f08640af380 2024-05-14T11:26:30.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08580047f0 con 0x7f08640af380 2024-05-14T11:26:30.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08640cd870 con 0x7f08640af380 2024-05-14T11:26:30.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858007680 con 0x7f08640af380 2024-05-14T11:26:30.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08640ce7d0 con 0x7f08640af380 2024-05-14T11:26:30.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086406a9c0 con 0x7f08640af380 2024-05-14T11:26:30.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.693+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0858011070 con 0x7f08640af380 2024-05-14T11:26:30.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.697+0000 7f0861600640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 0x7f0844071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:30.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.697+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6273+0+0 (secure 0 0 0) 0x7f085808efe0 con 0x7f08640af380 2024-05-14T11:26:30.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.697+0000 7f0869000640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 0x7f0844071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:30.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.697+0000 7f0869000640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 0x7f0844071910 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f086414f6f0 tx=0x7f0850009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:30.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.701+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f08640d1fc0 con 0x7f08640af380 2024-05-14T11:26:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.889+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f086415fe40 con 0x7f08640af380 2024-05-14T11:26:30.890 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-05-14T11:26:30.889+0000 7f7ecd600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T11:26:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.893+0000 7f0861600640 1 -- 172.21.15.116:0/843765206 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f085805b5a0 con 0x7f08640af380 2024-05-14T11:26:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 msgr2=0x7f0844071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 0x7f0844071910 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f086414f6f0 tx=0x7f0850009190 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 msgr2=0x7f08640ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f085800c3c0 tx=0x7f085800c880 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 shutdown_connections 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f084406f400 0x7f0844071910 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0864156fa0 0x7f086415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08640d1670 0x7f086415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 --2- 172.21.15.116:0/843765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08640af380 0x7f08640ce290 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 >> 172.21.15.116:0/843765206 conn(0x7f08640b4620 msgr2=0x7f08640b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 shutdown_connections 2024-05-14T11:26:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:30.897+0000 7f086c400640 1 -- 172.21.15.116:0/843765206 wait complete. 2024-05-14T11:26:30.900 INFO:tasks.ceph.mds.c.smithi116.stderr: -1378> 2024-05-14T11:26:24.189+0000 7fa9fc000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T11:26:30.918 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T11:26:31.025 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-05-14T11:26:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 Processor -- start 2024-05-14T11:26:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 -- start start 2024-05-14T11:26:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 0x7f9c80073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c80073ea0 0x7f9c80163f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c800d1410 con 0x7f9c800d0610 2024-05-14T11:26:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9c800742b0 con 0x7f9c80073ea0 2024-05-14T11:26:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c86600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9c800b6930 con 0x7f9c800d0200 2024-05-14T11:26:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c80073ea0 0x7f9c80163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c7fe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9c800d0610 0x7f9c80163820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36864/0 (socket says 172.21.15.116:36864) 2024-05-14T11:26:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c7fe00640 1 -- 172.21.15.116:0/2225665826 learned_addr learned my addr 172.21.15.116:0/2225665826 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85200640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 0x7f9c80073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85c00640 1 -- 172.21.15.116:0/2225665826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 msgr2=0x7f9c80073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85c00640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 0x7f9c80073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85c00640 1 -- 172.21.15.116:0/2225665826 >> v1:172.21.15.116:6789/0 conn(0x7f9c800d0610 legacy=0x7f9c80163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85c00640 1 -- 172.21.15.116:0/2225665826 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c800d0b70 con 0x7f9c80073ea0 2024-05-14T11:26:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c85200640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 0x7f9c80073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.073+0000 7f9c7f400640 1 -- 172.21.15.116:0/2225665826 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646872553 0 0) 0x7f9c800d1410 con 0x7f9c800d0610 2024-05-14T11:26:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c85c00640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80073ea0 0x7f9c80163f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c7801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2580aa682e6523d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c7f400640 1 -- 172.21.15.116:0/2225665826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b2070 con 0x7f9c80073ea0 2024-05-14T11:26:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c7f400640 1 -- 172.21.15.116:0/2225665826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c780a5a00 con 0x7f9c80073ea0 2024-05-14T11:26:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c7f400640 1 -- 172.21.15.116:0/2225665826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780a5dd0 con 0x7f9c80073ea0 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 -- 172.21.15.116:0/2225665826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80073ea0 msgr2=0x7f9c80163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80073ea0 0x7f9c80163f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c7801c940 comp rx=0 tx=0).stop 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 -- 172.21.15.116:0/2225665826 shutdown_connections 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80073ea0 0x7f9c80163f30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 --2- 172.21.15.116:0/2225665826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c800d0200 0x7f9c80073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 -- 172.21.15.116:0/2225665826 >> 172.21.15.116:0/2225665826 conn(0x7f9c800b4c20 msgr2=0x7f9c800d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 -- 172.21.15.116:0/2225665826 shutdown_connections 2024-05-14T11:26:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.077+0000 7f9c86600640 1 -- 172.21.15.116:0/2225665826 wait complete. 2024-05-14T11:26:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 Processor -- start 2024-05-14T11:26:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 -- start start 2024-05-14T11:26:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c80073ea0 0x7f9c80158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 0x7f9c80153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9c800af9f0 con 0x7f9c800d0610 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c800af870 con 0x7f9c800d0200 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c86600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9c80164470 con 0x7f9c80073ea0 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40812/0 (socket says 172.21.15.116:40812) 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 -- 172.21.15.116:0/2141366634 learned_addr learned my addr 172.21.15.116:0/2141366634 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85200640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c80073ea0 0x7f9c80158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 -- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c80073ea0 msgr2=0x7f9c80158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c80073ea0 0x7f9c80158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 -- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 msgr2=0x7f9c80153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c7fe00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 0x7f9c80153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 0x7f9c80153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c85c00640 1 -- 172.21.15.116:0/2141366634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c80158100 con 0x7f9c800d0610 2024-05-14T11:26:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.081+0000 7f9c7fe00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 0x7f9c80153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c85c00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c780196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b2050 con 0x7f9c800d0610 2024-05-14T11:26:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c780b3040 con 0x7f9c800d0610 2024-05-14T11:26:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801583e0 con 0x7f9c800d0610 2024-05-14T11:26:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780ad930 con 0x7f9c800d0610 2024-05-14T11:26:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8006d710 con 0x7f9c800d0610 2024-05-14T11:26:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8006a9c0 con 0x7f9c800d0610 2024-05-14T11:26:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c780c0020 con 0x7f9c800d0610 2024-05-14T11:26:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 0x7f9c60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.085+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6359+0+0 (secure 0 0 0) 0x7f9c78129ee0 con 0x7f9c800d0610 2024-05-14T11:26:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.089+0000 7f9c85200640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 0x7f9c60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.089+0000 7f9c85200640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 0x7f9c60071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c801648e0 tx=0x7f9c74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:31.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.093+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9c780c7020 con 0x7f9c800d0610 2024-05-14T11:26:31.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.289+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9c80070c80 con 0x7f9c800d0610 2024-05-14T11:26:31.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c7cc00640 1 -- 172.21.15.116:0/2141366634 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f9c78016560 con 0x7f9c800d0610 2024-05-14T11:26:31.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 msgr2=0x7f9c60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 0x7f9c60071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c801648e0 tx=0x7f9c74008040 comp rx=0 tx=0).stop 2024-05-14T11:26:31.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 msgr2=0x7f9c80155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:31.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c780196e0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 shutdown_connections 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f9c6006f400 0x7f9c60071910 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c800d0200 0x7f9c80153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c800d0610 0x7f9c80155890 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 --2- 172.21.15.116:0/2141366634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9c80073ea0 0x7f9c80158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 >> 172.21.15.116:0/2141366634 conn(0x7f9c800b4c20 msgr2=0x7f9c801565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 shutdown_connections 2024-05-14T11:26:31.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:31.897+0000 7f9c86600640 1 -- 172.21.15.116:0/2141366634 wait complete. 2024-05-14T11:26:31.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T11:26:32.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 Processor -- start 2024-05-14T11:26:32.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 -- start start 2024-05-14T11:26:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe92c070640 0x7fe92c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe92c0d0290 0x7fe92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c15d360 con 0x7fe92c070a50 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c15dc90 con 0x7fe92c0d0290 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe930a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c15d640 con 0x7fe92c070640 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe92c0d0290 0x7fe92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe92c0d0290 0x7fe92c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40824/0 (socket says 172.21.15.116:40824) 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 -- 172.21.15.116:0/274156300 learned_addr learned my addr 172.21.15.116:0/274156300 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe929e00640 1 --2- 172.21.15.116:0/274156300 >> v2:172.21.15.149:3300/0 conn(0x7fe92c070640 0x7fe92c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 -- 172.21.15.116:0/274156300 >> v2:172.21.15.149:3300/0 conn(0x7fe92c070640 msgr2=0x7fe92c06e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 --2- 172.21.15.116:0/274156300 >> v2:172.21.15.149:3300/0 conn(0x7fe92c070640 0x7fe92c06e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 -- 172.21.15.116:0/274156300 >> v1:172.21.15.149:6789/0 conn(0x7fe92c070a50 legacy=0x7fe92c1421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.073+0000 7fe92a800640 1 -- 172.21.15.116:0/274156300 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c15d920 con 0x7fe92c0d0290 2024-05-14T11:26:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe92a800640 1 --2- 172.21.15.116:0/274156300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c06edb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0722c0 tx=0x7fe92401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597f63160e8848c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe928a00640 1 -- 172.21.15.116:0/274156300 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240a5720 con 0x7fe92c0d0290 2024-05-14T11:26:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe928a00640 1 -- 172.21.15.116:0/274156300 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9240a58c0 con 0x7fe92c0d0290 2024-05-14T11:26:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe928a00640 1 -- 172.21.15.116:0/274156300 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240a5c90 con 0x7fe92c0d0290 2024-05-14T11:26:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 -- 172.21.15.116:0/274156300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 msgr2=0x7fe92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 --2- 172.21.15.116:0/274156300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c06edb0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0722c0 tx=0x7fe92401c960 comp rx=0 tx=0).stop 2024-05-14T11:26:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 -- 172.21.15.116:0/274156300 shutdown_connections 2024-05-14T11:26:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 --2- 172.21.15.116:0/274156300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c06edb0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 --2- 172.21.15.116:0/274156300 >> v2:172.21.15.149:3300/0 conn(0x7fe92c070640 0x7fe92c06e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 -- 172.21.15.116:0/274156300 >> 172.21.15.116:0/274156300 conn(0x7fe92c0cde80 msgr2=0x7fe92c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 -- 172.21.15.116:0/274156300 shutdown_connections 2024-05-14T11:26:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.077+0000 7fe930a00640 1 -- 172.21.15.116:0/274156300 wait complete. 2024-05-14T11:26:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 Processor -- start 2024-05-14T11:26:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 -- start start 2024-05-14T11:26:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c070640 0x7fe92c138700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c070a50 0x7fe92c13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1699a0 con 0x7fe92c0d0290 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c0cda10 con 0x7fe92c070a50 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe930a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c169b20 con 0x7fe92c070640 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe92a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe92a800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40826/0 (socket says 172.21.15.116:40826) 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe92a800640 1 -- 172.21.15.116:0/520699322 learned_addr learned my addr 172.21.15.116:0/520699322 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.081+0000 7fe929400640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c070a50 0x7fe92c13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe929e00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c070640 0x7fe92c138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 -- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c070640 msgr2=0x7fe92c138700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c070640 0x7fe92c138700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 -- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c070a50 msgr2=0x7fe92c13a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c070a50 0x7fe92c13a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 -- 172.21.15.116:0/520699322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0b2890 con 0x7fe92c0d0290 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe92a800640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe92401d200 tx=0x7fe924019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924009510 con 0x7fe92c0d0290 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9240bc070 con 0x7fe92c0d0290 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c0b17f0 con 0x7fe92c0d0290 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240b68c0 con 0x7fe92c0d0290 2024-05-14T11:26:32.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c0b1db0 con 0x7fe92c0d0290 2024-05-14T11:26:32.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c06aa30 con 0x7fe92c0d0290 2024-05-14T11:26:32.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.085+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9240b6a60 con 0x7fe92c0d0290 2024-05-14T11:26:32.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.089+0000 7fe922000640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:32.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.089+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6359+0+0 (secure 0 0 0) 0x7fe9241299b0 con 0x7fe92c0d0290 2024-05-14T11:26:32.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.089+0000 7fe929e00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:32.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.089+0000 7fe929e00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0d0f00 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:32.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.093+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe9240c5020 con 0x7fe92c0d0290 2024-05-14T11:26:32.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.249+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe92c06cd30 con 0x7fe92c0d0290 2024-05-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.897+0000 7fe922000640 1 -- 172.21.15.116:0/520699322 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7fe9240165e0 con 0x7fe92c0d0290 2024-05-14T11:26:32.897 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-05-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 msgr2=0x7fe90c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:32.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0d0f00 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 msgr2=0x7fe92c13a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe92401d200 tx=0x7fe924019700 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 shutdown_connections 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe90c06f430 0x7fe90c071940 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe92c070a50 0x7fe92c13a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe92c0d0290 0x7fe92c13a920 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 --2- 172.21.15.116:0/520699322 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe92c070640 0x7fe92c138700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 >> 172.21.15.116:0/520699322 conn(0x7fe92c0cde80 msgr2=0x7fe92c141510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 shutdown_connections 2024-05-14T11:26:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:32.901+0000 7fe930a00640 1 -- 172.21.15.116:0/520699322 wait complete. 2024-05-14T11:26:32.925 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T11:26:33.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 Processor -- start 2024-05-14T11:26:33.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 -- start start 2024-05-14T11:26:33.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f060c06cf90 0x7f060c06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c06d8e0 con 0x7f060c0b5f30 2024-05-14T11:26:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c15c800 con 0x7f060c15fce0 2024-05-14T11:26:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0613a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f060c15d0f0 con 0x7f060c06cf90 2024-05-14T11:26:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0611000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f060c0b5f30 0x7f060c0b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36872/0 (socket says 172.21.15.116:36872) 2024-05-14T11:26:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0611000640 1 -- 172.21.15.116:0/3827108794 learned_addr learned my addr 172.21.15.116:0/3827108794 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795111500 0 0) 0x7f060c06d8e0 con 0x7f060c0b5f30 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f0603e00640 1 --2- 172.21.15.116:0/3827108794 >> v2:172.21.15.149:3300/0 conn(0x7f060c06cf90 0x7f060c06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.081+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05f0003530 con 0x7f060c0b5f30 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361174697 0 0) 0x7f05f0003530 con 0x7f060c0b5f30 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 >> v2:172.21.15.149:3300/0 conn(0x7f060c06cf90 msgr2=0x7f060c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 --2- 172.21.15.116:0/3827108794 >> v2:172.21.15.149:3300/0 conn(0x7f060c06cf90 0x7f060c06d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 >> v1:172.21.15.149:6789/0 conn(0x7f060c15fce0 legacy=0x7f060c1600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c15caa0 con 0x7f060c0b5f30 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f05fc001700 con 0x7f060c0b5f30 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 >> v1:172.21.15.116:6789/0 conn(0x7f060c0b5f30 legacy=0x7f060c0b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 0x7f05f0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0004810 0x7f05f0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05f0005280 0x7f05f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f060c15d0f0 con 0x7f05f0003a60 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05f000c210 con 0x7f05f0005280 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f05f000c410 con 0x7f05f0004810 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 0x7f05f0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0603e00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0004810 0x7f05f0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611a00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05f0005280 0x7f05f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 -- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0004810 msgr2=0x7f05f0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0004810 0x7f05f0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 -- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05f0005280 msgr2=0x7f05f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05f0005280 0x7f05f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611a00640 1 -- 172.21.15.116:0/3827108794 reap_dead start 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 -- 172.21.15.116:0/3827108794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f0003530 con 0x7f05f0003a60 2024-05-14T11:26:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0603e00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0004810 0x7f05f0004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:33.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.085+0000 7f0611000640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 0x7f05f0003ef0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001060 tx=0x7f05fc01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf78c09d437bab55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc007d20 con 0x7f05f0003a60 2024-05-14T11:26:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc0a9020 con 0x7f05f0003a60 2024-05-14T11:26:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f060be00640 1 -- 172.21.15.116:0/3827108794 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005b20 con 0x7f05f0003a60 2024-05-14T11:26:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 msgr2=0x7f05f0003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 0x7f05f0003ef0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001060 tx=0x7f05fc01cba0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- 172.21.15.116:0/3827108794 shutdown_connections 2024-05-14T11:26:33.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 --2- 172.21.15.116:0/3827108794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f0003a60 0x7f05f0003ef0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- 172.21.15.116:0/3827108794 >> 172.21.15.116:0/3827108794 conn(0x7f060c0cde80 msgr2=0x7f060c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:33.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- 172.21.15.116:0/3827108794 shutdown_connections 2024-05-14T11:26:33.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- 172.21.15.116:0/3827108794 wait complete. 2024-05-14T11:26:33.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 Processor -- start 2024-05-14T11:26:33.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.089+0000 7f0613a00640 1 -- start start 2024-05-14T11:26:33.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0003a60 0x7f060c1384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 0x7f060c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f060c0b6750 con 0x7f060c13ae00 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f060c0b5200 con 0x7f060c0c68e0 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f060c13ab00 con 0x7f05f0003a60 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40850/0 (socket says 172.21.15.116:40850) 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 -- 172.21.15.116:0/2843546831 learned_addr learned my addr 172.21.15.116:0/2843546831 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0611000640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0003a60 0x7f060c1384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0611a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 0x7f060c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 -- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0003a60 msgr2=0x7f060c1384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0003a60 0x7f060c1384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 -- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 msgr2=0x7f060c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 0x7f060c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 -- 172.21.15.116:0/2843546831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c0c6cf0 con 0x7f060c13ae00 2024-05-14T11:26:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0611a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 0x7f060c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0603e00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f05f4011540 tx=0x7f05f4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004220 con 0x7f060c13ae00 2024-05-14T11:26:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f40043c0 con 0x7f060c13ae00 2024-05-14T11:26:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c13b900 con 0x7f060c13ae00 2024-05-14T11:26:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4015890 con 0x7f060c13ae00 2024-05-14T11:26:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c138fc0 con 0x7f060c13ae00 2024-05-14T11:26:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.093+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c138db0 con 0x7f060c13ae00 2024-05-14T11:26:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.097+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f05f4015a30 con 0x7f060c13ae00 2024-05-14T11:26:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.097+0000 7f0609600640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 0x7f05e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.097+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5921+0+0 (secure 0 0 0) 0x7f05f408f700 con 0x7f060c13ae00 2024-05-14T11:26:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.097+0000 7f0611000640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 0x7f05e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.097+0000 7f0611000640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 0x7f05e8071910 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0aa000 tx=0x7f05fc000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:33.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.101+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f060c0c6cf0 con 0x7f060c13ae00 2024-05-14T11:26:33.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:33.261+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f060c06aa30 con 0x7f060c13ae00 2024-05-14T11:26:34.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.253+0000 7f0609600640 1 -- 172.21.15.116:0/2843546831 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7f05f4019030 con 0x7f060c13ae00 2024-05-14T11:26:34.255 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-05-14T11:26:34.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 msgr2=0x7f05e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 0x7f05e8071910 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0aa000 tx=0x7f05fc000ff0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 msgr2=0x7f060c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f05f4011540 tx=0x7f05f4011a00 comp rx=0 tx=0).stop 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 shutdown_connections 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f05e806f400 0x7f05e8071910 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f060c0c68e0 0x7f060c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f060c13ae00 0x7f060c0c63a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 --2- 172.21.15.116:0/2843546831 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05f0003a60 0x7f060c1384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 >> 172.21.15.116:0/2843546831 conn(0x7f060c0cde80 msgr2=0x7f060c0b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 shutdown_connections 2024-05-14T11:26:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.257+0000 7f0613a00640 1 -- 172.21.15.116:0/2843546831 wait complete. 2024-05-14T11:26:34.285 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-14T11:26:34.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.437+0000 7f7d78000640 1 Processor -- start 2024-05-14T11:26:34.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.437+0000 7f7d78000640 1 -- start start 2024-05-14T11:26:34.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d78000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f7d7015db30 0x7f7d700aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d78000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7006ff20 con 0x7f7d7006c700 2024-05-14T11:26:34.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d78000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d70070470 con 0x7f7d700d1670 2024-05-14T11:26:34.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d78000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f7d700ae100 con 0x7f7d7015db30 2024-05-14T11:26:34.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d76000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7d7006c700 0x7f7d7006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36874/0 (socket says 172.21.15.116:36874) 2024-05-14T11:26:34.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d76000640 1 -- 172.21.15.116:0/3168061083 learned_addr learned my addr 172.21.15.116:0/3168061083 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d75600640 1 --2- 172.21.15.116:0/3168061083 >> v2:172.21.15.149:3300/0 conn(0x7f7d7015db30 0x7f7d700aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201592234 0 0) 0x7f7d7006ff20 con 0x7f7d7006c700 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d58003530 con 0x7f7d7006c700 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935825203 0 0) 0x7f7d70070470 con 0x7f7d700d1670 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d7006ff20 con 0x7f7d700d1670 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d75600640 1 -- 172.21.15.116:0/3168061083 >> v1:172.21.15.149:6789/0 conn(0x7f7d700d1670 legacy=0x7f7d700d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d75600640 1 -- 172.21.15.116:0/3168061083 >> v1:172.21.15.116:6789/0 conn(0x7f7d7006c700 legacy=0x7f7d7006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484344841 0 0) 0x7f7d58003530 con 0x7f7d7006c700 2024-05-14T11:26:34.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.441+0000 7f7d75600640 1 -- 172.21.15.116:0/3168061083 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d70070750 con 0x7f7d7015db30 2024-05-14T11:26:34.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d75600640 1 --2- 172.21.15.116:0/3168061083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d700aeaa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ae280 tx=0x7f7d6401c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543d659244bab0db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68001700 con 0x7f7d7015db30 2024-05-14T11:26:34.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d640a6430 con 0x7f7d7015db30 2024-05-14T11:26:34.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d6fe00640 1 -- 172.21.15.116:0/3168061083 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640a67e0 con 0x7f7d7015db30 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- 172.21.15.116:0/3168061083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 msgr2=0x7f7d700aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 --2- 172.21.15.116:0/3168061083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d700aeaa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ae280 tx=0x7f7d6401c7e0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- 172.21.15.116:0/3168061083 shutdown_connections 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 --2- 172.21.15.116:0/3168061083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d700aeaa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- 172.21.15.116:0/3168061083 >> 172.21.15.116:0/3168061083 conn(0x7f7d700b4620 msgr2=0x7f7d700b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- 172.21.15.116:0/3168061083 shutdown_connections 2024-05-14T11:26:34.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- 172.21.15.116:0/3168061083 wait complete. 2024-05-14T11:26:34.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 Processor -- start 2024-05-14T11:26:34.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.445+0000 7f7d78000640 1 -- start start 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d7006c700 0x7f7d700cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d70160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7016bf90 con 0x7f7d700d1670 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7015df40 con 0x7f7d7015db30 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7016c110 con 0x7f7d7006c700 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40864/0 (socket says 172.21.15.116:40864) 2024-05-14T11:26:34.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 -- 172.21.15.116:0/1761213557 learned_addr learned my addr 172.21.15.116:0/1761213557 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d75600640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d7006c700 0x7f7d700cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d76000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d70160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 -- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d7006c700 msgr2=0x7f7d700cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d7006c700 0x7f7d700cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 -- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 msgr2=0x7f7d70160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d70160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 -- 172.21.15.116:0/1761213557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d70151c70 con 0x7f7d700d1670 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d76000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d70160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d74c00640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c430 tx=0x7f7d5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c017070 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c0052d0 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d70166fa0 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003840 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d70168b00 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.449+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d701519e0 con 0x7f7d700d1670 2024-05-14T11:26:34.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.453+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d5c01a070 con 0x7f7d700d1670 2024-05-14T11:26:34.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.453+0000 7f7d6d600640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 0x7f7d50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.453+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f7d5c08e530 con 0x7f7d700d1670 2024-05-14T11:26:34.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.453+0000 7f7d75600640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 0x7f7d50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.453+0000 7f7d75600640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 0x7f7d50071940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7d640af000 tx=0x7f7d640a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.457+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7d5c005480 con 0x7f7d700d1670 2024-05-14T11:26:34.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.625+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7d7006a9c0 con 0x7f7d700d1670 2024-05-14T11:26:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.629+0000 7f7d6d600640 1 -- 172.21.15.116:0/1761213557 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f7d5c05b560 con 0x7f7d700d1670 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 msgr2=0x7f7d50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 0x7f7d50071940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7d640af000 tx=0x7f7d640a7000 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 msgr2=0x7f7d70160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c430 tx=0x7f7d5c00c8f0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 shutdown_connections 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f7d5006f430 0x7f7d50071940 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d7015db30 0x7f7d70160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d700d1670 0x7f7d70160130 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 --2- 172.21.15.116:0/1761213557 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d7006c700 0x7f7d700cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 >> 172.21.15.116:0/1761213557 conn(0x7f7d700b4620 msgr2=0x7f7d70073110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 shutdown_connections 2024-05-14T11:26:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.633+0000 7f7d78000640 1 -- 172.21.15.116:0/1761213557 wait complete. 2024-05-14T11:26:34.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-14T11:26:34.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 Processor -- start 2024-05-14T11:26:34.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 -- start start 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd9540b0770 0x7fd95406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd954072a30 0x7fd954072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95406cc40 con 0x7fd95406f730 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd95406db20 con 0x7fd954072a30 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd958a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd95406cf10 con 0x7fd9540b0770 2024-05-14T11:26:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd951400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd95406f730 0x7fd95406fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36890/0 (socket says 172.21.15.116:36890) 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd951400640 1 -- 172.21.15.116:0/1373600580 learned_addr learned my addr 172.21.15.116:0/1373600580 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.797+0000 7fd952800640 1 --2- 172.21.15.116:0/1373600580 >> v2:172.21.15.116:3300/0 conn(0x7fd954072a30 0x7fd954072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd951e00640 1 --2- 172.21.15.116:0/1373600580 >> v2:172.21.15.149:3300/0 conn(0x7fd9540b0770 0x7fd95406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd952800640 1 -- 172.21.15.116:0/1373600580 >> v2:172.21.15.149:3300/0 conn(0x7fd9540b0770 msgr2=0x7fd95406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd952800640 1 --2- 172.21.15.116:0/1373600580 >> v2:172.21.15.149:3300/0 conn(0x7fd9540b0770 0x7fd95406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd952800640 1 -- 172.21.15.116:0/1373600580 >> v1:172.21.15.116:6789/0 conn(0x7fd95406f730 legacy=0x7fd95406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd950a00640 1 -- 172.21.15.116:0/1373600580 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457032353 0 0) 0x7fd95406cc40 con 0x7fd95406f730 2024-05-14T11:26:34.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd952800640 1 -- 172.21.15.116:0/1373600580 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95406d1f0 con 0x7fd954072a30 2024-05-14T11:26:34.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd952800640 1 --2- 172.21.15.116:0/1373600580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd954072a30 0x7fd954072e40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd954071f20 tx=0x7fd94c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2578fe75bf305833 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd950a00640 1 -- 172.21.15.116:0/1373600580 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0b2070 con 0x7fd954072a30 2024-05-14T11:26:34.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd950a00640 1 -- 172.21.15.116:0/1373600580 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c0a57a0 con 0x7fd954072a30 2024-05-14T11:26:34.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.801+0000 7fd950a00640 1 -- 172.21.15.116:0/1373600580 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0a5b70 con 0x7fd954072a30 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- 172.21.15.116:0/1373600580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd954072a30 msgr2=0x7fd954072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 --2- 172.21.15.116:0/1373600580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd954072a30 0x7fd954072e40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd954071f20 tx=0x7fd94c01c870 comp rx=0 tx=0).stop 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- 172.21.15.116:0/1373600580 shutdown_connections 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 --2- 172.21.15.116:0/1373600580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd954072a30 0x7fd954072e40 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 --2- 172.21.15.116:0/1373600580 >> v2:172.21.15.149:3300/0 conn(0x7fd9540b0770 0x7fd95406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- 172.21.15.116:0/1373600580 >> 172.21.15.116:0/1373600580 conn(0x7fd9540b4620 msgr2=0x7fd9540b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- 172.21.15.116:0/1373600580 shutdown_connections 2024-05-14T11:26:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- 172.21.15.116:0/1373600580 wait complete. 2024-05-14T11:26:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 Processor -- start 2024-05-14T11:26:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.805+0000 7fd958a00640 1 -- start start 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 0x7fd95416a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd9540b0770 0x7fd95416a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd954073500 con 0x7fd95406f730 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd954073380 con 0x7fd954072a30 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd954073680 con 0x7fd9540b0770 2024-05-14T11:26:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40880/0 (socket says 172.21.15.116:40880) 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 -- 172.21.15.116:0/1039606897 learned_addr learned my addr 172.21.15.116:0/1039606897 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951400640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 0x7fd95416a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 -- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd9540b0770 msgr2=0x7fd95416a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd9540b0770 0x7fd95416a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 -- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 msgr2=0x7fd95416a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 0x7fd95416a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 -- 172.21.15.116:0/1039606897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9540cd930 con 0x7fd95406f730 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951400640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 0x7fd95416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd951e00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e870 tx=0x7fd94400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944017070 con 0x7fd95406f730 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94400c940 con 0x7fd95406f730 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9540cdc10 con 0x7fd95406f730 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400cd10 con 0x7fd95406f730 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95415db30 con 0x7fd95406f730 2024-05-14T11:26:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95406a9c0 con 0x7fd95406f730 2024-05-14T11:26:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.809+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd94401a070 con 0x7fd95406f730 2024-05-14T11:26:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.813+0000 7fd94a000640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 0x7fd930071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.813+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fd94408f4f0 con 0x7fd95406f730 2024-05-14T11:26:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.813+0000 7fd951400640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 0x7fd930071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.813+0000 7fd951400640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 0x7fd930071910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd9541fc920 tx=0x7fd93c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:34.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:34.817+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd94405c400 con 0x7fd95406f730 2024-05-14T11:26:35.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.029+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fd95406f510 con 0x7fd95406f730 2024-05-14T11:26:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.261+0000 7fd94a000640 1 -- 172.21.15.116:0/1039606897 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v491) ==== 144+0+0 (secure 0 0 0) 0x7fd94405bd00 con 0x7fd95406f730 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 msgr2=0x7fd930071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 0x7fd930071910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd9541fc920 tx=0x7fd93c009040 comp rx=0 tx=0).stop 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 msgr2=0x7fd954156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e870 tx=0x7fd94400ec70 comp rx=0 tx=0).stop 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 shutdown_connections 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd93006f400 0x7fd930071910 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd954072a30 0x7fd95416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd9540b0770 0x7fd95416a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 --2- 172.21.15.116:0/1039606897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd95406f730 0x7fd954156630 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 >> 172.21.15.116:0/1039606897 conn(0x7fd9540b4620 msgr2=0x7fd95415c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 shutdown_connections 2024-05-14T11:26:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:35.265+0000 7fd958a00640 1 -- 172.21.15.116:0/1039606897 wait complete. 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner: 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner:Ran 7 tests in 581.719s 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner: 2024-05-14T11:26:35.283 INFO:tasks.cephfs_test_runner:OK 2024-05-14T11:26:35.284 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-14T11:26:35.294 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-14T11:26:35.329 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-14T11:26:35.329 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:35.329 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:35.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:26:35.338 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-05-14T11:26:35.338 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:35.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-14T11:26:35.394 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:26:35.394 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:35.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:26:35.403 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-14T11:26:35.403 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T11:26:35.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=8105068k,nr_inodes=2026267,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6541 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1625944k,nr_inodes=406486,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=1625944k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-14T11:26:35.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.452 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-05-14T11:26:35.504 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-14T11:26:35.505 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-14T11:26:35.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.505 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-14T11:26:35.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.570 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-14T11:26:35.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.633 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-05-14T11:26:35.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.783 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-05-14T11:26:35.792 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T11:26:35.792 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-14T11:26:35.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.793 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T11:26:35.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.856 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-14T11:26:35.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T11:26:35.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-05-14T11:26:36.078 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-14T11:26:36.087 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-14T11:26:36.087 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:26:36.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.245+0000 7fd08bc00640 1 Processor -- start 2024-05-14T11:26:36.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.245+0000 7fd08bc00640 1 -- start start 2024-05-14T11:26:36.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd08bc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 0x7fd0840d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd08bc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 0x7fd08406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd08bc00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0840d1fc0 con 0x7fd08406cc40 2024-05-14T11:26:36.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd08bc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fd0841600a0 con 0x7fd0840d1bb0 2024-05-14T11:26:36.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd08bc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd08406f5d0 con 0x7fd084164220 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd08406cc40 0x7fd0840ae2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36894/0 (socket says 172.21.15.116:36894) 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089c00640 1 -- 172.21.15.116:0/4003350366 learned_addr learned my addr 172.21.15.116:0/4003350366 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089200640 1 --2- 172.21.15.116:0/4003350366 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 0x7fd0840d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd07be00640 1 --2- 172.21.15.116:0/4003350366 >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 0x7fd08406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd083e00640 1 -- 172.21.15.116:0/4003350366 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389932817 0 0) 0x7fd07c0014a0 con 0x7fd08406cc40 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd083e00640 1 -- 172.21.15.116:0/4003350366 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd068003530 con 0x7fd08406cc40 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089200640 1 -- 172.21.15.116:0/4003350366 >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 msgr2=0x7fd08406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089200640 1 --2- 172.21.15.116:0/4003350366 >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 0x7fd08406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089200640 1 -- 172.21.15.116:0/4003350366 >> v1:172.21.15.116:6789/0 conn(0x7fd08406cc40 legacy=0x7fd0840ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd089200640 1 -- 172.21.15.116:0/4003350366 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08406f8b0 con 0x7fd084164220 2024-05-14T11:26:36.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.249+0000 7fd07be00640 1 --2- 172.21.15.116:0/4003350366 >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 0x7fd08406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd089200640 1 --2- 172.21.15.116:0/4003350366 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 0x7fd0840d1670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907073d2cbc8d3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd083e00640 1 -- 172.21.15.116:0/4003350366 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b1070 con 0x7fd084164220 2024-05-14T11:26:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd083e00640 1 -- 172.21.15.116:0/4003350366 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0740a8050 con 0x7fd084164220 2024-05-14T11:26:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd083e00640 1 -- 172.21.15.116:0/4003350366 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0740ac860 con 0x7fd084164220 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 -- 172.21.15.116:0/4003350366 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 msgr2=0x7fd0840d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 --2- 172.21.15.116:0/4003350366 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 0x7fd0840d1670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c3e0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 -- 172.21.15.116:0/4003350366 shutdown_connections 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 --2- 172.21.15.116:0/4003350366 >> v2:172.21.15.149:3300/0 conn(0x7fd0840d1bb0 0x7fd08406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 --2- 172.21.15.116:0/4003350366 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd084164220 0x7fd0840d1670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 -- 172.21.15.116:0/4003350366 >> 172.21.15.116:0/4003350366 conn(0x7fd0840b4620 msgr2=0x7fd0840b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 -- 172.21.15.116:0/4003350366 shutdown_connections 2024-05-14T11:26:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.253+0000 7fd08bc00640 1 -- 172.21.15.116:0/4003350366 wait complete. 2024-05-14T11:26:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 Processor -- start 2024-05-14T11:26:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- start start 2024-05-14T11:26:36.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd08406cc40 0x7fd084157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 0x7fd084157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd08406d460 con 0x7fd084164220 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0840701d0 con 0x7fd0840d1bb0 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd084164630 con 0x7fd08406cc40 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40886/0 (socket says 172.21.15.116:40886) 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 -- 172.21.15.116:0/153220899 learned_addr learned my addr 172.21.15.116:0/153220899 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd07be00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 0x7fd084157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089200640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd08406cc40 0x7fd084157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 -- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd08406cc40 msgr2=0x7fd084157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd08406cc40 0x7fd084157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 -- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 msgr2=0x7fd084157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 0x7fd084157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 -- 172.21.15.116:0/153220899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd084071440 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd07be00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 0x7fd084157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd089c00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00cae0 tx=0x7fd07c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00ac40 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c00ade0 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd084156660 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c012620 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.257+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd084156c20 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08406a9c0 con 0x7fd084164220 2024-05-14T11:26:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd07c0127c0 con 0x7fd084164220 2024-05-14T11:26:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd081600640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 0x7fd060071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fd07c08f7b0 con 0x7fd084164220 2024-05-14T11:26:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd089200640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 0x7fd060071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.261+0000 7fd089200640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 0x7fd060071940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ae000 tx=0x7fd0740a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.265+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd0840d1fc0 con 0x7fd084164220 2024-05-14T11:26:36.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.421+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0841582b0 con 0x7fd06006f430 2024-05-14T11:26:36.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.421+0000 7fd081600640 1 -- 172.21.15.116:0/153220899 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7fd0841582b0 con 0x7fd06006f430 2024-05-14T11:26:36.422 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:36.423 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:26:36.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 msgr2=0x7fd060071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 0x7fd060071940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ae000 tx=0x7fd0740a7000 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 msgr2=0x7fd08414baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00cae0 tx=0x7fd07c00a060 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 shutdown_connections 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fd06006f430 0x7fd060071940 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0840d1bb0 0x7fd084157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd084164220 0x7fd08414baa0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 --2- 172.21.15.116:0/153220899 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd08406cc40 0x7fd084157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 >> 172.21.15.116:0/153220899 conn(0x7fd0840b4620 msgr2=0x7fd084072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 shutdown_connections 2024-05-14T11:26:36.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.425+0000 7fd08bc00640 1 -- 172.21.15.116:0/153220899 wait complete. 2024-05-14T11:26:36.444 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-05-14T11:26:34.518626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278256,"kb_used_data":28832,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696464,"statfs":{"total":773094113280,"available":772809179136,"internally_reserved":0,"allocated":29523968,"data_stored":23319072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390143,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-251,"num_write":-171,"num_write_kb":-8290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001220"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":284,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-14T11:26:18.300434+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:26:18.300434+0000","last_peered":"2024-05-14T11:26:18.300434+0000","last_clean":"2024-05-14T11:26:18.300434+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:26:18.300434+0000","last_undegraded":"2024-05-14T11:26:18.300434+0000","last_fullsized":"2024-05-14T11:26:18.300434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2048000,"data_stored":1260349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":5500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":5632000,"data_stored":4864084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36684,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":5636096,"data_stored":4863662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":5648384,"data_stored":4866467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1449984,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":669203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36684,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":5636096,"data_stored":4863529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2035712,"data_stored":1259593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:26:36.444 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:26:36.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 Processor -- start 2024-05-14T11:26:36.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 -- start start 2024-05-14T11:26:36.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c15fb30 0x7f831c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f831c0cdda0 0x7f831c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831c0cc030 con 0x7f831c0cbc20 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f831c0d1580 con 0x7f831c0cdda0 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f8322200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0d0970 con 0x7f831c15fb30 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f831c0cdda0 0x7f831c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f831c0cdda0 0x7f831c0b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40890/0 (socket says 172.21.15.116:40890) 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 -- 172.21.15.116:0/1584198486 learned_addr learned my addr 172.21.15.116:0/1584198486 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831b400640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c15fb30 0x7f831c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 -- 172.21.15.116:0/1584198486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c15fb30 msgr2=0x7f831c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:36.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c15fb30 0x7f831c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 -- 172.21.15.116:0/1584198486 >> v1:172.21.15.149:6789/0 conn(0x7f831c0cbc20 legacy=0x7f831c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.593+0000 7f831be00640 1 -- 172.21.15.116:0/1584198486 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0d0c50 con 0x7f831c0cdda0 2024-05-14T11:26:36.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f831be00640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cdda0 0x7f831c0b5c50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8314009e50 tx=0x7f831401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac691d05203eb07c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f831aa00640 1 -- 172.21.15.116:0/1584198486 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a57f0 con 0x7f831c0cdda0 2024-05-14T11:26:36.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f831aa00640 1 -- 172.21.15.116:0/1584198486 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83140a5990 con 0x7f831c0cdda0 2024-05-14T11:26:36.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f831aa00640 1 -- 172.21.15.116:0/1584198486 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140a5d60 con 0x7f831c0cdda0 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 -- 172.21.15.116:0/1584198486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cdda0 msgr2=0x7f831c0b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cdda0 0x7f831c0b5c50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8314009e50 tx=0x7f831401ca30 comp rx=0 tx=0).stop 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 -- 172.21.15.116:0/1584198486 shutdown_connections 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cdda0 0x7f831c0b5c50 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 --2- 172.21.15.116:0/1584198486 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c15fb30 0x7f831c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 -- 172.21.15.116:0/1584198486 >> 172.21.15.116:0/1584198486 conn(0x7f831c071270 msgr2=0x7f831c071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 -- 172.21.15.116:0/1584198486 shutdown_connections 2024-05-14T11:26:36.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.597+0000 7f8322200640 1 -- 172.21.15.116:0/1584198486 wait complete. 2024-05-14T11:26:36.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 Processor -- start 2024-05-14T11:26:36.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 -- start start 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c0cdda0 0x7f831c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 0x7f831c06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f831c15ff40 con 0x7f831c0cbc20 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f831c16c960 con 0x7f831c15fb30 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8322200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f831c06e700 con 0x7f831c0cdda0 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40904/0 (socket says 172.21.15.116:40904) 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 -- 172.21.15.116:0/360987438 learned_addr learned my addr 172.21.15.116:0/360987438 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f8312a00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c0cdda0 0x7f831c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831be00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 0x7f831c06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 -- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c0cdda0 msgr2=0x7f831c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c0cdda0 0x7f831c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 -- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 msgr2=0x7f831c06e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 0x7f831c06e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831b400640 1 -- 172.21.15.116:0/360987438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c159cc0 con 0x7f831c0cbc20 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.601+0000 7f831be00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 0x7f831c06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f831b400640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f830c00e870 tx=0x7f830c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c017070 con 0x7f831c0cbc20 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c00c940 con 0x7f831c0cbc20 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c15a420 con 0x7f831c0cbc20 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c00cd10 con 0x7f831c0cbc20 2024-05-14T11:26:36.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c06e8a0 con 0x7f831c0cbc20 2024-05-14T11:26:36.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c06eb00 con 0x7f831c0cbc20 2024-05-14T11:26:36.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.605+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f830c01a070 con 0x7f831c0cbc20 2024-05-14T11:26:36.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.609+0000 7f8313e00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 0x7f82f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.609+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f830c08f460 con 0x7f831c0cbc20 2024-05-14T11:26:36.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.609+0000 7f8312a00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 0x7f82f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.609+0000 7f8312a00640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 0x7f82f8071910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f831c06db40 tx=0x7f8304009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.613+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f830c05c400 con 0x7f831c0cbc20 2024-05-14T11:26:36.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.765+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f831c159fa0 con 0x7f82f806f400 2024-05-14T11:26:36.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8313e00640 1 -- 172.21.15.116:0/360987438 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13507 (secure 0 0 0) 0x7f831c159fa0 con 0x7f82f806f400 2024-05-14T11:26:36.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:36.768 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:26:36.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 msgr2=0x7f82f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 0x7f82f8071910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f831c06db40 tx=0x7f8304009040 comp rx=0 tx=0).stop 2024-05-14T11:26:36.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 msgr2=0x7f831c154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f830c00e870 tx=0x7f830c00ec70 comp rx=0 tx=0).stop 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 shutdown_connections 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f82f806f400 0x7f82f8071910 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f831c15fb30 0x7f831c06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f831c0cdda0 0x7f831c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 --2- 172.21.15.116:0/360987438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f831c0cbc20 0x7f831c154cf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 >> 172.21.15.116:0/360987438 conn(0x7f831c071270 msgr2=0x7f831c158940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 shutdown_connections 2024-05-14T11:26:36.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.769+0000 7f8322200640 1 -- 172.21.15.116:0/360987438 wait complete. 2024-05-14T11:26:36.788 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-05-14T11:26:36.519032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266204,"kb_used_data":16756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708516,"statfs":{"total":773094113280,"available":772821520384,"internally_reserved":0,"allocated":17158144,"data_stored":10854983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-179,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001297"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":289,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-14T11:26:35.216442+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:26:35.216442+0000","last_peered":"2024-05-14T11:26:35.216442+0000","last_clean":"2024-05-14T11:26:35.216442+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:26:35.216442+0000","last_undegraded":"2024-05-14T11:26:35.216442+0000","last_fullsized":"2024-05-14T11:26:35.216442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36696,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335144,"statfs":{"total":96636764160,"available":96599187456,"internally_reserved":0,"allocated":5648384,"data_stored":4870307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":678830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":682415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:26:36.789 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-14T11:26:36.789 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:26:36.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 Processor -- start 2024-05-14T11:26:36.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 -- start start 2024-05-14T11:26:36.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 0x7f4e6c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e6c0d1fc0 con 0x7f4e6c06e9e0 2024-05-14T11:26:36.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4e6c1573b0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e70a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f4e6c161120 con 0x7f4e6c156fa0 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.937+0000 7f4e63e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40912/0 (socket says 172.21.15.116:40912) 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 -- 172.21.15.116:0/488833109 learned_addr learned my addr 172.21.15.116:0/488833109 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e69e00640 1 --2- 172.21.15.116:0/488833109 >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 0x7f4e6c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 -- 172.21.15.116:0/488833109 >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 msgr2=0x7f4e6c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 --2- 172.21.15.116:0/488833109 >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 0x7f4e6c06ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 -- 172.21.15.116:0/488833109 >> v1:172.21.15.149:6789/0 conn(0x7f4e6c06e9e0 legacy=0x7f4e6c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 -- 172.21.15.116:0/488833109 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c160510 con 0x7f4e6c0d1670 2024-05-14T11:26:36.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e69e00640 1 --2- 172.21.15.116:0/488833109 >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 0x7f4e6c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e63e00640 1 --2- 172.21.15.116:0/488833109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c071f20 tx=0x7f4e5401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae97551d684f734e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e69400640 1 -- 172.21.15.116:0/488833109 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540aa650 con 0x7f4e6c0d1670 2024-05-14T11:26:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e69400640 1 -- 172.21.15.116:0/488833109 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e540af070 con 0x7f4e6c0d1670 2024-05-14T11:26:36.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.941+0000 7f4e69400640 1 -- 172.21.15.116:0/488833109 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540aaa10 con 0x7f4e6c0d1670 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- 172.21.15.116:0/488833109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 msgr2=0x7f4e6c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 --2- 172.21.15.116:0/488833109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c071f20 tx=0x7f4e5401c4f0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- 172.21.15.116:0/488833109 shutdown_connections 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 --2- 172.21.15.116:0/488833109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c0d1a80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 --2- 172.21.15.116:0/488833109 >> v2:172.21.15.149:3300/0 conn(0x7f4e6c156fa0 0x7f4e6c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- 172.21.15.116:0/488833109 >> 172.21.15.116:0/488833109 conn(0x7f4e6c0b4620 msgr2=0x7f4e6c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- 172.21.15.116:0/488833109 shutdown_connections 2024-05-14T11:26:36.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- 172.21.15.116:0/488833109 wait complete. 2024-05-14T11:26:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 Processor -- start 2024-05-14T11:26:36.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.945+0000 7f4e70a00640 1 -- start start 2024-05-14T11:26:36.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 0x7f4e6c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6c156fa0 0x7f4e6c14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c070520 con 0x7f4e6c0d1670 2024-05-14T11:26:36.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c0b07a0 con 0x7f4e6c156fa0 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c0706a0 con 0x7f4e6c06e9e0 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40920/0 (socket says 172.21.15.116:40920) 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 -- 172.21.15.116:0/2297677531 learned_addr learned my addr 172.21.15.116:0/2297677531 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e6a800640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6c156fa0 0x7f4e6c14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e69e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 0x7f4e6c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 -- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 msgr2=0x7f4e6c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 0x7f4e6c14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 -- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6c156fa0 msgr2=0x7f4e6c14f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6c156fa0 0x7f4e6c14f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 -- 172.21.15.116:0/2297677531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c14c0f0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e69e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 0x7f4e6c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:36.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e63e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ac040 tx=0x7f4e5401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540bd5f0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e540a6040 con 0x7f4e6c0d1670 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c14d380 con 0x7f4e6c0d1670 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540bd9d0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c14d940 con 0x7f4e6c0d1670 2024-05-14T11:26:36.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6c14d0f0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.949+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e540b33f0 con 0x7f4e6c0d1670 2024-05-14T11:26:36.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.953+0000 7f4e62000640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 0x7f4e48071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:36.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.953+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f4e54129270 con 0x7f4e6c0d1670 2024-05-14T11:26:36.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.953+0000 7f4e69e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 0x7f4e48071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:36.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.953+0000 7f4e69e00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 0x7f4e48071940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0b0d60 tx=0x7f4e5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:36.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:36.957+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4e540c3060 con 0x7f4e6c0d1670 2024-05-14T11:26:37.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.109+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e6c06a9c0 con 0x7f4e4806f430 2024-05-14T11:26:37.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.109+0000 7f4e62000640 1 -- 172.21.15.116:0/2297677531 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13507 (secure 0 0 0) 0x7f4e6c06a9c0 con 0x7f4e4806f430 2024-05-14T11:26:37.111 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:37.111 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 msgr2=0x7f4e48071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 0x7f4e48071940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0b0d60 tx=0x7f4e5c008040 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 msgr2=0x7f4e6c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ac040 tx=0x7f4e5401ad30 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 shutdown_connections 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f4e4806f430 0x7f4e48071940 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6c156fa0 0x7f4e6c14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e6c0d1670 0x7f4e6c152880 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 --2- 172.21.15.116:0/2297677531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4e6c06e9e0 0x7f4e6c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 >> 172.21.15.116:0/2297677531 conn(0x7f4e6c0b4620 msgr2=0x7f4e6c15dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:37.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 shutdown_connections 2024-05-14T11:26:37.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.113+0000 7f4e70a00640 1 -- 172.21.15.116:0/2297677531 wait complete. 2024-05-14T11:26:37.132 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-05-14T11:26:36.519032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266204,"kb_used_data":16756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708516,"statfs":{"total":773094113280,"available":772821520384,"internally_reserved":0,"allocated":17158144,"data_stored":10854983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-179,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001297"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":289,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-14T11:26:35.216442+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:26:35.216442+0000","last_peered":"2024-05-14T11:26:35.216442+0000","last_clean":"2024-05-14T11:26:35.216442+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:26:35.216442+0000","last_undegraded":"2024-05-14T11:26:35.216442+0000","last_fullsized":"2024-05-14T11:26:35.216442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574979,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36696,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335144,"statfs":{"total":96636764160,"available":96599187456,"internally_reserved":0,"allocated":5648384,"data_stored":4870307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":678830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":682415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:26:37.132 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T11:26:37.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.281+0000 7f4806e00640 1 Processor -- start 2024-05-14T11:26:37.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.281+0000 7f4806e00640 1 -- start start 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4806e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f48000d1670 0x7f48000d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4806e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48000af380 0x7f48000af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4806e00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48000d1fc0 con 0x7f4800156fa0 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4806e00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f48000706f0 con 0x7f48000af380 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4806e00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f48001573b0 con 0x7f48000d1670 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48000af380 0x7f48000af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f47ffe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4800156fa0 0x7f480006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36898/0 (socket says 172.21.15.116:36898) 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48000af380 0x7f48000af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40926/0 (socket says 172.21.15.116:40926) 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f47ffe00640 1 -- 172.21.15.116:0/863138937 learned_addr learned my addr 172.21.15.116:0/863138937 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f47ff400640 1 --2- 172.21.15.116:0/863138937 >> v2:172.21.15.149:3300/0 conn(0x7f48000d1670 0x7f48000d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 -- 172.21.15.116:0/863138937 >> v2:172.21.15.149:3300/0 conn(0x7f48000d1670 msgr2=0x7f48000d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:37.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 --2- 172.21.15.116:0/863138937 >> v2:172.21.15.149:3300/0 conn(0x7f48000d1670 0x7f48000d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 -- 172.21.15.116:0/863138937 >> v1:172.21.15.116:6789/0 conn(0x7f4800156fa0 legacy=0x7f480006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f4804e00640 1 -- 172.21.15.116:0/863138937 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4800160510 con 0x7f48000af380 2024-05-14T11:26:37.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.285+0000 7f47fea00640 1 -- 172.21.15.116:0/863138937 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204343855 0 0) 0x7f47f0001120 con 0x7f4800156fa0 2024-05-14T11:26:37.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4804e00640 1 --2- 172.21.15.116:0/863138937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000af380 0x7f48000af790 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4800070c70 tx=0x7f47f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ac45b4d7781b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:37.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f47fea00640 1 -- 172.21.15.116:0/863138937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f80b2070 con 0x7f48000af380 2024-05-14T11:26:37.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f47fea00640 1 -- 172.21.15.116:0/863138937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f80a58e0 con 0x7f48000af380 2024-05-14T11:26:37.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f47fea00640 1 -- 172.21.15.116:0/863138937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f80a5cb0 con 0x7f48000af380 2024-05-14T11:26:37.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- 172.21.15.116:0/863138937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000af380 msgr2=0x7f48000af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 --2- 172.21.15.116:0/863138937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000af380 0x7f48000af790 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4800070c70 tx=0x7f47f801c930 comp rx=0 tx=0).stop 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- 172.21.15.116:0/863138937 shutdown_connections 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 --2- 172.21.15.116:0/863138937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000af380 0x7f48000af790 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 --2- 172.21.15.116:0/863138937 >> v2:172.21.15.149:3300/0 conn(0x7f48000d1670 0x7f48000d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- 172.21.15.116:0/863138937 >> 172.21.15.116:0/863138937 conn(0x7f48000b4620 msgr2=0x7f48000b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- 172.21.15.116:0/863138937 shutdown_connections 2024-05-14T11:26:37.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- 172.21.15.116:0/863138937 wait complete. 2024-05-14T11:26:37.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 Processor -- start 2024-05-14T11:26:37.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.289+0000 7f4806e00640 1 -- start start 2024-05-14T11:26:37.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48000af380 0x7f4800152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48000d1670 0x7f480014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48000b0660 con 0x7f4800156fa0 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48000b04e0 con 0x7f48000d1670 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f480006c700 con 0x7f48000af380 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40928/0 (socket says 172.21.15.116:40928) 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 -- 172.21.15.116:0/2942191454 learned_addr learned my addr 172.21.15.116:0/2942191454 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f47ff400640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48000d1670 0x7f480014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f47ffe00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48000af380 0x7f4800152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 -- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48000af380 msgr2=0x7f4800152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48000af380 0x7f4800152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 -- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48000d1670 msgr2=0x7f480014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48000d1670 0x7f480014f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 -- 172.21.15.116:0/2942191454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480015c6d0 con 0x7f4800156fa0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4804e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f47f801ce10 tx=0x7f47f80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f80b2040 con 0x7f4800156fa0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f80039a0 con 0x7f4800156fa0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48001fca30 con 0x7f4800156fa0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8003d50 con 0x7f4800156fa0 2024-05-14T11:26:37.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001fcff0 con 0x7f4800156fa0 2024-05-14T11:26:37.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.293+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48001fc830 con 0x7f4800156fa0 2024-05-14T11:26:37.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.297+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47f80c1030 con 0x7f4800156fa0 2024-05-14T11:26:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.297+0000 7f47f7e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.297+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f47f8129960 con 0x7f4800156fa0 2024-05-14T11:26:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.297+0000 7f47ffe00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.297+0000 7f47ffe00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 0x7f47e0071910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47f0000f70 tx=0x7f47f0007df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.301+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f47f80c8020 con 0x7f4800156fa0 2024-05-14T11:26:37.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f480006a9c0 con 0x7f4800156fa0 2024-05-14T11:26:37.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f47f7e00640 1 -- 172.21.15.116:0/2942191454 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+13617 (secure 0 0 0) 0x7f47f8016520 con 0x7f4800156fa0 2024-05-14T11:26:37.464 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:37.464 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"8ea35704-38e5-41a3-b9f5-7d0f3a8c4277","created":"2024-05-14T11:15:47.341169+0000","modified":"2024-05-14T11:26:34.256868+0000","last_up_change":"2024-05-14T11:15:59.707805+0000","last_in_change":"2024-05-14T11:15:50.050628+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T11:16:00.384207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1924236-7cff-4e7b-a5d4-b5cb84beb123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6809","nonce":772722056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6811","nonce":772722056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6815","nonce":772722056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":772722056},{"type":"v1","addr":"172.21.15.116:6813","nonce":772722056}]},"public_addr":"172.21.15.116:6809/772722056","cluster_addr":"172.21.15.116:6811/772722056","heartbeat_back_addr":"172.21.15.116:6815/772722056","heartbeat_front_addr":"172.21.15.116:6813/772722056","state":["exists","up"]},{"osd":1,"uuid":"9f040550-c15e-418b-aaaa-27267ad5a416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6801","nonce":4228261718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6803","nonce":4228261718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6807","nonce":4228261718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":4228261718},{"type":"v1","addr":"172.21.15.116:6805","nonce":4228261718}]},"public_addr":"172.21.15.116:6801/4228261718","cluster_addr":"172.21.15.116:6803/4228261718","heartbeat_back_addr":"172.21.15.116:6807/4228261718","heartbeat_front_addr":"172.21.15.116:6805/4228261718","state":["exists","up"]},{"osd":2,"uuid":"b023f361-28b5-43af-96fe-d518ea584db2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6819","nonce":2784504541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6824","nonce":2784504541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6831","nonce":2784504541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2784504541},{"type":"v1","addr":"172.21.15.116:6828","nonce":2784504541}]},"public_addr":"172.21.15.116:6819/2784504541","cluster_addr":"172.21.15.116:6824/2784504541","heartbeat_back_addr":"172.21.15.116:6831/2784504541","heartbeat_front_addr":"172.21.15.116:6828/2784504541","state":["exists","up"]},{"osd":3,"uuid":"062cdb74-9d77-4afb-ba9b-ad53cb5be49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6818","nonce":1815220698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6822","nonce":1815220698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6827","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6829","nonce":1815220698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6823","nonce":1815220698},{"type":"v1","addr":"172.21.15.116:6825","nonce":1815220698}]},"public_addr":"172.21.15.116:6818/1815220698","cluster_addr":"172.21.15.116:6822/1815220698","heartbeat_back_addr":"172.21.15.116:6829/1815220698","heartbeat_front_addr":"172.21.15.116:6825/1815220698","state":["exists","up"]},{"osd":4,"uuid":"a42da358-4ee2-4bd3-9203-bcd1ad5ac93a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6812","nonce":3313182122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6817","nonce":3313182122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6823","nonce":3313182122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3313182122},{"type":"v1","addr":"172.21.15.149:6821","nonce":3313182122}]},"public_addr":"172.21.15.149:6812/3313182122","cluster_addr":"172.21.15.149:6817/3313182122","heartbeat_back_addr":"172.21.15.149:6823/3313182122","heartbeat_front_addr":"172.21.15.149:6821/3313182122","state":["exists","up"]},{"osd":5,"uuid":"f6f87186-b4aa-4e02-a8ab-e048753ba149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6825","nonce":4173680411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6827","nonce":4173680411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6831","nonce":4173680411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4173680411},{"type":"v1","addr":"172.21.15.149:6829","nonce":4173680411}]},"public_addr":"172.21.15.149:6825/4173680411","cluster_addr":"172.21.15.149:6827/4173680411","heartbeat_back_addr":"172.21.15.149:6831/4173680411","heartbeat_front_addr":"172.21.15.149:6829/4173680411","state":["exists","up"]},{"osd":6,"uuid":"500d5e45-1937-476c-99de-15b138f68674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6801","nonce":3938301688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6803","nonce":3938301688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6807","nonce":3938301688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3938301688},{"type":"v1","addr":"172.21.15.149:6805","nonce":3938301688}]},"public_addr":"172.21.15.149:6801/3938301688","cluster_addr":"172.21.15.149:6803/3938301688","heartbeat_back_addr":"172.21.15.149:6807/3938301688","heartbeat_front_addr":"172.21.15.149:6805/3938301688","state":["exists","up"]},{"osd":7,"uuid":"582cbeba-60c9-4f83-bf81-eb2ca3e9bd35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6810","nonce":2875938804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6811","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6813","nonce":2875938804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6819","nonce":2875938804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6815","nonce":2875938804},{"type":"v1","addr":"172.21.15.149:6816","nonce":2875938804}]},"public_addr":"172.21.15.149:6810/2875938804","cluster_addr":"172.21.15.149:6813/2875938804","heartbeat_back_addr":"172.21.15.149:6819/2875938804","heartbeat_front_addr":"172.21.15.149:6816/2875938804","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.729293+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.594786+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822571+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.822566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.955260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.722692+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.476270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T11:15:55.614605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6837/2487260442":"2024-05-15T11:26:30.894937+0000","172.21.15.116:6836/2487260442":"2024-05-15T11:26:30.894937+0000","172.21.15.116:6837/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6836/3756089525":"2024-05-15T11:26:18.174966+0000","172.21.15.116:6837/1742968683":"2024-05-15T11:26:01.594455+0000","172.21.15.116:6836/1742968683":"2024-05-15T11:26:01.594455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T11:26:37.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 msgr2=0x7f47e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 0x7f47e0071910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47f0000f70 tx=0x7f47f0007df0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 msgr2=0x7f4800073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:37.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f47f801ce10 tx=0x7f47f80196d0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 shutdown_connections 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48000d1670 0x7f480014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4800156fa0 0x7f4800073a80 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 --2- 172.21.15.116:0/2942191454 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48000af380 0x7f4800152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 >> 172.21.15.116:0/2942191454 conn(0x7f48000b4620 msgr2=0x7f48000b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 shutdown_connections 2024-05-14T11:26:37.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:37.465+0000 7f4806e00640 1 -- 172.21.15.116:0/2942191454 wait complete. 2024-05-14T11:26:38.487 INFO:tasks.ceph:Scrubbing osd.0 2024-05-14T11:26:38.487 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:38.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.641+0000 7f55f9800640 1 Processor -- start 2024-05-14T11:26:38.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.641+0000 7f55f9800640 1 -- start start 2024-05-14T11:26:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f9800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f406ef80 0x7f55f40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f9800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f415d360 con 0x7f55f4070a50 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f9800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f415d640 con 0x7f55f4070640 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f9800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f55f415dc90 con 0x7f55f406ef80 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f3400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f55f4070a50 0x7f55f40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41342/0 (socket says 172.21.15.116:41342) 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f3400640 1 -- 172.21.15.116:0/1184990245 learned_addr learned my addr 172.21.15.116:0/1184990245 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f406ef80 0x7f55f40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985107752 0 0) 0x7f55f415d360 con 0x7f55f4070a50 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55d8005740 con 0x7f55f4070a50 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560216271 0 0) 0x7f55f415d640 con 0x7f55f4070640 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f415d360 con 0x7f55f4070640 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2670304819 0 0) 0x7f55d8005740 con 0x7f55f4070a50 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f406ef80 msgr2=0x7f55f40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f406ef80 0x7f55f40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 >> v1:172.21.15.149:6789/0 conn(0x7f55f4070640 legacy=0x7f55f406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f415d920 con 0x7f55f4070a50 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f55ec001a80 con 0x7f55f4070a50 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f406ef80 0x7f55f40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 >> v1:172.21.15.116:6789/0 conn(0x7f55f4070a50 legacy=0x7f55f40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55d8005e30 0x7f55d8006280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55d8006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55d8007480 0x7f55d800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f55f415dc90 con 0x7f55d8006a20 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55d800e200 con 0x7f55d8005e30 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f55d800e400 con 0x7f55d8007480 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55d8006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2a00640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55d8005e30 0x7f55d8006280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f3400640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55d8007480 0x7f55d800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 -- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55d8007480 msgr2=0x7f55d800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55d8007480 0x7f55d800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 -- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55d8005e30 msgr2=0x7f55d8006280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55d8005e30 0x7f55d8006280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.645+0000 7f55f2000640 1 -- 172.21.15.116:0/1184990245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d8005740 con 0x7f55d8006a20 2024-05-14T11:26:38.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f2000640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55d8006e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009cb0 tx=0x7f55e001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f9e9ce9cfebe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00b2070 con 0x7f55d8006a20 2024-05-14T11:26:38.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e00a58f0 con 0x7f55d8006a20 2024-05-14T11:26:38.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f1600640 1 -- 172.21.15.116:0/1184990245 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0009880 con 0x7f55d8006a20 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 -- 172.21.15.116:0/1184990245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 msgr2=0x7f55d8006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 --2- 172.21.15.116:0/1184990245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55d8006e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009cb0 tx=0x7f55e001cc30 comp rx=0 tx=0).stop 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f3400640 1 -- 172.21.15.116:0/1184990245 reap_dead start 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 -- 172.21.15.116:0/1184990245 shutdown_connections 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 -- 172.21.15.116:0/1184990245 >> 172.21.15.116:0/1184990245 conn(0x7f55f40cde80 msgr2=0x7f55f40ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 -- 172.21.15.116:0/1184990245 shutdown_connections 2024-05-14T11:26:38.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.649+0000 7f55f9800640 1 -- 172.21.15.116:0/1184990245 wait complete. 2024-05-14T11:26:38.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 Processor -- start 2024-05-14T11:26:38.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 -- start start 2024-05-14T11:26:38.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55f41fa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f41e52a0 0x7f55f41e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f415bae0 0x7f55f415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f55f40cd800 con 0x7f55d8006a20 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55f413fda0 con 0x7f55f41e52a0 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f9800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f55f40cd980 con 0x7f55f415bae0 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55f41fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55f41fa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42642/0 (socket says 172.21.15.116:42642) 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 -- 172.21.15.116:0/103781827 learned_addr learned my addr 172.21.15.116:0/103781827 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2000640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f41e52a0 0x7f55f41e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f3400640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f415bae0 0x7f55f415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 -- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f415bae0 msgr2=0x7f55f415bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f55f415bae0 0x7f55f415bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 -- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f41e52a0 msgr2=0x7f55f41e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55f41e52a0 0x7f55f41e56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f415c430 con 0x7f55d8006a20 2024-05-14T11:26:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55f2a00640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55f41fa240 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f55e400f430 tx=0x7f55e400f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.653+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4010040 con 0x7f55d8006a20 2024-05-14T11:26:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e401a070 con 0x7f55d8006a20 2024-05-14T11:26:38.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40148b0 con 0x7f55d8006a20 2024-05-14T11:26:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f41e5fd0 con 0x7f55d8006a20 2024-05-14T11:26:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f41e6590 con 0x7f55d8006a20 2024-05-14T11:26:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55f406aa30 con 0x7f55d8006a20 2024-05-14T11:26:38.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f55e4014a50 con 0x7f55d8006a20 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55d406f400 0x7f55d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f55e4092680 con 0x7f55d8006a20 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55f2000640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55d406f400 0x7f55d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f55d4075a30 0x7f55d4077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55e4007680 con 0x7f55d4075a30 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f55e4092a30 con 0x7f55d8006a20 2024-05-14T11:26:38.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.657+0000 7f55f3400640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f55d4075a30 0x7f55d4077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.661+0000 7f55f2000640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55d406f400 0x7f55d4071910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f55e00af000 tx=0x7f55e00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.661+0000 7f55f3400640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f55d4075a30 0x7f55d4077ec0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.661+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== osd.0 v2:172.21.15.116:6808/772722056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55e4007680 con 0x7f55d4075a30 2024-05-14T11:26:38.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 --> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f55f41f7560 con 0x7f55d4075a30 2024-05-14T11:26:38.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55eaa00640 1 -- 172.21.15.116:0/103781827 <== osd.0 v2:172.21.15.116:6808/772722056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f55f41f7560 con 0x7f55d4075a30 2024-05-14T11:26:38.673 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:38.673 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:38.673 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f55d4075a30 msgr2=0x7f55d4077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6808/772722056,v1:172.21.15.116:6809/772722056] conn(0x7f55d4075a30 0x7f55d4077ec0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55d406f400 msgr2=0x7f55d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f55d406f400 0x7f55d4071910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f55e00af000 tx=0x7f55e00046d0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 msgr2=0x7f55f41fa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 --2- 172.21.15.116:0/103781827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55d8006a20 0x7f55f41fa240 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f55e400f430 tx=0x7f55e400f830 comp rx=0 tx=0).stop 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f3400640 1 -- 172.21.15.116:0/103781827 reap_dead start 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 shutdown_connections 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 >> 172.21.15.116:0/103781827 conn(0x7f55f40cde80 msgr2=0x7f55f40b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 shutdown_connections 2024-05-14T11:26:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.673+0000 7f55f9800640 1 -- 172.21.15.116:0/103781827 wait complete. 2024-05-14T11:26:38.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-14T11:26:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.837+0000 7fccf8000640 1 Processor -- start 2024-05-14T11:26:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.837+0000 7fccf8000640 1 -- start start 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf8000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf00b0390 0x7fccf0167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf8000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf0160510 con 0x7fccf0156fa0 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf8000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf01607f0 con 0x7fccf00d1670 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf8000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fccf00706f0 con 0x7fccf00b0390 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf00b0390 0x7fccf0167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf00b0390 0x7fccf0167800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42648/0 (socket says 172.21.15.116:42648) 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf5600640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fccf0156fa0 0x7fccf006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41346/0 (socket says 172.21.15.116:41346) 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 -- 172.21.15.116:0/1245764533 learned_addr learned my addr 172.21.15.116:0/1245764533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 -- 172.21.15.116:0/1245764533 >> v1:172.21.15.149:6789/0 conn(0x7fccf00d1670 legacy=0x7fccf00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 -- 172.21.15.116:0/1245764533 >> v1:172.21.15.116:6789/0 conn(0x7fccf0156fa0 legacy=0x7fccf006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.841+0000 7fccf6000640 1 -- 172.21.15.116:0/1245764533 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0160ad0 con 0x7fccf00b0390 2024-05-14T11:26:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf6000640 1 --2- 172.21.15.116:0/1245764533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf0167800 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009870 tx=0x7fcce801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0049e097e37f11c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccefe00640 1 -- 172.21.15.116:0/1245764533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80b2070 con 0x7fccf00b0390 2024-05-14T11:26:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccefe00640 1 -- 172.21.15.116:0/1245764533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce80a9070 con 0x7fccf00b0390 2024-05-14T11:26:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccefe00640 1 -- 172.21.15.116:0/1245764533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80ad880 con 0x7fccf00b0390 2024-05-14T11:26:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 -- 172.21.15.116:0/1245764533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 msgr2=0x7fccf0167800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 --2- 172.21.15.116:0/1245764533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf0167800 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009870 tx=0x7fcce801c560 comp rx=0 tx=0).stop 2024-05-14T11:26:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 -- 172.21.15.116:0/1245764533 shutdown_connections 2024-05-14T11:26:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 --2- 172.21.15.116:0/1245764533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf0167800 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 -- 172.21.15.116:0/1245764533 >> 172.21.15.116:0/1245764533 conn(0x7fccf00b4620 msgr2=0x7fccf00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 -- 172.21.15.116:0/1245764533 shutdown_connections 2024-05-14T11:26:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.845+0000 7fccf8000640 1 -- 172.21.15.116:0/1245764533 wait complete. 2024-05-14T11:26:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 Processor -- start 2024-05-14T11:26:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- start start 2024-05-14T11:26:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf00d1670 0x7fccf014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf0156fa0 0x7fccf009e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fccf016ad90 con 0x7fccf00b0390 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fccf00b07a0 con 0x7fccf0156fa0 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fccf016af10 con 0x7fccf00d1670 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42650/0 (socket says 172.21.15.116:42650) 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 -- 172.21.15.116:0/3347032576 learned_addr learned my addr 172.21.15.116:0/3347032576 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf4c00640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf00d1670 0x7fccf014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf6000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf0156fa0 0x7fccf009e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 -- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf00d1670 msgr2=0x7fccf014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf00d1670 0x7fccf014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 -- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf0156fa0 msgr2=0x7fccf009e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf0156fa0 0x7fccf009e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 -- 172.21.15.116:0/3347032576 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf014ec30 con 0x7fccf00b0390 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf5600640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c780 tx=0x7fcce400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4017070 con 0x7fccf00b0390 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce400a640 con 0x7fccf00b0390 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf014e390 con 0x7fccf00b0390 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce400aa90 con 0x7fccf00b0390 2024-05-14T11:26:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf014e950 con 0x7fccf00b0390 2024-05-14T11:26:38.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.849+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf006a9c0 con 0x7fccf00b0390 2024-05-14T11:26:38.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.853+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcce401a070 con 0x7fccf00b0390 2024-05-14T11:26:38.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.853+0000 7fcced600640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 0x7fccd0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:38.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.853+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fcce408e8d0 con 0x7fccf00b0390 2024-05-14T11:26:38.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.853+0000 7fccf4c00640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 0x7fccd0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:38.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.853+0000 7fccf4c00640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 0x7fccd0071910 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fccf00b2b90 tx=0x7fccdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:38.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:38.857+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcce405b870 con 0x7fccf00b0390 2024-05-14T11:26:39.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.013+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fccf00d1d00 con 0x7fccd006f400 2024-05-14T11:26:39.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.013+0000 7fcced600640 1 -- 172.21.15.116:0/3347032576 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fccf00d1d00 con 0x7fccd006f400 2024-05-14T11:26:39.015 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 0 to deep-scrub 2024-05-14T11:26:39.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 msgr2=0x7fccd0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 0x7fccd0071910 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fccf00b2b90 tx=0x7fccdc009040 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 msgr2=0x7fccf014d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c780 tx=0x7fcce400cc40 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 shutdown_connections 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd006f400 0x7fccd0071910 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf0156fa0 0x7fccf009e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf00d1670 0x7fccf014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 --2- 172.21.15.116:0/3347032576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf00b0390 0x7fccf014d270 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 >> 172.21.15.116:0/3347032576 conn(0x7fccf00b4620 msgr2=0x7fccf00b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 shutdown_connections 2024-05-14T11:26:39.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.017+0000 7fccf8000640 1 -- 172.21.15.116:0/3347032576 wait complete. 2024-05-14T11:26:39.036 INFO:tasks.ceph:Scrubbing osd.1 2024-05-14T11:26:39.036 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.185+0000 7f812de00640 1 Processor -- start 2024-05-14T11:26:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.185+0000 7f812de00640 1 -- start start 2024-05-14T11:26:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f812de00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 0x7f812806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f812de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f812de00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81280d04c0 con 0x7f81280cda30 2024-05-14T11:26:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f812de00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f81280d0270 con 0x7f81280cd620 2024-05-14T11:26:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f812de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f81280b0c30 con 0x7f812806d710 2024-05-14T11:26:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8127400640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8126a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 0x7f812806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f811e000640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f81280cda30 0x7f8128161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:53032/0 (socket says 172.21.15.116:53032) 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f811e000640 1 -- 172.21.15.116:0/1248877060 learned_addr learned my addr 172.21.15.116:0/1248877060 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8126000640 1 -- 172.21.15.116:0/1248877060 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455455606 0 0) 0x7f81280d04c0 con 0x7f81280cda30 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8127400640 1 -- 172.21.15.116:0/1248877060 >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 msgr2=0x7f812806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8127400640 1 --2- 172.21.15.116:0/1248877060 >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 0x7f812806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8127400640 1 -- 172.21.15.116:0/1248877060 >> v1:172.21.15.149:6789/0 conn(0x7f81280cda30 legacy=0x7f8128161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8127400640 1 -- 172.21.15.116:0/1248877060 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81280d07a0 con 0x7f812806d710 2024-05-14T11:26:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.189+0000 7f8126a00640 1 --2- 172.21.15.116:0/1248877060 >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 0x7f812806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:39.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f8127400640 1 --2- 172.21.15.116:0/1248877060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812806fbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8120009d90 tx=0x7f812001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c557bb4cf03afa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f8126000640 1 -- 172.21.15.116:0/1248877060 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200b2070 con 0x7f812806d710 2024-05-14T11:26:39.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f8126000640 1 -- 172.21.15.116:0/1248877060 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81200a5750 con 0x7f812806d710 2024-05-14T11:26:39.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f8126000640 1 -- 172.21.15.116:0/1248877060 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200a5b20 con 0x7f812806d710 2024-05-14T11:26:39.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 -- 172.21.15.116:0/1248877060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 msgr2=0x7f812806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 --2- 172.21.15.116:0/1248877060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812806fbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8120009d90 tx=0x7f812001c970 comp rx=0 tx=0).stop 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 -- 172.21.15.116:0/1248877060 shutdown_connections 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 --2- 172.21.15.116:0/1248877060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812806fbf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 --2- 172.21.15.116:0/1248877060 >> v2:172.21.15.149:3300/0 conn(0x7f81280cd620 0x7f812806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 -- 172.21.15.116:0/1248877060 >> 172.21.15.116:0/1248877060 conn(0x7f81280cde80 msgr2=0x7f81280ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 -- 172.21.15.116:0/1248877060 shutdown_connections 2024-05-14T11:26:39.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.193+0000 7f812de00640 1 -- 172.21.15.116:0/1248877060 wait complete. 2024-05-14T11:26:39.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 Processor -- start 2024-05-14T11:26:39.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- start start 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812813dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81280cd620 0x7f8128150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 0x7f81280b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f812816ae40 con 0x7f81280b2140 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f812816acc0 con 0x7f81280cd620 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f812816afc0 con 0x7f812806d710 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 0x7f81280b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 0x7f81280b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42654/0 (socket says 172.21.15.116:42654) 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 -- 172.21.15.116:0/1176134974 learned_addr learned my addr 172.21.15.116:0/1176134974 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f811e000640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81280cd620 0x7f8128150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8126a00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812813dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 -- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 msgr2=0x7f812813dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812813dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 -- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81280cd620 msgr2=0x7f8128150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81280cd620 0x7f8128150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81280b2a90 con 0x7f81280b2140 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8126a00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f812806d710 0x7f812813dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f8127400640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 0x7f81280b2550 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f81200b0040 tx=0x7f8120019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200b2050 con 0x7f81280b2140 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81200b3040 con 0x7f81280b2140 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812813e140 con 0x7f81280b2140 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200ad650 con 0x7f81280b2140 2024-05-14T11:26:39.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.197+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8128151890 con 0x7f81280b2140 2024-05-14T11:26:39.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f812806aa30 con 0x7f81280b2140 2024-05-14T11:26:39.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81200c0020 con 0x7f81280b2140 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f810406f400 0x7f8104071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f812012a530 con 0x7f81280b2140 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f8126a00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f810406f400 0x7f8104071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f8104075a30 0x7f8104077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81200a5750 con 0x7f8104075a30 2024-05-14T11:26:39.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f812012a8e0 con 0x7f81280b2140 2024-05-14T11:26:39.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811e000640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f8104075a30 0x7f8104077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f8126a00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f810406f400 0x7f8104071910 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8118009a40 tx=0x7f8118008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.201+0000 7f811e000640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f8104075a30 0x7f8104077ec0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.205+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== osd.1 v2:172.21.15.116:6800/4228261718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81200a5750 con 0x7f8104075a30 2024-05-14T11:26:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 --> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8128151360 con 0x7f8104075a30 2024-05-14T11:26:39.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f811fe00640 1 -- 172.21.15.116:0/1176134974 <== osd.1 v2:172.21.15.116:6800/4228261718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8128151360 con 0x7f8104075a30 2024-05-14T11:26:39.216 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:39.216 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:39.216 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f8104075a30 msgr2=0x7f8104077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6800/4228261718,v1:172.21.15.116:6801/4228261718] conn(0x7f8104075a30 0x7f8104077ec0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f810406f400 msgr2=0x7f8104071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f810406f400 0x7f8104071910 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8118009a40 tx=0x7f8118008040 comp rx=0 tx=0).stop 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 msgr2=0x7f81280b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 --2- 172.21.15.116:0/1176134974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81280b2140 0x7f81280b2550 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f81200b0040 tx=0x7f8120019710 comp rx=0 tx=0).stop 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f8127400640 1 -- 172.21.15.116:0/1176134974 reap_dead start 2024-05-14T11:26:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 shutdown_connections 2024-05-14T11:26:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 >> 172.21.15.116:0/1176134974 conn(0x7f81280cde80 msgr2=0x7f8128151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 shutdown_connections 2024-05-14T11:26:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.217+0000 7f812de00640 1 -- 172.21.15.116:0/1176134974 wait complete. 2024-05-14T11:26:39.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-14T11:26:39.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.381+0000 7f6db7000640 1 Processor -- start 2024-05-14T11:26:39.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.381+0000 7f6db7000640 1 -- start start 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db7000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6db00af380 0x7f6db00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db7000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db0160510 con 0x7f6db00d1670 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db7000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6db01573b0 con 0x7f6db00af380 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6db0161120 con 0x7f6db0156fa0 2024-05-14T11:26:39.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6db00af380 0x7f6db00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6db00af380 0x7f6db00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42670/0 (socket says 172.21.15.116:42670) 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6daf400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6db00d1670 0x7f6db00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41350/0 (socket says 172.21.15.116:41350) 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 -- 172.21.15.116:0/3991048748 learned_addr learned my addr 172.21.15.116:0/3991048748 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6dafe00640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 -- 172.21.15.116:0/3991048748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 msgr2=0x7f6db006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db006ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 -- 172.21.15.116:0/3991048748 >> v1:172.21.15.116:6789/0 conn(0x7f6db00d1670 legacy=0x7f6db00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.385+0000 7f6db5000640 1 -- 172.21.15.116:0/3991048748 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db01607f0 con 0x7f6db00af380 2024-05-14T11:26:39.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db5000640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00af380 0x7f6db00af790 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009870 tx=0x7f6da801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b58e3dc444cc57a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6daea00640 1 -- 172.21.15.116:0/3991048748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a95f0 con 0x7f6db00af380 2024-05-14T11:26:39.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6daea00640 1 -- 172.21.15.116:0/3991048748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da80a9790 con 0x7f6db00af380 2024-05-14T11:26:39.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6daea00640 1 -- 172.21.15.116:0/3991048748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a9b60 con 0x7f6db00af380 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 -- 172.21.15.116:0/3991048748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00af380 msgr2=0x7f6db00af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00af380 0x7f6db00af790 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009870 tx=0x7f6da801c530 comp rx=0 tx=0).stop 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 -- 172.21.15.116:0/3991048748 shutdown_connections 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00af380 0x7f6db00af790 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 --2- 172.21.15.116:0/3991048748 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 -- 172.21.15.116:0/3991048748 >> 172.21.15.116:0/3991048748 conn(0x7f6db00b4620 msgr2=0x7f6db00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 -- 172.21.15.116:0/3991048748 shutdown_connections 2024-05-14T11:26:39.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.389+0000 7f6db7000640 1 -- 172.21.15.116:0/3991048748 wait complete. 2024-05-14T11:26:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 Processor -- start 2024-05-14T11:26:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- start start 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db00af380 0x7f6db014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6db00b0660 con 0x7f6db00d1670 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6db00b04e0 con 0x7f6db00af380 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6db006c700 con 0x7f6db0156fa0 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42678/0 (socket says 172.21.15.116:42678) 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 -- 172.21.15.116:0/3473368678 learned_addr learned my addr 172.21.15.116:0/3473368678 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 -- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 msgr2=0x7f6db014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6dafe00640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db00af380 0x7f6db014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 -- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db00af380 msgr2=0x7f6db014d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db00af380 0x7f6db014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 -- 172.21.15.116:0/3473368678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db00cfa80 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6daf400640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c3f0 tx=0x7f6d9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980074e0 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d98007680 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004a10 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db00cfd60 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.393+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db00d0320 con 0x7f6db00d1670 2024-05-14T11:26:39.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db006a9c0 con 0x7f6db00d1670 2024-05-14T11:26:39.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d9801a020 con 0x7f6db00d1670 2024-05-14T11:26:39.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6da7e00640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 0x7f6d8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f6d9808ea30 con 0x7f6db00d1670 2024-05-14T11:26:39.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6dafe00640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 0x7f6d8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.397+0000 7f6dafe00640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 0x7f6d8c071910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0d60 tx=0x7f6da000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.401+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6d9805b940 con 0x7f6db00d1670 2024-05-14T11:26:39.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.561+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f6db0156860 con 0x7f6d8c06f400 2024-05-14T11:26:39.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.561+0000 7f6da7e00640 1 -- 172.21.15.116:0/3473368678 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6db0156860 con 0x7f6d8c06f400 2024-05-14T11:26:39.561 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 1 to deep-scrub 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.561+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 msgr2=0x7f6d8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 0x7f6d8c071910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0d60 tx=0x7f6da000b040 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 msgr2=0x7f6db014f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c3f0 tx=0x7f6d9800c8b0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 shutdown_connections 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d8c06f400 0x7f6d8c071910 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6db0156fa0 0x7f6db014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6db00d1670 0x7f6db014f4a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 --2- 172.21.15.116:0/3473368678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db00af380 0x7f6db014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 >> 172.21.15.116:0/3473368678 conn(0x7f6db00b4620 msgr2=0x7f6db00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 shutdown_connections 2024-05-14T11:26:39.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.565+0000 7f6db7000640 1 -- 172.21.15.116:0/3473368678 wait complete. 2024-05-14T11:26:39.582 INFO:tasks.ceph:Scrubbing osd.2 2024-05-14T11:26:39.582 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:39.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 Processor -- start 2024-05-14T11:26:39.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 -- start start 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6038070640 0x7f603806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f60380d0290 0x7f603806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603815d360 con 0x7f6038070a50 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f603815dc90 con 0x7f6038070640 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.733+0000 7f603dc00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f603815d640 con 0x7f60380d0290 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6038070640 0x7f603806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6038070640 0x7f603806e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42680/0 (socket says 172.21.15.116:42680) 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 -- 172.21.15.116:0/2423576277 learned_addr learned my addr 172.21.15.116:0/2423576277 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6037400640 1 --2- 172.21.15.116:0/2423576277 >> v2:172.21.15.149:3300/0 conn(0x7f60380d0290 0x7f603806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 -- 172.21.15.116:0/2423576277 >> v2:172.21.15.149:3300/0 conn(0x7f60380d0290 msgr2=0x7f603806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 --2- 172.21.15.116:0/2423576277 >> v2:172.21.15.149:3300/0 conn(0x7f60380d0290 0x7f603806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 -- 172.21.15.116:0/2423576277 >> v1:172.21.15.149:6789/0 conn(0x7f6038070a50 legacy=0x7f60381421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 -- 172.21.15.116:0/2423576277 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603815d920 con 0x7f6038070640 2024-05-14T11:26:39.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036a00640 1 --2- 172.21.15.116:0/2423576277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6038070640 0x7f603806e870 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f60380d06a0 tx=0x7f602801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bee4a318313dea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036000640 1 -- 172.21.15.116:0/2423576277 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280b1070 con 0x7f6038070640 2024-05-14T11:26:39.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036000640 1 -- 172.21.15.116:0/2423576277 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60280ac630 con 0x7f6038070640 2024-05-14T11:26:39.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.737+0000 7f6036000640 1 -- 172.21.15.116:0/2423576277 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280a8050 con 0x7f6038070640 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- 172.21.15.116:0/2423576277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6038070640 msgr2=0x7f603806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 --2- 172.21.15.116:0/2423576277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6038070640 0x7f603806e870 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f60380d06a0 tx=0x7f602801c490 comp rx=0 tx=0).stop 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- 172.21.15.116:0/2423576277 shutdown_connections 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 --2- 172.21.15.116:0/2423576277 >> v2:172.21.15.149:3300/0 conn(0x7f60380d0290 0x7f603806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 --2- 172.21.15.116:0/2423576277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6038070640 0x7f603806e870 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- 172.21.15.116:0/2423576277 >> 172.21.15.116:0/2423576277 conn(0x7f60380cde80 msgr2=0x7f60380ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- 172.21.15.116:0/2423576277 shutdown_connections 2024-05-14T11:26:39.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- 172.21.15.116:0/2423576277 wait complete. 2024-05-14T11:26:39.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 Processor -- start 2024-05-14T11:26:39.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.741+0000 7f603dc00640 1 -- start start 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038070640 0x7f60380aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6038070a50 0x7f603813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 0x7f60380afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60381699a0 con 0x7f60380d0290 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60380cda10 con 0x7f6038070640 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6038169b20 con 0x7f6038070a50 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 0x7f60380afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 0x7f60380afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42682/0 (socket says 172.21.15.116:42682) 2024-05-14T11:26:39.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 -- 172.21.15.116:0/2390561981 learned_addr learned my addr 172.21.15.116:0/2390561981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6036a00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038070640 0x7f60380aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f602e000640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6038070a50 0x7f603813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 -- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6038070a50 msgr2=0x7f603813ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6038070a50 0x7f603813ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 -- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038070640 msgr2=0x7f60380aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038070640 0x7f60380aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60380aed70 con 0x7f60380d0290 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6036a00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6038070640 0x7f60380aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:39.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f6037400640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 0x7f60380afb30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6030007e60 tx=0x7f603000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030014420 con 0x7f60380d0290 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030019070 con 0x7f60380d0290 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60380af050 con 0x7f60380d0290 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030014820 con 0x7f60380d0290 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60380aab10 con 0x7f60380d0290 2024-05-14T11:26:39.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.745+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f603806aa30 con 0x7f60380d0290 2024-05-14T11:26:39.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60300149c0 con 0x7f60380d0290 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f6030090880 con 0x7f60380d0290 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f6036a00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f6014075a30 0x7f6014077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6030014e50 con 0x7f6014075a30 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f6030090c30 con 0x7f60380d0290 2024-05-14T11:26:39.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602e000640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f6014075a30 0x7f6014077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f6036a00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f60280ae000 tx=0x7f60280a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602e000640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f6014075a30 0x7f6014077ec0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.749+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== osd.2 v2:172.21.15.116:6817/2784504541 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6030014e50 con 0x7f6014075a30 2024-05-14T11:26:39.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 --> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f60381fa8b0 con 0x7f6014075a30 2024-05-14T11:26:39.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f602f400640 1 -- 172.21.15.116:0/2390561981 <== osd.2 v2:172.21.15.116:6817/2784504541 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f60381fa8b0 con 0x7f6014075a30 2024-05-14T11:26:39.763 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:39.763 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:39.763 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f6014075a30 msgr2=0x7f6014077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6817/2784504541,v1:172.21.15.116:6819/2784504541] conn(0x7f6014075a30 0x7f6014077ec0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 msgr2=0x7f6014071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f601406f400 0x7f6014071910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f60280ae000 tx=0x7f60280a7000 comp rx=0 tx=0).stop 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 msgr2=0x7f60380afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 --2- 172.21.15.116:0/2390561981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60380d0290 0x7f60380afb30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6030007e60 tx=0x7f603000e4b0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f6037400640 1 -- 172.21.15.116:0/2390561981 reap_dead start 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 shutdown_connections 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 >> 172.21.15.116:0/2390561981 conn(0x7f60380cde80 msgr2=0x7f60380b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 shutdown_connections 2024-05-14T11:26:39.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.765+0000 7f603dc00640 1 -- 172.21.15.116:0/2390561981 wait complete. 2024-05-14T11:26:39.782 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-14T11:26:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 Processor -- start 2024-05-14T11:26:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 -- start start 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f13b81673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b806cf10 con 0x7f13b806f730 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b806d1f0 con 0x7f13b80b0770 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13bd800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13b806db20 con 0x7f13b8164f80 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13ae000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f13b806f730 0x7f13b806fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41356/0 (socket says 172.21.15.116:41356) 2024-05-14T11:26:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.929+0000 7f13ae000640 1 -- 172.21.15.116:0/756169160 learned_addr learned my addr 172.21.15.116:0/756169160 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b7400640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f13b81673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861382844 0 0) 0x7f13b806cf10 con 0x7f13b806f730 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f139c003530 con 0x7f13b806f730 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429805700 0 0) 0x7f13b806d1f0 con 0x7f13b80b0770 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b806cf10 con 0x7f13b80b0770 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199477154 0 0) 0x7f139c003530 con 0x7f13b806f730 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 msgr2=0x7f13b81673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f13b81673a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 >> v1:172.21.15.149:6789/0 conn(0x7f13b80b0770 legacy=0x7f13b806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b806d4d0 con 0x7f13b806f730 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f13a0001700 con 0x7f13b806f730 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 >> v1:172.21.15.116:6789/0 conn(0x7f13b806f730 legacy=0x7f13b806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c003c70 0x7f139c0040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004810 0x7f139c004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f139c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13b806db20 con 0x7f139c003c70 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f139c0055d0 con 0x7f139c004810 2024-05-14T11:26:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f139c00bf10 con 0x7f13b8164f80 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c003c70 0x7f139c0040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13ae000640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004810 0x7f139c004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b7400640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f139c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 -- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 msgr2=0x7f139c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13b8164f80 0x7f139c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 -- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004810 msgr2=0x7f139c004c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f139c004810 0x7f139c004c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.933+0000 7f13b6a00640 1 -- 172.21.15.116:0/756169160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c003530 con 0x7f139c003c70 2024-05-14T11:26:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13b6a00640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c003c70 0x7f139c0040c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f13a80018a0 tx=0x7f13a801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf13a5bae9a0d10 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80014f0 con 0x7f139c003c70 2024-05-14T11:26:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a80a9070 con 0x7f139c003c70 2024-05-14T11:26:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13b6000640 1 -- 172.21.15.116:0/756169160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8007020 con 0x7f139c003c70 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 -- 172.21.15.116:0/756169160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c003c70 msgr2=0x7f139c0040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 --2- 172.21.15.116:0/756169160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f139c003c70 0x7f139c0040c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f13a80018a0 tx=0x7f13a801c4c0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13b7400640 1 -- 172.21.15.116:0/756169160 reap_dead start 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 -- 172.21.15.116:0/756169160 shutdown_connections 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 -- 172.21.15.116:0/756169160 >> 172.21.15.116:0/756169160 conn(0x7f13b80b4620 msgr2=0x7f13b80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 -- 172.21.15.116:0/756169160 shutdown_connections 2024-05-14T11:26:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.937+0000 7f13bd800640 1 -- 172.21.15.116:0/756169160 wait complete. 2024-05-14T11:26:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 Processor -- start 2024-05-14T11:26:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- start start 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 0x7f13b815df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13b814a800 0x7f13b80cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8167520 con 0x7f13b80cb760 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13b81673a0 con 0x7f13b814a800 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13b81676a0 con 0x7f139c003c70 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42688/0 (socket says 172.21.15.116:42688) 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 -- 172.21.15.116:0/43255294 learned_addr learned my addr 172.21.15.116:0/43255294 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b6a00640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 0x7f13b815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13ae000640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13b814a800 0x7f13b80cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 -- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 msgr2=0x7f13b815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 0x7f13b815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 -- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13b814a800 msgr2=0x7f13b80cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13b814a800 0x7f13b80cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 -- 172.21.15.116:0/43255294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b80cbb70 con 0x7f13b80cb760 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b6a00640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 0x7f13b815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13b7400640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f13b8070d10 tx=0x7f13b0010580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0010de0 con 0x7f13b80cb760 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b001b070 con 0x7f13b80cb760 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b815e8f0 con 0x7f13b80cb760 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0015690 con 0x7f13b80cb760 2024-05-14T11:26:39.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.941+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b815eeb0 con 0x7f13b80cb760 2024-05-14T11:26:39.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b806a9c0 con 0x7f13b80cb760 2024-05-14T11:26:39.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13b0015830 con 0x7f13b80cb760 2024-05-14T11:26:39.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13af400640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 0x7f1394071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:39.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f13b008f7f0 con 0x7f13b80cb760 2024-05-14T11:26:39.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13b6a00640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 0x7f1394071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:39.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.945+0000 7f13b6a00640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 0x7f1394071940 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f13b80d1240 tx=0x7f13a8001370 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:39.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:39.949+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f13b80cbb70 con 0x7f13b80cb760 2024-05-14T11:26:40.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.105+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f13b815e680 con 0x7f139406f430 2024-05-14T11:26:40.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13af400640 1 -- 172.21.15.116:0/43255294 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f13b815e680 con 0x7f139406f430 2024-05-14T11:26:40.107 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 2 to deep-scrub 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 msgr2=0x7f1394071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 0x7f1394071940 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f13b80d1240 tx=0x7f13a8001370 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 msgr2=0x7f13b80ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f13b8070d10 tx=0x7f13b0010580 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 shutdown_connections 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f139406f430 0x7f1394071940 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13b814a800 0x7f13b80cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13b80cb760 0x7f13b80ce9e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 --2- 172.21.15.116:0/43255294 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f139c003c70 0x7f13b815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 >> 172.21.15.116:0/43255294 conn(0x7f13b80b4620 msgr2=0x7f13b815fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 shutdown_connections 2024-05-14T11:26:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.109+0000 7f13bd800640 1 -- 172.21.15.116:0/43255294 wait complete. 2024-05-14T11:26:40.128 INFO:tasks.ceph:Scrubbing osd.3 2024-05-14T11:26:40.128 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:40.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.277+0000 7fccfc000640 1 Processor -- start 2024-05-14T11:26:40.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 -- start start 2024-05-14T11:26:40.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40cdc30 0x7fccf415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf40ca6b0 0x7fccf40b8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fccf40caac0 0x7fccf40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fccf40b92d0 con 0x7fccf40ca6b0 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fccf40d11b0 con 0x7fccf40caac0 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfc000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fccf40d1490 con 0x7fccf40cdc30 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf40ca6b0 0x7fccf40b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fccf40ca6b0 0x7fccf40b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42692/0 (socket says 172.21.15.116:42692) 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 -- 172.21.15.116:0/2914984642 learned_addr learned my addr 172.21.15.116:0/2914984642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccfa000640 1 --2- 172.21.15.116:0/2914984642 >> v2:172.21.15.149:3300/0 conn(0x7fccf40caac0 0x7fccf40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf9600640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40cdc30 0x7fccf415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 -- 172.21.15.116:0/2914984642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40cdc30 msgr2=0x7fccf415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:40.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40cdc30 0x7fccf415fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 -- 172.21.15.116:0/2914984642 >> v2:172.21.15.149:3300/0 conn(0x7fccf40caac0 msgr2=0x7fccf40b8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:40.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 --2- 172.21.15.116:0/2914984642 >> v2:172.21.15.149:3300/0 conn(0x7fccf40caac0 0x7fccf40b8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.281+0000 7fccf8c00640 1 -- 172.21.15.116:0/2914984642 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf40d1770 con 0x7fccf40ca6b0 2024-05-14T11:26:40.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccf8c00640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40ca6b0 0x7fccf40b8850 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fccf4074cc0 tx=0x7fcce801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ef653361fd5709 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccf3e00640 1 -- 172.21.15.116:0/2914984642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80a6040 con 0x7fccf40ca6b0 2024-05-14T11:26:40.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccf3e00640 1 -- 172.21.15.116:0/2914984642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce80af070 con 0x7fccf40ca6b0 2024-05-14T11:26:40.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccf3e00640 1 -- 172.21.15.116:0/2914984642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80aa860 con 0x7fccf40ca6b0 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 -- 172.21.15.116:0/2914984642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40ca6b0 msgr2=0x7fccf40b8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40ca6b0 0x7fccf40b8850 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fccf4074cc0 tx=0x7fcce801c520 comp rx=0 tx=0).stop 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 -- 172.21.15.116:0/2914984642 shutdown_connections 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40ca6b0 0x7fccf40b8850 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 --2- 172.21.15.116:0/2914984642 >> v2:172.21.15.149:3300/0 conn(0x7fccf40caac0 0x7fccf40b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 --2- 172.21.15.116:0/2914984642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40cdc30 0x7fccf415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 -- 172.21.15.116:0/2914984642 >> 172.21.15.116:0/2914984642 conn(0x7fccf40d35e0 msgr2=0x7fccf40d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 -- 172.21.15.116:0/2914984642 shutdown_connections 2024-05-14T11:26:40.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.285+0000 7fccfc000640 1 -- 172.21.15.116:0/2914984642 wait complete. 2024-05-14T11:26:40.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 Processor -- start 2024-05-14T11:26:40.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 -- start start 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40ca6b0 0x7fccf414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 0x7fccf415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf40cdc30 0x7fccf415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fccf40d2be0 con 0x7fccf40caac0 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fccf40d2a60 con 0x7fccf40cdc30 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fccf40ce370 con 0x7fccf40ca6b0 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 0x7fccf415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 0x7fccf415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42702/0 (socket says 172.21.15.116:42702) 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 -- 172.21.15.116:0/44948646 learned_addr learned my addr 172.21.15.116:0/44948646 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf9600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40ca6b0 0x7fccf414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfa000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf40cdc30 0x7fccf415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 -- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40ca6b0 msgr2=0x7fccf414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40ca6b0 0x7fccf414cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 -- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf40cdc30 msgr2=0x7fccf415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf40cdc30 0x7fccf415bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf414d210 con 0x7fccf40caac0 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf9600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fccf40ca6b0 0x7fccf414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfa000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccf40cdc30 0x7fccf415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:40.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf8c00640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 0x7fccf415baa0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fcce801cdc0 tx=0x7fcce801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80aae10 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce80bd3f0 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf414f5d0 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.289+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80bd7e0 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf414fb90 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fccf406aa30 con 0x7fccf40caac0 2024-05-14T11:26:40.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcce80b33f0 con 0x7fccf40caac0 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd806f400 0x7fccd8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fcce81291a0 con 0x7fccf40caac0 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf9600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd806f400 0x7fccd8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7fccd8075a30 0x7fccd8077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcce8007e90 con 0x7fccd8075a30 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fcce8129550 con 0x7fccf40caac0 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.293+0000 7fccfa000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7fccd8075a30 0x7fccd8077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.297+0000 7fccf9600640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd806f400 0x7fccd8071910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009b10 tx=0x7fcce4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.297+0000 7fccfa000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7fccd8075a30 0x7fccd8077ec0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.297+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== osd.3 v2:172.21.15.116:6816/1815220698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcce8007e90 con 0x7fccd8075a30 2024-05-14T11:26:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 --> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fccf414f380 con 0x7fccd8075a30 2024-05-14T11:26:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccf1600640 1 -- 172.21.15.116:0/44948646 <== osd.3 v2:172.21.15.116:6816/1815220698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fccf414f380 con 0x7fccd8075a30 2024-05-14T11:26:40.309 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:40.309 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:40.309 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7fccd8075a30 msgr2=0x7fccd8077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6816/1815220698,v1:172.21.15.116:6818/1815220698] conn(0x7fccd8075a30 0x7fccd8077ec0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd806f400 msgr2=0x7fccd8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fccd806f400 0x7fccd8071910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009b10 tx=0x7fcce4008040 comp rx=0 tx=0).stop 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 msgr2=0x7fccf415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 --2- 172.21.15.116:0/44948646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fccf40caac0 0x7fccf415baa0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fcce801cdc0 tx=0x7fcce801ad60 comp rx=0 tx=0).stop 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfa000640 1 -- 172.21.15.116:0/44948646 reap_dead start 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 shutdown_connections 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 >> 172.21.15.116:0/44948646 conn(0x7fccf40d35e0 msgr2=0x7fccf415a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 shutdown_connections 2024-05-14T11:26:40.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.309+0000 7fccfc000640 1 -- 172.21.15.116:0/44948646 wait complete. 2024-05-14T11:26:40.328 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-14T11:26:40.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 Processor -- start 2024-05-14T11:26:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 -- start start 2024-05-14T11:26:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 0x7f03b80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03b8153c90 0x7f03b8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b80ccaa0 con 0x7f03b80d1810 2024-05-14T11:26:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f03b80b1f00 con 0x7f03b8153c90 2024-05-14T11:26:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03bf000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f03b815e070 con 0x7f03b8072470 2024-05-14T11:26:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03b8153c90 0x7f03b8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03b8153c90 0x7f03b8071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42704/0 (socket says 172.21.15.116:42704) 2024-05-14T11:26:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 -- 172.21.15.116:0/2297739602 learned_addr learned my addr 172.21.15.116:0/2297739602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03b7e00640 1 --2- 172.21.15.116:0/2297739602 >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 0x7f03b80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 -- 172.21.15.116:0/2297739602 >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 msgr2=0x7f03b80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 --2- 172.21.15.116:0/2297739602 >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 0x7f03b80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 -- 172.21.15.116:0/2297739602 >> v1:172.21.15.149:6789/0 conn(0x7f03b80d1810 legacy=0x7f03b80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03afe00640 1 -- 172.21.15.116:0/2297739602 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b80d2330 con 0x7f03b8153c90 2024-05-14T11:26:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.477+0000 7f03b7e00640 1 --2- 172.21.15.116:0/2297739602 >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 0x7f03b80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03afe00640 1 --2- 172.21.15.116:0/2297739602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b8153c90 0x7f03b8071a00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f03b8072400 tx=0x7f03a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af9a9c3ae0acc64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03b7400640 1 -- 172.21.15.116:0/2297739602 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00ab430 con 0x7f03b8153c90 2024-05-14T11:26:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03b7400640 1 -- 172.21.15.116:0/2297739602 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a00b0070 con 0x7f03b8153c90 2024-05-14T11:26:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03b7400640 1 -- 172.21.15.116:0/2297739602 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00ab7f0 con 0x7f03b8153c90 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 -- 172.21.15.116:0/2297739602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b8153c90 msgr2=0x7f03b8071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 --2- 172.21.15.116:0/2297739602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b8153c90 0x7f03b8071a00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f03b8072400 tx=0x7f03a001c520 comp rx=0 tx=0).stop 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 -- 172.21.15.116:0/2297739602 shutdown_connections 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 --2- 172.21.15.116:0/2297739602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b8153c90 0x7f03b8071a00 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 --2- 172.21.15.116:0/2297739602 >> v2:172.21.15.149:3300/0 conn(0x7f03b8072470 0x7f03b80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 -- 172.21.15.116:0/2297739602 >> 172.21.15.116:0/2297739602 conn(0x7f03b8073ca0 msgr2=0x7f03b80740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 -- 172.21.15.116:0/2297739602 shutdown_connections 2024-05-14T11:26:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.481+0000 7f03bf000640 1 -- 172.21.15.116:0/2297739602 wait complete. 2024-05-14T11:26:40.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 Processor -- start 2024-05-14T11:26:40.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 -- start start 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8072470 0x7f03b80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b8153c90 0x7f03b813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80c8ec0 con 0x7f03b80d1810 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03b815db50 con 0x7f03b8072470 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bf000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80c9040 con 0x7f03b8153c90 2024-05-14T11:26:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03afe00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42720/0 (socket says 172.21.15.116:42720) 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03afe00640 1 -- 172.21.15.116:0/970180513 learned_addr learned my addr 172.21.15.116:0/970180513 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03b7e00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8072470 0x7f03b80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.485+0000 7f03bd000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b8153c90 0x7f03b813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 -- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b8153c90 msgr2=0x7f03b813acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b8153c90 0x7f03b813acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 -- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8072470 msgr2=0x7f03b80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8072470 0x7f03b80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 -- 172.21.15.116:0/970180513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8149530 con 0x7f03b80d1810 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03afe00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f03a00ad040 tx=0x7f03a001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00be5f0 con 0x7f03b80d1810 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a00be790 con 0x7f03b80d1810 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00beb80 con 0x7f03b80d1810 2024-05-14T11:26:40.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b814ad20 con 0x7f03b80d1810 2024-05-14T11:26:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b814b2e0 con 0x7f03b80d1810 2024-05-14T11:26:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b806a9c0 con 0x7f03b80d1810 2024-05-14T11:26:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.489+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f03a00b43f0 con 0x7f03b80d1810 2024-05-14T11:26:40.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.493+0000 7f03b4c00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 0x7f0394071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.493+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f03a00a8070 con 0x7f03b80d1810 2024-05-14T11:26:40.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.493+0000 7f03b7e00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 0x7f0394071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.493+0000 7f03b7e00640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 0x7f0394071940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006e90 tx=0x7f03a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.497+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f03b8149530 con 0x7f03b80d1810 2024-05-14T11:26:40.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.653+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f03b806fa50 con 0x7f039406f430 2024-05-14T11:26:40.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.653+0000 7f03b4c00640 1 -- 172.21.15.116:0/970180513 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f03b806fa50 con 0x7f039406f430 2024-05-14T11:26:40.653 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 3 to deep-scrub 2024-05-14T11:26:40.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 msgr2=0x7f0394071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 0x7f0394071940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006e90 tx=0x7f03a8008040 comp rx=0 tx=0).stop 2024-05-14T11:26:40.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 msgr2=0x7f03b80cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f03a00ad040 tx=0x7f03a001ad60 comp rx=0 tx=0).stop 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 shutdown_connections 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f039406f430 0x7f0394071940 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b8153c90 0x7f03b813acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b80d1810 0x7f03b80cce40 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 --2- 172.21.15.116:0/970180513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b8072470 0x7f03b80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 >> 172.21.15.116:0/970180513 conn(0x7f03b8073ca0 msgr2=0x7f03b815e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 shutdown_connections 2024-05-14T11:26:40.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.657+0000 7f03bf000640 1 -- 172.21.15.116:0/970180513 wait complete. 2024-05-14T11:26:40.674 INFO:tasks.ceph:Scrubbing osd.4 2024-05-14T11:26:40.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:40.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.825+0000 7fbd82a00640 1 Processor -- start 2024-05-14T11:26:40.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.825+0000 7fbd82a00640 1 -- start start 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd82a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd82a00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fbd7c0d0290 0x7fbd7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd82a00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7c15d360 con 0x7fbd7c070a50 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd82a00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7c15dc90 con 0x7fbd7c0d0290 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd82a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c15d640 con 0x7fbd7c070640 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.116:48058/0 (socket says 172.21.15.116:48058) 2024-05-14T11:26:40.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 -- 172.21.15.116:0/474783855 learned_addr learned my addr 172.21.15.116:0/474783855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd80a00640 1 --2- 172.21.15.116:0/474783855 >> v2:172.21.15.149:3300/0 conn(0x7fbd7c0d0290 0x7fbd7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 -- 172.21.15.116:0/474783855 >> v2:172.21.15.149:3300/0 conn(0x7fbd7c0d0290 msgr2=0x7fbd7c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 --2- 172.21.15.116:0/474783855 >> v2:172.21.15.149:3300/0 conn(0x7fbd7c0d0290 0x7fbd7c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 -- 172.21.15.116:0/474783855 >> v1:172.21.15.149:6789/0 conn(0x7fbd7c070a50 legacy=0x7fbd7c1421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7be00640 1 -- 172.21.15.116:0/474783855 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c15d920 con 0x7fbd7c070640 2024-05-14T11:26:40.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.829+0000 7fbd7b400640 1 -- 172.21.15.116:0/474783855 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937609189 0 0) 0x7fbd7c15d360 con 0x7fbd7c070a50 2024-05-14T11:26:40.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd7be00640 1 --2- 172.21.15.116:0/474783855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009870 tx=0x7fbd6401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cab48524f877b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd7b400640 1 -- 172.21.15.116:0/474783855 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640aa600 con 0x7fbd7c070640 2024-05-14T11:26:40.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd7b400640 1 -- 172.21.15.116:0/474783855 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd640af070 con 0x7fbd7c070640 2024-05-14T11:26:40.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd7b400640 1 -- 172.21.15.116:0/474783855 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640aa9c0 con 0x7fbd7c070640 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- 172.21.15.116:0/474783855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 msgr2=0x7fbd7c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 --2- 172.21.15.116:0/474783855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009870 tx=0x7fbd6401c560 comp rx=0 tx=0).stop 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- 172.21.15.116:0/474783855 shutdown_connections 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 --2- 172.21.15.116:0/474783855 >> v2:172.21.15.149:3300/0 conn(0x7fbd7c0d0290 0x7fbd7c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 --2- 172.21.15.116:0/474783855 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c070640 0x7fbd7c06e870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- 172.21.15.116:0/474783855 >> 172.21.15.116:0/474783855 conn(0x7fbd7c0cde80 msgr2=0x7fbd7c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- 172.21.15.116:0/474783855 shutdown_connections 2024-05-14T11:26:40.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- 172.21.15.116:0/474783855 wait complete. 2024-05-14T11:26:40.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 Processor -- start 2024-05-14T11:26:40.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.833+0000 7fbd82a00640 1 -- start start 2024-05-14T11:26:40.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 0x7fbd7c143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd7c070a50 0x7fbd7c0af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c0d0290 0x7fbd7c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1699a0 con 0x7fbd7c070640 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0cda10 con 0x7fbd7c070a50 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c169b20 con 0x7fbd7c0d0290 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 0x7fbd7c143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 0x7fbd7c143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42732/0 (socket says 172.21.15.116:42732) 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 -- 172.21.15.116:0/4192421368 learned_addr learned my addr 172.21.15.116:0/4192421368 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd80a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c0d0290 0x7fbd7c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd73400640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd7c070a50 0x7fbd7c0af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 -- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c0d0290 msgr2=0x7fbd7c1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd7c0d0290 0x7fbd7c1458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 -- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd7c070a50 msgr2=0x7fbd7c0af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd7c070a50 0x7fbd7c0af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c156560 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd7be00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 0x7fbd7c143eb0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbd6401a190 tx=0x7fbd6401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640bd3f0 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd640bd590 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640bd980 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c0afef0 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c0b4b20 con 0x7fbd7c070640 2024-05-14T11:26:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.837+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd7c06aa30 con 0x7fbd7c070640 2024-05-14T11:26:40.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd640a9070 con 0x7fbd7c070640 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbd5806f400 0x7fbd58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fbd641294a0 con 0x7fbd7c070640 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd73400640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbd5806f400 0x7fbd58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7fbd58075a30 0x7fbd58077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd640b4e50 con 0x7fbd58075a30 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7fbd64129850 con 0x7fbd7c070640 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd80a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7fbd58075a30 0x7fbd58077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd73400640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbd5806f400 0x7fbd58071910 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c13b5b0 tx=0x7fbd6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.841+0000 7fbd80a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7fbd58075a30 0x7fbd58077ec0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.845+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== osd.4 v2:172.21.15.149:6809/3313182122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd640b4e50 con 0x7fbd58075a30 2024-05-14T11:26:40.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 --> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbd7c0b14b0 con 0x7fbd58075a30 2024-05-14T11:26:40.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd78c00640 1 -- 172.21.15.116:0/4192421368 <== osd.4 v2:172.21.15.149:6809/3313182122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbd7c0b14b0 con 0x7fbd58075a30 2024-05-14T11:26:40.857 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:40.857 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:40.857 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7fbd58075a30 msgr2=0x7fbd58077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.149:6809/3313182122,v1:172.21.15.149:6812/3313182122] conn(0x7fbd58075a30 0x7fbd58077ec0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbd5806f400 msgr2=0x7fbd58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fbd5806f400 0x7fbd58071910 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c13b5b0 tx=0x7fbd6c009040 comp rx=0 tx=0).stop 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 msgr2=0x7fbd7c143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 --2- 172.21.15.116:0/4192421368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7c070640 0x7fbd7c143eb0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbd6401a190 tx=0x7fbd6401ada0 comp rx=0 tx=0).stop 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd80a00640 1 -- 172.21.15.116:0/4192421368 reap_dead start 2024-05-14T11:26:40.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 shutdown_connections 2024-05-14T11:26:40.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 >> 172.21.15.116:0/4192421368 conn(0x7fbd7c0cde80 msgr2=0x7fbd7c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:40.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 shutdown_connections 2024-05-14T11:26:40.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:40.857+0000 7fbd82a00640 1 -- 172.21.15.116:0/4192421368 wait complete. 2024-05-14T11:26:40.876 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-14T11:26:41.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 Processor -- start 2024-05-14T11:26:41.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 -- start start 2024-05-14T11:26:41.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fe1e00d1670 0x7fe1e00d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e006ff20 con 0x7fe1e015db30 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00ae100 con 0x7fe1e00d1670 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e7600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e00ae870 con 0x7fe1e006c700 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e4c00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe1e015db30 0x7fe1e00aeaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41372/0 (socket says 172.21.15.116:41372) 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e4c00640 1 -- 172.21.15.116:0/3929586958 learned_addr learned my addr 172.21.15.116:0/3929586958 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e5600640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1dfe00640 1 --2- 172.21.15.116:0/3929586958 >> v2:172.21.15.149:3300/0 conn(0x7fe1e00d1670 0x7fe1e00d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1df400640 1 -- 172.21.15.116:0/3929586958 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867832846 0 0) 0x7fe1e006ff20 con 0x7fe1e015db30 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1df400640 1 -- 172.21.15.116:0/3929586958 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1c8003530 con 0x7fe1e015db30 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1dfe00640 1 -- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 msgr2=0x7fe1e006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1dfe00640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e006cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1dfe00640 1 -- 172.21.15.116:0/3929586958 >> v1:172.21.15.116:6789/0 conn(0x7fe1e015db30 legacy=0x7fe1e00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1dfe00640 1 -- 172.21.15.116:0/3929586958 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e00d1fc0 con 0x7fe1e00d1670 2024-05-14T11:26:41.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.025+0000 7fe1e5600640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:41.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1dfe00640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e00d1a80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0097c0 tx=0x7fe1cc01c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3511224c9b81c958 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1df400640 1 -- 172.21.15.116:0/3929586958 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0b0070 con 0x7fe1e00d1670 2024-05-14T11:26:41.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1df400640 1 -- 172.21.15.116:0/3929586958 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc0ab600 con 0x7fe1e00d1670 2024-05-14T11:26:41.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1df400640 1 -- 172.21.15.116:0/3929586958 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0ab9d0 con 0x7fe1e00d1670 2024-05-14T11:26:41.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 -- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 msgr2=0x7fe1e00d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e00d1a80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0097c0 tx=0x7fe1cc01c340 comp rx=0 tx=0).stop 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 -- 172.21.15.116:0/3929586958 shutdown_connections 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 --2- 172.21.15.116:0/3929586958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e00d1a80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 -- 172.21.15.116:0/3929586958 >> 172.21.15.116:0/3929586958 conn(0x7fe1e00b4620 msgr2=0x7fe1e00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 -- 172.21.15.116:0/3929586958 shutdown_connections 2024-05-14T11:26:41.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.029+0000 7fe1e7600640 1 -- 172.21.15.116:0/3929586958 wait complete. 2024-05-14T11:26:41.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 Processor -- start 2024-05-14T11:26:41.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 -- start start 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e0147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e016a770 con 0x7fe1e015db30 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e015df40 con 0x7fe1e00d1670 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.033+0000 7fe1e7600640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e016a8f0 con 0x7fe1e006c700 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42736/0 (socket says 172.21.15.116:42736) 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 -- 172.21.15.116:0/914241002 learned_addr learned my addr 172.21.15.116:0/914241002 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e4c00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1dfe00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e0147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 -- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 msgr2=0x7fe1e0149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e0149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 -- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 msgr2=0x7fe1e0147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e0147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 -- 172.21.15.116:0/914241002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0149f40 con 0x7fe1e015db30 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e4c00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e0149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e5600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800efc0 tx=0x7fe1d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8018020 con 0x7fe1e015db30 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8014480 con 0x7fe1e015db30 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8014870 con 0x7fe1e015db30 2024-05-14T11:26:41.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e014a220 con 0x7fe1e015db30 2024-05-14T11:26:41.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e014a7e0 con 0x7fe1e015db30 2024-05-14T11:26:41.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e006a9c0 con 0x7fe1e015db30 2024-05-14T11:26:41.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.037+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1d801b020 con 0x7fe1e015db30 2024-05-14T11:26:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.041+0000 7fe1dcc00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 0x7fe1c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.041+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fe1d808ffc0 con 0x7fe1e015db30 2024-05-14T11:26:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.041+0000 7fe1e4c00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 0x7fe1c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.041+0000 7fe1e4c00640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 0x7fe1c0071910 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005890 tx=0x7fe1d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.045+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe1d8022020 con 0x7fe1e015db30 2024-05-14T11:26:41.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.201+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe1e0148330 con 0x7fe1c006f400 2024-05-14T11:26:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.201+0000 7fe1dcc00640 1 -- 172.21.15.116:0/914241002 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1d40013a0 con 0x7fe1c006f400 2024-05-14T11:26:41.201 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 4 to deep-scrub 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 msgr2=0x7fe1c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 0x7fe1c0071910 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005890 tx=0x7fe1d4009040 comp rx=0 tx=0).stop 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 msgr2=0x7fe1e0147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800efc0 tx=0x7fe1d800c490 comp rx=0 tx=0).stop 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 shutdown_connections 2024-05-14T11:26:41.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fe1c006f400 0x7fe1c0071910 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1e00d1670 0x7fe1e0147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1e015db30 0x7fe1e0147560 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 --2- 172.21.15.116:0/914241002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1e006c700 0x7fe1e0149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 >> 172.21.15.116:0/914241002 conn(0x7fe1e00b4620 msgr2=0x7fe1e00b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 shutdown_connections 2024-05-14T11:26:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.205+0000 7fe1e7600640 1 -- 172.21.15.116:0/914241002 wait complete. 2024-05-14T11:26:41.222 INFO:tasks.ceph:Scrubbing osd.5 2024-05-14T11:26:41.222 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:41.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 Processor -- start 2024-05-14T11:26:41.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 -- start start 2024-05-14T11:26:41.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d48165060 0x7f6d48167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4815c800 con 0x7f6d480b5f30 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4815caa0 con 0x7f6d4806cf90 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ee00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4815d0f0 con 0x7f6d48165060 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d4ce00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d48165060 0x7f6d48167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d47e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6d480b5f30 0x7f6d480b6340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41378/0 (socket says 172.21.15.116:41378) 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.373+0000 7f6d47e00640 1 -- 172.21.15.116:0/3014440953 learned_addr learned my addr 172.21.15.116:0/3014440953 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d4ce00640 1 -- 172.21.15.116:0/3014440953 >> v1:172.21.15.149:6789/0 conn(0x7f6d4806cf90 legacy=0x7f6d4806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d4ce00640 1 -- 172.21.15.116:0/3014440953 >> v1:172.21.15.116:6789/0 conn(0x7f6d480b5f30 legacy=0x7f6d480b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d4ce00640 1 -- 172.21.15.116:0/3014440953 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4815cd80 con 0x7f6d48165060 2024-05-14T11:26:41.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d47400640 1 -- 172.21.15.116:0/3014440953 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177566283 0 0) 0x7f6d4815c800 con 0x7f6d480b5f30 2024-05-14T11:26:41.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d4ce00640 1 --2- 172.21.15.116:0/3014440953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d48165060 0x7f6d48167480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000a990 tx=0x7f6d4001d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d364f1d5c2f1d330 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d47400640 1 -- 172.21.15.116:0/3014440953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000b040 con 0x7f6d48165060 2024-05-14T11:26:41.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d47400640 1 -- 172.21.15.116:0/3014440953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d400a76d0 con 0x7f6d48165060 2024-05-14T11:26:41.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.377+0000 7f6d47400640 1 -- 172.21.15.116:0/3014440953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400a7aa0 con 0x7f6d48165060 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- 172.21.15.116:0/3014440953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d48165060 msgr2=0x7f6d48167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 --2- 172.21.15.116:0/3014440953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d48165060 0x7f6d48167480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000a990 tx=0x7f6d4001d870 comp rx=0 tx=0).stop 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- 172.21.15.116:0/3014440953 shutdown_connections 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 --2- 172.21.15.116:0/3014440953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d48165060 0x7f6d48167480 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- 172.21.15.116:0/3014440953 >> 172.21.15.116:0/3014440953 conn(0x7f6d480cde80 msgr2=0x7f6d480ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- 172.21.15.116:0/3014440953 shutdown_connections 2024-05-14T11:26:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- 172.21.15.116:0/3014440953 wait complete. 2024-05-14T11:26:41.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 Processor -- start 2024-05-14T11:26:41.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.381+0000 7f6d4ee00640 1 -- start start 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 0x7f6d48161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d480b5f30 0x7f6d480c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48165060 0x7f6d48146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48169b50 con 0x7f6d4806cf90 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481699d0 con 0x7f6d48165060 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6d48169cd0 con 0x7f6d480b5f30 2024-05-14T11:26:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 0x7f6d48161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 0x7f6d48161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42752/0 (socket says 172.21.15.116:42752) 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 -- 172.21.15.116:0/1088874562 learned_addr learned my addr 172.21.15.116:0/1088874562 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d3f400640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d480b5f30 0x7f6d480c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ce00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48165060 0x7f6d48146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 -- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d480b5f30 msgr2=0x7f6d480c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d480b5f30 0x7f6d480c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 -- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48165060 msgr2=0x7f6d48146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d48165060 0x7f6d48146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d481629d0 con 0x7f6d4806cf90 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d3f400640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d480b5f30 0x7f6d480c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d47e00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 0x7f6d48161840 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800ca80 tx=0x7f6d3800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800ac30 con 0x7f6d4806cf90 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3800add0 con 0x7f6d4806cf90 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38012640 con 0x7f6d4806cf90 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d48162cb0 con 0x7f6d4806cf90 2024-05-14T11:26:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d48163270 con 0x7f6d4806cf90 2024-05-14T11:26:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d48162780 con 0x7f6d4806cf90 2024-05-14T11:26:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.385+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d380127e0 con 0x7f6d4806cf90 2024-05-14T11:26:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d2406f400 0x7f6d24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f6d380902f0 con 0x7f6d4806cf90 2024-05-14T11:26:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d3f400640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d2406f400 0x7f6d24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f6d24075a30 0x7f6d24077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d38006e50 con 0x7f6d24075a30 2024-05-14T11:26:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f6d380906a0 con 0x7f6d4806cf90 2024-05-14T11:26:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d4ce00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f6d24075a30 0x7f6d24077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d3f400640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d2406f400 0x7f6d24071910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d480687c0 tx=0x7f6d30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d4ce00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f6d24075a30 0x7f6d24077ec0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.389+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== osd.5 v2:172.21.15.149:6824/4173680411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d38006e50 con 0x7f6d24075a30 2024-05-14T11:26:41.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.401+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 --> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6d4806aa30 con 0x7f6d24075a30 2024-05-14T11:26:41.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d44c00640 1 -- 172.21.15.116:0/1088874562 <== osd.5 v2:172.21.15.149:6824/4173680411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6d4806aa30 con 0x7f6d24075a30 2024-05-14T11:26:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:41.403 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:41.403 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f6d24075a30 msgr2=0x7f6d24077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.149:6824/4173680411,v1:172.21.15.149:6825/4173680411] conn(0x7f6d24075a30 0x7f6d24077ec0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d2406f400 msgr2=0x7f6d24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f6d2406f400 0x7f6d24071910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d480687c0 tx=0x7f6d30009040 comp rx=0 tx=0).stop 2024-05-14T11:26:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 msgr2=0x7f6d48161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 --2- 172.21.15.116:0/1088874562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d4806cf90 0x7f6d48161840 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800ca80 tx=0x7f6d3800cf40 comp rx=0 tx=0).stop 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ce00640 1 -- 172.21.15.116:0/1088874562 reap_dead start 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 shutdown_connections 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 >> 172.21.15.116:0/1088874562 conn(0x7f6d480cde80 msgr2=0x7f6d480730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 shutdown_connections 2024-05-14T11:26:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.405+0000 7f6d4ee00640 1 -- 172.21.15.116:0/1088874562 wait complete. 2024-05-14T11:26:41.421 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-14T11:26:41.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.569+0000 7fcf51c00640 1 Processor -- start 2024-05-14T11:26:41.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.569+0000 7fcf51c00640 1 -- start start 2024-05-14T11:26:41.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf51c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf4c0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf51c00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fcf4c0af380 0x7fcf4c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf51c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c0d1fc0 con 0x7fcf4c156fa0 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf51c00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c0706f0 con 0x7fcf4c0af380 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf51c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c1573b0 con 0x7fcf4c0d1670 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcf4c156fa0 0x7fcf4c06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41392/0 (socket says 172.21.15.116:41392) 2024-05-14T11:26:41.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 -- 172.21.15.116:0/4088541843 learned_addr learned my addr 172.21.15.116:0/4088541843 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4a000640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf4c0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2041065083 0 0) 0x7fcf3c0014a0 con 0x7fcf4c156fa0 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4b400640 1 --2- 172.21.15.116:0/4088541843 >> v2:172.21.15.149:3300/0 conn(0x7fcf4c0af380 0x7fcf4c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf30003530 con 0x7fcf4c156fa0 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 413142369 0 0) 0x7fcf30003530 con 0x7fcf4c156fa0 2024-05-14T11:26:41.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 msgr2=0x7fcf4c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf4c0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 >> v2:172.21.15.149:3300/0 conn(0x7fcf4c0af380 msgr2=0x7fcf4c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 --2- 172.21.15.116:0/4088541843 >> v2:172.21.15.149:3300/0 conn(0x7fcf4c0af380 0x7fcf4c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c160510 con 0x7fcf4c156fa0 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fcf3c001c50 con 0x7fcf4c156fa0 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 >> v1:172.21.15.116:6789/0 conn(0x7fcf4c156fa0 legacy=0x7fcf4c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf30003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf30004810 0x7fcf30004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf30005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0706f0 con 0x7fcf30003a10 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3000ba80 con 0x7fcf30004810 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3000bc00 con 0x7fcf4c0d1670 2024-05-14T11:26:41.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf30003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4a000640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf30004810 0x7fcf30004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4b400640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf30005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 -- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 msgr2=0x7fcf30005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c0d1670 0x7fcf30005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 -- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf30004810 msgr2=0x7fcf30004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf30004810 0x7fcf30004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.573+0000 7fcf4aa00640 1 -- 172.21.15.116:0/4088541843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf30003530 con 0x7fcf30003a10 2024-05-14T11:26:41.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf30003e80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00aa40 tx=0x7fcf3c01d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78a6a6aa27af183 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00a690 con 0x7fcf30003a10 2024-05-14T11:26:41.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c0072d0 con 0x7fcf30003a10 2024-05-14T11:26:41.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf49600640 1 -- 172.21.15.116:0/4088541843 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0076c0 con 0x7fcf30003a10 2024-05-14T11:26:41.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 -- 172.21.15.116:0/4088541843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 msgr2=0x7fcf30003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 --2- 172.21.15.116:0/4088541843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf30003e80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00aa40 tx=0x7fcf3c01d2e0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf4b400640 1 -- 172.21.15.116:0/4088541843 reap_dead start 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 -- 172.21.15.116:0/4088541843 shutdown_connections 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 -- 172.21.15.116:0/4088541843 >> 172.21.15.116:0/4088541843 conn(0x7fcf4c0b4620 msgr2=0x7fcf4c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 -- 172.21.15.116:0/4088541843 shutdown_connections 2024-05-14T11:26:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.577+0000 7fcf51c00640 1 -- 172.21.15.116:0/4088541843 wait complete. 2024-05-14T11:26:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 Processor -- start 2024-05-14T11:26:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 -- start start 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c073020 0x7fcf4c073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 0x7fcf4c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b0660 con 0x7fcf30003a10 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b04e0 con 0x7fcf4c157840 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c167800 con 0x7fcf4c073020 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42764/0 (socket says 172.21.15.116:42764) 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 -- 172.21.15.116:0/2692536750 learned_addr learned my addr 172.21.15.116:0/2692536750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4a000640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c073020 0x7fcf4c073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4b400640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 0x7fcf4c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 -- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c073020 msgr2=0x7fcf4c073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c073020 0x7fcf4c073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 -- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 msgr2=0x7fcf4c157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 0x7fcf4c157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 -- 172.21.15.116:0/2692536750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c073970 con 0x7fcf30003a10 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4b400640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 0x7fcf4c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf4aa00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c01b9e0 tx=0x7fcf30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0bc070 con 0x7fcf30003a10 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c0014a0 con 0x7fcf30003a10 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c14fee0 con 0x7fcf30003a10 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.581+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0b2040 con 0x7fcf30003a10 2024-05-14T11:26:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c15ba10 con 0x7fcf30003a10 2024-05-14T11:26:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c06a9c0 con 0x7fcf30003a10 2024-05-14T11:26:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf3c0bf070 con 0x7fcf30003a10 2024-05-14T11:26:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf42a00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 0x7fcf2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fcf3c128f50 con 0x7fcf30003a10 2024-05-14T11:26:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf4a000640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 0x7fcf2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.585+0000 7fcf4a000640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 0x7fcf2c071910 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0b2890 tx=0x7fcf38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.589+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcf4c073970 con 0x7fcf30003a10 2024-05-14T11:26:41.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.749+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fcf4c06fdc0 con 0x7fcf2c06f400 2024-05-14T11:26:41.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.749+0000 7fcf42a00640 1 -- 172.21.15.116:0/2692536750 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcf4c06fdc0 con 0x7fcf2c06f400 2024-05-14T11:26:41.749 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 5 to deep-scrub 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 msgr2=0x7fcf2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 0x7fcf2c071910 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0b2890 tx=0x7fcf38008040 comp rx=0 tx=0).stop 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 msgr2=0x7fcf4c15ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c01b9e0 tx=0x7fcf30005840 comp rx=0 tx=0).stop 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 shutdown_connections 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fcf2c06f400 0x7fcf2c071910 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf4c157840 0x7fcf4c157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf4c073020 0x7fcf4c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 --2- 172.21.15.116:0/2692536750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf30003a10 0x7fcf4c15ab50 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 >> 172.21.15.116:0/2692536750 conn(0x7fcf4c0b4620 msgr2=0x7fcf4c166cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 shutdown_connections 2024-05-14T11:26:41.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.753+0000 7fcf51c00640 1 -- 172.21.15.116:0/2692536750 wait complete. 2024-05-14T11:26:41.770 INFO:tasks.ceph:Scrubbing osd.6 2024-05-14T11:26:41.770 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:41.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.921+0000 7f588b800640 1 Processor -- start 2024-05-14T11:26:41.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.921+0000 7f588b800640 1 -- start start 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f588b800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f58840b5f30 0x7f58840b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f588b800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f588415fce0 0x7f58841600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f588b800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58840b6880 con 0x7f588406cf90 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f588b800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f588415d0f0 con 0x7f588415fce0 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f588b800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f588415c800 con 0x7f58840b5f30 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f588415fce0 0x7f58841600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f588415fce0 0x7f58841600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42776/0 (socket says 172.21.15.116:42776) 2024-05-14T11:26:41.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 -- 172.21.15.116:0/1526758226 learned_addr learned my addr 172.21.15.116:0/1526758226 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:41.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 -- 172.21.15.116:0/1526758226 >> v2:172.21.15.149:3300/0 conn(0x7f58840b5f30 msgr2=0x7f58840b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 --2- 172.21.15.116:0/1526758226 >> v2:172.21.15.149:3300/0 conn(0x7f58840b5f30 0x7f58840b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 -- 172.21.15.116:0/1526758226 >> v1:172.21.15.149:6789/0 conn(0x7f588406cf90 legacy=0x7f588406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:41.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.925+0000 7f5889800640 1 -- 172.21.15.116:0/1526758226 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588415caa0 con 0x7f588415fce0 2024-05-14T11:26:41.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f5889800640 1 --2- 172.21.15.116:0/1526758226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588415fce0 0x7f58841600f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81ce2fe8085a9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f5883400640 1 -- 172.21.15.116:0/1526758226 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0ac020 con 0x7f588415fce0 2024-05-14T11:26:41.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f5883400640 1 -- 172.21.15.116:0/1526758226 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587c0a6420 con 0x7f588415fce0 2024-05-14T11:26:41.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f5883400640 1 -- 172.21.15.116:0/1526758226 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0a67f0 con 0x7f588415fce0 2024-05-14T11:26:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 -- 172.21.15.116:0/1526758226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588415fce0 msgr2=0x7f58841600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 --2- 172.21.15.116:0/1526758226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588415fce0 0x7f58841600f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e50 tx=0x7f587c01c810 comp rx=0 tx=0).stop 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 -- 172.21.15.116:0/1526758226 shutdown_connections 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 --2- 172.21.15.116:0/1526758226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588415fce0 0x7f58841600f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 --2- 172.21.15.116:0/1526758226 >> v2:172.21.15.149:3300/0 conn(0x7f58840b5f30 0x7f58840b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 -- 172.21.15.116:0/1526758226 >> 172.21.15.116:0/1526758226 conn(0x7f58840cde80 msgr2=0x7f58840ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 -- 172.21.15.116:0/1526758226 shutdown_connections 2024-05-14T11:26:41.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.929+0000 7f588b800640 1 -- 172.21.15.116:0/1526758226 wait complete. 2024-05-14T11:26:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 Processor -- start 2024-05-14T11:26:41.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- start start 2024-05-14T11:26:41.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 0x7f588414c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f58840b5f30 0x7f5884158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f588415fce0 0x7f5884157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f58840b5380 con 0x7f588406cf90 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58840b5200 con 0x7f588415fce0 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5884160a40 con 0x7f58840b5f30 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 0x7f588414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 0x7f588414c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42788/0 (socket says 172.21.15.116:42788) 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 -- 172.21.15.116:0/1582728582 learned_addr learned my addr 172.21.15.116:0/1582728582 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5883e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f58840b5f30 0x7f5884158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5889800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f588415fce0 0x7f5884157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 -- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f58840b5f30 msgr2=0x7f5884158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f58840b5f30 0x7f5884158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 -- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f588415fce0 msgr2=0x7f5884157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f588415fce0 0x7f5884157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5884150930 con 0x7f588406cf90 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5883e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f58840b5f30 0x7f5884158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5889800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f588415fce0 0x7f5884157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5888e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 0x7f588414c470 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f587400b440 tx=0x7f587400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874015020 con 0x7f588406cf90 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58740118b0 con 0x7f588406cf90 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5884159080 con 0x7f588406cf90 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874011ca0 con 0x7f588406cf90 2024-05-14T11:26:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5884157460 con 0x7f588406cf90 2024-05-14T11:26:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.933+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f588406aa30 con 0x7f588406cf90 2024-05-14T11:26:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5874018020 con 0x7f588406cf90 2024-05-14T11:26:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f586406f400 0x7f5864071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f587408f990 con 0x7f588406cf90 2024-05-14T11:26:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5883e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f586406f400 0x7f5864071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7f5864075a30 0x7f5864077ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f587400ee50 con 0x7f5864075a30 2024-05-14T11:26:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f587408fe20 con 0x7f588406cf90 2024-05-14T11:26:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5889800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7f5864075a30 0x7f5864077ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.937+0000 7f5883e00640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f586406f400 0x7f5864071910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f588414de20 tx=0x7f5870008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.941+0000 7f5889800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7f5864075a30 0x7f5864077ec0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.941+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== osd.6 v2:172.21.15.149:6800/3938301688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f587400ee50 con 0x7f5864075a30 2024-05-14T11:26:41.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 --> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58840d0790 con 0x7f5864075a30 2024-05-14T11:26:41.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f5880c00640 1 -- 172.21.15.116:0/1582728582 <== osd.6 v2:172.21.15.149:6800/3938301688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58840d0790 con 0x7f5864075a30 2024-05-14T11:26:41.953 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:41.953 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:41.953 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7f5864075a30 msgr2=0x7f5864077ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.149:6800/3938301688,v1:172.21.15.149:6801/3938301688] conn(0x7f5864075a30 0x7f5864077ec0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f586406f400 msgr2=0x7f5864071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f586406f400 0x7f5864071910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f588414de20 tx=0x7f5870008040 comp rx=0 tx=0).stop 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 msgr2=0x7f588414c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 --2- 172.21.15.116:0/1582728582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f588406cf90 0x7f588414c470 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f587400b440 tx=0x7f587400b900 comp rx=0 tx=0).stop 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f5889800640 1 -- 172.21.15.116:0/1582728582 reap_dead start 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 shutdown_connections 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 >> 172.21.15.116:0/1582728582 conn(0x7f58840cde80 msgr2=0x7f58841559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 shutdown_connections 2024-05-14T11:26:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:41.953+0000 7f588b800640 1 -- 172.21.15.116:0/1582728582 wait complete. 2024-05-14T11:26:41.972 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-14T11:26:42.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 Processor -- start 2024-05-14T11:26:42.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 -- start start 2024-05-14T11:26:42.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 0x7f639806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63980c74f0 con 0x7f63980c70e0 2024-05-14T11:26:42.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f63981602d0 con 0x7f6398163980 2024-05-14T11:26:42.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f63980cee00 con 0x7f63981642d0 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f6397e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639d800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 0x7f639806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f6397e00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.116:50416/0 (socket says 172.21.15.116:50416) 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639ce00640 1 --1- >> v1:172.21.15.149:6789/0 conn(0x7f63980c70e0 0x7f639815fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.149:6789/0 says I am v1:172.21.15.116:53122/0 (socket says 172.21.15.116:53122) 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f6397e00640 1 -- 172.21.15.116:0/3350577512 learned_addr learned my addr 172.21.15.116:0/3350577512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639d800640 1 -- 172.21.15.116:0/3350577512 >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 msgr2=0x7f6398163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639d800640 1 --2- 172.21.15.116:0/3350577512 >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639d800640 1 -- 172.21.15.116:0/3350577512 >> v1:172.21.15.149:6789/0 conn(0x7f63980c70e0 legacy=0x7f639815fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f639d800640 1 -- 172.21.15.116:0/3350577512 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639806e260 con 0x7f63981642d0 2024-05-14T11:26:42.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.121+0000 7f6397e00640 1 --2- 172.21.15.116:0/3350577512 >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:26:42.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639d800640 1 --2- 172.21.15.116:0/3350577512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 0x7f639806efe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6390009d90 tx=0x7f639001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf42eb46a7c07af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f6397400640 1 -- 172.21.15.116:0/3350577512 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900b4070 con 0x7f63981642d0 2024-05-14T11:26:42.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f6397400640 1 -- 172.21.15.116:0/3350577512 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900af600 con 0x7f63981642d0 2024-05-14T11:26:42.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f6397400640 1 -- 172.21.15.116:0/3350577512 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900af9d0 con 0x7f63981642d0 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 -- 172.21.15.116:0/3350577512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 msgr2=0x7f639806efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 --2- 172.21.15.116:0/3350577512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 0x7f639806efe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6390009d90 tx=0x7f639001f150 comp rx=0 tx=0).stop 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 -- 172.21.15.116:0/3350577512 shutdown_connections 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 --2- 172.21.15.116:0/3350577512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63981642d0 0x7f639806efe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 --2- 172.21.15.116:0/3350577512 >> v2:172.21.15.149:3300/0 conn(0x7f6398163980 0x7f6398163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 -- 172.21.15.116:0/3350577512 >> 172.21.15.116:0/3350577512 conn(0x7f63980d0670 msgr2=0x7f63980d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 -- 172.21.15.116:0/3350577512 shutdown_connections 2024-05-14T11:26:42.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.125+0000 7f639f800640 1 -- 172.21.15.116:0/3350577512 wait complete. 2024-05-14T11:26:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 Processor -- start 2024-05-14T11:26:42.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 -- start start 2024-05-14T11:26:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63980c70e0 0x7f63981482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63981642d0 0x7f6398162890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f639806eac0 con 0x7f6398163980 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f639806e7e0 con 0x7f63981642d0 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639f800640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6398164870 con 0x7f63980c70e0 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f6397e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f6397e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42802/0 (socket says 172.21.15.116:42802) 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f6397e00640 1 -- 172.21.15.116:0/3310220138 learned_addr learned my addr 172.21.15.116:0/3310220138 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639d800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63981642d0 0x7f6398162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.129+0000 7f639ce00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63980c70e0 0x7f63981482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 -- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63980c70e0 msgr2=0x7f63981482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63980c70e0 0x7f63981482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 -- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63981642d0 msgr2=0x7f6398162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63981642d0 0x7f6398162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 -- 172.21.15.116:0/3310220138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63981617f0 con 0x7f6398163980 2024-05-14T11:26:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6397e00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f638400e980 tx=0x7f638400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384017070 con 0x7f6398163980 2024-05-14T11:26:42.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638400c9a0 con 0x7f6398163980 2024-05-14T11:26:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6398161ad0 con 0x7f6398163980 2024-05-14T11:26:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638400cd90 con 0x7f6398163980 2024-05-14T11:26:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6398162090 con 0x7f6398163980 2024-05-14T11:26:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639806a9c0 con 0x7f6398163980 2024-05-14T11:26:42.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f638401a070 con 0x7f6398163980 2024-05-14T11:26:42.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.133+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f638408f370 con 0x7f6398163980 2024-05-14T11:26:42.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.137+0000 7f639ce00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.137+0000 7f639ce00640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f70 tx=0x7f638c007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.141+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f638405bcc0 con 0x7f6398163980 2024-05-14T11:26:42.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.297+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f63980c6800 con 0x7f637806f400 2024-05-14T11:26:42.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.297+0000 7f6394c00640 1 -- 172.21.15.116:0/3310220138 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f63980c6800 con 0x7f637806f400 2024-05-14T11:26:42.298 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 6 to deep-scrub 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 msgr2=0x7f6378071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f638c000f70 tx=0x7f638c007e50 comp rx=0 tx=0).stop 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 msgr2=0x7f6398160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f638400e980 tx=0x7f638400ee40 comp rx=0 tx=0).stop 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 shutdown_connections 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63981642d0 0x7f6398162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398163980 0x7f6398160d30 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 --2- 172.21.15.116:0/3310220138 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63980c70e0 0x7f63981482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 >> 172.21.15.116:0/3310220138 conn(0x7f63980d0670 msgr2=0x7f639815df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 shutdown_connections 2024-05-14T11:26:42.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.301+0000 7f639f800640 1 -- 172.21.15.116:0/3310220138 wait complete. 2024-05-14T11:26:42.319 INFO:tasks.ceph:Scrubbing osd.7 2024-05-14T11:26:42.319 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-14T11:26:42.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.469+0000 7f88b1c00640 1 Processor -- start 2024-05-14T11:26:42.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.469+0000 7f88b1c00640 1 -- start start 2024-05-14T11:26:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88b1c00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88ac0d1bc0 0x7f88ac0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88b1c00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ac06fa90 con 0x7f88ac15cd00 2024-05-14T11:26:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88b1c00640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ac06fd70 con 0x7f88ac0cbf70 2024-05-14T11:26:42.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88b1c00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f88ac0d00d0 con 0x7f88ac0d1bc0 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88a3400640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88ac0d1bc0 0x7f88ac0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88abe00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f88ac15cd00 0x7f88ac0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41408/0 (socket says 172.21.15.116:41408) 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88abe00640 1 -- 172.21.15.116:0/591700010 learned_addr learned my addr 172.21.15.116:0/591700010 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88a3400640 1 -- 172.21.15.116:0/591700010 >> v1:172.21.15.149:6789/0 conn(0x7f88ac0cbf70 legacy=0x7f88ac0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88a3400640 1 -- 172.21.15.116:0/591700010 >> v1:172.21.15.116:6789/0 conn(0x7f88ac15cd00 legacy=0x7f88ac0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88a3400640 1 -- 172.21.15.116:0/591700010 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac070050 con 0x7f88ac0d1bc0 2024-05-14T11:26:42.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.473+0000 7f88ab400640 1 -- 172.21.15.116:0/591700010 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278949476 0 0) 0x7f88ac06fa90 con 0x7f88ac15cd00 2024-05-14T11:26:42.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88a3400640 1 --2- 172.21.15.116:0/591700010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac0d1bc0 0x7f88ac0cfb90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f889401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff39b2a84d5620d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88ab400640 1 -- 172.21.15.116:0/591700010 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940a7070 con 0x7f88ac0d1bc0 2024-05-14T11:26:42.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88ab400640 1 -- 172.21.15.116:0/591700010 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88940b0070 con 0x7f88ac0d1bc0 2024-05-14T11:26:42.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88ab400640 1 -- 172.21.15.116:0/591700010 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0cd2a0 con 0x7f88ac0d1bc0 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- 172.21.15.116:0/591700010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac0d1bc0 msgr2=0x7f88ac0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 --2- 172.21.15.116:0/591700010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac0d1bc0 0x7f88ac0cfb90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f88940097c0 tx=0x7f889401c470 comp rx=0 tx=0).stop 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- 172.21.15.116:0/591700010 shutdown_connections 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 --2- 172.21.15.116:0/591700010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac0d1bc0 0x7f88ac0cfb90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- 172.21.15.116:0/591700010 >> 172.21.15.116:0/591700010 conn(0x7f88ac0b43a0 msgr2=0x7f88ac0b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- 172.21.15.116:0/591700010 shutdown_connections 2024-05-14T11:26:42.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- 172.21.15.116:0/591700010 wait complete. 2024-05-14T11:26:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 Processor -- start 2024-05-14T11:26:42.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.477+0000 7f88b1c00640 1 -- start start 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac0cbf70 0x7f88ac1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 0x7f88ac0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f88ac153c70 0x7f88ac0985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac16a8f0 con 0x7f88ac15cd00 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac16a770 con 0x7f88ac0cbf70 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f88ac16aa70 con 0x7f88ac153c70 2024-05-14T11:26:42.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 0x7f88ac0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 0x7f88ac0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42822/0 (socket says 172.21.15.116:42822) 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 -- 172.21.15.116:0/368692171 learned_addr learned my addr 172.21.15.116:0/368692171 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88abe00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac0cbf70 0x7f88ac1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1200640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f88ac153c70 0x7f88ac0985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 -- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f88ac153c70 msgr2=0x7f88ac0985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f88ac153c70 0x7f88ac0985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 -- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac0cbf70 msgr2=0x7f88ac1583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac0cbf70 0x7f88ac1583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac097500 con 0x7f88ac15cd00 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88abe00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac0cbf70 0x7f88ac1583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1200640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f88ac153c70 0x7f88ac0985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a3400640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 0x7f88ac0cb280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f889401a0a0 tx=0x7f889401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940bf610 con 0x7f88ac15cd00 2024-05-14T11:26:42.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88940bf7d0 con 0x7f88ac15cd00 2024-05-14T11:26:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88940bfbc0 con 0x7f88ac15cd00 2024-05-14T11:26:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac0977e0 con 0x7f88ac15cd00 2024-05-14T11:26:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac097da0 con 0x7f88ac15cd00 2024-05-14T11:26:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.481+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88ac050880 con 0x7f88ac15cd00 2024-05-14T11:26:42.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f88940bd020 con 0x7f88ac15cd00 2024-05-14T11:26:42.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f888806f430 0x7f8888071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f889412a2f0 con 0x7f88ac15cd00 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88abe00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f888806f430 0x7f8888071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7f8888075a60 0x7f8888077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88940aaec0 con 0x7f8888075a60 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f889412a6a0 con 0x7f88ac15cd00 2024-05-14T11:26:42.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88abe00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f888806f430 0x7f8888071940 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f889c000f70 tx=0x7f889c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88b1200640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7f8888075a60 0x7f8888077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.485+0000 7f88b1200640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7f8888075a60 0x7f8888077ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.489+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== osd.7 v2:172.21.15.149:6808/2875938804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88940aaec0 con 0x7f8888075a60 2024-05-14T11:26:42.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 --> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f88ac06ccd0 con 0x7f8888075a60 2024-05-14T11:26:42.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88a8c00640 1 -- 172.21.15.116:0/368692171 <== osd.7 v2:172.21.15.149:6808/2875938804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f88ac06ccd0 con 0x7f8888075a60 2024-05-14T11:26:42.500 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-05-14T11:26:42.501 INFO:teuthology.orchestra.run.smithi116.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-14T11:26:42.501 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-05-14T11:26:42.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7f8888075a60 msgr2=0x7f8888077ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.149:6808/2875938804,v1:172.21.15.149:6810/2875938804] conn(0x7f8888075a60 0x7f8888077ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f888806f430 msgr2=0x7f8888071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f888806f430 0x7f8888071940 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f889c000f70 tx=0x7f889c009040 comp rx=0 tx=0).stop 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 msgr2=0x7f88ac0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 --2- 172.21.15.116:0/368692171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88ac15cd00 0x7f88ac0cb280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f889401a0a0 tx=0x7f889401acb0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1200640 1 -- 172.21.15.116:0/368692171 reap_dead start 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 shutdown_connections 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 >> 172.21.15.116:0/368692171 conn(0x7f88ac0b43a0 msgr2=0x7f88ac0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 shutdown_connections 2024-05-14T11:26:42.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.501+0000 7f88b1c00640 1 -- 172.21.15.116:0/368692171 wait complete. 2024-05-14T11:26:42.519 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-14T11:26:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.665+0000 7fb273200640 1 Processor -- start 2024-05-14T11:26:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.665+0000 7fb273200640 1 -- start start 2024-05-14T11:26:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb273200640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb26c156fa0 0x7fb26c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb273200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0af380 0x7fb26c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb273200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb26c160510 con 0x7fb26c0d1670 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb273200640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb26c1573b0 con 0x7fb26c156fa0 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb273200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb26c161120 con 0x7fb26c0af380 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26be00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb26c156fa0 0x7fb26c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb263e00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb26c0d1670 0x7fb26c0d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:41410/0 (socket says 172.21.15.116:41410) 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb263e00640 1 -- 172.21.15.116:0/4112363728 learned_addr learned my addr 172.21.15.116:0/4112363728 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26be00640 1 -- 172.21.15.116:0/4112363728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0af380 msgr2=0x7fb26c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26be00640 1 --2- 172.21.15.116:0/4112363728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0af380 0x7fb26c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26be00640 1 -- 172.21.15.116:0/4112363728 >> v1:172.21.15.116:6789/0 conn(0x7fb26c0d1670 legacy=0x7fb26c0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26be00640 1 -- 172.21.15.116:0/4112363728 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1607f0 con 0x7fb26c156fa0 2024-05-14T11:26:42.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.669+0000 7fb26b400640 1 -- 172.21.15.116:0/4112363728 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963798006 0 0) 0x7fb26c160510 con 0x7fb26c0d1670 2024-05-14T11:26:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb26be00640 1 --2- 172.21.15.116:0/4112363728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c156fa0 0x7fb26c06ffe0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0097c0 tx=0x7fb25c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c761abcaa0b7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb26b400640 1 -- 172.21.15.116:0/4112363728 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0a7070 con 0x7fb26c156fa0 2024-05-14T11:26:42.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb26b400640 1 -- 172.21.15.116:0/4112363728 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c0b0070 con 0x7fb26c156fa0 2024-05-14T11:26:42.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb26b400640 1 -- 172.21.15.116:0/4112363728 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0ab890 con 0x7fb26c156fa0 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 -- 172.21.15.116:0/4112363728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c156fa0 msgr2=0x7fb26c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 --2- 172.21.15.116:0/4112363728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c156fa0 0x7fb26c06ffe0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0097c0 tx=0x7fb25c01c470 comp rx=0 tx=0).stop 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 -- 172.21.15.116:0/4112363728 shutdown_connections 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 --2- 172.21.15.116:0/4112363728 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0af380 0x7fb26c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 --2- 172.21.15.116:0/4112363728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c156fa0 0x7fb26c06ffe0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 -- 172.21.15.116:0/4112363728 >> 172.21.15.116:0/4112363728 conn(0x7fb26c0b4620 msgr2=0x7fb26c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 -- 172.21.15.116:0/4112363728 shutdown_connections 2024-05-14T11:26:42.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.673+0000 7fb273200640 1 -- 172.21.15.116:0/4112363728 wait complete. 2024-05-14T11:26:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 Processor -- start 2024-05-14T11:26:42.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- start start 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0d1670 0x7fb26c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 0x7fb26c15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c0b0660 con 0x7fb26c0af380 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c0b04e0 con 0x7fb26c156fa0 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb26c169810 con 0x7fb26c0d1670 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42852/0 (socket says 172.21.15.116:42852) 2024-05-14T11:26:42.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 -- 172.21.15.116:0/1361000859 learned_addr learned my addr 172.21.15.116:0/1361000859 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 -- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0d1670 msgr2=0x7fb26c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0d1670 0x7fb26c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 -- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 msgr2=0x7fb26c15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb271200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 0x7fb26c15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 0x7fb26c15c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 -- 172.21.15.116:0/1361000859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c0b1760 con 0x7fb26c0af380 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb271200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 0x7fb26c15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb26be00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ad040 tx=0x7fb25c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0bf610 con 0x7fb26c0af380 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c0bf7d0 con 0x7fb26c0af380 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c0b1a40 con 0x7fb26c0af380 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0bfbc0 con 0x7fb26c0af380 2024-05-14T11:26:42.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.677+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c154ab0 con 0x7fb26c0af380 2024-05-14T11:26:42.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c06a9c0 con 0x7fb26c0af380 2024-05-14T11:26:42.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb25c0bd020 con 0x7fb26c0af380 2024-05-14T11:26:42.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb268c00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 0x7fb248071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:42.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fb25c129d80 con 0x7fb26c0af380 2024-05-14T11:26:42.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb263e00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 0x7fb248071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:42.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.681+0000 7fb263e00640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 0x7fb248071940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b5110 tx=0x7fb254007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:42.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.685+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb25c016090 con 0x7fb26c0af380 2024-05-14T11:26:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.841+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb26c1f92b0 con 0x7fb24806f430 2024-05-14T11:26:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.841+0000 7fb268c00640 1 -- 172.21.15.116:0/1361000859 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb26c1f92b0 con 0x7fb24806f430 2024-05-14T11:26:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:instructed osd(s) 7 to deep-scrub 2024-05-14T11:26:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 msgr2=0x7fb248071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 0x7fb248071940 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0b5110 tx=0x7fb254007970 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 msgr2=0x7fb26c1f8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ad040 tx=0x7fb25c01acb0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 shutdown_connections 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb24806f430 0x7fb248071940 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb26c156fa0 0x7fb26c15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb26c0d1670 0x7fb26c0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 --2- 172.21.15.116:0/1361000859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb26c0af380 0x7fb26c1f8d70 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 >> 172.21.15.116:0/1361000859 conn(0x7fb26c0b4620 msgr2=0x7fb26c166ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 shutdown_connections 2024-05-14T11:26:42.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:42.845+0000 7fb273200640 1 -- 172.21.15.116:0/1361000859 wait complete. 2024-05-14T11:26:42.864 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:26:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.009+0000 7f90dea00640 1 Processor -- start 2024-05-14T11:26:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.009+0000 7f90dea00640 1 -- start start 2024-05-14T11:26:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 0x7f90d806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d80d1670 0x7f90d80d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90d806e9e0 0x7f90d806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f90d81573b0 con 0x7f90d806e9e0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f90d8161120 con 0x7f90d8156fa0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90d80d1fc0 con 0x7f90d80d1670 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90d806e9e0 0x7f90d806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90d806e9e0 0x7f90d806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42868/0 (socket says 172.21.15.116:42868) 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 -- 172.21.15.116:0/691529511 learned_addr learned my addr 172.21.15.116:0/691529511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90d7e00640 1 --2- 172.21.15.116:0/691529511 >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 0x7f90d806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90d7400640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d80d1670 0x7f90d80d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 -- 172.21.15.116:0/691529511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d80d1670 msgr2=0x7f90d80d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d80d1670 0x7f90d80d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 -- 172.21.15.116:0/691529511 >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 msgr2=0x7f90d806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 --2- 172.21.15.116:0/691529511 >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 0x7f90d806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 -- 172.21.15.116:0/691529511 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d8160510 con 0x7f90d806e9e0 2024-05-14T11:26:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90d7e00640 1 --2- 172.21.15.116:0/691529511 >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 0x7f90d806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T11:26:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.013+0000 7f90dca00640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d806e9e0 0x7f90d806edf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f90d80d2170 tx=0x7f90d001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c62a6a7b008437 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90d6a00640 1 -- 172.21.15.116:0/691529511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aa610 con 0x7f90d806e9e0 2024-05-14T11:26:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90d6a00640 1 -- 172.21.15.116:0/691529511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d00af070 con 0x7f90d806e9e0 2024-05-14T11:26:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90d6a00640 1 -- 172.21.15.116:0/691529511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aa9d0 con 0x7f90d806e9e0 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- 172.21.15.116:0/691529511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d806e9e0 msgr2=0x7f90d806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d806e9e0 0x7f90d806edf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f90d80d2170 tx=0x7f90d001c4f0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- 172.21.15.116:0/691529511 shutdown_connections 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d806e9e0 0x7f90d806edf0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 --2- 172.21.15.116:0/691529511 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d80d1670 0x7f90d80d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 --2- 172.21.15.116:0/691529511 >> v2:172.21.15.149:3300/0 conn(0x7f90d8156fa0 0x7f90d806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- 172.21.15.116:0/691529511 >> 172.21.15.116:0/691529511 conn(0x7f90d80b4620 msgr2=0x7f90d80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- 172.21.15.116:0/691529511 shutdown_connections 2024-05-14T11:26:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- 172.21.15.116:0/691529511 wait complete. 2024-05-14T11:26:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 Processor -- start 2024-05-14T11:26:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.017+0000 7f90dea00640 1 -- start start 2024-05-14T11:26:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d806e9e0 0x7f90d815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d80d1670 0x7f90d806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8070520 con 0x7f90d8156fa0 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90d806f740 con 0x7f90d80d1670 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90d80706a0 con 0x7f90d806e9e0 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42882/0 (socket says 172.21.15.116:42882) 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 -- 172.21.15.116:0/3077122545 learned_addr learned my addr 172.21.15.116:0/3077122545 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90d7400640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d80d1670 0x7f90d806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90d7e00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d806e9e0 0x7f90d815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 -- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d806e9e0 msgr2=0x7f90d815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d806e9e0 0x7f90d815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 -- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d80d1670 msgr2=0x7f90d806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d80d1670 0x7f90d806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 -- 172.21.15.116:0/3077122545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d809b280 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dca00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f90d001aba0 tx=0x7f90d001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00bd5f0 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d00bd790 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00bdb80 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d81f7720 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81f7ce0 con 0x7f90d8156fa0 2024-05-14T11:26:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.021+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d81f7490 con 0x7f90d8156fa0 2024-05-14T11:26:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.025+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f90d00b33f0 con 0x7f90d8156fa0 2024-05-14T11:26:43.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.025+0000 7f90cfe00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 0x7f90b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:26:43.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.025+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f90d0129b00 con 0x7f90d8156fa0 2024-05-14T11:26:43.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.025+0000 7f90d7e00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 0x7f90b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:26:43.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.025+0000 7f90d7e00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 0x7f90b8071940 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90d80b0d60 tx=0x7f90c80098f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:26:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.029+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f90d00a7070 con 0x7f90d8156fa0 2024-05-14T11:26:43.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.185+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90d806a9c0 con 0x7f90b806f430 2024-05-14T11:26:43.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.185+0000 7f90cfe00640 1 -- 172.21.15.116:0/3077122545 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13377 (secure 0 0 0) 0x7f90d806a9c0 con 0x7f90b806f430 2024-05-14T11:26:43.186 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:26:43.186 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 msgr2=0x7f90b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 0x7f90b8071940 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90d80b0d60 tx=0x7f90c80098f0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 msgr2=0x7f90d8150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f90d001aba0 tx=0x7f90d001ad30 comp rx=0 tx=0).stop 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 shutdown_connections 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f90b806f430 0x7f90b8071940 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d80d1670 0x7f90d806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8156fa0 0x7f90d8150040 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 --2- 172.21.15.116:0/3077122545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90d806e9e0 0x7f90d815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:26:43.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 >> 172.21.15.116:0/3077122545 conn(0x7f90d80b4620 msgr2=0x7f90d80b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:26:43.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 shutdown_connections 2024-05-14T11:26:43.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:26:43.189+0000 7f90dea00640 1 -- 172.21.15.116:0/3077122545 wait complete. 2024-05-14T11:26:43.207 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-05-14T11:26:42.520256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262164,"kb_used_data":12712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712556,"statfs":{"total":773094113280,"available":772825657344,"internally_reserved":0,"allocated":13017088,"data_stored":6689048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.617814"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":289,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-14T11:26:35.216442+0000","last_change":"2024-05-14T11:16:01.856100+0000","last_active":"2024-05-14T11:26:35.216442+0000","last_peered":"2024-05-14T11:26:35.216442+0000","last_clean":"2024-05-14T11:26:35.216442+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:26:35.216442+0000","last_undegraded":"2024-05-14T11:26:35.216442+0000","last_fullsized":"2024-05-14T11:26:35.216442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T11:16:00.721842+0000","last_clean_scrub_stamp":"2024-05-14T11:16:00.721842+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:43:45.462226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:26:43.211 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-14T11:16:00.721842+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=14, tm_hour=11, tm_min=16, tm_sec=0, tm_wday=1, tm_yday=135, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=14, tm_hour=11, tm_min=26, tm_sec=37, tm_wday=1, tm_yday=135, tm_isdst=0) 2024-05-14T11:26:43.211 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-14T11:26:43.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:26:43.861 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:03.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T11:27:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 Processor -- start 2024-05-14T11:27:03.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 -- start start 2024-05-14T11:27:03.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7fb9f00d1400 0x7fb9f00d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9f00d1d50 con 0x7fb9f00cf370 2024-05-14T11:27:03.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9f006eea0 con 0x7fb9f015ffa0 2024-05-14T11:27:03.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f8200640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f01611d0 con 0x7fb9f00d1400 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f6200640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb9f00cf370 0x7fb9f0160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39244/0 (socket says 172.21.15.116:39244) 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f6200640 1 -- 172.21.15.116:0/660603551 learned_addr learned my addr 172.21.15.116:0/660603551 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9f5800640 1 --2- 172.21.15.116:0/660603551 >> v2:172.21.15.149:3300/0 conn(0x7fb9f00d1400 0x7fb9f00d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826043356 0 0) 0x7fb9e80014a0 con 0x7fb9f00cf370 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.373+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d8003530 con 0x7fb9f00cf370 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675100423 0 0) 0x7fb9f006eea0 con 0x7fb9f015ffa0 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9e80014a0 con 0x7fb9f015ffa0 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2506004667 0 0) 0x7fb9d8003530 con 0x7fb9f00cf370 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 >> v2:172.21.15.149:3300/0 conn(0x7fb9f00d1400 msgr2=0x7fb9f00d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 --2- 172.21.15.116:0/660603551 >> v2:172.21.15.149:3300/0 conn(0x7fb9f00d1400 0x7fb9f00d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 >> v1:172.21.15.149:6789/0 conn(0x7fb9f015ffa0 legacy=0x7fb9f01603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f006f180 con 0x7fb9f00cf370 2024-05-14T11:27:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7fb9e8001c50 con 0x7fb9f00cf370 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f5800640 1 --2- 172.21.15.116:0/660603551 >> v2:172.21.15.149:3300/0 conn(0x7fb9f00d1400 0x7fb9f00d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 >> v1:172.21.15.116:6789/0 conn(0x7fb9f00cf370 legacy=0x7fb9f0160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9d8003c40 0x7fb9d8004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 0x7fb9d8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8005290 0x7fb9d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01611d0 con 0x7fb9d8004810 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d800c240 con 0x7fb9d8003c40 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d800c440 con 0x7fb9d8005290 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 0x7fb9d8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f5800640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9d8003c40 0x7fb9d8004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f6200640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8005290 0x7fb9d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 -- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8005290 msgr2=0x7fb9d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8005290 0x7fb9d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 -- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9d8003c40 msgr2=0x7fb9d8004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9d8003c40 0x7fb9d8004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f4e00640 1 -- 172.21.15.116:0/660603551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d8003530 con 0x7fb9d8004810 2024-05-14T11:27:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.377+0000 7fb9f6200640 1 -- 172.21.15.116:0/660603551 reap_dead start 2024-05-14T11:27:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f4e00640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 0x7fb9d8004c20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc001900 tx=0x7fb9dc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60541b35623da75f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc001550 con 0x7fb9d8004810 2024-05-14T11:27:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc0052d0 con 0x7fb9d8004810 2024-05-14T11:27:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9efe00640 1 -- 172.21.15.116:0/660603551 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0054c0 con 0x7fb9d8004810 2024-05-14T11:27:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 msgr2=0x7fb9d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 0x7fb9d8004c20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc001900 tx=0x7fb9dc01ec70 comp rx=0 tx=0).stop 2024-05-14T11:27:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- 172.21.15.116:0/660603551 shutdown_connections 2024-05-14T11:27:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 --2- 172.21.15.116:0/660603551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9d8004810 0x7fb9d8004c20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- 172.21.15.116:0/660603551 >> 172.21.15.116:0/660603551 conn(0x7fb9f00d0670 msgr2=0x7fb9f00d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- 172.21.15.116:0/660603551 shutdown_connections 2024-05-14T11:27:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- 172.21.15.116:0/660603551 wait complete. 2024-05-14T11:27:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 Processor -- start 2024-05-14T11:27:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.381+0000 7fb9f8200640 1 -- start start 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8004810 0x7fb9f0153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9f0157260 0x7fb9f00ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f00cddf0 con 0x7fb9f0157670 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f006f740 con 0x7fb9f0157260 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f00d2260 con 0x7fb9d8004810 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58770/0 (socket says 172.21.15.116:58770) 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 -- 172.21.15.116:0/1567066278 learned_addr learned my addr 172.21.15.116:0/1567066278 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f4e00640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9f0157260 0x7fb9f00ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f5800640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8004810 0x7fb9f0153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 -- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8004810 msgr2=0x7fb9f0153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8004810 0x7fb9f0153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 -- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9f0157260 msgr2=0x7fb9f00ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9f0157260 0x7fb9f00ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 -- 172.21.15.116:0/1567066278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f0158070 con 0x7fb9f0157670 2024-05-14T11:27:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f6200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0070280 tx=0x7fb9e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8007ac0 con 0x7fb9f0157670 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e800ee50 con 0x7fb9f0157670 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f0158570 con 0x7fb9f0157670 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8013420 con 0x7fb9f0157670 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f0155ef0 con 0x7fb9f0157670 2024-05-14T11:27:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.385+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f006a9c0 con 0x7fb9f0157670 2024-05-14T11:27:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.389+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb9e801a020 con 0x7fb9f0157670 2024-05-14T11:27:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.389+0000 7fb9ed600640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 0x7fb9d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.389+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7fb9e808edb0 con 0x7fb9f0157670 2024-05-14T11:27:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.389+0000 7fb9f5800640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 0x7fb9d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.389+0000 7fb9f5800640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 0x7fb9d0071940 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b4e20 tx=0x7fb9e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.393+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb9e8021020 con 0x7fb9f0157670 2024-05-14T11:27:03.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 --> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9f00b9510 con 0x7fb9d006f430 2024-05-14T11:27:03.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9ed600640 1 -- 172.21.15.116:0/1567066278 <== mgr.4104 v2:172.21.15.116:6832/1273922728 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7fb9f00b9510 con 0x7fb9d006f430 2024-05-14T11:27:03.548 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:27:03.548 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 msgr2=0x7fb9d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 0x7fb9d0071940 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b4e20 tx=0x7fb9e4008040 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 msgr2=0x7fb9f015dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0070280 tx=0x7fb9e800e530 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 shutdown_connections 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7fb9d006f430 0x7fb9d0071940 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9f0157260 0x7fb9f00ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0157670 0x7fb9f015dc60 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 --2- 172.21.15.116:0/1567066278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb9d8004810 0x7fb9f0153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.549+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 >> 172.21.15.116:0/1567066278 conn(0x7fb9f00d0670 msgr2=0x7fb9f01549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.553+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 shutdown_connections 2024-05-14T11:27:03.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.553+0000 7fb9f8200640 1 -- 172.21.15.116:0/1567066278 wait complete. 2024-05-14T11:27:03.570 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-05-14T11:27:02.524944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262156,"kb_used_data":12712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712564,"statfs":{"total":773094113280,"available":772825665536,"internally_reserved":0,"allocated":13017088,"data_stored":6689048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002972"},"pg_stats":[{"pgid":"1.0","version":"118'74","reported_seq":300,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-14T11:26:43.375280+0000","last_change":"2024-05-14T11:26:43.375316+0000","last_active":"2024-05-14T11:26:43.375280+0000","last_peered":"2024-05-14T11:26:43.375280+0000","last_clean":"2024-05-14T11:26:43.375280+0000","last_became_active":"2024-05-14T11:16:01.855065+0000","last_became_peered":"2024-05-14T11:16:01.855065+0000","last_unstale":"2024-05-14T11:26:43.375280+0000","last_undegraded":"2024-05-14T11:26:43.375280+0000","last_fullsized":"2024-05-14T11:26:43.375280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'74","last_scrub_stamp":"2024-05-14T11:26:43.375201+0000","last_deep_scrub":"118'74","last_deep_scrub_stamp":"2024-05-14T11:26:43.375201+0000","last_clean_scrub_stamp":"2024-05-14T11:26:43.375201+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:45:56.604467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":688539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1278907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T11:27:03.570 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-14T11:27:03.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 Processor -- start 2024-05-14T11:27:03.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 -- start start 2024-05-14T11:27:03.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f49b00cd1c0 0x7f49b00cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b006f0a0 con 0x7f49b006c700 2024-05-14T11:27:03.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 -- --> v1:172.21.15.149:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b00cbb60 con 0x7f49b00d1730 2024-05-14T11:27:03.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b7000640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f49b006f420 con 0x7f49b00cd1c0 2024-05-14T11:27:03.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b5000640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f49b006c700 0x7f49b006cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39248/0 (socket says 172.21.15.116:39248) 2024-05-14T11:27:03.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49b5000640 1 -- 172.21.15.116:0/3147356507 learned_addr learned my addr 172.21.15.116:0/3147356507 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49afe00640 1 --2- 172.21.15.116:0/3147356507 >> v2:172.21.15.149:3300/0 conn(0x7f49b00cd1c0 0x7f49b00cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832033184 0 0) 0x7f49b006f0a0 con 0x7f49b006c700 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4994003530 con 0x7f49b006c700 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 <== mon.1 v1:172.21.15.149:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595479453 0 0) 0x7f499c001120 con 0x7f49b00d1730 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 --> v1:172.21.15.149:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49b006f0a0 con 0x7f49b00d1730 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49afe00640 1 -- 172.21.15.116:0/3147356507 >> v1:172.21.15.149:6789/0 conn(0x7f49b00d1730 legacy=0x7f49b00d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49afe00640 1 -- 172.21.15.116:0/3147356507 >> v1:172.21.15.116:6789/0 conn(0x7f49b006c700 legacy=0x7f49b006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.717+0000 7f49afe00640 1 -- 172.21.15.116:0/3147356507 --> v2:172.21.15.149:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b006f700 con 0x7f49b00cd1c0 2024-05-14T11:27:03.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49afe00640 1 --2- 172.21.15.116:0/3147356507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b00cd1c0 0x7f49b00cd5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49a00097c0 tx=0x7f49a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3a8be574859dab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00aa600 con 0x7f49b00cd1c0 2024-05-14T11:27:03.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a00af070 con 0x7f49b00cd1c0 2024-05-14T11:27:03.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49aea00640 1 -- 172.21.15.116:0/3147356507 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00aa9c0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 -- 172.21.15.116:0/3147356507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b00cd1c0 msgr2=0x7f49b00cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 --2- 172.21.15.116:0/3147356507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b00cd1c0 0x7f49b00cd5d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49a00097c0 tx=0x7f49a001c4a0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 -- 172.21.15.116:0/3147356507 shutdown_connections 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 --2- 172.21.15.116:0/3147356507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b00cd1c0 0x7f49b00cd5d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 -- 172.21.15.116:0/3147356507 >> 172.21.15.116:0/3147356507 conn(0x7f49b00b4720 msgr2=0x7f49b00b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 -- 172.21.15.116:0/3147356507 shutdown_connections 2024-05-14T11:27:03.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.721+0000 7f49b7000640 1 -- 172.21.15.116:0/3147356507 wait complete. 2024-05-14T11:27:03.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 Processor -- start 2024-05-14T11:27:03.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 -- start start 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b006c700 0x7f49b00cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49b00d1730 0x7f49b015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00cddb0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015cf10 con 0x7f49b006c700 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b7000640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00cdf30 con 0x7f49b00d1730 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49af400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49af400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58780/0 (socket says 172.21.15.116:58780) 2024-05-14T11:27:03.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49af400640 1 -- 172.21.15.116:0/2577602114 learned_addr learned my addr 172.21.15.116:0/2577602114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:27:03.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49b5000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49b00d1730 0x7f49b015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.725+0000 7f49afe00640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b006c700 0x7f49b00cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 -- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49b00d1730 msgr2=0x7f49b015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49b00d1730 0x7f49b015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 -- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b006c700 msgr2=0x7f49b00cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b006c700 0x7f49b00cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 -- 172.21.15.116:0/2577602114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01fd100 con 0x7f49b00cd1c0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49af400640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c4d0 tx=0x7f499c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0052d0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c005470 con 0x7f49b00cd1c0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c012870 con 0x7f49b00cd1c0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01fd3e0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01fd9a0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b01fdd80 con 0x7f49b00cd1c0 2024-05-14T11:27:03.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.729+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f499c012a10 con 0x7f49b00cd1c0 2024-05-14T11:27:03.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.733+0000 7f49a7e00640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 0x7f4990071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:03.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.733+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f499c08ed90 con 0x7f49b00cd1c0 2024-05-14T11:27:03.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.733+0000 7f49afe00640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 0x7f4990071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:03.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.733+0000 7f49afe00640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 0x7f4990071910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f49a00ae000 tx=0x7f49a00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:03.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.737+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f49b01fdd80 con 0x7f49b00cd1c0 2024-05-14T11:27:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.893+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f49b006a9c0 con 0x7f49b00cd1c0 2024-05-14T11:27:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.897+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f49b00cbb60 con 0x7f49b00cd1c0 2024-05-14T11:27:03.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.897+0000 7f49a7e00640 1 -- 172.21.15.116:0/2577602114 <== mon.0 v2:172.21.15.116:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f499c007e40 con 0x7f49b00cd1c0 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 msgr2=0x7f4990071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 0x7f4990071910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f49a00ae000 tx=0x7f49a00a7000 comp rx=0 tx=0).stop 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 msgr2=0x7f49b0157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c4d0 tx=0x7f499c00c990 comp rx=0 tx=0).stop 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 shutdown_connections 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f499006f400 0x7f4990071910 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f49b00d1730 0x7f49b015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49b00cd1c0 0x7f49b0157640 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 --2- 172.21.15.116:0/2577602114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49b006c700 0x7f49b00cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:03.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 >> 172.21.15.116:0/2577602114 conn(0x7f49b00b4720 msgr2=0x7f49b015bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:03.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 shutdown_connections 2024-05-14T11:27:03.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:03.901+0000 7f49b7000640 1 -- 172.21.15.116:0/2577602114 wait complete. 2024-05-14T11:27:03.919 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T11:27:03.920 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-14T11:27:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.065+0000 7f984ec00640 1 Processor -- start 2024-05-14T11:27:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.065+0000 7f984ec00640 1 -- start start 2024-05-14T11:27:04.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984ec00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f984806c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984ec00640 1 --2- >> v2:172.21.15.149:3300/0 conn(0x7f9848072a30 0x7f9848072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984ec00640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f984806cc40 con 0x7f984806f730 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984ec00640 1 -- --> v2:172.21.15.149:3300/0 -- mon_getmap magic: 0 -- 0x7f984806db20 con 0x7f9848072a30 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984ec00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f984806cf10 con 0x7f98480b0770 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f983f400640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f984806f730 0x7f984806fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:39258/0 (socket says 172.21.15.116:39258) 2024-05-14T11:27:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f983f400640 1 -- 172.21.15.116:0/1005163104 learned_addr learned my addr 172.21.15.116:0/1005163104 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f984806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984cc00640 1 --2- 172.21.15.116:0/1005163104 >> v2:172.21.15.149:3300/0 conn(0x7f9848072a30 0x7f9848072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544431574 0 0) 0x7f984806cc40 con 0x7f984806f730 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c003530 con 0x7f984806f730 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1018430827 0 0) 0x7f982c003530 con 0x7f984806f730 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 msgr2=0x7f984806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f984806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 >> v2:172.21.15.149:3300/0 conn(0x7f9848072a30 msgr2=0x7f9848072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 --2- 172.21.15.116:0/1005163104 >> v2:172.21.15.149:3300/0 conn(0x7f9848072a30 0x7f9848072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984806d1f0 con 0x7f984806f730 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 383175348 0 0) 0x7f9830001700 con 0x7f984806f730 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 >> v1:172.21.15.116:6789/0 conn(0x7f984806f730 legacy=0x7f984806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f982c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f982c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f982c004d50 0x7f982c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f984806db20 con 0x7f982c003a10 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f984806cf10 con 0x7f982c004d50 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f982c00bb00 con 0x7f98480b0770 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f982c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f983f400640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f982c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f984cc00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f982c004d50 0x7f982c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 -- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 msgr2=0x7f982c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98480b0770 0x7f982c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 -- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f982c004d50 msgr2=0x7f982c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f982c004d50 0x7f982c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.069+0000 7f9847e00640 1 -- 172.21.15.116:0/1005163104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c003530 con 0x7f982c003a10 2024-05-14T11:27:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f9847e00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f982c003e80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9838009ab0 tx=0x7f983801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c16c46f8556162 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380096b0 con 0x7f982c003a10 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f98380a64b0 con 0x7f982c003a10 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f9847400640 1 -- 172.21.15.116:0/1005163104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380a6880 con 0x7f982c003a10 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 -- 172.21.15.116:0/1005163104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 msgr2=0x7f982c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 --2- 172.21.15.116:0/1005163104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f982c003e80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9838009ab0 tx=0x7f983801ca40 comp rx=0 tx=0).stop 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984cc00640 1 -- 172.21.15.116:0/1005163104 reap_dead start 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 -- 172.21.15.116:0/1005163104 shutdown_connections 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 -- 172.21.15.116:0/1005163104 >> 172.21.15.116:0/1005163104 conn(0x7f98480b4620 msgr2=0x7f98480b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 -- 172.21.15.116:0/1005163104 shutdown_connections 2024-05-14T11:27:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.073+0000 7f984ec00640 1 -- 172.21.15.116:0/1005163104 wait complete. 2024-05-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 Processor -- start 2024-05-14T11:27:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 -- start start 2024-05-14T11:27:04.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 0x7f9848074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f984814cd90 0x7f984814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9848073500 con 0x7f982c003a10 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9848073380 con 0x7f984814d2c0 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984ec00640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9848073680 con 0x7f984814cd90 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58792/0 (socket says 172.21.15.116:58792) 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 -- 172.21.15.116:0/3698825535 learned_addr learned my addr 172.21.15.116:0/3698825535 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f984cc00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f984814cd90 0x7f984814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f983f400640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 0x7f9848074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 -- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f984814cd90 msgr2=0x7f984814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f984814cd90 0x7f984814e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 -- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 msgr2=0x7f9848074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 0x7f9848074050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 -- 172.21.15.116:0/3698825535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984815f1b0 con 0x7f982c003a10 2024-05-14T11:27:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f983f400640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 0x7f9848074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.077+0000 7f9847e00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9838009ab0 tx=0x7f982c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838005a60 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f98380bf070 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838005e20 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848156a20 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984814d810 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984815f490 con 0x7f982c003a10 2024-05-14T11:27:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9838006040 con 0x7f982c003a10 2024-05-14T11:27:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 0x7f9824071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T11:27:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5507+0+0 (secure 0 0 0) 0x7f98381299f0 con 0x7f982c003a10 2024-05-14T11:27:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.081+0000 7f983f400640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 0x7f9824071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T11:27:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.085+0000 7f983f400640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 0x7f9824071940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f984815b440 tx=0x7f9830009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T11:27:04.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.085+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f984815f490 con 0x7f982c003a10 2024-05-14T11:27:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9848156810 con 0x7f982c003a10 2024-05-14T11:27:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f9844c00640 1 -- 172.21.15.116:0/3698825535 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3883 (secure 0 0 0) 0x7f9838016780 con 0x7f982c003a10 2024-05-14T11:27:04.280 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-05-14T11:27:04.280 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6834/3484256661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3484256661},{"type":"v1","addr":"172.21.15.149:6834","nonce":3484256661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3542491539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6833","nonce":3542491539},{"type":"v1","addr":"172.21.15.149:6835","nonce":3542491539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2210951049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2210951049},{"type":"v1","addr":"172.21.15.116:6835","nonce":2210951049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3407718246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3407718246},{"type":"v1","addr":"172.21.15.116:6837","nonce":3407718246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-05-14T11:27:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 115 2024-05-14T11:27:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 msgr2=0x7f9824071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 0x7f9824071940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f984815b440 tx=0x7f9830009040 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 msgr2=0x7f9848073b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9838009ab0 tx=0x7f982c005840 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 shutdown_connections 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:6832/1273922728,v1:172.21.15.116:6833/1273922728] conn(0x7f982406f430 0x7f9824071940 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f984814d2c0 0x7f9848074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f984814cd90 0x7f984814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 --2- 172.21.15.116:0/3698825535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f982c003a10 0x7f9848073b10 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 >> 172.21.15.116:0/3698825535 conn(0x7f98480b4620 msgr2=0x7f984815c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 shutdown_connections 2024-05-14T11:27:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-05-14T11:27:04.281+0000 7f984ec00640 1 -- 172.21.15.116:0/3698825535 wait complete. 2024-05-14T11:27:04.301 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-14T11:27:04.301 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-14T11:27:04.302 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-14T11:27:04.302 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:04.313 INFO:tasks.ceph.mds.a:Stopped 2024-05-14T11:27:04.313 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-14T11:27:04.313 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:04.375 INFO:tasks.ceph.mds.c:Stopped 2024-05-14T11:27:04.375 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-14T11:27:04.375 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:04.389 INFO:tasks.ceph.mds.b:Stopped 2024-05-14T11:27:04.389 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-14T11:27:04.389 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:04.444 INFO:tasks.ceph.mds.d:Stopped 2024-05-14T11:27:04.444 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-14T11:27:04.445 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-14T11:27:04.445 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:04.510 INFO:tasks.ceph.osd.0:Stopped 2024-05-14T11:27:04.510 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-14T11:27:04.510 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:06.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-14T11:27:10.612 INFO:tasks.ceph.osd.1:Stopped 2024-05-14T11:27:10.612 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-14T11:27:10.612 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:10.685 INFO:tasks.ceph.osd.2:Stopped 2024-05-14T11:27:10.686 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-14T11:27:10.686 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:11.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-14T11:27:11.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-14T11:27:14.162 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:14.165 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:16.788 INFO:tasks.ceph.osd.3:Stopped 2024-05-14T11:27:16.788 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-14T11:27:16.788 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:16.854 INFO:tasks.ceph.osd.4:Stopped 2024-05-14T11:27:16.855 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-14T11:27:16.855 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:17.699 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-14T11:27:17.700 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-14T11:27:22.956 INFO:tasks.ceph.osd.5:Stopped 2024-05-14T11:27:22.956 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-14T11:27:22.957 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:23.032 INFO:tasks.ceph.osd.6:Stopped 2024-05-14T11:27:23.033 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-14T11:27:23.033 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:23.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2024-05-14T11:27:28.909 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-14T11:27:28.910 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-14T11:27:29.134 INFO:tasks.ceph.osd.7:Stopped 2024-05-14T11:27:29.134 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-14T11:27:29.134 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-14T11:27:29.134 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:29.194 INFO:tasks.ceph.mgr.y:Stopped 2024-05-14T11:27:29.194 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-14T11:27:29.194 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:29.249 INFO:tasks.ceph.mgr.x:Stopped 2024-05-14T11:27:29.249 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-14T11:27:29.250 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-14T11:27:29.250 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:29.270 INFO:tasks.ceph.mon.a:Stopped 2024-05-14T11:27:29.270 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-14T11:27:29.270 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:29.287 INFO:tasks.ceph.mon.b:Stopped 2024-05-14T11:27:29.287 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-14T11:27:29.287 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T11:27:29.348 INFO:tasks.ceph.mon.c:Stopped 2024-05-14T11:27:29.349 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-14T11:27:29.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-14T11:27:29.386 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:20:28.519345+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-05-14T11:27:29.387 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-05-14T11:27:29.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-14T11:27:29.469 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-14T11:27:29.548 INFO:teuthology.orchestra.run.smithi116.stdout:2024-05-14T11:22:24.028712+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-14T11:27:29.549 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:27:29.549 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-14T11:27:29.983 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:27:29.983 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-14T11:27:30.167 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:27:30.167 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-14T11:27:30.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:27:30.334 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-14T11:27:30.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:27:30.511 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-14T11:27:30.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:27:30.797 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-14T11:27:31.010 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:27:31.010 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-14T11:27:31.185 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:27:31.185 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-14T11:27:31.352 INFO:tasks.ceph:Archiving mon data... 2024-05-14T11:27:31.353 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/data/mon.a.tgz 2024-05-14T11:27:31.353 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-05-14T11:27:31.362 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.3rHqZQchhP 2024-05-14T11:27:31.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3rHqZQchhP 2024-05-14T11:27:31.504 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.3rHqZQchhP 2024-05-14T11:27:31.582 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.3rHqZQchhP is 411KB 2024-05-14T11:27:31.645 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.3rHqZQchhP 2024-05-14T11:27:31.653 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/data/mon.b.tgz 2024-05-14T11:27:31.653 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-05-14T11:27:31.662 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.FpftWLtQhj 2024-05-14T11:27:31.662 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FpftWLtQhj 2024-05-14T11:27:31.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.FpftWLtQhj 2024-05-14T11:27:31.875 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.FpftWLtQhj is 416KB 2024-05-14T11:27:31.938 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.FpftWLtQhj 2024-05-14T11:27:31.946 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/data/mon.c.tgz 2024-05-14T11:27:31.946 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-05-14T11:27:31.994 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.vtsZfRodFK 2024-05-14T11:27:31.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vtsZfRodFK 2024-05-14T11:27:32.131 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.vtsZfRodFK 2024-05-14T11:27:32.203 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.vtsZfRodFK is 416KB 2024-05-14T11:27:32.266 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.vtsZfRodFK 2024-05-14T11:27:32.274 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-14T11:27:32.274 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-14T11:27:32.277 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-14T11:27:32.346 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-14T11:27:32.383 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-14T11:27:32.383 INFO:tasks.ceph:Archiving crash dumps... 2024-05-14T11:27:32.385 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi116/crash 2024-05-14T11:27:32.385 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-14T11:27:32.409 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi149/crash 2024-05-14T11:27:32.410 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-14T11:27:32.441 INFO:tasks.ceph:Shutting down logrotate 2024-05-14T11:27:32.441 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:32.461 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:32.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:32.561 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-14T11:27:32.585 INFO:tasks.ceph:Compressing logs... 2024-05-14T11:27:32.585 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T11:27:32.609 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T11:27:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40674.log 2024-05-14T11:27:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38862.log 2024-05-14T11:27:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2024-05-14T11:27:32.632 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40674.log.gz 2024-05-14T11:27:32.633 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38394.log 2024-05-14T11:27:32.633 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38862.log.gz 2024-05-14T11:27:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-05-14T11:27:32.635 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gz 2024-05-14T11:27:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46229.log 2024-05-14T11:27:32.636 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38394.log.gz 2024-05-14T11:27:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41582.log 2024-05-14T11:27:32.636 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-05-14T11:27:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-05-14T11:27:32.637 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46229.log.gz 2024-05-14T11:27:32.637 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50904.log 2024-05-14T11:27:32.637 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41582.log.gz 2024-05-14T11:27:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38658.log 2024-05-14T11:27:32.638 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-05-14T11:27:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33443.log 2024-05-14T11:27:32.639 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50904.log.gz 2024-05-14T11:27:32.639 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51295.log 2024-05-14T11:27:32.640 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33443.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38658.log.gz 2024-05-14T11:27:32.640 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36517.log 2024-05-14T11:27:32.640 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33443.log.gz 2024-05-14T11:27:32.641 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41660.log 2024-05-14T11:27:32.641 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51295.log.gz 2024-05-14T11:27:32.641 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39111.log 2024-05-14T11:27:32.641 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36517.log.gz 2024-05-14T11:27:32.642 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45386.log 2024-05-14T11:27:32.642 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41660.log.gz 2024-05-14T11:27:32.643 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41878.log 2024-05-14T11:27:32.643 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39111.log.gz 2024-05-14T11:27:32.644 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2024-05-14T11:27:32.644 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45386.log.gz 2024-05-14T11:27:32.644 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-05-14T11:27:32.645 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41878.log.gz 2024-05-14T11:27:32.645 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38631.log 2024-05-14T11:27:32.646 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2024-05-14T11:27:32.646 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-05-14T11:27:32.646 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz 2024-05-14T11:27:32.647 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-05-14T11:27:32.647 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38631.log.gz 2024-05-14T11:27:32.647 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50748.log 2024-05-14T11:27:32.648 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz 2024-05-14T11:27:32.648 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-05-14T11:27:32.649 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50748.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-14T11:27:32.649 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50748.log.gz 2024-05-14T11:27:32.650 INFO:teuthology.orchestra.run.smithi116.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-05-14T11:27:32.650 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-05-14T11:27:32.650 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-05-14T11:27:32.651 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-05-14T11:27:32.652 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-05-14T11:27:32.652 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-05-14T11:27:32.652 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-05-14T11:27:32.653 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-05-14T11:27:32.653 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2024-05-14T11:27:32.653 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-05-14T11:27:32.654 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46281.log 2024-05-14T11:27:32.654 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-05-14T11:27:32.655 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-05-14T11:27:32.655 INFO:teuthology.orchestra.run.smithi116.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2024-05-14T11:27:32.656 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39085.log 2024-05-14T11:27:32.656 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46281.log.gz 2024-05-14T11:27:32.657 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-05-14T11:27:32.657 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-05-14T11:27:32.658 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35813.log 2024-05-14T11:27:32.658 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39085.log.gz 2024-05-14T11:27:32.659 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49036.log 2024-05-14T11:27:32.659 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-05-14T11:27:32.659 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46741.log 2024-05-14T11:27:32.660 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35813.log.gz 2024-05-14T11:27:32.660 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.49036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-05-14T11:27:32.660 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27847.log 2024-05-14T11:27:32.661 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44315.log 2024-05-14T11:27:32.661 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46741.log.gz 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27775.log 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27847.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27847.log.gz 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27775.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27775.log.gz 2024-05-14T11:27:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50521.log 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44315.log.gz 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34790.log 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50521.log.gz 2024-05-14T11:27:32.665 INFO:teuthology.orchestra.run.smithi116.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.49036.log.gz 2024-05-14T11:27:32.666 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz 2024-05-14T11:27:32.666 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-05-14T11:27:32.668 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-05-14T11:27:32.668 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34790.log.gz 2024-05-14T11:27:32.668 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41736.log 2024-05-14T11:27:32.668 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27751.log 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27799.log 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27751.log.gz 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:gzip/var/log/ceph/ceph-client.admin.27799.log: -5 --verbose -- /var/log/ceph/ceph.log 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27799.log.gz 2024-05-14T11:27:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27871.log 2024-05-14T11:27:32.670 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27727.log 2024-05-14T11:27:32.673 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41736.log.gz 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36848.log 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45227.log 2024-05-14T11:27:32.674 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36848.log.gz 2024-05-14T11:27:32.677 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27871.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27871.log.gz 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33408.log 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27727.log.gz 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.3.33408.log: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.3.33408.log.gz 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33289.log 2024-05-14T11:27:32.678 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-14T11:27:32.680 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-05-14T11:27:32.680 INFO:teuthology.orchestra.run.smithi116.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-05-14T11:27:32.680 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37525.log 2024-05-14T11:27:32.680 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45227.log.gz 2024-05-14T11:27:32.680 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49561.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37525.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49561.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42230.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48051.log 2024-05-14T11:27:32.681 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.42230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35981.log 2024-05-14T11:27:32.682 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.1.33289.log: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-14T11:27:32.682 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mds.d.log: 88.7% -- replaced with /var/log/ceph/ceph-client.1.33289.log.gz 2024-05-14T11:27:32.682 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-14T11:27:32.682 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-14T11:27:32.682 INFO:teuthology.orchestra.run.smithi149.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-14T11:27:32.683 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-14T11:27:32.685 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48051.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48051.log.gz 2024-05-14T11:27:32.685 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39850.log 2024-05-14T11:27:32.685 INFO:teuthology.orchestra.run.smithi116.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.42230.log.gz 2024-05-14T11:27:32.685 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34173.log 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35981.log: /var/log/ceph/ceph-client.admin.39850.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35981.log.gz 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39850.log.gz 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46715.log 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34173.log.gz 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43079.log 2024-05-14T11:27:32.686 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-05-14T11:27:32.687 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-14T11:27:32.687 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33750.log 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46715.log.gz 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43079.log.gz 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37421.log 2024-05-14T11:27:32.688 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33750.log.gz 2024-05-14T11:27:32.689 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27703.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2024-05-14T11:27:32.689 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27823.log 2024-05-14T11:27:32.690 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43727.log 2024-05-14T11:27:32.690 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-05-14T11:27:32.690 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-05-14T11:27:32.690 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37421.log.gz 2024-05-14T11:27:32.691 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51112.log 2024-05-14T11:27:32.691 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43727.log.gz 2024-05-14T11:27:32.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.27823.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27823.log.gz 2024-05-14T11:27:32.692 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35701.log 2024-05-14T11:27:32.692 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2024-05-14T11:27:32.692 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51112.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.51112.log.gz 2024-05-14T11:27:32.692 INFO:teuthology.orchestra.run.smithi116.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43753.log 2024-05-14T11:27:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41213.log 2024-05-14T11:27:32.693 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35701.log.gz 2024-05-14T11:27:32.694 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33292.log 2024-05-14T11:27:32.694 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43753.log.gz 2024-05-14T11:27:32.695 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39059.log 2024-05-14T11:27:32.695 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41213.log.gz 2024-05-14T11:27:32.695 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50163.log 2024-05-14T11:27:32.696 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33292.log.gz 2024-05-14T11:27:32.696 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45995.log 2024-05-14T11:27:32.696 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39059.log.gz 2024-05-14T11:27:32.697 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43647.log 2024-05-14T11:27:32.697 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50163.log.gz 2024-05-14T11:27:32.698 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47501.log 2024-05-14T11:27:32.698 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45995.log.gz 2024-05-14T11:27:32.699 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34303.log 2024-05-14T11:27:32.699 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43647.log.gz 2024-05-14T11:27:32.700 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43961.log 2024-05-14T11:27:32.700 INFO:teuthology.orchestra.run.smithi116.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.47501.log.gz 2024-05-14T11:27:32.701 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34303.log.gz 2024-05-14T11:27:32.701 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2024-05-14T11:27:32.701 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41448.log 2024-05-14T11:27:32.702 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43961.log.gz 2024-05-14T11:27:32.702 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-05-14T11:27:32.703 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2024-05-14T11:27:32.703 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34979.log 2024-05-14T11:27:32.704 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41448.log.gz 2024-05-14T11:27:32.704 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-05-14T11:27:32.704 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-05-14T11:27:32.705 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-05-14T11:27:32.705 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34979.log.gz 2024-05-14T11:27:32.706 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-05-14T11:27:32.706 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-05-14T11:27:32.707 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36822.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-05-14T11:27:32.707 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36740.log 2024-05-14T11:27:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-05-14T11:27:32.708 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-05-14T11:27:32.709 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2024-05-14T11:27:32.709 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36740.log.gz 2024-05-14T11:27:32.709 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41044.log 2024-05-14T11:27:32.710 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-05-14T11:27:32.711 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41148.log 2024-05-14T11:27:32.711 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2024-05-14T11:27:32.711 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-05-14T11:27:32.711 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41044.log.gz 2024-05-14T11:27:32.712 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-05-14T11:27:32.712 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41148.log.gz 2024-05-14T11:27:32.713 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49442.log 2024-05-14T11:27:32.713 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-05-14T11:27:32.714 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37934.log 2024-05-14T11:27:32.714 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-05-14T11:27:32.715 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40265.log 2024-05-14T11:27:32.715 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49442.log.gz 2024-05-14T11:27:32.716 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39371.log 2024-05-14T11:27:32.716 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37934.log.gz 2024-05-14T11:27:32.716 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41634.log 2024-05-14T11:27:32.717 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40265.log.gz 2024-05-14T11:27:32.718 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36321.log 2024-05-14T11:27:32.718 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39371.log.gz 2024-05-14T11:27:32.719 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34590.log 2024-05-14T11:27:32.719 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41634.log.gz 2024-05-14T11:27:32.720 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-05-14T11:27:32.720 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36321.log.gz 2024-05-14T11:27:32.721 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34225.log 2024-05-14T11:27:32.721 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34590.log.gz 2024-05-14T11:27:32.721 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49210.log 2024-05-14T11:27:32.721 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-05-14T11:27:32.722 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2024-05-14T11:27:32.722 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34225.log.gz 2024-05-14T11:27:32.723 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35870.log 2024-05-14T11:27:32.723 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49210.log.gz 2024-05-14T11:27:32.724 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-05-14T11:27:32.724 INFO:teuthology.orchestra.run.smithi116.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2024-05-14T11:27:32.725 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-05-14T11:27:32.725 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35870.log.gz 2024-05-14T11:27:32.726 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2024-05-14T11:27:32.726 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-05-14T11:27:32.727 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36411.log 2024-05-14T11:27:32.727 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-05-14T11:27:32.728 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51008.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2024-05-14T11:27:32.728 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48181.log 2024-05-14T11:27:32.728 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40817.log 2024-05-14T11:27:32.729 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36411.log.gz 2024-05-14T11:27:32.729 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48181.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-14T11:27:32.730 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48181.log.gz 2024-05-14T11:27:32.730 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44360.log 2024-05-14T11:27:32.731 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40817.log.gz 2024-05-14T11:27:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-05-14T11:27:32.733 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-05-14T11:27:32.733 INFO:teuthology.orchestra.run.smithi116.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.44360.log.gz 2024-05-14T11:27:32.733 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48774.log 2024-05-14T11:27:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46125.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gzgzip 2024-05-14T11:27:32.735 INFO:teuthology.orchestra.run.smithi116.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42124.log 2024-05-14T11:27:32.735 INFO:teuthology.orchestra.run.smithi116.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-05-14T11:27:32.735 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-05-14T11:27:32.736 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42124.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48774.log.gz 2024-05-14T11:27:32.736 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2024-05-14T11:27:32.737 INFO:teuthology.orchestra.run.smithi116.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.42124.log.gz 2024-05-14T11:27:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-05-14T11:27:32.738 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-05-14T11:27:32.738 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-05-14T11:27:32.739 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2024-05-14T11:27:32.739 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-05-14T11:27:32.740 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-05-14T11:27:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-05-14T11:27:32.742 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36952.log: /var/log/ceph/ceph-client.admin.35753.log: gzip -5 --verbose 91.1% -- -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz /var/log/ceph/ceph-client.admin.35164.log 2024-05-14T11:27:32.742 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-05-14T11:27:32.742 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-05-14T11:27:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-05-14T11:27:32.743 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-05-14T11:27:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51060.log 2024-05-14T11:27:32.745 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51138.log: 91.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35164.log.gz 2024-05-14T11:27:32.745 INFO:teuthology.orchestra.run.smithi116.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33225.log 2024-05-14T11:27:32.745 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-05-14T11:27:32.746 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-05-14T11:27:32.746 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51060.log.gz 2024-05-14T11:27:32.747 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45092.log 2024-05-14T11:27:32.747 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33225.log: /var/log/ceph/ceph-client.admin.35786.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33225.log.gz 2024-05-14T11:27:32.748 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-14T11:27:32.748 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-05-14T11:27:32.749 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-05-14T11:27:32.749 INFO:teuthology.orchestra.run.smithi116.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45092.log.gz 2024-05-14T11:27:32.750 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50677.log 2024-05-14T11:27:32.751 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43501.log 2024-05-14T11:27:32.751 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-05-14T11:27:32.752 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51269.log 2024-05-14T11:27:32.752 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50677.log.gz 2024-05-14T11:27:32.753 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47870.log 2024-05-14T11:27:32.753 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43501.log.gz 2024-05-14T11:27:32.755 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2024-05-14T11:27:32.755 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51269.log.gz 2024-05-14T11:27:32.755 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42994.log 2024-05-14T11:27:32.756 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47870.log.gz 2024-05-14T11:27:32.756 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-05-14T11:27:32.757 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2024-05-14T11:27:32.758 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-05-14T11:27:32.758 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42994.log.gz 2024-05-14T11:27:32.759 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-05-14T11:27:32.759 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-05-14T11:27:32.760 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47998.log 2024-05-14T11:27:32.760 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-05-14T11:27:32.761 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43469.log 2024-05-14T11:27:32.761 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-05-14T11:27:32.762 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42782.log 2024-05-14T11:27:32.762 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47998.log.gz 2024-05-14T11:27:32.763 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48904.log 2024-05-14T11:27:32.764 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43469.log.gz 2024-05-14T11:27:32.764 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40522.log 2024-05-14T11:27:32.765 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48904.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42782.log.gz 2024-05-14T11:27:32.765 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46463.log 2024-05-14T11:27:32.766 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48904.log.gz 2024-05-14T11:27:32.766 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2024-05-14T11:27:32.767 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40522.log.gz 2024-05-14T11:27:32.768 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40739.log 2024-05-14T11:27:32.768 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46463.log.gz 2024-05-14T11:27:32.769 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38659.log 2024-05-14T11:27:32.769 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2024-05-14T11:27:32.770 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40739.log.gz 2024-05-14T11:27:32.770 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-05-14T11:27:32.771 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36033.log 2024-05-14T11:27:32.771 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38659.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-14T11:27:32.772 INFO:teuthology.orchestra.run.smithi116.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38659.log.gz 2024-05-14T11:27:32.772 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47115.log 2024-05-14T11:27:32.772 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-05-14T11:27:32.772 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38730.log 2024-05-14T11:27:32.773 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36033.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36033.log.gz 2024-05-14T11:27:32.774 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-05-14T11:27:32.774 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47115.log.gz 2024-05-14T11:27:32.774 INFO:teuthology.orchestra.run.smithi116.stderr:gzip/var/log/ceph/ceph-client.admin.38730.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27801.log 2024-05-14T11:27:32.775 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48077.log 2024-05-14T11:27:32.776 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-05-14T11:27:32.776 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38730.log.gz 2024-05-14T11:27:32.776 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.27801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41018.log 2024-05-14T11:27:32.777 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27801.log.gz 2024-05-14T11:27:32.777 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51086.log 2024-05-14T11:27:32.777 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48077.log.gz 2024-05-14T11:27:32.778 INFO:teuthology.orchestra.run.smithi149.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-14T11:27:32.778 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-05-14T11:27:32.778 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41018.log.gz 2024-05-14T11:27:32.779 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41278.log 2024-05-14T11:27:32.779 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51086.log.gz 2024-05-14T11:27:32.780 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47815.log 2024-05-14T11:27:32.780 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-05-14T11:27:32.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35649.log 2024-05-14T11:27:32.781 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41278.log.gz 2024-05-14T11:27:32.782 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-05-14T11:27:32.782 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35649.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47815.log.gz 2024-05-14T11:27:32.783 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35649.log.gz 2024-05-14T11:27:32.783 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36688.log 2024-05-14T11:27:32.784 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42577.log 2024-05-14T11:27:32.784 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-05-14T11:27:32.785 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38785.log 2024-05-14T11:27:32.785 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36688.log.gz 2024-05-14T11:27:32.786 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38052.log 2024-05-14T11:27:32.786 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42577.log.gz 2024-05-14T11:27:32.787 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39772.log 2024-05-14T11:27:32.788 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38785.log.gz 2024-05-14T11:27:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-05-14T11:27:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38052.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38052.log.gz 2024-05-14T11:27:32.789 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-05-14T11:27:32.789 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39772.log.gz 2024-05-14T11:27:32.790 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40572.log 2024-05-14T11:27:32.790 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-05-14T11:27:32.791 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51401.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-05-14T11:27:32.791 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36085.log 2024-05-14T11:27:32.791 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38316.log 2024-05-14T11:27:32.792 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40572.log: /var/log/ceph/ceph-client.admin.36085.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40572.log.gz 2024-05-14T11:27:32.792 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33672.log 2024-05-14T11:27:32.792 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36085.log.gz 2024-05-14T11:27:32.793 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-05-14T11:27:32.793 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38316.log.gz 2024-05-14T11:27:32.794 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33672.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33672.log.gz 2024-05-14T11:27:32.794 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-05-14T11:27:32.795 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43332.log 2024-05-14T11:27:32.795 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-05-14T11:27:32.796 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-05-14T11:27:32.796 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2024-05-14T11:27:32.797 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-05-14T11:27:32.797 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43332.log.gz 2024-05-14T11:27:32.798 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-05-14T11:27:32.798 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-05-14T11:27:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51347.log 2024-05-14T11:27:32.802 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-05-14T11:27:32.803 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47067.log 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51347.log.gz 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50956.log 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2024-05-14T11:27:32.804 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-05-14T11:27:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2024-05-14T11:27:32.805 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50956.log.gz 2024-05-14T11:27:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-05-14T11:27:32.805 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2024-05-14T11:27:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-05-14T11:27:32.806 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2024-05-14T11:27:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37989.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-05-14T11:27:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-05-14T11:27:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34377.log 2024-05-14T11:27:32.808 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-05-14T11:27:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35675.log 2024-05-14T11:27:32.809 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-05-14T11:27:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-05-14T11:27:32.810 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34377.log.gz 2024-05-14T11:27:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2024-05-14T11:27:32.811 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35675.log.gz 2024-05-14T11:27:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44606.log 2024-05-14T11:27:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-05-14T11:27:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34371.log 2024-05-14T11:27:32.813 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2024-05-14T11:27:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.44606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-05-14T11:27:32.815 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34371.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34371.log.gz 2024-05-14T11:27:32.815 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39954.log 2024-05-14T11:27:32.816 INFO:teuthology.orchestra.run.smithi116.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.44606.log.gz 2024-05-14T11:27:32.816 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-05-14T11:27:32.816 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz 2024-05-14T11:27:32.816 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-05-14T11:27:32.817 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39954.log.gz 2024-05-14T11:27:32.817 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37674.log 2024-05-14T11:27:32.818 INFO:teuthology.orchestra.run.smithi116.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-05-14T11:27:32.818 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36437.log 2024-05-14T11:27:32.819 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2024-05-14T11:27:32.820 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46314.log 2024-05-14T11:27:32.820 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37674.log.gz 2024-05-14T11:27:32.820 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45360.log 2024-05-14T11:27:32.821 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36437.log.gz 2024-05-14T11:27:32.821 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47401.log 2024-05-14T11:27:32.821 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46314.log.gz 2024-05-14T11:27:32.822 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-05-14T11:27:32.823 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45360.log.gz 2024-05-14T11:27:32.823 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-05-14T11:27:32.824 INFO:teuthology.orchestra.run.smithi116.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47401.log.gz 2024-05-14T11:27:32.824 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-05-14T11:27:32.824 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-05-14T11:27:32.825 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50043.log 2024-05-14T11:27:32.825 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-05-14T11:27:32.826 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48103.log 2024-05-14T11:27:32.827 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50043.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-05-14T11:27:32.827 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33501.log 2024-05-14T11:27:32.828 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-05-14T11:27:32.828 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48103.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.50043.log.gz 2024-05-14T11:27:32.829 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33501.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48103.log.gz 2024-05-14T11:27:32.829 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33776.log 2024-05-14T11:27:32.829 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33501.log.gz 2024-05-14T11:27:32.830 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2024-05-14T11:27:32.831 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36714.log 2024-05-14T11:27:32.831 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-05-14T11:27:32.831 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39824.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.33776.log.gz 2024-05-14T11:27:32.832 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38212.log 2024-05-14T11:27:32.832 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2024-05-14T11:27:32.833 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-05-14T11:27:32.833 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36714.log.gz 2024-05-14T11:27:32.834 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36653.log 2024-05-14T11:27:32.834 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38212.log.gz 2024-05-14T11:27:32.835 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35109.log 2024-05-14T11:27:32.835 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-05-14T11:27:32.835 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-05-14T11:27:32.836 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36653.log.gz 2024-05-14T11:27:32.837 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44289.log 2024-05-14T11:27:32.837 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35109.log.gz 2024-05-14T11:27:32.837 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39033.log 2024-05-14T11:27:32.838 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44289.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-05-14T11:27:32.839 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44289.log.gz 2024-05-14T11:27:32.839 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-14T11:27:32.839 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2024-05-14T11:27:32.840 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39033.log.gz 2024-05-14T11:27:32.840 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48024.log 2024-05-14T11:27:32.842 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51243.log 2024-05-14T11:27:32.842 INFO:teuthology.orchestra.run.smithi116.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz 2024-05-14T11:27:32.843 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40869.log 2024-05-14T11:27:32.843 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48024.log.gz 2024-05-14T11:27:32.844 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46520.log 2024-05-14T11:27:32.844 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51243.log.gz 2024-05-14T11:27:32.845 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40598.log 2024-05-14T11:27:32.845 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40869.log.gz 2024-05-14T11:27:32.845 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42914.log 2024-05-14T11:27:32.846 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46520.log.gz 2024-05-14T11:27:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2024-05-14T11:27:32.847 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40598.log.gz 2024-05-14T11:27:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.42914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-05-14T11:27:32.849 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35495.log 2024-05-14T11:27:32.849 INFO:teuthology.orchestra.run.smithi116.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.42914.log.gz 2024-05-14T11:27:32.849 INFO:teuthology.orchestra.run.smithi116.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2024-05-14T11:27:32.850 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49760.log 2024-05-14T11:27:32.850 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-05-14T11:27:32.851 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.2.35495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-05-14T11:27:32.852 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-05-14T11:27:32.852 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.2.35495.log.gz 2024-05-14T11:27:32.853 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49760.log.gz 2024-05-14T11:27:32.853 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-05-14T11:27:32.854 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-05-14T11:27:32.854 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39286.log 2024-05-14T11:27:32.855 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz 2024-05-14T11:27:32.855 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33594.log 2024-05-14T11:27:32.855 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-05-14T11:27:32.856 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51034.log 2024-05-14T11:27:32.857 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39286.log.gz 2024-05-14T11:27:32.858 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40843.log 2024-05-14T11:27:32.858 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33594.log.gz 2024-05-14T11:27:32.858 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-05-14T11:27:32.859 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51034.log.gz 2024-05-14T11:27:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:gzip/var/log/ceph/ceph-client.admin.40843.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2024-05-14T11:27:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43565.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2024-05-14T11:27:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51321.log 2024-05-14T11:27:32.861 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-05-14T11:27:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34379.log 2024-05-14T11:27:32.862 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2024-05-14T11:27:32.863 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-05-14T11:27:32.863 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51321.log.gz 2024-05-14T11:27:32.864 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35844.log 2024-05-14T11:27:32.864 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34379.log.gz 2024-05-14T11:27:32.864 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47546.log 2024-05-14T11:27:32.865 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-05-14T11:27:32.866 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41304.log 2024-05-14T11:27:32.866 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35844.log.gz 2024-05-14T11:27:32.867 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37058.log 2024-05-14T11:27:32.867 INFO:teuthology.orchestra.run.smithi116.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47546.log.gz 2024-05-14T11:27:32.868 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-05-14T11:27:32.868 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41304.log.gz 2024-05-14T11:27:32.869 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.37058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-05-14T11:27:32.870 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49383.log 2024-05-14T11:27:32.870 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-05-14T11:27:32.872 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47171.log 2024-05-14T11:27:32.873 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49383.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-05-14T11:27:32.873 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.49383.log.gz 2024-05-14T11:27:32.873 INFO:teuthology.orchestra.run.smithi116.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42174.log 2024-05-14T11:27:32.874 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.47171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45143.log 2024-05-14T11:27:32.876 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42174.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.37058.log.gz 2024-05-14T11:27:32.876 INFO:teuthology.orchestra.run.smithi116.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42174.log.gz 2024-05-14T11:27:32.877 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45143.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45143.log.gz 2024-05-14T11:27:32.877 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-05-14T11:27:32.878 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36059.log 2024-05-14T11:27:32.880 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41474.log 2024-05-14T11:27:32.880 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-05-14T11:27:32.881 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-05-14T11:27:32.881 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36059.log.gz 2024-05-14T11:27:32.881 INFO:teuthology.orchestra.run.smithi116.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.47171.log.gz 2024-05-14T11:27:32.882 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-05-14T11:27:32.882 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41474.log.gz 2024-05-14T11:27:32.883 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38897.log 2024-05-14T11:27:32.885 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37447.log 2024-05-14T11:27:32.885 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-05-14T11:27:32.885 INFO:teuthology.orchestra.run.smithi116.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-05-14T11:27:32.885 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38897.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-14T11:27:32.885 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38897.log.gz 2024-05-14T11:27:32.886 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36140.log 2024-05-14T11:27:32.887 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37447.log.gz 2024-05-14T11:27:32.887 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-05-14T11:27:32.889 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45826.log 2024-05-14T11:27:32.889 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36140.log.gz 2024-05-14T11:27:32.890 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37473.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-05-14T11:27:32.891 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45826.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45826.log.gz 2024-05-14T11:27:32.891 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-05-14T11:27:32.893 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40368.log 2024-05-14T11:27:32.894 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-05-14T11:27:32.894 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-05-14T11:27:32.895 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40368.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2024-05-14T11:27:32.896 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34588.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-05-14T11:27:32.897 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49987.log 2024-05-14T11:27:32.898 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45859.log 2024-05-14T11:27:32.900 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49987.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49987.log.gz 2024-05-14T11:27:32.901 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45859.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-05-14T11:27:32.901 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34585.log 2024-05-14T11:27:32.902 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-05-14T11:27:32.903 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44171.log 2024-05-14T11:27:32.903 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34585.log.gz 2024-05-14T11:27:32.905 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37395.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-05-14T11:27:32.908 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.44171.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.44171.log.gz 2024-05-14T11:27:32.909 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-05-14T11:27:32.910 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34251.log 2024-05-14T11:27:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-05-14T11:27:32.912 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-05-14T11:27:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34251.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34251.log.gz 2024-05-14T11:27:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43591.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-05-14T11:27:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2024-05-14T11:27:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44836.log 2024-05-14T11:27:32.916 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45015.log 2024-05-14T11:27:32.917 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2024-05-14T11:27:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39549.log 2024-05-14T11:27:32.919 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44836.log.gz 2024-05-14T11:27:32.920 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-05-14T11:27:32.920 INFO:teuthology.orchestra.run.smithi116.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45015.log.gz 2024-05-14T11:27:32.921 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51191.log 2024-05-14T11:27:32.921 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39549.log.gz 2024-05-14T11:27:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34381.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-05-14T11:27:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34095.log 2024-05-14T11:27:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43935.log 2024-05-14T11:27:32.924 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51191.log.gz 2024-05-14T11:27:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-05-14T11:27:32.925 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34095.log.gz 2024-05-14T11:27:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45776.log 2024-05-14T11:27:32.926 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2024-05-14T11:27:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39338.log 2024-05-14T11:27:32.928 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-05-14T11:27:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40086.log 2024-05-14T11:27:32.929 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45776.log.gz/var/log/ceph/ceph-client.admin.39338.log: 2024-05-14T11:27:32.930 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39338.log.gz 2024-05-14T11:27:32.933 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.40086.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.40086.log.gz 2024-05-14T11:27:32.934 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-05-14T11:27:32.935 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39007.log 2024-05-14T11:27:32.937 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34375.log 2024-05-14T11:27:32.937 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-05-14T11:27:32.937 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39694.log 2024-05-14T11:27:32.937 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2024-05-14T11:27:32.939 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34375.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34375.log.gz 2024-05-14T11:27:32.939 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34329.log 2024-05-14T11:27:32.940 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39694.log.gz 2024-05-14T11:27:32.940 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-05-14T11:27:32.942 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2024-05-14T11:27:32.942 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34329.log.gz 2024-05-14T11:27:32.943 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36289.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-05-14T11:27:32.944 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50599.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2024-05-14T11:27:32.944 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38420.log 2024-05-14T11:27:32.945 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-05-14T11:27:32.946 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40215.log 2024-05-14T11:27:32.947 INFO:teuthology.orchestra.run.smithi116.stderr: 91.2%/var/log/ceph/ceph-client.admin.46637.log: -- replaced with /var/log/ceph/ceph-client.admin.38420.log.gz 2024-05-14T11:27:32.948 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-05-14T11:27:32.949 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40215.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.40215.log.gz 2024-05-14T11:27:32.950 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35350.log 2024-05-14T11:27:32.951 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-05-14T11:27:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.35350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35597.log 2024-05-14T11:27:32.953 INFO:teuthology.orchestra.run.smithi116.stderr: 89.9%/var/log/ceph/ceph-client.admin.44886.log: -- replaced with /var/log/ceph/ceph-client.0.35350.log.gz 2024-05-14T11:27:32.953 INFO:teuthology.orchestra.run.smithi116.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-05-14T11:27:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35597.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35597.log.gz 2024-05-14T11:27:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44115.log 2024-05-14T11:27:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35031.log 2024-05-14T11:27:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38017.log 2024-05-14T11:27:32.958 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44115.log.gz 2024-05-14T11:27:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35031.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35031.log.gz 2024-05-14T11:27:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38017.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38017.log.gz 2024-05-14T11:27:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-05-14T11:27:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41096.log 2024-05-14T11:27:32.963 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-05-14T11:27:32.963 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41096.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-05-14T11:27:32.963 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41096.log.gz 2024-05-14T11:27:32.965 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48207.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-05-14T11:27:32.965 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38160.log 2024-05-14T11:27:32.966 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-05-14T11:27:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38160.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38160.log.gz 2024-05-14T11:27:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37596.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-05-14T11:27:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-05-14T11:27:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37189.log 2024-05-14T11:27:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-05-14T11:27:32.971 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-05-14T11:27:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37189.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37189.log.gz 2024-05-14T11:27:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49103.log/var/log/ceph/ceph-client.admin.39403.log: 2024-05-14T11:27:32.973 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-05-14T11:27:32.974 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-05-14T11:27:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50464.log 2024-05-14T11:27:32.975 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49103.log.gz 2024-05-14T11:27:32.977 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46411.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gz 2024-05-14T11:27:32.978 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50464.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-05-14T11:27:32.978 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48664.log 2024-05-14T11:27:32.979 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2024-05-14T11:27:32.981 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43987.log 2024-05-14T11:27:32.981 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48664.log.gz 2024-05-14T11:27:32.981 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41608.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2024-05-14T11:27:32.982 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45595.log 2024-05-14T11:27:32.983 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43987.log.gz 2024-05-14T11:27:32.983 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40966.log 2024-05-14T11:27:32.986 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47635.log 2024-05-14T11:27:32.986 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40966.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45595.log.gz 2024-05-14T11:27:32.986 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-05-14T11:27:32.987 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47635.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47635.log.gz 2024-05-14T11:27:32.987 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40470.log 2024-05-14T11:27:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42398.log 2024-05-14T11:27:32.990 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33566.log 2024-05-14T11:27:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42398.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40470.log.gz 2024-05-14T11:27:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33566.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.42398.log.gz 2024-05-14T11:27:32.992 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33566.log.gz 2024-05-14T11:27:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39980.log 2024-05-14T11:27:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41330.log 2024-05-14T11:27:32.994 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39980.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-14T11:27:32.995 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-14T11:27:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41330.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.39980.log.gz 2024-05-14T11:27:32.996 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41330.log.gz 2024-05-14T11:27:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-05-14T11:27:33.000 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48800.log 2024-05-14T11:27:33.000 INFO:teuthology.orchestra.run.smithi116.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-14T11:27:33.001 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33471.log 2024-05-14T11:27:33.001 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-05-14T11:27:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48800.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48800.log.gz 2024-05-14T11:27:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33471.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33471.log.gz 2024-05-14T11:27:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37752.log 2024-05-14T11:27:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42318.log 2024-05-14T11:27:33.006 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-05-14T11:27:33.006 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37752.log.gz 2024-05-14T11:27:33.007 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42318.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42318.log.gz 2024-05-14T11:27:33.008 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36926.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-05-14T11:27:33.008 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2024-05-14T11:27:33.010 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-05-14T11:27:33.011 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-05-14T11:27:33.011 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz 2024-05-14T11:27:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34383.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-05-14T11:27:33.013 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38186.log 2024-05-14T11:27:33.013 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-05-14T11:27:33.014 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-05-14T11:27:33.015 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43131.log 2024-05-14T11:27:33.016 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38186.log.gz 2024-05-14T11:27:33.016 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41504.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-05-14T11:27:33.017 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43131.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43131.log.gz 2024-05-14T11:27:33.018 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-05-14T11:27:33.019 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50010.log 2024-05-14T11:27:33.021 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40190.log 2024-05-14T11:27:33.021 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-05-14T11:27:33.021 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-05-14T11:27:33.022 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40190.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.50010.log.gz 2024-05-14T11:27:33.023 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40190.log.gz 2024-05-14T11:27:33.023 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50982.log: /var/log/ceph/ceph-client.admin.40895.log 2024-05-14T11:27:33.024 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-05-14T11:27:33.024 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33536.log 2024-05-14T11:27:33.026 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41422.log 2024-05-14T11:27:33.026 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40895.log.gz 2024-05-14T11:27:33.026 INFO:teuthology.orchestra.run.smithi149.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-14T11:27:33.027 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33536.log.gz 2024-05-14T11:27:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41422.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41422.log.gz 2024-05-14T11:27:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-05-14T11:27:33.029 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41686.log 2024-05-14T11:27:33.032 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-05-14T11:27:33.032 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41686.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-05-14T11:27:33.032 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46177.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41686.log.gz 2024-05-14T11:27:33.033 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gz 2024-05-14T11:27:33.033 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36625.log 2024-05-14T11:27:33.034 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-05-14T11:27:33.036 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2024-05-14T11:27:33.036 INFO:teuthology.orchestra.run.smithi116.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36625.log.gz 2024-05-14T11:27:33.037 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38835.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-05-14T11:27:33.038 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44656.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2024-05-14T11:27:33.039 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37159.log 2024-05-14T11:27:33.040 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-05-14T11:27:33.042 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35727.log 2024-05-14T11:27:33.042 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37159.log.gz 2024-05-14T11:27:33.043 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2024-05-14T11:27:33.043 INFO:teuthology.orchestra.run.smithi116.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-05-14T11:27:33.043 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35727.log.gz 2024-05-14T11:27:33.044 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36465.log 2024-05-14T11:27:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38470.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2024-05-14T11:27:33.045 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41387.log 2024-05-14T11:27:33.046 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2024-05-14T11:27:33.046 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36465.log.gz 2024-05-14T11:27:33.048 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41387.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41387.log.gz 2024-05-14T11:27:33.049 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-05-14T11:27:33.049 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2024-05-14T11:27:33.049 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38527.log 2024-05-14T11:27:33.051 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36491.log 2024-05-14T11:27:33.051 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz 2024-05-14T11:27:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.38527.log: /var/log/ceph/ceph-client.admin.36491.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36491.log.gz 2024-05-14T11:27:33.053 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35057.log 2024-05-14T11:27:33.054 INFO:teuthology.orchestra.run.smithi116.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.38527.log.gz 2024-05-14T11:27:33.055 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47294.log 2024-05-14T11:27:33.056 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37726.log 2024-05-14T11:27:33.056 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35057.log.gz 2024-05-14T11:27:33.057 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47294.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47294.log.gz 2024-05-14T11:27:33.058 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37778.log 2024-05-14T11:27:33.058 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37726.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37726.log.gz 2024-05-14T11:27:33.059 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41792.log 2024-05-14T11:27:33.060 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-05-14T11:27:33.060 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37778.log.gz 2024-05-14T11:27:33.062 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.41792.log: /var/log/ceph/ceph-client.admin.47004.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-05-14T11:27:33.063 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37843.log 2024-05-14T11:27:33.064 INFO:teuthology.orchestra.run.smithi116.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.41792.log.gz 2024-05-14T11:27:33.064 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33391.log 2024-05-14T11:27:33.066 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47330.log 2024-05-14T11:27:33.066 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37843.log.gz 2024-05-14T11:27:33.067 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33391.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33391.log.gz 2024-05-14T11:27:33.068 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47330.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47330.log.gz 2024-05-14T11:27:33.068 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45440.log 2024-05-14T11:27:33.069 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-05-14T11:27:33.072 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49693.log 2024-05-14T11:27:33.072 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42527.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45440.log.gz 2024-05-14T11:27:33.072 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.49693.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-05-14T11:27:33.074 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34199.log 2024-05-14T11:27:33.075 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-05-14T11:27:33.077 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49587.log 2024-05-14T11:27:33.077 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34199.log: 93.5% -- replaced with /var/log/ceph/ceph-client.0.49693.log.gz 2024-05-14T11:27:33.078 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34199.log.gz 2024-05-14T11:27:33.078 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz 2024-05-14T11:27:33.079 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-05-14T11:27:33.080 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39720.log 2024-05-14T11:27:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49587.log: /var/log/ceph/ceph-client.admin.47609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39928.log 2024-05-14T11:27:33.082 INFO:teuthology.orchestra.run.smithi116.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-05-14T11:27:33.082 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39720.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49587.log.gz 2024-05-14T11:27:33.082 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39720.log.gz 2024-05-14T11:27:33.083 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50878.log 2024-05-14T11:27:33.084 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34573.log 2024-05-14T11:27:33.084 INFO:teuthology.orchestra.run.smithi116.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39928.log.gz 2024-05-14T11:27:33.085 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50878.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50878.log.gz 2024-05-14T11:27:33.086 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33826.log 2024-05-14T11:27:33.086 INFO:teuthology.orchestra.run.smithi116.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34573.log.gz 2024-05-14T11:27:33.087 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2024-05-14T11:27:33.089 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45748.log 2024-05-14T11:27:33.089 INFO:teuthology.orchestra.run.smithi116.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33826.log.gz 2024-05-14T11:27:33.090 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43306.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2024-05-14T11:27:33.091 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45748.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45748.log.gz 2024-05-14T11:27:33.091 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-05-14T11:27:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27582.log 2024-05-14T11:27:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2024-05-14T11:27:33.094 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-05-14T11:27:33.095 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.27582.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.27582.log.gz 2024-05-14T11:27:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46346.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2024-05-14T11:27:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38078.log 2024-05-14T11:27:33.098 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37960.log 2024-05-14T11:27:33.100 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-05-14T11:27:33.100 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-05-14T11:27:33.100 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37960.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37960.log.gz 2024-05-14T11:27:33.101 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49467.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-05-14T11:27:33.102 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2024-05-14T11:27:33.103 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-05-14T11:27:33.105 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-05-14T11:27:33.105 INFO:teuthology.orchestra.run.smithi116.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2024-05-14T11:27:33.105 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-05-14T11:27:33.107 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34147.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-05-14T11:27:33.107 INFO:teuthology.orchestra.run.smithi116.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-05-14T11:27:33.107 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-05-14T11:27:33.108 INFO:teuthology.orchestra.run.smithi116.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-05-14T11:27:33.108 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38929.log 2024-05-14T11:27:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34365.log 2024-05-14T11:27:33.110 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-05-14T11:27:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38929.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38929.log.gz 2024-05-14T11:27:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34365.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34365.log.gz 2024-05-14T11:27:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41181.log 2024-05-14T11:27:33.113 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34039.log 2024-05-14T11:27:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45645.log 2024-05-14T11:27:33.115 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41181.log.gz 2024-05-14T11:27:33.116 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34039.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34039.log.gz 2024-05-14T11:27:33.117 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45645.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45645.log.gz 2024-05-14T11:27:33.118 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-05-14T11:27:33.119 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48748.log 2024-05-14T11:27:33.122 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-05-14T11:27:33.122 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48748.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-05-14T11:27:33.122 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48748.log.gz 2024-05-14T11:27:33.122 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44448.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-05-14T11:27:33.123 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22775.log 2024-05-14T11:27:33.124 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-05-14T11:27:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.tmp-client.admin.22775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35005.log 2024-05-14T11:27:33.125 INFO:teuthology.orchestra.run.smithi116.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22775.log.gz 2024-05-14T11:27:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41122.log 2024-05-14T11:27:33.127 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-05-14T11:27:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49323.log 2024-05-14T11:27:33.128 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35005.log.gz 2024-05-14T11:27:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41122.log.gz 2024-05-14T11:27:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35225.log 2024-05-14T11:27:33.130 INFO:teuthology.orchestra.run.smithi116.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49323.log.gz 2024-05-14T11:27:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40648.log 2024-05-14T11:27:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-05-14T11:27:33.132 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35225.log.gz 2024-05-14T11:27:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40648.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40648.log.gz 2024-05-14T11:27:33.134 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50547.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-05-14T11:27:33.135 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37219.log 2024-05-14T11:27:33.136 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-05-14T11:27:33.137 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35955.log 2024-05-14T11:27:33.138 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37219.log.gz 2024-05-14T11:27:33.138 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2024-05-14T11:27:33.138 INFO:teuthology.orchestra.run.smithi116.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-05-14T11:27:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35955.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35955.log.gz 2024-05-14T11:27:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35925.log 2024-05-14T11:27:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48519.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2024-05-14T11:27:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42962.log 2024-05-14T11:27:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34798.log 2024-05-14T11:27:33.143 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35925.log.gz 2024-05-14T11:27:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42962.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.42962.log.gz 2024-05-14T11:27:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34798.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34798.log.gz 2024-05-14T11:27:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-05-14T11:27:33.146 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-14T11:27:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-05-14T11:27:33.147 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-05-14T11:27:33.150 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.36796.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-05-14T11:27:33.150 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-05-14T11:27:33.151 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-14T11:27:33.151 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-14T11:27:33.153 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2024-05-14T11:27:33.153 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-05-14T11:27:33.154 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36900.log 2024-05-14T11:27:33.155 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48389.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2024-05-14T11:27:33.156 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36900.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36900.log.gz 2024-05-14T11:27:33.157 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-05-14T11:27:33.158 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50625.log 2024-05-14T11:27:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45412.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-05-14T11:27:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50625.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50625.log.gz 2024-05-14T11:27:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-05-14T11:27:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46689.log 2024-05-14T11:27:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44913.log: 91.5%/var/log/ceph/ceph-client.admin.46689.log: -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz 2024-05-14T11:27:33.164 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46689.log.gz 2024-05-14T11:27:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2024-05-14T11:27:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-05-14T11:27:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2024-05-14T11:27:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48571.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2024-05-14T11:27:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-14T11:27:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46437.log 2024-05-14T11:27:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.46437.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46437.log.gz 2024-05-14T11:27:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38581.log 2024-05-14T11:27:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38581.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38581.log.gz 2024-05-14T11:27:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48716.log 2024-05-14T11:27:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-05-14T11:27:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48716.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48716.log.gz 2024-05-14T11:27:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50389.log 2024-05-14T11:27:33.180 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-05-14T11:27:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.50389.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.50389.log.gz 2024-05-14T11:27:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-05-14T11:27:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43675.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-05-14T11:27:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2024-05-14T11:27:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2024-05-14T11:27:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34567.log 2024-05-14T11:27:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-05-14T11:27:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34567.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34567.log.gz 2024-05-14T11:27:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50495.log 2024-05-14T11:27:33.196 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-05-14T11:27:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50495.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50495.log.gz 2024-05-14T11:27:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-05-14T11:27:33.202 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44474.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-05-14T11:27:33.202 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50573.log 2024-05-14T11:27:33.205 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36007.log 2024-05-14T11:27:33.205 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50573.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50573.log.gz 2024-05-14T11:27:33.208 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36007.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36007.log.gz 2024-05-14T11:27:33.208 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40765.log 2024-05-14T11:27:33.211 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40765.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40765.log.gz 2024-05-14T11:27:33.211 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-05-14T11:27:33.214 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43025.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-05-14T11:27:33.214 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41070.log 2024-05-14T11:27:33.218 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34121.log 2024-05-14T11:27:33.218 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41070.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41070.log.gz 2024-05-14T11:27:33.219 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48129.log 2024-05-14T11:27:33.219 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34121.log.gz 2024-05-14T11:27:33.221 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48129.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48129.log.gz 2024-05-14T11:27:33.222 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36874.log 2024-05-14T11:27:33.224 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41530.log 2024-05-14T11:27:33.226 INFO:teuthology.orchestra.run.smithi116.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36874.log.gz 2024-05-14T11:27:33.227 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-05-14T11:27:33.229 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41530.log.gz 2024-05-14T11:27:33.232 INFO:teuthology.orchestra.run.smithi116.stderr:gzip/var/log/ceph/ceph-client.admin.45491.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-05-14T11:27:33.232 INFO:teuthology.orchestra.run.smithi116.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50227.log 2024-05-14T11:27:33.236 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50227.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50227.log.gz 2024-05-14T11:27:33.236 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-05-14T11:27:33.238 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47264.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-05-14T11:27:33.239 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51165.log 2024-05-14T11:27:33.240 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2024-05-14T11:27:33.242 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51165.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51165.log.gz 2024-05-14T11:27:33.244 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34575.log 2024-05-14T11:27:33.245 INFO:teuthology.orchestra.run.smithi116.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz 2024-05-14T11:27:33.247 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39234.log 2024-05-14T11:27:33.248 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34575.log.gz 2024-05-14T11:27:33.251 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39234.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39234.log.gz 2024-05-14T11:27:33.251 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39902.log 2024-05-14T11:27:33.254 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39902.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39902.log.gz 2024-05-14T11:27:33.254 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33698.log 2024-05-14T11:27:33.257 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33698.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33698.log.gz 2024-05-14T11:27:33.257 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50774.log 2024-05-14T11:27:33.258 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38264.log 2024-05-14T11:27:33.260 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50774.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50774.log.gz 2024-05-14T11:27:33.263 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38264.log.gz 2024-05-14T11:27:33.264 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49266.log 2024-05-14T11:27:33.270 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.49266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46047.log 2024-05-14T11:27:33.271 INFO:teuthology.orchestra.run.smithi116.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.49266.log.gz 2024-05-14T11:27:33.273 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35898.log 2024-05-14T11:27:33.275 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46047.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41556.log 2024-05-14T11:27:33.275 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-05-14T11:27:33.277 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35898.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35898.log.gz 2024-05-14T11:27:33.278 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50257.log 2024-05-14T11:27:33.279 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46869.log 2024-05-14T11:27:33.280 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50257.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41556.log.gz 2024-05-14T11:27:33.281 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50257.log.gz 2024-05-14T11:27:33.282 INFO:teuthology.orchestra.run.smithi116.stderr: 91.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50333.log 2024-05-14T11:27:33.282 INFO:teuthology.orchestra.run.smithi116.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-14T11:27:33.284 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-05-14T11:27:33.285 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50333.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46869.log.gz 2024-05-14T11:27:33.285 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50333.log.gz 2024-05-14T11:27:33.286 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-05-14T11:27:33.288 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37315.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-05-14T11:27:33.289 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37293.log 2024-05-14T11:27:33.289 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-05-14T11:27:33.289 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39494.log 2024-05-14T11:27:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37293.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37293.log.gz 2024-05-14T11:27:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49819.log 2024-05-14T11:27:33.292 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39494.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gz 2024-05-14T11:27:33.293 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38368.log 2024-05-14T11:27:33.294 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49819.log.gz 2024-05-14T11:27:33.295 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-05-14T11:27:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38368.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38368.log.gz 2024-05-14T11:27:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44230.log 2024-05-14T11:27:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44706.log: 91.0%/var/log/ceph/ceph-client.admin.44230.log: -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2024-05-14T11:27:33.299 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35539.log 2024-05-14T11:27:33.299 INFO:teuthology.orchestra.run.smithi116.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44230.log.gz 2024-05-14T11:27:33.300 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35136.log 2024-05-14T11:27:33.301 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35539.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35539.log.gz 2024-05-14T11:27:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39577.log 2024-05-14T11:27:33.303 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35136.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35136.log.gz 2024-05-14T11:27:33.303 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2024-05-14T11:27:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39577.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39577.log.gz 2024-05-14T11:27:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33724.log 2024-05-14T11:27:33.306 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40394.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-05-14T11:27:33.307 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34368.log 2024-05-14T11:27:33.308 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33724.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33724.log.gz 2024-05-14T11:27:33.309 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-05-14T11:27:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34368.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34368.log.gz 2024-05-14T11:27:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47329.log 2024-05-14T11:27:33.311 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-05-14T11:27:33.312 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-05-14T11:27:33.314 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47329.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47329.log.gz 2024-05-14T11:27:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33646.log 2024-05-14T11:27:33.315 INFO:teuthology.orchestra.run.smithi116.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-05-14T11:27:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-05-14T11:27:33.317 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33646.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33646.log.gz 2024-05-14T11:27:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42603.log 2024-05-14T11:27:33.319 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46793.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-05-14T11:27:33.319 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33954.log 2024-05-14T11:27:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42603.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.42603.log.gz 2024-05-14T11:27:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37132.log 2024-05-14T11:27:33.323 INFO:teuthology.orchestra.run.smithi116.stderr: 92.6%/var/log/ceph/ceph-client.admin.37132.log: -- replaced with /var/log/ceph/ceph-client.admin.33954.log.gz 2024-05-14T11:27:33.324 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-05-14T11:27:33.324 INFO:teuthology.orchestra.run.smithi116.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37132.log.gz 2024-05-14T11:27:33.325 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-05-14T11:27:33.327 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35083.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-05-14T11:27:33.327 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-05-14T11:27:33.328 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47920.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-05-14T11:27:33.328 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-05-14T11:27:33.330 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44963.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-05-14T11:27:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-05-14T11:27:33.331 INFO:teuthology.orchestra.run.smithi116.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-05-14T11:27:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38955.log 2024-05-14T11:27:33.333 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43701.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-05-14T11:27:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46925.log 2024-05-14T11:27:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38955.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38955.log.gz 2024-05-14T11:27:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47451.log 2024-05-14T11:27:33.338 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.46925.log: /var/log/ceph/ceph-client.admin.47451.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47451.log.gz 2024-05-14T11:27:33.338 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40791.log 2024-05-14T11:27:33.340 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40791.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40791.log.gz 2024-05-14T11:27:33.341 INFO:teuthology.orchestra.run.smithi116.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.46925.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49160.log 2024-05-14T11:27:33.341 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-05-14T11:27:33.342 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-05-14T11:27:33.344 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49160.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49160.log.gz 2024-05-14T11:27:33.345 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48454.log 2024-05-14T11:27:33.346 INFO:teuthology.orchestra.run.smithi116.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-05-14T11:27:33.346 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-05-14T11:27:33.348 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48454.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48454.log.gz 2024-05-14T11:27:33.348 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43831.log 2024-05-14T11:27:33.349 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48363.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2024-05-14T11:27:33.350 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2024-05-14T11:27:33.351 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43831.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43831.log.gz 2024-05-14T11:27:33.351 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33620.log 2024-05-14T11:27:33.353 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50283.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2024-05-14T11:27:33.353 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39260.log 2024-05-14T11:27:33.354 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33620.log.gz 2024-05-14T11:27:33.355 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38238.log 2024-05-14T11:27:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39260.log.gz 2024-05-14T11:27:33.356 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-05-14T11:27:33.358 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38238.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38238.log.gz 2024-05-14T11:27:33.358 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40921.log 2024-05-14T11:27:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34781.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-05-14T11:27:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-05-14T11:27:33.361 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40921.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40921.log.gz 2024-05-14T11:27:33.361 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38608.log 2024-05-14T11:27:33.363 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43384.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2024-05-14T11:27:33.363 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46203.log 2024-05-14T11:27:33.364 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38608.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.38608.log.gz 2024-05-14T11:27:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45066.log 2024-05-14T11:27:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46203.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46203.log.gz 2024-05-14T11:27:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-05-14T11:27:33.368 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.45066.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45066.log.gz 2024-05-14T11:27:33.368 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2024-05-14T11:27:33.369 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.42042.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-05-14T11:27:33.370 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40318.log 2024-05-14T11:27:33.371 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.51217.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2024-05-14T11:27:33.371 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50851.log 2024-05-14T11:27:33.373 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40318.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40318.log.gz 2024-05-14T11:27:33.373 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33325.log 2024-05-14T11:27:33.374 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50851.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50851.log.gz 2024-05-14T11:27:33.375 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-05-14T11:27:33.376 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.33325.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33325.log.gz 2024-05-14T11:27:33.377 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2024-05-14T11:27:33.377 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50651.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-05-14T11:27:33.378 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46607.log 2024-05-14T11:27:33.379 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50201.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2024-05-14T11:27:33.380 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-05-14T11:27:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46607.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46607.log.gz 2024-05-14T11:27:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36597.log 2024-05-14T11:27:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37551.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-05-14T11:27:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50088.log 2024-05-14T11:27:33.384 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36597.log.gz 2024-05-14T11:27:33.384 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-05-14T11:27:33.387 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.50088.log: /var/log/ceph/ceph-client.admin.48980.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.50088.log.gz 2024-05-14T11:27:33.387 INFO:teuthology.orchestra.run.smithi116.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-05-14T11:27:33.387 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34800.log 2024-05-14T11:27:33.388 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2024-05-14T11:27:33.390 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34800.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34800.log.gz 2024-05-14T11:27:33.390 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34277.log 2024-05-14T11:27:33.391 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49868.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2024-05-14T11:27:33.392 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47743.log 2024-05-14T11:27:33.393 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34277.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34277.log.gz 2024-05-14T11:27:33.393 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37337.log 2024-05-14T11:27:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47743.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47743.log.gz 2024-05-14T11:27:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-05-14T11:27:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37337.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2024-05-14T11:27:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40444.log 2024-05-14T11:27:33.399 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41961.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-05-14T11:27:33.399 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34783.log 2024-05-14T11:27:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40444.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40444.log.gz 2024-05-14T11:27:33.400 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37271.log 2024-05-14T11:27:33.402 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34783.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34783.log.gz 2024-05-14T11:27:33.402 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39468.log 2024-05-14T11:27:33.403 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.37271.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37271.log.gz 2024-05-14T11:27:33.404 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48878.log 2024-05-14T11:27:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39468.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39468.log.gz 2024-05-14T11:27:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44039.log 2024-05-14T11:27:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48878.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48878.log.gz 2024-05-14T11:27:33.407 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-05-14T11:27:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44039.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44039.log.gz 2024-05-14T11:27:33.409 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34012.log 2024-05-14T11:27:33.410 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49846.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-05-14T11:27:33.410 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36770.log 2024-05-14T11:27:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34012.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34012.log.gz 2024-05-14T11:27:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39612.log 2024-05-14T11:27:33.413 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36770.log.gz 2024-05-14T11:27:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2024-05-14T11:27:33.415 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.39612.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39612.log.gz 2024-05-14T11:27:33.415 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-05-14T11:27:33.417 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43779.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2024-05-14T11:27:33.417 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2024-05-14T11:27:33.418 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43209.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-05-14T11:27:33.419 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46663.log 2024-05-14T11:27:33.420 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38342.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz 2024-05-14T11:27:33.421 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46663.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46663.log.gz 2024-05-14T11:27:33.421 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40167.log 2024-05-14T11:27:33.423 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-05-14T11:27:33.424 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40167.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40167.log.gz 2024-05-14T11:27:33.425 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.47090.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz 2024-05-14T11:27:33.426 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-05-14T11:27:33.427 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27698.log 2024-05-14T11:27:33.429 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43183.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-05-14T11:27:33.429 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.27698.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27698.log.gz 2024-05-14T11:27:33.430 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38134.log 2024-05-14T11:27:33.431 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-05-14T11:27:33.433 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38134.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38134.log.gz 2024-05-14T11:27:33.433 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46977.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-05-14T11:27:33.434 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38290.log 2024-05-14T11:27:33.435 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-05-14T11:27:33.437 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38290.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38290.log.gz 2024-05-14T11:27:33.437 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-05-14T11:27:33.438 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44338.log 2024-05-14T11:27:33.439 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49637.log 2024-05-14T11:27:33.441 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44338.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44338.log.gz 2024-05-14T11:27:33.441 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49637.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49637.log.gz 2024-05-14T11:27:33.442 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44013.log 2024-05-14T11:27:33.443 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45547.log 2024-05-14T11:27:33.445 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.44013.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44013.log.gz 2024-05-14T11:27:33.446 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.0.45547.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-14T11:27:33.446 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40030.log 2024-05-14T11:27:33.447 INFO:teuthology.orchestra.run.smithi116.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.0.45547.log.gz 2024-05-14T11:27:33.447 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-05-14T11:27:33.449 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.40030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38108.log 2024-05-14T11:27:33.449 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40030.log.gz 2024-05-14T11:27:33.450 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.43410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-05-14T11:27:33.451 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.38108.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38108.log.gz 2024-05-14T11:27:33.451 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35565.log 2024-05-14T11:27:33.452 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2024-05-14T11:27:33.454 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.35565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-05-14T11:27:33.454 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35565.log.gz 2024-05-14T11:27:33.455 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.41359.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2024-05-14T11:27:33.456 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.36263.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-05-14T11:27:33.456 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34591.log 2024-05-14T11:27:33.458 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-05-14T11:27:33.459 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.34591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2024-05-14T11:27:33.459 INFO:teuthology.orchestra.run.smithi116.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34591.log.gz 2024-05-14T11:27:33.460 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.46255.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-05-14T11:27:33.461 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48690.log 2024-05-14T11:27:33.461 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.49531.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2024-05-14T11:27:33.463 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.48690.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48690.log.gz 2024-05-14T11:27:33.603 INFO:teuthology.orchestra.run.smithi116.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-14T11:27:33.862 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-14T11:27:33.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-14T11:27:34.013 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-14T11:27:34.070 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-14T11:27:34.076 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-14T11:27:34.078 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-14T11:27:34.078 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m1.441s 2024-05-14T11:27:34.079 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m5.695s 2024-05-14T11:27:34.079 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.508s 2024-05-14T11:27:34.127 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-14T11:27:34.211 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-14T11:27:34.361 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-14T11:27:34.393 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-14T11:27:34.395 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-05-14T11:27:34.395 INFO:teuthology.orchestra.run.smithi116.stderr:real 0m1.782s 2024-05-14T11:27:34.395 INFO:teuthology.orchestra.run.smithi116.stderr:user 0m7.450s 2024-05-14T11:27:34.395 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m1.536s 2024-05-14T11:27:34.396 INFO:tasks.ceph:Archiving logs... 2024-05-14T11:27:34.396 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi116/log 2024-05-14T11:27:34.397 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T11:27:35.670 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi149/log 2024-05-14T11:27:35.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T11:27:36.371 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T11:27:36.382 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T11:27:36.382 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T11:27:36.386 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T11:27:36.456 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T11:27:36.467 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T11:27:36.467 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T11:27:36.470 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi116.stdout: remote refid st t when poll reach delay offset jitter 2024-05-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================== 2024-05-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi116.stdout:+hv01.front.sepi 67.205.162.81 3 u 15 64 377 0.063 -2.787 0.752 2024-05-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi116.stdout:+hv02.front.sepi 63.231.80.2 3 u 10 64 377 0.091 -0.965 1.122 2024-05-14T11:27:36.482 INFO:teuthology.orchestra.run.smithi116.stdout:*hv03.front.sepi 74.6.168.72 3 u 20 64 377 0.084 -2.497 0.729 2024-05-14T11:27:36.483 INFO:teuthology.orchestra.run.smithi116.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-14T11:27:36.484 INFO:teuthology.orchestra.run.smithi149.stdout: remote refid st t when poll reach delay offset jitter 2024-05-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================== 2024-05-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi149.stdout:+hv01.front.sepi 67.205.162.81 3 u 19 64 377 0.090 -3.940 0.566 2024-05-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi149.stdout:*hv02.front.sepi 63.231.80.2 3 u 12 64 377 0.064 -2.448 0.500 2024-05-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi149.stdout:+hv03.front.sepi 74.6.168.72 3 u 11 64 377 0.091 -2.727 0.595 2024-05-14T11:27:36.485 INFO:teuthology.orchestra.run.smithi149.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-14T11:27:36.486 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T11:27:36.496 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T11:27:36.497 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T11:27:36.531 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T11:27:36.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T11:27:36.597 INFO:teuthology.task.internal:Duration was 1395.931459 seconds 2024-05-14T11:27:36.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T11:27:36.631 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T11:27:36.631 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T11:27:36.634 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T11:27:36.725 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T11:27:36.726 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-05-14T11:27:36.726 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T11:27:36.746 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-05-14T11:27:36.747 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T11:27:36.771 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T11:27:36.771 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T11:27:36.793 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T11:27:36.849 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T11:27:36.849 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T11:27:36.876 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T11:27:36.965 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T11:27:36.976 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T11:27:36.976 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T11:27:36.996 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T11:27:37.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T11:27:37.026 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T11:27:37.041 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T11:27:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-05-14T11:27:37.080 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-05-14T11:27:37.098 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T11:27:37.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:27:37.124 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T11:27:37.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T11:27:37.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T11:27:37.160 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T11:27:37.161 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi116 2024-05-14T11:27:37.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T11:27:37.195 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738/remote/smithi149 2024-05-14T11:27:37.195 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T11:27:37.221 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T11:27:37.221 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T11:27:37.232 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T11:27:37.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T11:27:37.279 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T11:27:37.279 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T11:27:37.307 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T11:27:37.307 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T11:27:37.310 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T11:27:37.315 INFO:teuthology.orchestra.run.smithi116.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 11:27 /home/ubuntu/cephtest 2024-05-14T11:27:37.319 INFO:teuthology.orchestra.run.smithi149.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 11:27 /home/ubuntu/cephtest 2024-05-14T11:27:37.321 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T11:27:37.341 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T11:27:37.416 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T11:27:37.437 DEBUG:teuthology.nuke:shortname: smithi116 2024-05-14T11:27:37.438 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T11:27:37.462 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 10:53:45.720940', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUFfQvC/Eeg5MY0zk5NfdO/V0tles3akRJGBV52ASHZhy74RNaRRuEH/JG/4tzI0p1mm2wuxczjBQZrvBgj6oI='} 2024-05-14T11:27:37.467 DEBUG:teuthology.nuke:shortname: smithi149 2024-05-14T11:27:37.467 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T11:27:37.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 10:53:45.719103', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMQCUVx0Oaewy2yohiXHF0W2rtGXKf/aJNUvheYs+LseaKr4ShASChwzQCxpjGn9M/gpHNvNQa1uUJxEZ4KHn4='} 2024-05-14T11:27:37.516 INFO:teuthology.orchestra.console.smithi116:Power off 2024-05-14T11:27:37.517 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T11:27:37.532 INFO:teuthology.orchestra.console.smithi149:Power off 2024-05-14T11:27:37.532 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T11:27:37.541 DEBUG:teuthology.orchestra.console.smithi116:power off output: Chassis Power Control: Down/Off 2024-05-14T11:27:37.541 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:37.553 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-05-14T11:27:37.553 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:37.563 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-05-14T11:27:37.574 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-14T11:27:41.564 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:41.575 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:41.675 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-05-14T11:27:41.685 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-14T11:27:45.678 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:45.687 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:45.787 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-05-14T11:27:45.796 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-05-14T11:27:49.789 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:49.797 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T11:27:49.899 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2024-05-14T11:27:49.906 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-05-14T11:27:50.000 INFO:teuthology.orchestra.console.smithi116:Power off completed 2024-05-14T11:27:50.006 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-05-14T11:27:50.111 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1395.9314589500427 failure_reason: '"2024-05-14T11:22:24.028712+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e mon.b-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e owner: scheduled_vshankar@teuthology success: false 2024-05-14T11:27:50.112 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T11:27:50.212 INFO:teuthology.run:FAIL